At the request of the United States Agency for International Development (USAID), this publication was prepared independently by International Business and Technical Consultants, Inc. (IBTCI). © 2015 K. Ochel/MI, Courtesy of Photoshare EVALUATION FINAL REPORT PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES February 2019 Performance Evaluation of USAID Ebola Pillar II Activities: Final Report USAID CONTRACT # AID-OAA￾I-15-00022 Task Order # AID-OAA￾TO-16-00040 Authors: Zhuzhi Moore, Annette Bongiovanni, Swati Sadaphal, Orlando Hernandez, Donna Espeut, Carol Levin, and Michael Toole Submitted December 31, 2018 Updated February 15, 2019 Prepared for: United States Agency for International Development Ronald Reagan Building and International Trade Center 1300 Pennsylvania Ave NW Washington, DC 20004 Prepared by: International Business & Technical Consultants, Inc. (IBTCI) 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA Tel: +1 (703) 749-0100 Citation: Moore Z et al. “Performance Evaluation of USAID Ebola Pillar II Activities: Final Report.” Evaluation Report to USAID, February 2019, International Business & Technical Consultants, Inc., Vienna, VA. Project Description: The Ebola Pillar II, Monitoring, Evaluation and Learning (MEL) activity is a three-year USAID-funded contract addressing USAID-coordinated efforts in mitigating the second-order impacts of the Ebola virus outbreak in Guinea, Liberia, and Sierra Leone. The activity focuses on four main components: evaluation, routine monitoring, data quality assurance, and improved knowledge management and learning. The activity is led and managed by International Business and Technical Consultants Inc. (IBTCI), with partners StatView International in Guinea, Global Research Insights, LTD (GRI) in Sierra Leone and Liberia, and Opinion Research Business (ORB) International in all three countries. Cover Photo: Two young children are orphans after losing their parents to Ebola virus disease in Monrovia, Liberia. © 2015 K. Ochel/MI, Courtesy of Photoshare. PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES FINAL REPORT February 2019 This publication is made possible by the generous support of the American people through the United States Agency for International Development (USAID) under the terms of contract AID-OAA-I-15-00022/AID-OAA-TO-16-00040 managed by International Business and Technical Consultants Inc. (IBTCI) and does not necessarily reflect the views of USAID or the United States Government. ACKNOWLEDGMENTS The authors would like to acknowledge the diligent involvement of the full technical team: Richard Columbia, Barbara de Zalduondo, Salima Mutima, Elisabeth Nolan, Naomi Rutenberg, Suzanne Essama-Bibi, Andrew Reuter, Edward Allan, Robert Grossman-Vermaas, Kapil Ahmed, Roger Emmanuel Millimono, Richard Ngafuan, Aaron Kokolie, and Samuel Dilito Turay. The authors are grateful to Cara Carter, Yuliya Dudaronak, and Ellie Bailey of ORB International and the data collection teams for implementing household and health facility surveys in Guinea, Liberia, and Sierra Leone and for conducting key informant interviews and focus groups discussions in Liberia and Sierra Leone. In addition, we want to thank Aliou Barry of StatView International and the data collection teams for conducting key informant interviews and focus groups discussions in Guinea. The authors would like to thank the project coordination staff for their continued support and commitment, including Katherine Labombarde and Meredith Kerrigan. Special thanks to the editors Mary Burket and Charlotte Wilkins and the graphic designer, Erin Dowling. Amadou Bakayoko, Amanda Boachie, Latrisha Chappin and Nadine Ritcheson from USAID’s Africa Bureau (USAID/AFR) and representatives from the bureaus for Global Health, Global Development Lab, Food Security, Democracy, Conflict, and Humanitarian Assistance, and Economic Growth, Education, and Environment, the Guinea, Liberia, and Sierra Leone missions, and various implementing partners provided valuable support and constructive comments throughout the preparation of this report. Finally, our gratitude goes to the many individuals who participated in the key informant interviews, focus group discussions, and surveys. This evaluation would not have been possible without your contributions. n i TABLE OF CONTENTS LIST OF TABLES AND FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii ACRONYMS AND ABBREVIATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Evaluation Purpose and Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Project Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Evaluation Design and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Analytical Domain: Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Analytical Domain: Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Analytical Domain: Gaps and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Analytical Domain: Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Evaluation Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Evaluation Objectives and Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 United States Government Activities Addressing EVD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 METHODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Evaluation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Data Collection Methodology and Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Data Management and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 FINDINGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Evaluation Question 1: Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Activity Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Ebola Pillar II Expenditure Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34 Evaluation Question 2: Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Evaluation Question 3: Gaps and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 ii n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES LIST OF TABLES AND FIGURES Figure 1. Regional map of the West Africa EVD outbreak . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Figure 2. Pillar II M&E Results Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Figure 3. Overall Ebola Pillar II theory of change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Figure 4. Map of the Ebola Pillar II priority areas in Guinea, Liberia, and Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Figure 5. Overview of the six Pillar II intervention types used in the PE1 analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Figure 6. Links between intervention types, themes, and Pillar II outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Figure 7. Overview of Pillar II activities by thematic area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 8. Distribution of Pillar II activities by thematic area, intervention type, and country . . . . . . . . . . . . . . . . . . . . . 22 Figure 9. Ebola Pillar II obligations and disbursements by country as of September 2017 . . . . . . . . . . . . . . . . . . . . . . . 23 Figure 10. Ebola Pillar II obligations and disbursements by intervention type as of September 2017 . . . . . . . . . . . . . . 23 Figure 11. Ebola Pillar II disbursements by country/region and intervention typology as of September 2017 . . . . . . . 24 Figure 12. Mean number of outpatient clients and the percentage change from previous year (Guinea) . . . . . . . . . . . . 25 Figure 13. Mean number of outpatient clients and the percentage change from previous year (Liberia) . . . . . . . . . . . . 29 Figure 14. Changes in HDDS between baseline and endline (Liberia) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Figure 15. Mean number of outpatient clients and the percentage change from previous year (Sierra Leone) . . . . . . . 34 Figure 16. Analytical framework to measure sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Table 1. Evaluation questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Table 2. Data collection methods and sources of information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Table 3. Respondent stakeholder groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Table 4. Mapping of Pillar II activities by thematic area(s) and intervention type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Table 5. Ebola Pillar II O&E by sector as of September 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Table 6. Reported Ebola Pillar II obligations and disbursements by intervention typology as of September 2017 . . . . . 23 Evaluation Question 4: Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 USAID/Washington . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Gaps and Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 NOTE: Annexes are supplied in a separate document. n iii ACRONYMS AND ABBREVIATIONS AEU Africa Ebola Unit AFR Africa Bureau AFS Agriculture and Food Security AfT Agenda for Transformation ANC Antenatal Care AOR Agreement Officer’s Representative APC Advancing Partners and Communities Project BCC Behavior Communication Change BFS Bureau for Food Security (USAID) CaLP Cash Learning Activity CAO County Agriculture Officer CBO Community-based Organizations CII Center for Innovation and Impact COR Contracting Officer’s Representative CDC Center for Disease Control and Prevention CEIDC Center of Excellence for Infectious Disease Control CEPPS Consortium for Elections and Political Processes Strengthening CfW Cash for Work CLTS Community-led Total Sanitation CRS Catholic Relief Services CSH Collaborative Support for Health CSML Civil Society and Media Leadership CSO Civil Society Organization CT Cash Transfer DFID United Kingdom Department for International Development DG Democracy & Governance DHHS Department of Health and Human Services DHIS 2 District Health Information Software, version 2 DHMT District Health Management Team DHS Demographic and Health Survey DOD Department of Defense DTRA Defense Threat Reduction Agency ECOWAS Economic Community of West African States ECRL Education Crisis Response in Liberia EDU Education EFSP Emergency Access to Food for EVD-Affected Guineans EOC Emergency Operation Center EPHS Essential Package of Health Services EQ Evaluation Question EREL Economic Recovery from Ebola for Liberia ERRP Ebola Recovery and Resilience Program ERSP Economic Recovery and Stabilization Plan ESF Economic Support Funds ETU Ebola Treatment Unit ETP&SS Ebola Transmission Prevention and Survivor Services EU European Union EVD Ebola Virus Disease FDA Food and Drug Administration FFP Food for Peace FGD Focus Group Discussion FMC Facility Management Committees FY Fiscal Year GAO Government Accountability Office GBV Gender-Based Violence gCHVs General Community Health Volunteers GECM Governance, Economic Crisis and Mitigation GH Global Health GHET Global Health Ebola Team GHP Global Health Programs (USAID) GHSC/PSM Global Health Supply Chain Program/ Procurement and Supply Management GNF Guinea Franc GOG Government of Guinea GOL Government of Liberia GOSL Government of Sierra Leone GSA General Services Agency HC3 Health Communication Capacity Collaborative HCW Health Care Worker HDDS Household Dietary Diversity Score HF Health Facility HFG Health Finance and Governance HH Household HHS Household Hunger Score HIS Health Information System HMIS Health Management Information System HRH Human Resources for Health HSD Health Service Delivery HSS Health System Strengthening IBTCI International Business and Technical Consultants, Inc. ICT Information and Communications Technology IDA International Disaster Assistance iv n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ILADP Improved Livelihoods and Agriculture Development Program IOM International Organization for Migration IP Implementing Partner IPC Infection Prevention and Control ITP Innovation, Technology & Partnership JSI John Snow, Inc. JSI R&T JSI Research &Training Institute JHUCCP Johns Hopkins University Center for Communication Programs KAP Knowledge, Attitudes and Practices KII Key Informant Interview LAB U.S. Global Development Lab LIPA Liberia Institute for Public Administration LOP Life of Project MACS Ministries, Agencies and Commissions MCP Management, Coordination and Partnerships MCSP Maternal and Child Survival Program MCSP/HRH Maternal and Child Survival Program/Human Resources for Health MCSP/RHS Maternal and Child Survival Program/ Restoration of Health Services MEL Monitoring, Evaluation and Learning MICS Multiple Indicator Cluster Survey MOAg Ministry of Agriculture MOE Ministry of Education MOH Ministry of Health (Liberia and Guinea) MOHS Ministry of Health and Sanitation (Sierra Leone) MSH Management Sciences for Health NaCSA National Commission for Social Action NDI National Democratic Institute NGO Nongovernmental Organization NIH National Institutes of Health O&E Obligations and Expenditures OD Organizational Development OFDA Office of Foreign Disaster Assistance OIG Office of the Inspector General OVC Orphans and Vulnerable Children OU Operating Unit PACS Partnership for Advancing Community-based Services PCI Project Concern International PDTTR Presidential Delivery Team on Transition and Recovery PERHS Post-Ebola Recovery of Health Services PHU Primary Health Care Unit PPE Personal Protective Equipment PPP Public Private Partnership PTA Parent Teacher Association RESSNER Rapid Ebola Social Safety Net and Economic Recovery RMNCH Reproductive Maternal Neonatal and Child Health SARA Service Availability and Readiness Assessment SBCC Social and Behavior Change Communication SECHNs State-enrolled Community Health Nurses SFG Search for Common Ground SIAPS System for Improved Access to Pharmaceuticals and Services SNAP Sustainable Nutrition and Agriculture Promotion SOP Standard Operating Procedure STAMP2 Sensor Technology and Analytics to Monitor, Predict and Protect Ebola STC Save the Children UCT Unconditional Cash Transfer UNICEF United Nations Children’s Fund UNMEER United Nations Mission for Ebola Emergency Response UNOPS United Nations Office for Project Services USAID United States Agency for International Development USG United States Government VSLA Village Savings and Loan Association WAHO West Africa Health Organization WAR Western Area Rural WASH Water, Sanitation, and Hygiene WAU Western Area Urban WB World Bank WFP World Food Programme WHO World Health Organization WinS WASH in Schools EXECUTIVE SUMMARY n 1 EXECUTIVE SUMMARY Evaluation Purpose and Questions 1. IBTCI collected reports and documents from the operating units (OUs) and implementing partners (IPs) before December 2017. The output and outcome figures stated in the report therefore do not reflect figures reached by December 2017. In October 2016, International Business & Technical Consultants, Inc. (IBTCI) received a contract from the United States Agency for International Development (USAID) to conduct an independent performance evaluation of activities funded by USAID aimed at assisting the governments of Guinea, Liberia, and Sierra Leone to recover from the 2014– 2016 Ebola Virus Disease (EVD) outbreak. These activities are referred to as Pillar II, whereas Pillar I was focused on the response to control the outbreak. Based on guidance in the scope of work and discussions with USAID’s Africa Bureau (USAID/AFR), the evaluation is primarily focused on USAID Ebola Pillar II activities implemented between March 2015 and December 2017.1 The purpose of the evaluation is to document the overall performance of Pillar II activities in each of the three countries—Guinea, Liberia, and Sierra Leone— and provide information and lessons learned to inform and improve USAID’s ability to respond effectively to future global health emergencies. The evaluation was guided by four questions: 1. How are USAID’s Pillar II Ebola Recovery activities in Guinea, Liberia, and Sierra Leone contributing to the achievement of the Ebola Pillar II Strategic Framework’s three objectives: to halt the loss of development gains; recover and strengthen key institutions and infrastructures; and, build sustained systems through public private partnerships (PPPs), innovation, and capacity building? 2. What results of Pillar II activities have endured after the activities have closed, and why? 3. What gaps and opportunities arose in the course of Pillar II activities that influenced the achievement of Pillar II objectives? 4. What lessons have been learned regarding how best to use the strengths of USAID’s emergency and long-term development mechanisms in a complex emergency? Project Background Guinea, Liberia, and Sierra Leone were the three countries most affected by the 2014–2016 EVD outbreak. This, in turn, led to the collapse of multiple government systems in each country, including health, agriculture, democracy and governance, trade and markets, and education, many of which were already fragile before the crisis. As such, the population of each country suffered significant second-order impacts on household welfare, human development, and their economy. As the U.S. Government (USG) lead for the post-EVD recovery, USAID designed, implemented, and oversaw activities with these objectives: prevent the loss of development gains; recover and strengthen existing institutions and infrastructures; and build sustainable systems through public-private partnerships, innovation, and capacity building. These activities aimed to strengthen key institutions and infrastructure (already weak before the start of the outbreak), restore citizens’ trust in their governments, rebuild communities’ willingness to accept social messaging on EVD, and ensure that recovery efforts increased each country’s ability to respond to and recover from similar crises in the future. Pillar II activities focused on six thematic areas: Agriculture and food security; non-EVD essential health and health system strengthening; governance and economic crisis mitigation; education; water and sanitation; and innovation, technology and partnerships. Pillar II activities were programmed by numerous USAID bureaus, offices, and the three affected missions, and implemented by a diverse group of implementing partners (IPs) aligned with international and host-country government EVD recovery strategies. Evaluation Design and Methods The performance evaluation used a mix of qualitative and quantitative methods in a three-phase design to create a cross-sectional picture of Pillar II accomplishments in each country. An evaluability assessment was conducted in Phase One (January through September, 2017), when the team identified relevant data sets, reviewed background documents, and conducted key informant interviews (KIIs) within USAID and selected IPs, host-country government officials, and 2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES other stakeholders in each country to finalize the evaluation design and theory of change. During Phase Two (October 2017 through January 2018), the evaluation team focused on implementation of the evaluation, which measures the progress of and identifies challenges to the implementation of USAID￾funded Pillar II activities. The data collection methods included desk reviews, KIIs, focus group discussions (FGDs), health facility surveys (IBTCI’s 2017 HFS), and household surveys (IBTCI’s 2017 HHS). During Phase 3 (February through March 2018), the team triangulated qualitative and quantitative data across six thematic areas and six intervention types (social protection; frontline worker support; management, coordination 2. For the household survey data, IBTCI used previous DHSs as pre-EVD baseline. The sampling methodology and questions are similar between DHSs and 2017 IBTCI’s household surveys and results are comparable. IBTCI’s 2017 HHS total figures refer to priority areas only (13 prefectures in Guinea, eight counties in Liberia, and eight districts in Sierra Leone), while the DHS total figures refer to the entire country. Subnational trend comparisons are shown in Annex L tables for Liberia and Sierra Leone. Guinea 2012 DHS disaggregated data at the regional level and not at the prefecture level. Hence, prefecture-level trend comparisons are not available. 3. This report focuses on the 148 USAID-funded recovery activities funded under Ebola Pillar II (Economic Support Funds (ESF) 73%; International Disaster Assistance (IDA) 25%, Global Health Programs (GHP) 2%); see September 2017 Obligations & Expenditures spreadsheet. A total of $1.86 billion was obligated across all four pillars by September 2017. and partnership (MCP); information, communication and technology (ICT); social and behavior change communication (SBCC); and institutional enhancements) in all three countries to answer the evaluation questions. The evaluation questions were categorized into four analytical domains: 1) performance; 2) sustainability; 3) gaps and opportunities; and, 4) management. Detailed information on the service delivery baseline data for outcome indicators, disaggregated by counties, prefectures, or districts is not available. It is therefore a challenge to assess and compare results of individual activities that vary greatly in duration and scale. Findings2 ANALYTICAL DOMAIN: PERFORMANCE Pillar II activities were designed to combine proven interventions in six thematic areas: agriculture and food security (AFS), non-EVD essential health services and health system strengthening, governance and economic crisis mitigation (ECM), education, water and sanitation, and innovation, technology and partnerships (ITP). The largest proportion of USAID’s Pillar II activities categorized by thematic area (calculated by dividing the number of activities in a thematic area by the total number of activities), were health (60 percent; of these 78 percent were in health systems recovery, 17 percent focused on non-EVD health services, and 5 percent were survivors’ programs). Fourteen percent of all Pillar II activities were implemented in the AFS sector. The governance and ECM sectors each accounted for nine percent of recovery activities. Five percent of Pillar II activities fell under ITP and one activity in Liberia was implemented in the education sector. Though many activities had ITP elements, several of these were categorized as ICT interventions in health activities. The most recent USAID obligation and disbursement table shows that as of September 2017, approximately 46 percent of the $475 million3 obligated for Pillar II activities had been disbursed (see Figure 10). GUINEA In Guinea, USAID Pillar II activities largely work in health, governance, and food security. Health activities focused primarily on frontline worker support, MCP, SBCC, and institutional enhancements. IBTCI’s 2017 HFS found that in the 13 prefectures sampled, the mean number of outpatient visits to HFs increased by 74 percent between 2013 and 2017. The HHS found that the percentage of pregnant women with at least four antenatal care visits (ANC4) increased from 57 percent in the 2012 Demographic Health Survey (DHS) to 66 percent. Skilled birth attendant coverage increased from 45 percent to 80 percent and health facility deliveries increased from 40 percent to 73 percent between the two surveys. Pillar II-funded activities achieved many successes. The Health Finance and Governance activity supported the National Assembly’s Health Commission to increase the Government of Guinea (GOG) health budget by 2.4 percent over Parliament’s initial proposal. Under the Fighting Ebola Grand Challenge, 3D Family Productions, working with the Ministry of Health (MOH) and Médecins Sans Frontières (MSF), held a national song-writing competition that increased awareness of and trust in Guinea’s restored health services. The mHero interoperable mobile platform was incorporated into the MOH’s HMIS strategic plan. The Consortium for Elections and Political Processes Strengthening, implemented by the National Democratic Institute (CEPPS/NDI) built the capacity of Guinea’s Election Commission and political and civil society organizations (CSOs) and educated communities about the political process in the lead-up to the February 2018 municipal elections, which helped the electoral process move smoothly and improved women’s participation. Twenty-three percent of the 30,000 candidates were women, not far from the 30 percent quota set by the electoral commission (USAID Guinea, 2016). However, IPs often cited the short duration of Pillar II activities as a challenge, compared to a longer and more traditional duration for development work. Only one AFS activity remains active, and the Health Communication and Capacity Collaborative (HC3) activity closed after two years with demonstrated success in reestablishing trust in health services and strengthening the capacity of the country’s health workforce. EXECUTIVE SUMMARY n 3 LIBERIA In Liberia, Pillar II health activities focused primarily on frontline worker support, MCP, SBCC, and institutional enhancements, although ICT-based activities were also employed. Reports and indicator matrices show that Pillar II health activities achieved most of their targets, including successful testing of improved personal protective equipment (PPE) and patient management devices, and national scale-up of the mHero platform for MOH-health care worker (HCW) communication. IBTCI’s 2017 HFS found positive trends as well, including a gradual increase in outpatient attendance in surveyed HFs since 2014 and growth in the use of skilled birth attendants (to 93 percent, from the Liberia 2013 DHS national estimate of 61 percent). At the time of this evaluation, both quantitative and qualitative data indicate an unmet need for care and support services for EVD survivors. However, during the data collection for this evaluation, the John Snow International Research and Training Institute Advancing Partners and Communities project (JSI R&T/APC) Ebola survivor activity was just getting started and, therefore, it was not fully operational. UNICEF’s Education Crisis Response in Liberia (ECRL) activity was the only Pillar II activity focused on basic education. ECRL improved water, sanitation, and hygiene (WASH) infrastructure in schools, distributed teaching materials, and provided limited teacher training. According to IBTCI’s 2017 HHS, at least 77 percent of primary school-aged children in the eight counties covered by the survey attend school—a vast improvement from the Liberia 2013 DHS national estimate of 38 percent. Completed AFS activities achieved or came close to achieving their targets and showed improvements in household dietary diversity and reductions in the magnitude of moderate and severe hunger. Pillar II AFS activities in Liberia implemented cash transfer (CT) programs, cash-for-work interventions, and provided agricultural inputs, vocational training, and other social protection activities. The activities were regarded by beneficiaries and implementers as having both a positive impact on the recipient HHs and success in reaching women and female-headed HHs. For example, Mercy Corps’ Economic Recovery from Ebola for Liberia (EREL) provided cash to 30,077 HHs, of which 70.5 percent were female-headed. IBTCI’s 2017 HHS indicates that, while 52 percent of HHs in the eight targeted counties received some form of social assistance in the past 12 months (cash, food, and educational support are the most common types of assistance), formal sources of support (e.g., GOL, NGOs, and CSOs) were mentioned by only five percent of HHs. SIERRA LEONE In Sierra Leone, USAID Pillar II investments primarily addressed health system recovery, restoration of non-EVD essential health services, and food security. The Advancing Partners and Communities – Post-Ebola Recovery of Health Services (PERHS) activity revitalized 305 of 365 peripheral health units (PHUs) in the five districts covered by the activity by providing basic medical equipment and furniture. Of the 305 PHUs targeted, more than 240 also received training and 110 received training and were renovated to improve infection prevention and control (IPC) standards, including installation or rehabilitation of boreholes/wells, installation or rehabilitation of toilets, waste pits, and incinerators, and installing or rehabilitating solar panels. The activity also trained more than 900 health professionals, 1,500 community health workers, and 2,500 members (predominantly women) of 214 facility management committees (FMCs). To collate morbidity and context data at the facility level, the EPIC platform and dashboard, developed under the Fighting Ebola Grand Challenge, were piloted in one district. These initiatives have improved health services for approximately two million Sierra Leoneans. KIIs with IPs and the Government of Sierra Leone (GOSL) highlighted the success of WASH services. The AFS sector IPs supported a mix of targeted cash transfers, agricultural input vouchers, and other complementary activities. In coordination with the GOSL and with input from participating communities, the Office of Food for Peace (FFP) partnered with ACDI/VOCA, CARE, Catholic Relief Services, Save the Children, and World Vision to implement food security activities. More than 67,000 HHs (364,000 individuals) received cash transfers. Recipients used the money to purchase food and agricultural inputs. In addition, the Save the Children activity provided 400 female small-scale traders with small conditional cash grants and business training as a complementary activity. ANALYTICAL DOMAIN: SUSTAINABILITY Many of the activities in Guinea were designed to support existing policies and work with the GOG to develop new policies as needed. Health-sector activities were aligned with the GOG’s Recovery and Resilience Strategy and worked toward growing the capacity of national-level legislators to build their capacity for health financing. An FGD with USAID and KIIs with the GOG stated institutional support, such as training and capacity building of prefecture health teams and local health facility staff, will have long-lasting effects. Outcomes that GOG respondents believe are sustainable include IPC in HFs, facility rehabilitation, and health worker training. The country’s new health management information system (HMIS), built on District Health Information Software, version 2 (DHIS 2) and developed with Pillar II funds, rolled out nationally in 2018, incorporating the mHero platform for two-way communication between MOH and HCWs. The HMIS is expected to improve the quality of the data collected and increase the use of data for informed decision-making. However, infrastructure problems such as lack of electricity remain. KIIs mentioned that social mobilization on the electoral process and in conflict resolution may have long- 4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES lasting effects. AFS activities were primarily designed to provide emergency food assistance with only a few livelihood interventions. In Liberia, it is not possible to map Pillar II activities along a sustainability pathway reflective of all determinants included in the sustainability framework (see Evaluation Question 2: Sustainability in main body of the report). However, institutional enhancement interventions such as physical upgrades of HFs and WASH infrastructure strengthening in schools and communities were intended to have a longer “shelf-life” than other activities. In contrast, CT programs are not a sustainable social protection mechanism, and they generally yielded shorter-term effects (e.g., they enabled HHs to avert starvation in the midst of acute food shortages). Other forms of social protection that addressed livelihoods are linked to longer-lasting benefits (greater resilience and increased ability to meet basic household needs and send children to school). As illustrated by a case study on Maternal and Child Survival Program/Human Resources for Health (MCSP/HRH), frontline worker support activities that involve capacity building can leave a legacy if institutional support requirements are addressed. However, other sustainability determinants, such as the financing of HRH, functional accountability mechanisms, and reviews of existing policies with an eye toward health-sector readiness to rapidly mobilize and deploy HRH (in the event of another crisis) are also necessary. In Sierra Leone, as in the other two countries, Pillar II interventions can be analyzed through a systems lens, which allows us to see that investments influenced many of the six WHO health system building blocks. That is, different policies were either put into place or revitalized, including the Community Health Workers Policy, the National Guidelines for WASH Services in Health Facilities, and the Integrated Disease Surveillance and Report Technical Guidelines. In addition, there were multiple interventions improving institutional performance and support, and community involvement was garnered through the FMCs and a national campaign promoted care-seeking behavior, thus addressing the human dimension. Further, reporting of service data was improved, thus alleviating health information system concerns. All of these elements contributed to health system strengthening, the effects of which will be explored further during the performance evaluation follow-up. ANALYTICAL DOMAIN: GAPS AND OPPORTUNITIES EVD-related death rates among health workers were high (1.45% versus 0.02% for general population). Pillar II health sector IPs designed interventions to promote frontline worker safety through IPC training and rehabilitation of HFs to include basic infrastructure and equipment. In addition, 10 of the 14 innovations funded under the Fighting Ebola Grand Challenge, from improved PPE and Ebola treatment units (ETUs) to patient monitoring devices, aimed to improve frontline worker safety. Institutional enhancements in the governance sector responded to gaps related to the country’s legal and electoral procedures and institutions. The realization that inadequate community engagement was a key factor in mistrust and misconception, seen during the EVD outbreak, led IPs to include social mobilization as a key element in health and governance activities. An opportunity related to the recovery and cited by respondents was the continuation of partner coordination established during the EVD response. EVD struck Liberia during the annual planting and harvesting cycle (FEWS Net, 2017a), and many HHs lost crops— frequently because people could not go to their farms or get to markets to sell or trade any surplus due to border closures, quarantines, and other restrictions on their movement. Pillar II AFS activities (social protection interventions, in particular) addressed a gap that emerged due to food insecurity and loss of income. WASH and IPC, which were addressed through institutional enhancements, frontline worker support, and SBCC in the health and basic education thematic areas, also responded to gaps related to effective hygiene practices. There were, however, missed opportunities. In some instances, seeds and other agricultural inputs were received too late for planting in the first year, and thus were saved for the next planting season. As recovery efforts evolved, some Pillar II IPs adapted to implementation challenges and seized opportunities to amplify results. The use of mobile money for CT programs capitalized on growing ICT momentum in the country. While showing great potential, this method was hampered by limited availability of cash transfer agents in rural areas and their limited liquidity, as documented in a separate review of FFP￾supported responses to the EVD crises in Liberia and Sierra Leone (Radice, 2017). To mitigate these issues, IPs developed creative solutions such as increasing incentives to cash-out agents. “WASH in Schools” (WinS) activities centered on institutional enhancements; however, the engagement of local groups such as parent-teacher associations is an example of a creative partnership approach employed to explore sustainable financing for WASH activities in schools. In Sierra Leone, there were different gaps in AFS mentioned by key informants. They can be grouped in the following categories: the felt need for wider and more timely coverage and support via the CTP; the importance of involving potential beneficiaries in the planning process of recovery interventions; a desire for the expansion of support for farming and processing equipment to add value to agricultural production; and the importance of feeder-road rehabilitation activities. In the health sector, the perceived gaps centered around two major issues—expanding health provider training and assisting with retention of personnel, especially those recruited as nurse volunteers and improving the availability of pharmaceutical supplies at the facility level. EXECUTIVE SUMMARY n 5 Respondents in all three countries and in Washington noted that the evaluation team’s process of vetting and refining the Pillar II theory of change (conducted by the evaluation team in Phase One) was useful, as they had generally focused on work and plans within their own sector. Neither the missions nor the Washington-based operating units (OUs) used the USAID/AFR Pillar II Monitoring and Evaluation (M&E) Results Framework (Figure 2)to create a more granular activity￾specific M&E framework, as was the original intent of the Africa Ebola Unit (AEU) in USAID/AFR. The majority of support was core funded through USAID/Global Health (GH) and USAID/ Bureau for Food Security (BFS) and much of the USAID/GH funding was added on to existing projects/activities. As such, from the perspective of funding OUs, it was more logical to follow existing M&E results frameworks rather than retrofit one to the AEU’s Results Framework. ANALYTICAL DOMAIN: MANAGEMENT In Guinea, Pillar II activities aligned with the GOG’s post￾Ebola Recovery and Resilience Strategy. The supportive role of the Mission’s program planning, monitoring and evaluation was highlighted. In general, respondents praised USAID for its coordination with government and other development partners. Pillar II activities have heavily targeted the health sector and GOG respondents stated that they appreciated the flexibility of USAID funding, but felt that the Pillar II activities were too short in duration. KIIs reported concern that the effects of post-EVD health sector recovery efforts will likely stop or fade away without continued support. However, some improvements, such as the use of the mHero mobile platform for MOH-HCW communication, have been incorporated into national plans, which enhances the likelihood of their continuation. GOG stakeholders responded that USAID was consultative and supportive with other non-USG development partners and government stakeholders and highlighted the supportive role of the Mission’s M&E section and program planning. Respondents from a subnational level CSO stated that there was inadequate consultation with government stakeholders, especially outside Conakry. AFS and governance IP respondents reported that the Pillar II activities were too short in duration. The pre-EVD presence of AFS activities in Liberia was an important strength on which to build. However, some applications for Pillar II AFS activities took several months to be approved, which delayed start-up and thus the ability to meet acute needs of individuals, HHs, and communities affected by the EVD outbreak. The Liberia Mission worked closely with the Ministry of Health to align Pillar II activities with the government’s Investment Plan for Building a Resilient Health System, Liberia 2015–2021. At the time of the data collection (late 2017/early 2018), qualitative evaluation data reveal generally positive views on coordination, although stakeholders noted that coordination is not now occurring as smoothly as it did in the midst of the crisis, leading to some current duplication and cost-inefficiencies. USAID’s structured reporting, M&E processes, and tools have helped to establish clear expectations and are helpful as an objective means of tracking progress. However, some GOL and CSO KII respondents noted that there is suboptimal transparency on how much funding is allocated, what impact each activity is making/has made, and at what level of quality activities have been implemented. In Sierra Leone, the public sector played a crucial role in the coordination of recovery efforts. The President’s Delivery Team on Transition and Recovery (PDTTR) coordinated recovery priorities outlined by the government and provided technical support in monitoring to all involved ministries, departments, and agencies, where they embedded its members. USAID’s efforts and those of other donors were coordinated through this mechanism. USAID’s FFP activities worked very closely with the National Commission for Social Action (NaCSA), a semi-autonomous agency in charge of coordinating and implementing all social protection interventions, to target the right beneficiaries. Cash transfer working groups were used by FFP IPs to coordinate many of their activities. In the health sector, IPs made substantial efforts to leverage synergies, technical expertise, and other resources to avoid duplication and maximize effectiveness of activities on the ground. The EPIC platform and analytics were piloted in Sierra Leone, demonstrating one approach to improving coordination and evidence-based health decision-making. district councils were the main mechanism for coordination of donor-supported efforts at the district level. These councils organized district forums to review progress of program interventions and resolve issues. INTRA-AGENCY COORDINATION In addition to the robust inter-agency coordination that was in situ from the onset of the outbreak, recovery efforts also resulted in strong intra-agency coordination (among three missions, three OUs, a regional bureau, and the AEU). The team observed that the respective roles of the Agreement Officer Representatives (AORs) and the Mission were ill￾defined. There were distinct differences noted between the management and leadership of the Secretariat; for example, daily tasks for each OU were assigned in writing to OUs in contrast to the AEU which tasks were less prescribed and not delineated in writing. Under the Secretariat, the leadership followed up daily to assess progress against the daily tasks assigned in writing and ensured execution. While the AEU was recognized across the Agency as the coordinating body for the recovery efforts, the roles and responsibilities of the AEU were not clearly set out. Further, USAID staff on the ground did not consistently comprehend the role that the Global Health Ebola Team (GHET) played vis-à-vis the activity managers of Ebola Pillar II-funded 6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES activities; this was despite efforts to make it clear that the Global Health Bureau was in charge of the scope and program direction of EVD activities and would provide their AORs. The varying procurement mechanisms and differing contractual arrangements across USAID OUs made it difficult to obtain consistent information from quarterly financial reports on individual activities. However, the Bureau of Resource Management (BRM) provided IBTCI with obligation and expenditure information and the official list of Pillar-funded activities as of September 2017. Conclusions Pillar II activities were designed to address key recovery needs in six thematic areas: AFS; non-EVD essential health services and health system strengthening; governance; ECM; basic education; and ITP. Pillar II health activities have done more than merely prevent the loss of development gains; they have helped to elevate health behavior and inspired substantial gains in health-seeking behaviors. Various AFS and health service activities supported each other. AFS activities addressed HHs’ immediate basic needs and the loss of agricultural productivity and markets. Cash transfers allowed beneficiaries to purchase food and other essentials for health, as well as to infuse money back into the markets. Assistance to market traders helped ensure that there were products to purchase. In locations where Pillar II AFS activities were implemented, standard measures of household food security improved considerably in a relatively short period of time; it is yet unknown to what extent this will translate into improved nutritional status. Research and development initiatives under the Fighting Ebola Grand Challenge introduced improvements in ETUs, medical devices, and ICT platforms that now strengthen the countries’ health systems and make them more resilient to shocks in the future. ECM and governance activities were few in comparison to other thematic areas, funding was delayed, and achievements more difficult to link directly to ultimate outcomes of health and development at the population level. Through Pillar II investments, the government agencies, CSOs, and the media have strengthened advocacy, transparency, and accountability. ITP activities, though a small part of each country’s Pillar II funding allocation, were higher in number, implemented by multiple IPs and common to all three countries. Given local challenges (e.g., cell phone coverage, network connectivity) and the time requirements to roll out ICT interventions, the trickle-down effects of those interventions are not widely documented at this time. Overall, Pillar II recovery activities were of relatively short duration and the actual implementation period was often further compressed by necessary start-up activities. Nonetheless, relative to activities for which there is data, IPs report achieving their outputs. Pillar II provided a rare opportunity to observe support from and within multiple sectors directed toward achieving specific outcomes with cross-cutting relevance (e.g., accountability, citizen trust, citizen engagement). At this stage, health sector activities have bolstered key building blocks of a functional health system (e.g., health workforce, health information), strengthened health service quality, and restored trust in the health system. The AFS activities addressed poor welfare and food insecurity issues initiated by EVD and the related loss of agricultural productivity and markets. Infusions of cash, seeds, equipment, drugs, and infrastructure helped to restart and rebuild livelihoods. The questions remaining are about the sustainability of these intervention components after the AFS staff and added resources are no longer available. To bring effective innovations to the global marketplace, there is evidence of “Grand Challenge” innovations, spurring additional partnerships between innovators and private-sector entities. Experience with mobile money in the CT programs has shown that quite a number of obstacles must be overcome before e-payment works smoothly for vulnerable HHs in rural areas. Some of the applications for the AFS activities took several months to be approved and required some iteration on planning to align them with USAID/FFP’s objectives and meet quality standards. These delays resulted in activities starting later than initially envisioned. Although the intention was to cover food insecurity during the lean season, many of the early transfers did not reach beneficiaries until after the most challenging time had passed. The weakness of health systems in the three countries in containing the EVD outbreak was not exclusively a health issue; it was also a function of leadership and management (governance and economy issues). Yet ECM and governance activities were sparsely funded in the recovery efforts, and thus their response was not fully mobilized for the recovery. EXECUTIVE SUMMARY n 7 Recommendations USAID’s future emergency coordinating bodies, such as the AEU, should be based in the front office and comprise a team of leaders with senior staff experienced in development and humanitarian relief. The team should be multi-sectoral, capable of drawing on the full range of the USAID’s expertise in development, with clear rules of engagement and specific roles for the team. This central team should have the authority to solicit cooperation from OUs across the Agency. This central coordinating team should be bigger than the AEU was and should be able to exercise certain authorities for the tasks at hand without approval from each of the OUs. The team should be charged with invoking special procurement and staffing regulations and redesigned for emergencies, including explicit criteria for clearances and decision-making processes that specify the authorities of Mission and Washington OUs. OUs should identify ways to expedite access to funding while maintaining accountability and the minimum due process to protect the use of USG funds. This discussion needs to happen above the level of the individual OUs and procedures would need to be in place to avoid use of the recovery funds without constraints, misappropriation of funds, inefficiency, or mismanagement at the IP level. The agency can consider waivers for routine activity design and implementation during emergency situations. A time frame for addressing the waived actions (e.g., instituting an M&E plan) should be finalized within one to two years of implementation. USAID/AFR should create standard operating procedures to be available for immediate use in future emergencies. Protocols would include variations for applying to short-, medium-, and long-term phases of recovery. USAID should judiciously consider supporting with non-emer￾gency funds limited follow-on activities at the national and district levels in each country to prepare government agencies and communities for the sunset of Pillar II resources. Com￾munication and dialogue activities, using skills built during the response and recovery, can help communities to understand that the time limit was a condition of the EVD response funding from the start, the resources have been used as planned, and they have produced benefits for the people. These communication and consultation activities can invite and engage stakeholders in planning ways to sustain and extend those benefits. While it is understandable that M&E and reporting might get set aside during a health emergency, de-prioritizing M&E is more a result of staffing shortage than of difficulty in establishing M&E basics. IPs must include qualified M&E staff in all staffing plans and during every phase of the activity lifecycle. Their functions are critical to sound and rapid evidence-based decision making in all aspects of development and emergency programming. It is recommended that IPs continue to invest in trust￾building activities with government and civil society. They should improve and expand the capacity and frequency of contacts between CSOs and community leaders. USAID and IPs should take every opportunity to bolster the value and mandate of government officials’ willingness to listen and respond to their communities’ needs. Photo courtesy of Michael Duff 8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES INTRODUCTION Evaluation Purpose 4. IBTCI collected reports and documents from the OUs and implementing partners before December 2017. The output and outcome figures stated in the report therefore do not reflect figures reached by December 2017. In addition to claiming thousands of lives in Guinea, Liberia, and Sierra Leone, the 2014–2016 Ebola Virus Disease (EVD) outbreak resulted in agricultural and economic losses, loss of trust in government agencies, and further deterioration of already weak health systems in the three countries (Figure 1). In close collaboration with the host-country governments, the U.S. Government (USG) based its response on four pillars: I) controlling the epidemic at its source; II) mitigating second-order impacts, including blunting the economic, social, and political consequences in the region; III) engaging and coordinating with a broader global audience; and IV) fortifying the health security infrastructure in the region and beyond. In October 2016, the United States Agency for International Development’s Africa Bureau (USAID/AFR) awarded International Business and Technical Consultants, Inc. (IBTCI) a three-year contract for the Ebola Pillar II Monitoring, Evaluation and Learning (MEL) activity (AID-OAA-I-15-00022), which provides evaluation, routine monitoring, data quality assurance, and knowledge management and learning support for USAID￾funded Pillar II activities. The scope of work (SOW) for the activity’s evaluation component calls for a macro-level performance evaluation in 2017 (PE1) and a follow-up evaluation in 2019 (PE2). These evaluations will document the performance of Pillar II activities in each of the three countries, the views of various stakeholder groups regarding the value of these USAID investments, and lessons learned. The primary audiences for this evaluation are USAID Washington operating units (OUs) including, Africa, Global Health, Democracy, Conflict, and Humanitarian Assistance, and Economic Growth, Education, the Global Development Lab and USAID’s Liberia, Guinea, and Sierra Leone Missions. The results of the PE1 and PE2 are intended to guide future USAID decisions regarding the type and extent of support provided to future outbreaks of similar magnitude and complexity, with the full understanding that public health emergencies of international concern are relatively rare events. Evaluation Objectives and Questions Based on the activity SOW (Annex A) and discussions with USAID/AFR, PE1 focused on Ebola Pillar II activities implemented between March 2015 and December 2017.4 The evaluation questions (EQs) were categorized into four analytical domains: 1) performance, 2) sustainability, 3) gaps and opportunities, and 4) management (see Table 1, page 9). Figure 1. Regional map of the West Africa EVD outbreak Maryland Grand Kru River Gee Grand Gedeh Sinoe Nimba River Cess Grand Bassa Bong Margibi Montserrado Bomi Grand Cape Mount Gbarpolu Lofa Guéckédou Kailahun Macenta N’Zérékoré Yomou Lola Beyla Kerouane Kissidougou Kankan Mandiana Siguiri Kouroussa Dinguiraye Dabola Faranah Tougue Mamou Dalaba Koubia Mali Koundara Gaoual Lelouma Labé Pita Boké Telemele Fria Boffa Dubreka Kindia Goyah Ratoma Natoto Ioum Forécariah Koinadugu Bombali Kambia Port Loko Tonkolili Moyamba Bo Western Area Bonthe Pujehun Kenema Kono 1 – 5 6 – 20 21 – 100 101 – 500 501 – 4,000 TOTAL CASES* *Source: WHO Ebola Response Roadmap 16 March 2016 0 GUINEA SIERRA LEONE LIBERIA INTRODUCTION n 9 Table 1. Evaluation questions EQ1 Performance: How are USAID’s Pillar II Ebola Recovery activities in Guinea, Liberia, and Sierra Leone contributing to the achievement of the Ebola Pillar II Strategic Framework’s three objectives: to halt the loss of development gains; recover and strengthen key institutions and infrastructures; and build sustained systems through public private partnerships (PPPs), innovation, and capacity building? EQ2 Sustainability: What results of Pillar II activities have endured after the activities have closed, and why? EQ3 Gaps and Opportunities: What gaps and opportunities arose in the course of Pillar II activities that influenced the achievement of Pillar II objectives? EQ4 Management: What lessons have been learned regarding how best to use the strengths of USAID’s emergency and long-term development mechanisms in a complex emergency? Photo courtesy of Michael Duff 10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES BACKGROUND Context Guinea, Liberia, and Sierra Leone were the three countries most affected by the 2014–2016 EVD outbreak, which claimed 11,300 lives and infected at least 28,000 people (WHO, 2016). As the largest EVD outbreak in history and the first in an urban setting, the West Africa EVD outbreak led to the collapse of multiple systems including health, agriculture, democracy and governance, trade and markets, and education, many of which were in a fragile state before EVD struck. Fears associated with EVD led governments to restrict residents’ movement, furlough non-essential public-sector workers, and close borders. International businesses reduced their investments and citizens were less engaged in trade, stalling the economies of the three countries. Services from government and nongovernmental organizations (NGOs) were curtailed, as many staff stayed home or left the country. With health facilities (HFs) closed, health workers ill or absent, and patients afraid of contracting the virus, many failed to receive routine health services, leading to additional morbidity and mortality (Brolin Ribacke KJ, 2016; Parpia AS, 2016). The EVD outbreak highlighted existing institutional weaknesses in each of these countries and exposed their vulnerability to crises. While economic activity had begun to rebound and many official restrictions, including border closures, were lifted by early 2015, recovery was slowed by the depletion of household and business-sector assets, the interruption of large-scale investment plans, and the reservations of investors (USAID, 2015). United States Government Activities Addressing EVD As discussed in the introduction, to confront the cross-sectoral challenges posed by the EVD outbreak, the USG mounted a whole-of-government response which was quickly organized into four pillars. Between March and September 2014, USAID had already mobilized more than US $21 million for the response to the outbreak; by August 2014, the Agency had begun work under the Ebola Task Force. The Task Force was established to help coordinate agency activities related to EVD under the leadership of Senior Foreign Service Officer Patricia Reeder from USAID’s Bureau for Policy, Planning and Learning (PPL) (CDC, 2014; OIG, 2015). Concurrently, between July and September 2014 the U.S. Centers for Disease Control and Prevention (CDC) had mobilized over 500 staff members to assist in the response (CDC, 2014). Prior to the Obama administration’s Emergency Appropriations Request for Ebola for Fiscal Year (FY) 2015 (November 5, 2014), USAID, CDC, the Department of Defense (DOD), Department of State, and Department of Health and Human Services (DHHS) were already working together under the coordination of the National Security Council (NSC) to address the outbreak, but the epidemic was outpacing the response (White House, 2014; CDC, 2014). In order to respond to the growing need for funds and action in West Africa, the drafting of the administration’s emergency request was coordinated through NSC and the Office of Management and Budget (OMB), as the result of clear and direct conversations with all agencies involved, to determine who would lead which efforts and how much money was needed. After the administration’s Emergency Request in November 2014, the USAID Ebola Secretariat was convened, which included members of the Ebola Task Force and others from USAID OUs who were dedicated to the Ebola response and recovery efforts. This new Secretariat was led by Dirk Dijkerman, who coordinated the activities of key personnel from all involved OUs to respond to daily changes on the ground. USAID’s Office of Foreign Disaster Assistance (OFDA) led the USG response to control the outbreak (Pillar I) with the CDC, the DHHS, National Institutes of Health (NIH), and DOD. In March 2015, USAID’s Africa Bureau (USAID/AFR) established the Africa Ebola Unit (AEU) to lead and coordinate the many USAID OUs involved in Pillar II activities. Under the AEU, the coordination of the recovery efforts shifted to a new Senior Coordinator, Denise Rollins. As of September 30, 2016, the AEU’s responsibilities were transferred to USAID/AFR’s Office of West African Affairs, which continues to oversee and coordinate Pillar II activities. The USG Ebola response and recovery strategic framework can be found in Annex C. The USG designed its multidimensional response to the outbreak using a robust inter-agency process—by far BACKGROUND n 11 the largest in terms of funds and human resources—in collaboration with the governments of Guinea, Liberia, and Sierra Leone and integrated its activities with support provided by other international partners, including the WHO, United Nations Mission for Ebola Emergency Response (UNMEER), World Bank (WB), European Union (EU), governments of the UK, Japan, Germany, China, and France, The Paul Allen Foundation, and The Bill & Melinda Gates Foundation. Pillar II funded 14 innovations through the regionally focused Fighting Ebola Grand Challenge initiative out of the Center for Innovation and Impact in the Global Health Bureau (CII). This business-minded approach has helped CII to spark new solutions to serious global health challenges from malaria to HIV to maternal, newborn, and child health. Including this challenge, CII has leveraged over $300 million in outside capital to develop and test 150 innovations in 35 countries, of which 25 were scaling or transitioning to scale in 2018. These challenges jump-start innovative product developments by providing seed funding for research, development and testing. With input from the White House Office of Science and Technology Policy, the DOD, the CDC, OFDA, Offices in the Bureau of Global Health, the Global Development Lab, the Africa Bureau, and experts from USAID missions, which informed the team about current conditions in West Africa, the CII selected 14 winner innovations (out of 1,500 applicants). Twelve of the 14 innovations have been field-tested in West Africa. The USAID Ebola Pillar II M&E Results Framework (Figure 2) laid out a recovery approach that included measures to revive economic activity, restore inclusive growth, and strengthen the capacities and resilience of public institutions. Pillar II activities focus on food security, non-EVD health services, health systems recovery, governance and economic crisis mitigation, and innovation, technology and partnerships (see details in Annex A). They combine proven recovery interventions, with the goals of strengthening key institutions and infrastructure weakened during the outbreak, renewing trust in government, and ensuring that countries are recovering in a way that leaves them better prepared for, and resilient to, similar crises in the future. Pillar II activities are managed by numerous USAID Bureaus, offices, and three affected missions, and implemented by a diverse group of implementing partners (IPs). Figure 2. Pillar II M&E Results Framework (Source: Ebola Pillar II MEL Scope of Work) Food Security Non-Ebola Health Services Health Systems Recovery Governance and Economic Crisis Management Innovation, Technology and Partnerships Objective 1 Prevent the loss of development gains Objective 2 Recover and strengthen existing institutions and infrastructure Objective 3 Build sustained systems through PPPs, innovation, & capacity building • Household food security & market functions restored • Household/individuals standards of living improved • School attendance restored/improved • MCH service delivery restored • Behavior change gains enhanced & reinforced through communication • Health infrastructure strengthened at community and district levels • Health workers capacity to deliver both EVD and non-EVD essential services increased • Policy & Judicial system reformed • Special social protection provided & comprehensive care developed • Health surveillance system developed • CSOs strengthened to provide effective advocacy, transparency, and accountability activities Cash Transfers/Food Vouchers # EVD-affected households that received cash transfers/food vouchers to increase household access to food Agricultural Support # EVD-affected small￾holder farmers who received agricultural inputs to promote food production to improve household availability of food Maternal Child Health # of pregnant females who receive prenatal care Immunizations # children under 5 who received the required immunizations Behavior Change Communication % increase in demand for health services at the national level Health Facility Restoration % of health facilities meeting country-specific quality standards Health Policy # of laws, policies, procedures updated or adopted to restore services Social Protection Process % of EVD-affected communities benefitting from social protection services as a result of USG support Private Sector Engagement # of PPPs harnessed to address second-order of Ebola impacts Civil Society # of CBOs/CBOs staff trained in developing and implementing effective Ebola health-related advocacy activities Water & Sanitation Supply % of schools with clean water sources implementing safe hygiene (WASH) practices Election Process # of domestic election observers/party agents trained with USAID assistance Safe Learning Environment # of schools with developed WASH facilities that meet MOE standards Governance # of USG-assisted CSOs that participate in legislative proceedings to improve IHR HIS # of health workers regularly communicating through interoperable digital HIS platforms E-payments # of secure electronic disbursements for CHWs utilizing national civil servant identity per year Connectivity # of reliable internet connections for hospitals, universities, and government ministries Pillar II Goal: Mitigate Second-order Impacts of Ebola Capacity Building in IPC Protocols # of health workers trained in new IPC protocols 12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES METHODOLOGY Evaluation Design 5. “Thematic area” refers to a body of technical knowledge and activities, often associated with specialized, though often overlapping, groups of experts and organizations. The objectives of the PE1 are to: 1) compile existing information about where, when, and for whom Pillar II activities were implemented; 2) collect stories, qualitative data, and quantitative evidence about the outputs and outcomes of the activities; and (3) document the challenges encountered during implementation, as well as successes. IBTCI designed the Ebola Pillar II performance evaluation to capture, aggregate, and report on Pillar II activities’ achievement of macro-level objectives in each of the thematic areas5 as stated in the Pillar II M&E Results Framework (Figure 2, page 11); it was not designed to assess the results of specific interventions or activities. Overall Pillar II Theory of Change: During the inception phase, the evaluation team constructed the Pillar II theory of change (Figure 3, page 13) with inputs from USAID. This TOC was a tool to analyze and present the findings in this report. The Pillar II theory of change illustrates the interactions implicit in the Pillar II M&E Results Framework. It lists the types of Table 2. Data collection methods and sources of information Methods Sources of Information/Stakeholder Groups Scope (n) and Sample Size Desk review Implementing partner reports, USG documents including Congressional Reports and Notifications from 2015 - 2017, published demographic and health surveys (DHS) and multi-ple indicator cluster surveys (MICS), government strategies, plans, and other documents, peer-reviewed and gray litera-ture ~800 documents Key informant interviews USAID (missions and Washington OUs), Pillar II program implementers, including implementing partners (IPs), governments of Guinea/Liberia/ Sierra Leone, beneficiaries (recipients and service providers), civil society organizations, non-USG and other USG partners, including WHO, UNICEF Guinea: 166 Liberia: 91 Sierra Leone: 93 USAID/Washington OUs: 19 Focus group discussions Community leaders, beneficiaries (recipients and service providers), local government staff, community members Guinea: 83 Liberia: 74 Sierra Leone: 58 Quantitative surveys HH surveys (three countries) Guinea: 5,189 HHs in 13 priority* prefectures Liberia: 3,297 HHs in 8 priority* counties Sierra Leone: 3,235 HHs in 8 priority* districts HF surveys (three countries) Guinea: 248 HFs in 13 priority* prefectures Liberia: 153 HFs in 8 priority* counties Sierra Leone: 128 HFs in 8 priority* districts * Priority prefectures/counties/districts: —Guinea (prefectures): Boffa, Boké, Faranah, Kankan, Forécariah, Kindia, Labé, Beyla, Guéckédou, Lola, Macenta, N’Zérékoré, and Conakry —Liberia (counties): Bomi, Grand Bassa, Grand Cape Mount, Bong, Lofa, Margibi, Montserrado, and Nimba. Bomi and Grand Cape Mount were added to USAID/Liberia’s six focus counties based on activity mapping and local experts advise that both counties were important settings for understanding the EVD recovery efforts in Liberia. —Sierra Leone (districts): Kailahun, Kenema, Bombali, Koinadugu, Port Loko, Tonkolili, Western Area Rural, and Western Area Urban EVALUATION METHODOLOGY n 13 Figure 3. Overall Ebola Pillar II theory of change Ebola Pillar II Activities (in six interacting thematic areas) Activities & Input Types (Used across several thematic areas) Basic health service delivery restored Health worker capacity to deliver EVD and non-EVD services Comprehensive services developed and provided for EVD survivors Health behavior gains enhanced and reinforced Health infrastructure (including management systems) strengthened Health information system developed and supported Health financing & resource allocation improved School attendance restored and gender balance improved Literacy & numeracy improved Household food security increased Nutritional status improved Household standard of living improved Enabling gvt policies developed & harmful policies & practices revised Community service organizations strengthened to provide advocacy, transparency, accountability Management systems & accountability of ministries & stakeholders improved at all levels Agriculture & import market function restored & improved Social protection provided for EVD-affected groups ICT systems for health, gender equity, & civic participation improved Power, water, roads, & communica￾tion infrastructure restored & expanded Outcomes Ebola Pillar II Objectives Loss of development gains prevented Institutions and infrastructure recovered and strengthened Sustained systems built through partnerships, innovation, and capacity building Technical Assistance Training (leadership, infection protection & control, management, gender equity) Clinical Care M&E Systems Development Consultation with Community Leaders Education Refurbishing/Restocking Facilities Nutritional Support Income Support (cash transfers, vouchers, food for work) Agricultural Inputs Social and Behavior Change Communication Staffing Support Restoring/Improving Infrastructure (roads, power) Procurement, Logistics, & Distribution Support Partnership Development ICT Inputs Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery, & Survivor Programs 14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES activities, which vary extensively across the three countries, implemented in each thematic area to contribute to the achievement of the three objectives of Pillar II. The thematic areas in the Pillar II theory of change have been disaggregated to facilitate data analysis and triangulation; while the context is the same, they are slightly different from those in the Pillar II M&E Results Framework. The Ebola Pillar II activity mapping (shown in Annex J) was done by the evaluation team to provide more detailed information on the contribution of activities for each sector or thematic area and served as a tool for this 6. An education study is planned in Liberia in early 2019. The results will be included in the PE2 report. analysis. The activity mapping shows both USAID’s categorization according to sector, as well as IBTCI’s categorization by thematic area. While there are some differences in the activity categorization, the majority of activities fall into the same sector and thematic area. This evaluation used a mix of qualitative and quantitative methods in a three-phase design to create a cross-sectional picture of the Pillar II accomplishments. Table 2, page 12 presents the sources of information, scope or sample sizes, and locations for each data collection method.6 Data Collection Methodology and Implementation The evaluation design incorporates five data collection methods: 1) Pillar II activity mapping; 2) desk review of activity documents and IP reports; 3) semi-structured key informant interviews (KIIs); 4) semi-structured focus group discussions (FGDs); and 5) quantitative household (HH) and HF surveys. The team also identified several secondary data sources to serve as proxy baselines for Pillar II outcome indicators. An in-depth evaluation design that describes the data collection methods, data sources, and data analysis methods for each evaluation question can be found in Annex E. All team members signed non-disclosure agreements (see Annex G). Activity mapping: The purpose of activity mapping was to compile existing information about where, when, and for whom Pillar II activities were implemented. To determine which activities to include in the mapping exercise, the evaluation team first consulted Congressional Reports and Notifications regarding Ebola Pillar II activity funding and then identified the activities that might be Pillar II-funded through KIIs, communication with USAID staff, and activity and funding documentation received from USAID. IBTCI used USAID’s Office of Budget and Resource Management (BRM) obligations and expenditures (O&E) spreadsheet, dated September 2017 (updated in December 2017) to inform the activity mapping and expenditure tracking. The final mapping includes only the activities in the O&E spreadsheet. As mentioned above, the activity mapping shows both USAID’s categorization according to sector, as well as IBTCI’s categorization by thematic area. While there are some differences in the activity categorization, the majority of activities fall into the same sectors and thematic areas. Desk review: From March to September 2017, IBTCI requested contract and grant agreements, quarterly and annual reports, monitoring and evaluation (M&E) plans and reports, baseline and endline studies, and other reports for all Pillar II￾funded activities from points of contact in USAID/AFR, the three missions, and IPs (through their respective Contracting or Agreement Officer Representative (C/AOR)). The team considered all documents received by January 24, 2018 for the desk reviews. This list is included in Annex F. KIIs and FGDs: Qualitative data collection activities took place between September 2017 and January 2018. The team collected data in 13 prefectures in Guinea, eight counties in Liberia, and eight districts in Sierra Leone (Figure 4), which were selected in consultation with USAID, based on four criteria: 1) geographical areas prioritized in USAID’s country development strategy; 2) level of EVD burden; 3) presence of USAID Ebola Pillar II activities; and 4) importance of geographic areas for understanding contextual factors that have potentially affected recovery. Figure 4. Map of the Ebola Pillar II priority areas in Guinea, Liberia, and Sierra Leone Nimba Grand Bassa Bong Margibi Montserrado Bomi Grand Cape Mount Lofa Faranah Labé Boké Boffa Kindia Conakry Forécariah Koinadugu Bombali Port Loko LIBERIA SIERRA LEONE GUINEA EVALUATION METHODOLOGY n 15 To capture insights from a broad range of knowledgeable and experienced people, the evaluation team selected a purposive sample that included members of all stakeholder groups to participate in KIIs and FGDs (Table 3). A full list of the KII respondents and FGD stakeholder groups and location is included in Annex I. Quantitative surveys: The teams collected HH and HF data between December 2017 and January 2018 in the priority areas shown in Figure 4. The evaluation team identified participants for the HH surveys (HHSs) through a multi-stage sampling approach proportionate to population size, designed to be representative of the priority area level (i.e., prefecture/ county/district).7 The Lot Quality Assurance Sampling (LQAS) methodology was used to determine sample sizes for the HF surveys (HFSs). The samples were representative of each type of health facility (hospital, clinic, health center, and health post).8 Annex E includes the sample size calculations for both surveys. IBTCI trained local interviewers and field supervisors on qualitative and quantitative (survey) methodology, research ethics, survey instruments, and data management using 7. For the household data, previous demographic and health surveys (DHS) were used as pre-EVD baselines. The sampling methodology and questions are similar and therefore, the results are comparable. IBTCI’s 2017 HHS total figures refer to priority areas only (13 prefectures in Guinea, eight counties in Liberia, and eight districts in Sierra Leone), while the DHS total figures refer to the entire country. Subnational trend comparisons are shown in Annex L tables for Liberia and Sierra Leone. Guinea 2012 DHS disaggregated data at the regional level and not at the prefecture level. Hence, prefecture-level trend comparisons are not available. 8. Country-specific. electronic data collection devices. The qualitative and quantitative tools were translated into local languages and pre￾tested in each country. IBTCI developed protocols to comply with “do no harm” principles and ensure the protection of respondents. Verbal informed consent was obtained from each HH, HF, key informant, and FGD participant. To protect respondents’ privacy, IBTCI used unique identifiers in place of names. The activity obtained approval for the study design and data collection activities from the National Statistics Institute in the Ministry of Planning and Cooperation in Guinea, the Institute of Statistics and Geo-Information Services in Liberia, and the Statistician General and the Ministry of Health and Sanitation (MOHS) in Sierra Leone. All data collection tools can be found in Annex H. A detailed description of the data collection methodology and fieldwork implementation is included in Annex E. Review of secondary data sources: IBTCI attempted to identify existing data sources that could serve as proxy baseline figures for the outcome indicators. Due to incomplete information in various USAID activity reports, the team Table 3. Respondent stakeholder groups U.S. Government Beneficiaries USAID Washington: Bureaus of AFR, Global Health, including Center for Innovation and Impact; E3; administrator/front office; OFDA; Global Development Lab; Food for Peace Recipients of Pillar II training and technical support at central, district, and community levels (including implementers) Department of State, Department of Defense EVD survivors and their households USAID missions in Liberia, Guinea, and Sierra Leone Farmers, small-scale traders and merchants, young people Millennium Challenge Corporation Community organizations (women’s groups, faith-based networks) media, community radio Congress, the Government Accountability Office, the National Security Council Pillar II Implementers Governments of: Liberia, Guinea, and Sierra Leone (policy guid- ance, decision-making and oversight bodies at national, county/ prefecture, and local levels IP program managers Health, education, agriculture, parliament/governance, trade/ finance, ICT/telecommunications, EVD-focused bodies Health workers (government-employed and other) Educators Partners Cash transfer implementers World Bank ICT providers UNICEF, UNDP, OCHA, WHO Civil Society (not direct beneficiaries) World Food Programme (WFSP) and other agriculture and food security Women’s and youth groups, religious and traditional leaders, EU, DfID, Irish Aid, France, Nordics media (national and community radio) 16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES relied on secondary data sources such as the Demographic Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS). These surveys were conducted between 2008 and 9. Lists of USAID Ebola Pillar II-supported HFs were provided by the Global Health Ebola Team (GHET) for Guinea, by the Liberia mission for Liberia, and by JSI R&T for Sierra Leone. For Sierra Leone, only Tier 1 HFs were considered as supported by USAID Ebola Pillar II funds. 2013 and used similar indicator definitions as the Pillar II M&E framework and theory of change. Annex L shows the baseline and secondary data for each country. Data Management and Analysis Activity mapping: The evaluation team mapped Ebola Pillar II activities by country, IP, OU, duration, activity type, and funding amount (see Annex J). The activity documents, Congressional reports, and O&E spreadsheet informed the activity mapping and desk reviews. Activity mapping was an iterative process that included several calls over the course of a year. Desk reviews: The evaluation team conducted one desk review per country, focusing on quarterly and annual reports, IPs’ indicator data, midline and endline reports, and documents from other sources, including lessons learned from local government and other international partners. The full versions of the three desk reviews are included in Annex M. To analyze the Pillar II project/activity documents, the team: 1. extracted the activities’ approach, achievements (outputs and outcomes), opportunities, gaps, and management issues; 2. captured the documented output- and outcome-level changes, challenges, and successes; and 3. assessed interventions’ contributions to sustainability. KIIs and FGDs: IBTCI established standardized procedures for interviewing, note-taking, data quality assurance, and data analy￾sis to ensure consistency and objectivity in interpreting findings. KII and FGD facilitators prepared transcripts immediately fol￾lowing the interview or group discussion. The qualitative data were coded and analyzed using Atlas.ti (version 8). Quantitative surveys: The surveys collected quantifiable information on key output and outcome variables at the HH and HF level. Data collectors used electronic tablets using SurveyToGo software, which has a built-in response validation system to minimize errors. IBTCI used STATA version 14 for the quantitative data analysis. The team compared indicators in the priority areas within each country and, as necessary, across countries. When possible, the team compared the collect￾ed data to proxy baseline data from activity documents and secondary data sources to identify trends over time. The full results of the HHS and HFS are included in Annex K. Survey data are disaggregated by prefecture/county/district. One of the advantages of using the LQAS for HFS is its ability to assess prefecture/county/district performance against a set cut-point or threshold. The analysis applied the three LQAS decision rules: 1) less than 50 percent coverage (below acceptable level or poor); 2) between 50 to 79 percent coverage (above acceptable level and average); and 3) 80 percent or more cov￾erage (above acceptable level and above average or good). All HFS data are disaggregated by their funding source (whether the facility was or was not supported through Pillar II funds).9 Annex E, Part 1 provides an in-depth description of the data collection methods, sources, collection locations, and sampling methods for each evaluation question. Annex E, Part 2 provides a detailed data analysis and data quality assurance methodology. The team triangulated qualitative and quantitative data and information from activity documents, the literature review, and primary data collection. Therefore, each finding in this report is supported by two or more data sources and each conclusion is supported by data triangulation and interpretation of two or more findings. The team was in contact with each IP that implemented activities mentioned in this report (71 activities in all) in order to verify IP-specific information. Thirty-three (33) responses were received; of these, 10 affirmed all facts were correct and 23 made minor clarifications or edits which are reflected in this report. Limitations Perhaps due to the multi-sectoral nature of Pillar II activities and multiple USAID funding streams, it has been difficult to obtain access to detailed activity-level financial reporting. Additionally, IBTCI has found inconsistencies in data between sources: information on the start date and end date is often incomplete and indicator data for most activities was not received. Another critical challenge is that some Pillar II funding is channeled through ongoing cooperative agreements that do not disaggregate Pillar II funding from other funding sources. USAID has noted this issue and some IPs now disaggregate Pillar II expenditures in quarterly reports. Reporting is inconsistent, however, and cooperative agreements do not require that the IPs provide disaggregated figures. Another very important limitation was that at the end of February 2017, the Liberia Mission announced that it would EVALUATION METHODOLOGY n 17 be unable to work with the evaluation team during five of the following ten months. During the situation analysis conducted in 2016, the Liberia Mission expressed interest in an IBTCI-led school-based study as part of the PE1. However, at the request of the Mission, the education study was postponed. Its results will be included in the PE2 and tabular data will be made available to the Mission once available. Many of the outputs reported are “numerator only.” For instance, IPs provided the number of people or HFs served, but there are no denominators to assess how much of the need was met. Few reports offer detailed geographic information on the service delivery area or baseline data, particularly health services utilization. In all IP reports, the data are not disaggregated by county, prefecture, or district. Assessing and comparing results of activities that varied greatly in duration and scale is therefore challenging. Baseline data are not available for all countries for all indicators. The HF and community-level outcome indicators are especially scarce. During the evaluation, many activities were ongoing or were closing, which limited the availability of endline or final evaluation reports. IBTCI requested activity documentation over the course of a year. However, IBTCI was unable to obtain complete documentation for each of the Pillar II-funded activities. 10. Halo is a cognitive bias, when respondents to a survey have a tendency to agree with all the questions in a measure. As in any other study, information on health practices and behaviors reported by HH, HF, and key informants and FGD participants may be subject to social desirability bias. Halo bias10 may be a factor, since respondents might have reported what they should have done rather than what they actually did. There is possible recall bias for self-reported behavior in the HHS. The health provider interviews in the HFS might be subject to respondent bias. To mitigate these biases, the questionnaire included some measures of direct observation (e.g., facility registers and reporting documents, verification of medication and inventory stocks). As noted, a different group of data collectors and interviewers was selected and trained in each country. This approach introduced the potential for intra-interviewer bias. If IBTCI had had the same team collect data in all countries, it is likely to have introduced respondent bias. Local respondents may not have been comfortable speaking openly and honestly with interviewers who did not share their culture and language. Moreover, questions would have needed to be asked through a translator, which would have introduced interviewer bias, since the team would not know what was being asked of the respondents. Thus, the approach that would introduce the least amount of interviewer (and respondent) bias was to employ local field personnel. The field supervisors remained the same throughout all study sites. Photo courtesy of Michael Duff 18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES FINDINGS Evaluation Question 1: Performance 11. https://www.usaid.gov/sites/default/files/documents/1870/201man.pdf 12. As noted earlier, individual Pillar II activities are expected to provide their own evaluations at the activity level. EQ1: How are USAID’s Pillar II Ebola Recovery activities in Guinea, Liberia, and Sierra Leone contributing to the achievement of the Ebola Pillar II Strategic Framework’s three objectives: to halt the loss of development gains; recover and strengthen key institutions and infrastructures; and build sustained systems through PPPs, innovation and capacity building? Activities funded by USAID under Ebola Pillar II were spread across the different sectors (thematic areas) that USAID supports. USAID uses the term “activity” to identify project components that are being implemented by country partners or by other organizations USAID has funded to carry out specific tasks under a contract, cooperative agreement, grant, or other arrangement.11 To focus the evaluation on the achievement of the AEU’s Pillar II objectives, the evaluation team grouped similar activities into intervention types (or stream-of-work themes) for analysis. These aggregated intervention types are: social protection; frontline worker support; management, coordination, and partnerships (MCP); information and communication technology (ICT); social and behavior change communication (SBCC); and institutional enhancements (Figure 5). These intervention types are not mutually exclusive and there are areas of overlap. A detailed definition of the intervention types is shown in Annex E, Part 2. To conduct the thematic analysis for EQ1, IBTCI plotted the 19 Pillar II outcome indicators identified in the USAID Pillar II M&E framework and Ebola Pillar II theory of change according to the relevant intervention types and color coded them according to the thematic area. The team used this classification to evaluate the Ebola Pillar II program’s progress toward achievement of the three Pillar II strategic objectives.12 Table 4 (page 19) maps Pillar II activities by thematic areas and intervention types. Figure 6 (page 20) shows the linkage between intervention types and Pillar II outcomes. A full mapping of Pillar II activities by thematic area and intervention typology is shown in Annex J. Figure 5. Overview of the six Pillar II intervention types used in the PE1 analysis Gender-sensitive economic strengthening for vulnerable groups and non-economic strengthening support Social Protection Interventions to enhance worker motivation, safety, and performance Frontline Worker Support Policy development, leadership and governance interventions, including organization development as well as partnerships and coordination among stakeholders Management, Coordination, and Partnerships Strengthening information systems and applying technological innovations to promote the use of data for decision-making Information, Communication, and Technology Use of diverse communication methods (education, media, community dialogue, etc.) to change knowledge, attitudes, and norms toward the intervention’s objectives Social and Behavior Change Communication Upgrading or refurbishment of physical infrastructure and provision of commodities, equipment, and supplies Institutional Enhancements EVALUATION FINDINGS : EQ 1– PERFORMANCE n 19 Innovation, Technology, Economic Crisis Mitigation & Partnerships Governance Agriculture & Food Security Non-EVD Health Services, Basic Education Health Systems Recovery & Survivor Programs Table 4. Mapping of Pillar II activities by thematic area(s) and intervention type Interventions Social Protn Frontline Wkr Spt MCP ICT SBCC Instl Enhcmts Health worker capacity building: • IPC protocols • Pre-service training in reproductive, maternal, newborn, and child health service delivery • Training and capacity development in SBCC Distance learn￾ing for health programming Health policies, laws, procedures updated or adapted Management and organizational development training and technical assistance to national and local government Organizational development & leadership training for HF managers & health workers Supply chain management Support to health information systems SBCC activities to increase demand for health services HF restorations/upgrades Community radio training and message development Safe learning environment by promoting water, sanitation, and hygiene in schools Cash transfer programs: • Unconditional • Conditional Agricultural support to promote local markets/trade: • Agricultural inputs/vouchers • VSLAs • Cash for work • Small grants for traders and womens’ groups Health, nutrition, and household financial management training Direct food support • In-kind food support • Therapeutic food supplements Election process observers training; organizational develop￾ment, management, and leadership CSOs to participate in local legislative proceedings CSOs trained in health advocacy Social protection services to EVD-affected communities: • Cash transfers • Food vouchers Training and engaging women and girls in health, education, and governance systems Expansion of mobile money platform and program to serve community health workers Multiple Grand Challenge innovations Building tools for data use and dissemination, including HIS and other ICT products LEGEND 20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Figure 6. Links between intervention types, themes, and Pillar II outcomes The top row of this figure describes the six intervention types in gray. Below each intervention type is the list of corresponding outcome indicators. IBTCI plotted the 19 Pillar II outcome indicators identified in the USAID Pillar II M&E framework and Ebola Pillar II theory of change according to the relevant intervention type, and color coded them according to the thematic area (legend appears lower right). Social Protn Frontline Wkr Spt MCP ICT SBCC Instl Enhcmts Social protection provided Health workers’ capacity to deliver EVD and non-EVD services increased Enabling governance policies developed & harmful poli￾cies and practices revised Health information system developed & supported Health behavior gains enhanced and reinforced School attendance restored & gender balance improved Specialty services provided to EVD survivors in specific health facilities Basic health service delivery restored HH standard of living improved Management sys￾tems & accountabil￾ity levels, ministries & stakeholders improved ICT for health sys￾tems developed Health infrastructure and community involvement strengthened HH food security increased Power, roads and communication infra￾structure expanded Nutritional status improved Health financing & resource allocation improved Literacy & numeracy improved Ag & import market function restored & improved CSOs strength- ened to provide effective advocacy, transparency, and accountability Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Non-EVD Health Services, Basic Education Health Systems Recovery & Survivor Programs LEGEND Photo courtesy of Michael Duff Photo courtesy of Michael Duff EVALUATION FINDINGS : EQ 1– PERFORMANCE n 21 ACTIVITY MAPPING A diverse group of IPs, managed by seven OUs within USAID, implemented 148 Pillar II activities: 47 regional, 24 in Guinea, 53 in Liberia, and 24 in Sierra Leone (see Figures 7 and 8).13 Sixty percent of Pillar II activities were in the health sector (of these, 78 percent contributed to health systems recovery, 17 percent focused on non-EVD health services, and 5 percent were for survivors programs). Fourteen percent of all Pillar II activities were in the agriculture and food security (AFS) sector. The governance sector and the economic crisis mitigation (ECM) area both held nine percent of recovery activities. Five percent of Pillar II activities fell under innovation, technology, and partnerships (ITP) and one percent were in the education sector (one activity in Liberia). Although many activities had ITP elements, many were categorized as health (their main sector) with the intervention type categorized as ICT. The average duration of the activities varied greatly between sector and country. EBOLA PILLAR II EXPENDITURE TRACKING The September 2017 Obligation Report shows that approximately 46 percent of the $475 million14 Pillar II funding that has been obligated had been disbursed (Table 5).15 The 13. Ref: USAID O&E spreadsheet, September 2017. The spreadsheet shows all activities, including those ending in 2020 or 2021. Regional activities are core funded by USAID/W and implemented in two or more countries. The “number of activity (N)” is based on classification of activities per USAID’s strategic framework categories. 14. This report focuses on the 148 USAID-funded recovery activities funded under Ebola Pillar II (ESF 73%; IDA 25%, GHP 2%); see September 2017 Obligations & Expenditures spreadsheet. A total of $1.86 billion was obligated across all four pillars by September 2017. 15. The September 2017 O&E are higher than previous March 2017 O&E with the inclusion of the CDC Interagency agreement, Ebola Transmission Prevention (Survivors), and IBTCI Monitoring, Evaluation, and Learning activities and adjustments to existing activities. health sector represented the largest share of obligations (a total of 33 percent: 15 percent non-EVD health services; 13 percent health systems recovery; 6 percent survivors programs), followed by the AFS sector (25 percent). AFS disbursements account for 50 percent of the total to date, in part due to cash transfers. All AFS activities with cash transfer interventions have closed, whereas several health and governance and education activities are ongoing. As shown in Figure 9 on page 23, Liberia received the largest total obligation and currently has the highest level of disbursement. Overall, Pillar II obligations were highest for interventions that strengthened MCP (49 percent), followed by provision of social protection (26 percent). Social protection interventions had the highest disbursement rate, accounting for 51 percent of total expenditures as of 2017. This was true in all three countries; Guinea disbursed 33 percent of funds on social protection, still their largest area of disbursement. Social protection accounted for 62 percent of funds disbursed in Liberia and Sierra Leone. Guinea has disbursed 27 percent of Pillar II funding for strengthening MCP (Figure 10). As of September 2017, ICT, SBCC, support to frontline workers, and institutional enhancements accounted for the remaining 25 percent of total disbursements across the three countries (see Table 6). Table 5. Ebola Pillar II O&E by sector as of September 2017 Sector Obligations ($) Share (%) Disbursements ($) Share (%) Agriculture & Food Security 119,176,750 25 110,331822 50 Health Systems Recovery 61,437,096 13 22,734,870 10 Non-EVD Health Services 70,053,691 15 40,796,855 19 EVD Transmission Prevention (Survivors) 29,500,000 6 21,975,078 10 Governance & Economic Crisis Mitigation 75,000,000 16 5,873,657 3 Innovation & Communications Technology 29,974,342 6 17,289,754 8 CDC Inter-Agency Agreement/Ebola Re￾sponse* 80,000,000 17 0 0 Monitoring and Evaluation Activities* 10,000,000 2 0 0 Grand Total $475,141,879 100% $219,002,035 100% Source: USAID’s Office of Budget and Resource Management O&E spreadsheet, September 2017. * Obligations to US Centers for Disease Control and Prevention (CDC) for overall support and to IBTCI for Monitoring, Evaluation and Learning activity were recently added to the Obligations and Expenditures (O&E) reporting template and are added here for completeness but will not be considered in this evaluation. We used the September 2017 O&E spreadsheet received at the time of the report by USAID’s Office of Budget and Resource Management. For the financial analysis, we used the same sectors as the O&E spreadsheet, to ensure matching of the financial data. For the rest of the report, we have used the thematic areas in the Pillar II theory of change (Figure 3), which are slightly different than the ones used in the O&E spreadsheet. 22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Figure 8. Distribution of Pillar II activities by thematic area, intervention type, and country Source: Activity documents available to IBTCI and O&E spreadsheet (September 2017 version) Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services Health Systems Recovery & Survivor Programs Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology SBCC Institutional Enhancements Guinea Liberia Sierra Leone 5 10 Regional 5 28 Source: Activity documents available to IBTCI and O&E spreadsheet (September 2017 version) NOTES: These figures represent the total number of activities funded without consideration of the total dollar value of the particular activity(ies). Five unclassified: Two Guinea activities listed as Health Systems Recovery funded to the GoG had no description and could not be classified into intervention type. One activity funded as Governance and Economic Crisis Mitigation in Sierra Leone had no description and was not classified into a thematic area. Two regional activities, Project Support from to USAID/AFR and the IBTCI Pillar II MEL activities, were not classified into a thematic area. Seven Guinea activities, 12 Liberia activities, five Sierra Leone activities, and eight regional activities were classified as more than one intervention type. Number indicates highest number of projects in each country. These figures represent the total number of activities funded (without consideration of the total dollar value of the particular activity(ies)). Five are unclassified: Two Guinea activities listed as Health Systems Recovery funded to the GOG had no description and could not be classified into intervention type. One activity funded as Governance and Economic Crisis Mitigation in Sierra Leone had no description and was not classified into a thematic area. Two regional activities, Project Support from to USAID/AFR and the IBTCI Pillar II MEL activities, were not classified into a thematic area. Seven Guinea activities, 12 Liberia activities, five Sierra Leone activities, and eight regional activities were classified as more than one intervention type. Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Non-EVD Health Services Basic Education Health Systems Recovery & Survivor Programs LEGEND Figure 7. Overview of Pillar II activities by thematic area Source: Activity documents available to IBTCI and O&E spreadsheet (September 2017 version) Health 60% AFS 14% Gov 9% ECM 9% ITP 5% Education 1% Emergency Non￾Ebola Health Services activities 17% Survivor Program activities 5% Health Systems Recovery activities 78% EVALUATION FINDINGS : EQ 1– PERFORMANCE n 23 Table 6. Reported Ebola Pillar II obligations and disbursements by intervention typology as of September 2017 Sector Obligations ($) Share (%) Disbursements ($) Share (%) Frontline Worker Support 33,806,853 7 11,993,062 5 Innovation & Communications Technology 27,295,935 6 16,962,995 8 Institutional Enhancement 40,648,825 9 11,715,658 5 Management, Coordination & Partnerships 228,033,705 49 52,592,120 24 Social & Behavior Change Communications 14,379,810 3 14,255,815 7 Social Protection 121,476,750 26 111,482,386 51 Grand Total $465,641,879 100% $219,002,035 100% Unspecified 9,500,000 Source: USAID’s Office of Budget and Resource Management O&E spreadsheet, September 2017. Note: To focus the evaluation on the achievement of the AEU’s Pillar II objectives, the evaluation team grouped similar activities into intervention types (or stream-of-work themes) for this analysis. These aggregated intervention types are: (1) frontline worker support; (2) information and communication technology; (3) institutional enhancement; management, coordination, and partnerships; social and behavior change communication; and social protection (Figure 5). These intervention types are not mutually exclusive and there are areas of overlap. Figure 9. Ebola Pillar II obligations and disbursements by country as of September 2017 Source: USAID’s Office of Budget and Resource Management (O&E spreadsheet, September 2017). Note: Figure 9 data may not be up to date for all countries. Regional activities are funded by USAID and implemented in all three countries. Obligations Disbursements Guinea 14% Liberia 35% Sierra Leone 19% Regional 32% Guinea 16% Liberia 37% Sierra Leone 24% Regional 22% Figure 10. Ebola Pillar II obligations and disbursements by intervention type as of September 2017 Source: USAID’s Office of Budget and Resource Management (O&E spreadsheet, September 2017) Obligations Disbursements ICT Institutional Enhancement MCP SBCC Social Protection MCP Social Protection Frontline Worker Support Institutional Enhancement SBCC Frontline Worker Support ICT 24 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Figure 11 shows that in Liberia and Sierra Leone, spending has been highest on AFS activities, while in Guinea spending is highest in health systems recovery. When looking at specific interventions, the largest share of expenditures in Guinea and Sierra Leone went to cash transfer social protection interventions (40 and 57 percent, respectively). GUINEA In Guinea, USAID Pillar II investments mainly focused on health systems recovery, AFS, restoration of non-EVD essential health services, and governance, in descending order. Looking at the number of activities by intervention, Figure 8 on page 22 shows that MCP is the most prominent type of intervention, followed by SBCC and institutional enhancements. NON-EVD HEALTH SERVICES, HEALTH SYSTEMS RECOVERY, AND SURVIVOR PROGRAMS Pillar II health activities included frontline worker support, MCP, SBCC, and institutional enhancements. Only major findings on outcome-level data are presented in this section; details of the main activities and their output and outcome results are described in the Guinea Desk Review (Annex M). Results from IBTCI’s 2017 HHS and HFS for Guinea are shown in Annex K tables. Trend tables with baseline (pre-EVD, 2008–2013), midline (during outbreak, 2014–2015), if available, and post￾EVD (2016–2018) are included in Annex L. Restoration of basic health services: In December 2014, the Government of Guinea (GOG) reported a 20 percent decline in the use of HFs for assisted births, a 25 percent decline in prenatal consultations, a 30 percent decline in vaccination coverage when compared to previous months, and the closing of 94 health centers and district hospitals (GOG, 2015a). In 2017, and endline survey conducted by HC3 in five regions severely impacted by EVD—Conakry, Kindia, Boké, N’Zérékoré, and Faranah—found that one-third of the 3,000 respondents reported improvements in the quality of services in their local health facility during the duration of the activity (JHUCCP, 2017). HC3 supported the Ministry of Health (MOH) to develop and disseminate the Gold Star logo, to signal that information and HFs were trustworthy. Misinformation was a key source of fear and conflict in the EVD crisis; encouraging citizens to look for a government stamp of approval may help avert similar problems in the future. Reports from the USAID end-of-year portfolio review Figure 11. Ebola Pillar II disbursements by country/region and intervention typology as of September 2017 Source: USAID’s Office of Budget and Resource Management (O&E spreadsheet, September 2017) SBCC ICT MCP Frontline Worker Support Social Protection Institutional Enhancements 0 20 40 60 80 100 million Disbursements by Country and Intervention Type Guinea Guinea Liberia Sierra Leone Regional Governance & Economic Crisis Mitigation Agriculture & Food Security EVD Transmission Prevention (Survivor Programs) Non-EVD Health Services Health Systems Recovery Liberia Sierra Leone Regional Disbursements by Country and Sector Innovation & Communications Technology 0 20 40 60 80 100 million LEGEND LEGEND For the financial analysis by sector, IBTCI used the September 2017 O&E spreadsheet received at the time of the report by USAID’s Office of Budget and Resource Management. For the financial analysis only, we used the same sectors as the O&E spreadsheet, to ensure that the financial data match. For the rest of the report, we have used the thematic areas as shown in the Pillar II theory of change (Figure 3), which are slightly different than the ones used in the O&E spreadsheet. EVALUATION FINDINGS : EQ 1– PERFORMANCE n 25 2016 show that from the first half of 2015 to the first half of 2016, patient consultations in targeted health care facilities increased by 57 percent (from 116,545 to 183,149) in Kindia region. The number of women who received uterotonics at delivery in five regions (Kindia, Faranah, Kankan, Boké, and Mamou) increased almost from 17,375 in 2015 to 47,133 in 2016 (171 percent increase). The number of assisted deliveries in N’Zérékoré and Conakry regions increased from 44,226 in the period Oct 14–Sept 15 to 57,080 in Oct 15–Sept 16 (an increase of 29 percent) (USAID Guinea, 2016). IBTCI’s 2017 HFS showed a gradual increase in outpatient attendance in surveyed HFs since 2014 (Figure 12). The survey showed that the mean number of outpatient clients in HFs across all 13 prefectures increased by 74 percent between 2013 and 2017. “RMNCH services were decimated during the Ebola epidemic but they have now recovered to the extent that they are better than pre-Ebola.” — USAID FGD respondent Strengthening health infrastructure and management systems: The Maternal and Child Survival Program Restoration of Health Services (MCSP/RHS) activity improved infection prevention and control (IPC) through training and supportive supervision of 1,430 health care providers and 271 support staff in 249 health care facilities (MCSP/RHS, 2017). A total of 23 HFs were renovated through the HC3 activity. According to IBTCI’s 2017 HFS, HF renovations conducted over the past three years were supported primarily by NGOs in Boké, Faranah, and Kindia. The survey found that the availability of basic amenities such as improved water sources and safe methods of waste disposal in HFs in all surveyed Guinean prefectures is above average (80 percent or more) based on LQAS decision rules described in the methodology section. Solar energy is the primary mode of electricity in all prefectures, except Conakry, Forécariah, and Kindia, which primarily use generators. Prefectures fare below average (less than 50 percent) in availability of telephone, internet access, and backup sources of electricity at the HFs. The System for Improved Access to Pharmaceuticals and Services (SIAPS) activity and USAID’s Global Health Supply Chain Program Procurement and Supply Management (GHSC/ PSM) activity both received partial funding from the Global Health Ebola Team (GHET) and aim to strengthen Guinea’s national pharmaceutical logistics management system (Global Health Ebola Team, 2016). According to IPs, the activity has not yet engaged the private sector and is facing challenges in registering private pharmacies due to long customs procedures. Data from IBTCI’s 2017 HFS show seven of the 13 surveyed prefectures have below acceptable (less than 50 percent) coverage for timely delivery of medicines and commodities to HFs. Improving health financing and resource allocation: Achievements from the Health Finance and Governance (HFG) activity include mentorship and capacity building in health budgeting for the Legislative Assembly Health Commission. The commission advocated for an increase in the health budget from 2.5 percent of national expenditure pre￾EVD to 8.2 percent in 2017, with a commitment to reach 15 percent by 2020. However, the percent of the national budget allocated to health fell to 6 percent in FY 2018, mostly due to a huge increase in the national budget (from 16.2 trillion Guinean Francs (GFN) in FY 2017 to 21.1 trillion in FY 2018). The health budget increased nominally from 1.1 trillion GFN in FY 2017 to 1.3 trillion GFN in FY 2018 (LFI, 2017; MOH, Guinea 2017a; MOH Guinea, 2017b). Increasing health worker capacity to deliver EVD and non￾EVD services: IBTCI’s 2017 HFS LQAS analysis shows that nine of the 13 priority prefectures were assessed as having achieved the indicator of “at least 80 percent of HFs received health worker IPC protocol training in the last 2 years,” whereas Boké, Boffa, Lola, and Macenta were assessed as acceptable (coverage ranging from 50–79 percent). Seven of the 13 priority prefectures were assessed as having standard guidelines for IPC in more than 80 percent of HFs. All prefectures performed well (80 percent or more coverage) for “HFs having received supervision from a higher level (supervisors from government) at least once in last 3 months.” Conakry was the exception, with 50–79 percent. ICT for the health system: The MEASURE Evaluation activity Strengthening Data Availability and Use in Guinea provided customized training for statistics officers on data entry, analysis, and reporting using DHIS 2, estimation of key health management information systems (HMIS) costing by category, and system integration and data warehousing. The activity compiled a master facility list for public facilities and is in the process of developing the same for private facilities (MEASURE, 2017). Figure 12. Mean number of outpatient clients and the percentage change from previous year (Guinea) Source: IBTCI 2017 HFS; n=248 2013 2014 2015 2016 2017 Percentage change from previous year Mean number of outpatient clients each year 1,035 ▲ 32% 837 ▼ 19% 1,101 ▲ 18% 1,304 ▲ 38% 1,804 Source: IBTCI 2017 HFS; n=248 26 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Under the Fighting Ebola Grand Challenge, 3D Family Productions, working with the MOH and Médecins Sans Frontières (MSF), held a national song-writing competition that increased awareness of and trust in Guinea’s restored health services. The mHero inter-operable mobile platform was incorporated into the MOH’s HMIS strategic plan. Provision of specialty services to EVD survivors: IBTCI’s 2017 HHS found that 1.2 percent of sampled HHs had at least one member diagnosed with EVD. In Guinea, John Snow International’s (JSI’s) Research and Training Institute (R&T) Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services (APC/ETP&SS) activity focused on 14 public HFs in N’Zérékoré, Kindia, and Conakry that serve areas with high volumes of EVD survivors as of 2017, and provided national coverage for selected activities. Community-based surveillance is a critical component of the program. Using sentinel sites in 21 prefectures and five urban settings with the largest number of EVD survivors, the system monitors survivors and their close contacts to track their health status and ensure access to services (JSI, 2018). Nevertheless, gaps in the provision of services to EVD survivors remain. IBTCI’s 2017 HFS found that eight prefectures, out of 13 surveyed, had no facility to offer care and support to EVD survivors. Only Conakry, Guéckédou, Faranah, Forécariah, and Labé have at least one facility offering these services. This is to be expected given that the APC/ETP&SS project was focused only on a limited number of prefectures and was just getting started at the time of the data collection for this evaluation. An additional nine facilities were added in 2018, bringing the total number of clinics to 23, well beyond the target of 10 clinics originally requested by GHET. APC/ ETP&SS has supported the survivor associations through learning visits to Liberia and Sierra Leone for the purpose of learning the management and promotion of survivor association. A new headquarters for the national umbrella organization of EVD survivor associations opened in 2017. A clinical guide for the treatment of EVD complications has been developed for use by health facility workers and an extensive network of community surveillance agents has been trained to monitor the health of survivors. Additionally, considerable capacity building support for survivor associations was implemented in all three countries after the period of data collection for this report. For example, in Guinea, this included an organizational capacity assessment and action plan road map creation, a collaborative workshop, regional meetings with survivor association leaders from all three countries, and sponsorship of the president of the survivors’ association to attend a conference in Thailand on stigma and bring back lessons learned. 16. https://dhsprogram.com/pubs/pdf/FR280/FR280.pdf Guinea 2012 DHS collected information from the entire country and totals are country-wide. IBTCI’s 2017 HHS collected information in 13 priority prefectures and totals refer to those prefectures only. Guinea 2012 DHS disaggregated data at the regional level and not at the prefecture level. Hence, prefecture-level trend comparisons are not available. Health behavior gains: Three core maternal health indicators have improved in all 13 priority prefectures in Guinea. The proportion of pregnant women who reported attending at least four antenatal care visits (ANC4) increased from 57 percent in 2012 (Guinea 2012 DHS16) to 66 percent in 2017 (IBTCI’s 2017 HHS). The lowest percentage is in Faranah (47 percent) and the highest in Conakry (79 percent). Skilled provider attendance during delivery increased from 45 percent (Guinea 2012 DHS) to 80 percent (IBTCI’s 2017 HHS) over the same period, and ranges from 40 percent in Faranah to 95 percent in Conakry. The proportion of births occurring in HFs increased from 40 percent (Guinea 2012 DHS) to 73 percent (IBTCI’s 2017 HHS) between 2012 and 2017, and ranges from 52 percent in Faranah to 88 percent in Conakry in the latter year. The HHS revealed that 95 percent of interviewees stated that they would go to an HF if they had a fever (ranges between 79 percent in Faranah and 99 percent in Boké (baseline not available)). Among those who said they would not seek care, about 78 percent indicated a lack of money for treatment as their major obstacle. About 21 percent indicated absence of an HF nearby, and 1 percent reported they believed the HF would be contaminated by EVD. Eleven of the 41 KII respondents from the GOG reported that the activities that have contributed the most to recovery were in the areas of sanitation, care, and support to people affected by EVD, health infrastructures, and logistics. “During the crisis, there was total reticence of population to the entire medical body, community agents included. From the recruitment and equipment of additional community agents and the rehabilitation of health posts, they (people) started listening, understanding and collaborating. Now in sub-prefectures, there is a good attendance in health facilities. We can even experience that by the numerous cases that are referred at the regional hospital.” —GOG respondent BASIC EDUCATION In Guinea, Pillar II investments did not contribute specifically to the education sector. AGRICULTURE AND FOOD SECURITY Between March 2015 and December 2016, the USAID Office of Food for Peace (FFP) funded Catholic Relief Services (CRS) to implement the Emergency Access to Food for EVD￾Affected Guineans (EFSP) activity in N’Zérékoré and Macenta. In addition, USAID/Guinea partnered with the UN World Food Programme (WFP) to provide emergency in-kind food assistance (USAID Guinea, 2016). Pillar II funds also provided additional support to the WFP’s school feeding program. This activity’s detailed description is provided in the Guinea Desk Review Report (Annex M). EVALUATION FINDINGS : EQ 1– PERFORMANCE n 27 Restoring and improving agriculture: Only one AFS activity remains active. The Improved Livelihoods and Agriculture Development Program (ILADP) is implemented by OIC International (OICI) in Boffa and started in October 2017. KIIs with OICI indicate that the activity has been successful. Its main achievement has been the stabilization of rice prices, from which both rice producers and consumers are benefiting. The activity formed cooperatives of female rice producers, enabling them to negotiate fair prices with buyers from Conakry. The price received by female rice producers has risen from around 4,000 GNF to 5,000 GNF per kilogram between 2015 and 2017;17 the market price for rice consumers dropped from 9,000 GNF to around 7,000 GNF per kilogram during the same period. IBTCI’s 2017 HHS survey found that three percent of HHs received agricultural inputs in the last year. This percentage was higher in Boffa (7 percent) and Boké (6 percent) than in other prefectures. Changes in household food security: The final EFSP evaluation results showed improvements in all categories. Reported food insecurity dropped from 95 percent at baseline (April 2015) to 57 percent in the endline HHS (IBTCI’s 2017 HHS). The proportion of HHs with a high household dietary diversity score (HDDS), defined as those who consume foods from six or more food groups, increased from 68 percent to 82 percent. The proportion of HHs found to be either food secure or mildly food insecure increased from 1 to 32 percent. Respondents reported a significant increase their family’s health status (Catholic Relief Services, 2017). Despite these improvements, IBTCI’s 2017 HHS found higher hunger scores in Macenta and N’Zérékoré (1.4 and 1.2, respectively) than the total figure (0.9) for all 13 prefectures and lower HDDS (5.5 in both) than the total figure (6.5) for all 13 prefectures. Fourteen of the 247 respondents highlighted the WFP as a source for food assistance during the EVD outbreak. “The assistance comforted us a lot and we are getting back to normal little by little.”—FGD participant GOVERNANCE Improving management systems and accountability: The Support to Local Election Reconciliation and National Unity in Guinea activity was implemented by Search for Common Ground (SFG) and the Consortium for Elections and Political Processes Strengthening (CEPPS). A final evaluation conducted in February 2017 showed that the activity: 1) increased citizen’s knowledge about the sources of violence, its consequences, and ways that violence can be avoided; 2) enhanced citizens’ opportunities to talk about their differences; and 3) increased citizens’ information about the election process, mostly through the media. Respondents reported a reduction in the level of violence between communities throughout the area where the activity was implemented. More than 60 percent of respondents found the information around the election reliable and used it 17. IBTCI does not have documentation for this activity. This information was gathered from the KII with IP staff in Oct 2017. to guide their voting decisions (SFG, 2017). IBTCI conducted 28 KIIs on governance, which corroborated these findings. About 66 percent of respondents agreed that they are free to express their opinion. Fifty-eight percent reported feeling free to join a political organization of their choice and 46 percent said they felt free to choose a candidate without any pressure during voting. Thirty￾five percent of respondents suggested that SFG messaging had an impact on perceptions of national unity and peace and were particularly effective in the run-up to the election. Strengthening civil society organizations (CSOs) to provide advocacy, transparency, and accountability activities: In 2017, the CEPPS activity implemented by the National Democratic Institute (NDI) reached 646 political party and civil society representatives, including 228 women and 280 youth, through seven regional public forums. The activity supported a two￾day Women in Politics event in which 67 women from 15 political parties, 17 CSOs, three political associations, and four government ministries—representing all seven administrative regions of Guinea and Conakry—exchanged experiences. The activity trained 77 youth active in 16 different political parties, including 31 women, on effective leadership and communication skills for political organizing (CEPPS, 2017). More than 60 percent of the 28 KII respondents stated opposition to participation in campaign rallies, but support of meetings with candidates and education of the public on their rights and duties. Respondents stated they were aware that USAID contributed (through its partners) to strengthening political dialogue, consensus building, and civic education for the electoral process. “They talk about peace with local elected people, religious leaders, youths and women. They do sensitization. During the past elections they worked with different components of the population for dialogue and consensus between political parties. Sometimes they even invite the Mayor to talk about issues relating to peace, national unity and good cohabitation. In the scheme of elections observation, they go through NGOs and Civil Society to recruit some youths and send them to the ground for sensitization and they convey all information from the ground.” —CSO respondent The Citizens Engagement in Health Governance activity, implemented by FHI 360, began in October 2017. It is too early to report on the performance of this activity. ECONOMIC CRISIS MITIGATION Social protection for EVD survivors and other vulnerable groups: The EFSP activity reached 7,008 HHs in Macenta and N’Zérékoré with livelihood interventions. The activity supported 66 vendors (42 percent women) during two livelihood fairs held between January and December 2016 to boost local economic transactions. However, the report acknowledged challenges to improve the livelihoods of the 28 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES population in a sustainable manner (the challenges were not specified). IBTCI’s 2017 HHS found that 35 percent of HHs reported receiving some form of social assistance within the past 12 months. Respondents most commonly reported receiving cash assistance and food (19 percent each). Almost 27 percent of HHs in Macenta and 38 percent of HHs in N’Zérékoré reported receiving some form of social assistance in the past 12 months (these are the two prefectures covered by EFSP). Friends and family members were the most-often reported source and were the sole source of assistance in at least 26 percent of HHs in 11 prefectures. In the Forécariah and Beyla prefectures, family and friends were given as the sole source of support for just 9 percent of HHs. Government, NGOs, community-based organizations (CBOs), and community groups were mentioned as a source of social assistance by 10 percent of HHs in Macenta and 38 percent in N’Zérékoré. FGDs with beneficiaries (18 of 54 FGDs) highlighted the high value placed on food assistance, cash, and in-kind items distributed by Pillar II activities. Employment is lowest in Macenta and N’Zérékoré (19 percent and 17 percent, respectively). The rate of employment within the last 12 months across all 13 prefectures is 80 percent. Household standard of living: The presence of electricity and mobile telephone are considered to be indicators of a household’s standard of living. According to IBTCI’s 2017 HHS, 49 percent of HHs reported having electricity, compared to 26 percent reported in the Guinea 2012 DHS. This figure ranges from of 95 percent in Conakry to 5 percent in Lola. Eighty-eight percent of HHs reported having a mobile phone, compared to 65 percent reported in the Guinea 2012 DHS. Reported access to improved drinking water sources improved from 75 percent as reported in the Guinea 2012 DHS to 90 percent in IBTCI’s 2017 HHS. Substantial variations still exist; 98 percent of HHs in Conakry reported having access to improved drinking water, compared to just 61 percent in Boffa. Approximately 57 percent of HHs reported having access to improved sanitation facilities, compared to just 19 percent in 2012 (Guinea 2012 DHS). The proportion of HHs with observed handwashing stations with water and soap increased from 34 percent in 2012 (Guinea 2012 DHS) to 43 percent in 2017. This proportion ranged from 10 percent in Macenta to 63 percent in N’Zérékoré. INNOVATION, TECHNOLOGY, AND PARTNERSHIPS Pillar II supported the introduction of CommCare, a Grand Challenge innovation and an open-source mobile platform that supported community-based surveillance (USAID Guinea, 2016). KIIs with GOG and CSO representatives (N=21) and FGDs with beneficiaries (N=4) said that Pillar II activities revitalized community radio in Guinea and community workers harmonized their messages with radio spots and local radio to raise awareness. At the grassroots level of disease surveillance, village protection committees used cell phones to relay information to the prefectural level. Three GOG officials noted during KIIs that the country established the National Agency for Health Security and USAID provided support in strengthening the capacity of country to fight against epidemics and disasters by rebuilding prefectural hospitals and staffing them with qualified agents. The WFP distributed SIM cards to facilitate cash transfers electronically. Through Fighting Ebola: A Grand Challenge for Development, Pillar II funded 14 promising innovations, identified for their potential to reinforce the response to the EVD outbreak and future epidemics. Pillar II funding was provided to support the accelerated development, testing, and deployment of these solutions across all three countries. The original challenge issued to innovators was to “develop new practical and cost￾effective solutions to improve infection treatment and control that can be rapidly deployed 1) to help health care workers (HCWs) provide better care and 2) to transform our ability to combat Ebola” (USAID, 2014). IntraHealth received one of the largest Fighting Ebola Grand Challenge grants, with which it supported the development and adaptation of mHero, a two-way SMS platform that enhances existing national HMIS by connecting HCWs with their MOHs. IntraHealth piloted mHero in Liberia, adapted and further developed it in Guinea and Sierra Leone, and incorporated the application into broader health information strengthening efforts (IntraHealth, 2018). mHero continues to be used on a regular basis by the Liberian Ministry of Health, and the platform has been included in the 2016–2021 Liberian HIS and ICT Strategic Plan. There were noted challenges reported in all three countries in implementing mHero, including a misunderstanding about the tool’s open-source nature in Guinea (USAID Office of Inspector General, 2016b). Twelve of the 14 Grand Challenge innovations have been field-tested in West Africa in at least one site. Some highlights in Guinea include: ■ Columbia University/Kinnos, Inc. Tested its colorized bleach solution, Highlight, in hospitals and ETUs in Guinea and Liberia; ■ Johns Hopkins University (JHU) gained user feedback on their redesigned personal protective equipment (PPE) in Guinea, and conducted more extensive user testing in partnership with Jhpiego in Liberia; ■ mHero reached 7,729 HCWs regionally by the end of the third quarter of FY 2016. CROSS-CUTTING Findings on gender consideration in Pillar II activities are supported by multiple respondents and stakeholder groups (GOG, CSOs, and male and female beneficiaries). The respondents said that development assistance was focused on the needs of women and children and provided access to EVALUATION FINDINGS : EQ 1– PERFORMANCE n 29 health services, food, vocational training, and finance. No sector tended to dominate the statements from respondents. There was consensus that focusing development assistance toward women was effective in: 1) identifying potential beneficiaries; 2) ensuring support for the most vulnerable; and 3) ensuring aid was distributed at the family level. The two most commonly cited achievements were in providing women and children with health services and improving women’s access to income-generating activities in agriculture and home industry. Increasing women’s participation in community organizing and decision-making was also identified as a success by FGD respondents within men and women’s groups. Common themes on overall Pillar II successes in Guinea extracted from the KII and FGD transcripts (n=247) include epidemiological surveillance, rehabilitation of HFs, the training of health personnel, social mobilization, community hygiene education, restoration of reproductive health services, political and social reconciliation, and electoral sensitization. Challenges cited by KII and FGD participants include coordination at the prefectural level, short duration of activities, and inadequate support for EVD survivors and the families of the deceased. LIBERIA In Liberia, Pillar II activities have focused on health and AFS. ICT and MCP are the most prominent type of intervention, though many counties received multiple interventions (see Figure 8). Pillar II health activities entailed frontline worker support, MCP, SBCC, and institutional enhancements and are described in the Liberia Desk Review (Annex M). Results from IBTCI’s 2017 HHS and HFS in Liberia are shown in Annex K. Trend tables with baseline (pre-EVD, 2008–2013), midline, if available (during the outbreak, 2014–2015) and post-EVD (2016–2018) are included in Annex L. HEALTH SYSTEMS AND HEALTH SERVICES Restoration of basic health services: At least one health facility from each county sampled in IBTCI’s 2017 HHS was closed in 2014–2015, with the exception of Grand Cape Mount. Central and county-level GOL stakeholders and many beneficiaries reported improved health service availability and quality in the post-EVD era (e.g., higher HCW performance, better triaging of clients). UNICEF received funding from GHET to restore routine immunizations. IBTCI’s 2017 HFS shows that seven of the eight counties met an 80-percent threshold (based on LQAS performance decision rules) in the provision of immunization services, and in the provision of the minimum package of reproductive, maternal, newborn, and child health (RMNCH) services (Montserrado is the exception). All counties met the 80-percent threshold for provision of malaria diagnosis and treatment services. In contrast, HFs in five out of eight counties had coverage of child survival and adolescent health services below the 80-percent threshold (ranging from 50–79 percent). IBTCI’s 2017 HFS showed that there has been a gradual increase in outpatient attendance in surveyed HFs since 2014 (Figure 13). The mean number of outpatient clients in HFs across the eight counties more than doubled between 2013 and 2017. Strengthening health infrastructure and management system: Seven KIIs and 13 FGDs mentioned institutional enhancement achievements in the health sector. According to IBTCI’s 2017 HFS, renovation/upgrading conducted over the past three years was supported primarily by NGOs in Grand Bassa and Lofa, by the MOH in Bong, Margibi, and Nimba, and by sources other than NGOs or the MOH in Bomi, Grand Cape Mount, and Montserrado. The survey found that all counties were above average (80 percent or more) in the availability of basic amenities, such as improved water sources and safe methods of waste disposal at HFs. Solar energy is the primary source of electricity in all counties except Montserrado. Counties fare below average (less than 50 percent) in availability of telephone, internet access, and backup sources of electricity in HFs. Pillar II resources were also allocated to support supply-chain management through DELIVER and the Coca-Cola and The Bill and Melinda Gates Foundation partnership. DELIVER reported a decline in stock-out rates from 93 percent in the first round of supply to 5 percent in the 9th round of supply in 2016. Twelve KIIs and one FGD raised the issue of essential drug shortages. Data from IBTCI’s 2017 HFS show all eight counties have below acceptable (less than 50 percent) coverage for timely delivery of medicines and commodities to HFs. The issue is particularly problematic in Margibi and Montserrado. “There are still prominent challenges in the health sector, we have issues of unavailability of drugs presently, and the sad thing is that people from the deep rural communities visit the health center Figure 13. Mean number of outpatient clients and the percentage change from previous year (Liberia) Source: IBTCI 2017 HFS; n=153 2013 2014 2015 2016 2017 ▼ 34% ▲ 27% Percentage change from previous year 2,740 3,486 5,110 8,652 4,166 Mean number of outpatient clients each year ▲ 47% ▲ 69% Source: IBTCI 2017 HFS; n=153 30 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES hoping to get drugs or be treated but unfortunately…instructing them to go look for the drugs all by themselves … most of them do not even have money to transport themselves back to their communities let alone having money to purchase the drugs.” —KII respondent, educator in Gbarnga “… our hospitals here [are] doing well in terms of care taking but when it comes to having drugs in our places, it is one of the major challenge that we have. If we go the hospital they do all tests but at the end they will give you the description to go and look for the drugs.” —KII respondent, farmer in Bomi Increasing health worker capacity to deliver non-EVD services: Twelve KIIs and three FGDs highlighted HCW training and capacity strengthening as observed improvements. For example, by September 2017, the MCSP/RHS activity brought 74 of 77 HFs in Lofa, Grand Bassa and Nimba up to MOH standards through training, facility refurbishment, incentives for providers, and ensuring that the MOH provided supportive supervision (MCSP/RHS, 2017). The RHS activity reported the percentage of HFs meeting at least 80 percent of the minimum standards for IPC rose from 46 percent at baseline to 65 percent at the end of year one and 78 percent by the end of the second activity year. MCSP/RHS trained 722 health workers (Jhpiego, 2017). HC3 supported the MOH to develop and disseminate the Healthy Lives logo to signal that information and HFs were trustworthy in order to encourage citizens to access HFs. mHero was piloted in Liberia and has been incorporated as a health communication tool into broader health information strengthening efforts (IntraHealth, 2018). Significant turnover of Liberian MOH staff trained in mHero has been a reported challenge (USAID Office of Inspector General, 2016b). According to IBTCI’s 2017 HFS, at least 80 percent of HFs in all eight counties reported having HCW IPC protocol training within the past two years. Similarly, all eight counties were assessed favorably (above 80 percent) with respect to the existence of essential IPC infrastructure. However, two counties (Bomi and Bong) were only average (ranging from 50–79 percent) in terms of the existence of IPC standard guidelines on site. Only 49 percent of the 35 GHET-supported HFs and 72 percent of the 118 non-GHET-supported facilities surveyed had standard IPC guidelines that were observed by the survey team. There are only four counties (Bomi, Grand Bassa, Margibi, and Montserrado) where at least 80 percent of HFs have sufficient human resources. Provision of specialty services to EVD survivors: IBTCI’s 2017 HHS found that 1.6 percent of sampled HHs had at least one member diagnosed with EVD. JSI R&T’s APC/ETP&SS strengthened health service delivery for EVD survivors in Montserrado, Margibi, Bong, and Lofa counties, supported the establishment of a CSO, the National Ebola Survivors Network, and worked with the GOL’s National Ebola Survivor Secretariat to disseminate information to EVD survivors (APC, 2017b). IBTCI’s 2017 HFS found that fewer than 50 percent of EVD survivors are able to access care and support services in all eight priority counties. These findings are to be expected given the fact that at the time of the data collection for this evaluation, the APC/ETP&SS project had just started. Respondents in four KIIs and five FGDs with beneficiaries and service providers reported that the need for EVD survivor care and support far outstrips availability. One health IP key informant also noted that there has been an under-emphasis on diverse survivor needs. In three separate KIIs and one FGD, EVD survivors conveyed a perception that NGOs have borne the responsibility of supporting EVD survivors, and that GOL participation related to EVD care and support is suboptimal. Health behavior gains: In 21 KIIs and 13 FGDs, respondents stated that development assistance focused on the needs of women and children’s health. IBTCI’s 2017 HHS found that reported skilled birth attendance coverage was 93 percent, much higher than the 2013 Liberia DHS national estimate (61 percent). ANC4 coverage was similar to the 78 percent reported in the 2013 DHS, 79 percent in the 2016 MICS and 79 percent in IBTCI’s 2017 HHS. The rate ranged from 54 percent in Grand Bassa to 87 percent in Montserrado. AGRICULTURE AND FOOD SECURITY As described in the Liberia Desk Review (Annex M), there are four Pillar II/FFP AFS activities for which IBTCI received activity documents. Social protection was a prominent feature of those activities. Issues and outcomes related to non-agriculture￾specific social protection efforts undertaken by Pillar II AFS activities are included in the ECM discussion for Liberia on page 32. This section focuses solely on efforts aimed at enhancing agricultural production and food security. Restoring and improving agriculture: Between July 2015 and September 2016, ACDI/VOCA’s Ebola Recovery and Resilience Program (ERRP) in Bong and Nimba distributed 6,132 agricultural input vouchers (AIVs), falling short of its target of 10,275 (ACDI/VOCA, 2016). However, Mercy Corps’ Economic Recovery from Ebola for Liberia (EREL) activity in Lofa, Margibi, and Montserrado nearly met its target of 10,000, having distributed AIVs to 9,931 farming HHs between January 2015 and December 2016 (Mercy Corps, 2017). Between February 2015 and September 2016, Project Concern International (PCI) distributed 5,861 AIVs, exceeding its target of 5,279 (PCI, 2016). Save the Children’s Emergency Food Assistance for Ebola Affected Families activity in Bong and Margibi provided 24,978 individuals with AIVs or cash between August 2015 and September 2016, very close to its target of 25,129 (STC, 2016). IBTCI’s 2017 HHS survey indicates that 7 percent of HHs across the eight counties reported receiving agricultural inputs in the past 12 months. The percentage ranged from 5 percent in Montserrado to 13 percent in Bomi. With the exception of Bong (6 percent), in AFS-targeted counties HHs receiving EVALUATION FINDINGS : EQ 1– PERFORMANCE n 31 agricultural inputs in the past 12 months ranged from 8 to 10 percent. One IP, two County Agriculture Coordinators (CAOs), and participants in two FGDs with farming students—each from different counties—suggested that Pillar II interventions were not always demand-driven or rooted in the realities of the local context (e.g., understanding the local planting calendar, aligning crops being promoted with public demand/market preferences). Farmers described challenges such as hectic registration processes, paying for goods but not receiving them, and being asked to pay for seeds and fertilizer when they had no money. In 13 KIIs and 1 FGD, CAOs and farmers noted that structural factors such as poor road infrastructure limit farmers’ and traders’ links to markets. One farming group highlighted that Pillar II AFS community engagement activities have led to greater community unity. “USAID has done extremely well. . . They have improved a lot of people’s life, personally I benefited from the agriculture sector, I received a lot of materials for my farming. I plant pepper, cassava and okra and at different times they provided fertilizers and tools, so I benefited.” —KII respondent, farmer in Gbarnga Changes in household food security: IBTCI’s 2017 HHS found that the average HDDS reported across all counties is 5.9 and ranged between 5.3 (Lofa) and 6.4 in Nimba. Montserrado had the second-highest HDDS at 6.0. Households that received social assistance from both informal and formal sources in the past 12 months reported the highest average HDDS (7.0), followed by HHs with assistance only from formal sources (6.7), those with assistance only from informal sources (6.3), and those that received no social assistance (5.4). As shown in Figure 14, AFS activities documented improved HDDS between baseline and endline. IBTCI’s 2017 HHS also revealed low hunger scores; the average household hunger score was 1.3. County-specific scores ranged from 1.1 to 1.8, indicative of a low degree of hunger (see Desk Review, Annex M, for more detail). BASIC EDUCATION Pillar II education activities in Liberia focused on frontline worker support, social protection efforts to spur school attendance, and institutional enhancements. UNICEF’s Education Crisis Response in Liberia (ECRL) improved WASH facilities and IPC through interventions in 120 schools, delivered teaching and learning materials to 4,460 schools, and reached 517,249 primary school children (5 percent over￾achievement compared to the LOP target) with “school in a box” kits (UNICEF, 2017). The activity trained 10,064 teachers on psychological first aid to support children affected by EVD and other trauma such as gender-based violence (GBV) (UNICEF Liberia, 2015). Institutional enhancements in the education sector: Fifteen KIIs respondents from the education, health, and agriculture sectors and five FGDs with young male EVD survivors, farmers, and students identified gender-sensitive WASH infrastructure strengthening and the promotion of handwashing as meaningful achievements. Respondents did not, however, link those achievements to improved educational outcomes. School-based hygiene and behavior gains: In 14 KIIs and 4 FGDs, respondents placed a high value on the positive spillover effect that school-based efforts had on community WASH practices. Measures to sustain these efforts are described in the section on sustainability, page 41. Provision of social protection support through educational assistance: IBTCI’s 2017 HHS found that 20 percent of HHs reported receiving educational assistance in the past 12 months; 16 percent received that assistance in the past three months. Grand Bassa had the lowest coverage (9 percent) and Lofa, Montserrado, Grand Cape Mount had the highest coverage rates (23–25 percent). Respondents from 5 FGDs and 13 KIIs with representatives of county-level GOL, the education and agriculture sectors, and CSO cited examples of educational assistance (monetary or in-kind) that they believe were instrumental in getting children in school/back to school after the EVD crisis. However, some vulnerable children have not fully benefited. Respondents in four FGDs and three KIIs identified educational assistance for girls as a contributor to women’s empowerment. Thirteen KIIs and two FGDs mentioned investment in women and girls as a benefit to HHs and the community at large. Restoration of school attendance: According to the 2013 Liberia DHS, Liberia’s primary school net attendance ratio (NAR) was 38.3 percent. IBTCI’s 2017 HHS reported an average NAR of 77 percent. This rate was highest in Lofa, where it reached almost 90 percent. GOVERNANCE Pillar II governance efforts involved MCP, ICT, and SBCC interven￾tions and targeted government, civil society, media, and communities. Figure 14. Changes in HDDS between baseline and endline (Liberia) Source: IBTCI 2017 HFS; n=153 Mercy Corps (EREL) Jan 15–Dec 16 PCI (PEER) Feb 15–Sep 16 SC Aug 15–Sep 16 4.7 8.6 3.9 5.6 5.8 4.5 BASELINE ENDLINE 32 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Improving management systems and accountability: IBTCI’s 2017 HFS found that 91 percent of the 35 GHET-supported HFs sampled and 81 percent of the 118 non-GHET-supported HFs reported the existence of government regulations for facility operations. Forty percent of GHET-supported HFs and 38 percent of non-GHET-supported HFs reported having village development committees in their catchment areas. Six out of the eight counties reported an acceptable coverage (50–79 percent) of facilities with systems for determining clients’ opinions about health facility services. However, all eight counties scored below acceptable (less than 50 percent) in the availability of evidence (e.g., reports) confirming the above finding. Strengthening CSOs to provide advocacy, transparency, and accountability activities: IREX’s Civil Society and Media Leadership (CSML) activity found that despite citizens’ increased interest in and engagement with government, the proportion of the population with positive perceptions of government declined from 35 percent in 2010 to 21 percent in 2016 (IREX, 2016). Monrovia and Community Radio journalists produced 221 news stories on accountability, far beyond the activity’s target of 120 (IREX, 2016). CSML also reported that 32 female journalists participated in accountability reporting, exceeding its target of 30. The media’s coverage of gender issues and the uptake of gender equity overall did not change (IREX, 2016). CSML reported that 41 communities participated in at least two events related to accountability over the previous four months, exceeding the original activity target of 38 (IREX, 2016). The same number of communities also engaged in advocacy and watchdog functions at least two times over the same period. IREX reported that in the Gbolokai Ta community, where 100 people died of EVD, CSML facilitated dialogue between community members who disagreed about whether to bury the dead or alert the authorities, helping to heal a serious rift (IREX, 2016). ECONOMIC CRISIS MITIGATION Disruptions caused by the EVD outbreak led to a dip in Liberia’s GDP from 8.7 percent in 2013 (pre-EVD) to 0.0 percent in 2014 (during EVD). The country’s 2018 GDP is forecasted to be 3.9 percent (see Desk Review, Annex M). Social protection for EVD survivors and other vulnerable groups: Unconditional cash transfers (UCTs) via direct cash payment and, to a lesser degree, mobile money, were a core component of ECM activities in Liberia. Those activities have either exceeded or came very close to achieving their targets (see Desk Review, Annex M, for more detail). Mercy Corps’ EREL provided UCTs to 30,077 HHs, of which 70.5 percent were female-headed (Mercy Corps, 2017). Through its cash-for￾work intervention, PCI-PEER supported 2,519 individuals (33 percent female) who rehabilitated 187 kilometers (km) of roads, exceeding the target of 121 km (PCI, 2016). IBTCI’s 2017 HHS data indicate that 52 percent of HHs in eight counties reported receiving some form of social assistance within the past 12 months. Cash accounted for 35 percent of this assistance and food an additional 34 percent. Grand Bassa had the lowest proportion of HHs that received cash (14 percent). Montserrado and Margibi, two counties covered by EREL, had the highest proportions (34 and 30 percent, respectively). “During and after the Ebola crisis, there was no food, so the food the partners gave us was what stopped us from dying of starvation.”— FGD participant, student group in Buchanan IBTCI’s 2017 HHS found that 60 percent of household heads reported being employed; 14 percent were not currently employed but had been with in the past 12 months. Twenty￾six percent had not been employed for at least one year. Employment rates were lowest in Grand Bassa, the same county with the lowest proportion of HHs that received social assistance. Nimba, had the highest proportion of HHs receiving any assistance in the past 12 months and the third-highest percentage of unemployment. Forty percent of HHs in Nimba are female-headed. Grand Cape Mount, which has the second￾highest percentage of HHs receiving assistance (69 percent) also has the highest percentage of female-headed HHs (55 percent). Family and friends were reported as the sole source of assistance for at least 46 percent of HHs in seven of the eight surveyed counties. The proportion in Montserrado was 25 percent. Five percent of HHs mentioned government, NGOs, CBOs, or community groups as a source of social assistance. Despite the low coverage of formal sources of assistance, two KIIs and five FGDs highlighted the high value placed on food assistance, cash, and in-kind items distributed by Pillar II activities. Four KIIs and two FGDs mentioned the importance of livelihoods activities such as vocational training, micro-finance support, and improved access to agricultural markets. In 21 KIIs and 13 FGDs, respondents said that women and girls rightfully benefited from ECM activities more than males. EVD survivors, farmers, petty traders, and CSO staff raised concerns about occasional inappropriate distribution of resources at the grassroots level. “When the people come here, they only pay attention to those that were certificated. But those of us that were not opportune [sic] to get that certificate, we are marginalized. We are abandoned people... At times, we find it difficult to find food to eat. —FGD participant, young male EVD survivors in Tubmanburg Household standard of living: In four FGDs, participants who benefited directly from social protection support, particularly livelihoods training, reported a satisfactory quality of life, citing issues such improved ability to address household basic needs and send children to school. However, participants in seven FGDs said that household and community resilience is tenuous EVALUATION FINDINGS : EQ 1– PERFORMANCE n 33 or that quality of life is worse than before the outbreak, particularly for those not reached by recovery efforts. According to IBTCI’s 2017 HHS, only 27 percent of all HHs reported having electricity compared to 10 percent in 2013 (Liberia 2013 DHS18). The highest proportion is in Montserrado (56 percent) and the lowest is in Grand Bassa (4 percent). Seventy percent of HHs reported having a mobile phone; rates are lowest in Grand Bassa and Nimba (53 and 58 percent, respectively). Reported access to improved drinking water sources has not changed dramatically over time. Substantial variations still exist; Montserrado reported the lowest access to improved sources of drinking water (52 percent). Grand Cape Mount reported the highest (95 percent). Note that HHs in Montserrado rely heavily on bottled/packaged water, which is not regarded as an improved water source. IBTCI’s 2017 HHS found that Bomi, Grand Bassa, and Grand Cape Mount reported the lowest proportion of HHs with access to improved sanitation facilities (19 to 21 percent). According to the Liberia 2013 DHS report, HHs reporting improved sanitation were 19 percent in 2013. The rate varied between 39 and 50 percent in the other five counties. The percent of HHs reporting handwashing stations has increased from a baseline of two percent reported in the Liberia 2013 DHS. IBTCI’s 2017 HHS found the rate to range from 7 percent in Bong to 50 percent in Grand Bassa. In 51 percent of HHs with handwashing stations, water and soap or other cleansing agents were reported present, comparable to the Liberia 2013 DHS finding of 49 percent. INNOVATION, TECHNOLOGY, AND PARTNERSHIPS Few KII respondents referenced innovation. Infrastructure challenges related to electricity, road networks, or telecommu￾nications were identified in 13 KIIs and 1 FGD. Respondents noted that radio and traditional leaders are vital in disseminat￾ing information and messages, as was evident during the EVD crisis. NetHope received Pillar II funds to implement three activities focused on ICT infrastructure, policy, and capacity improvements. The World Wide Web Consortium received funding to support development of public private partnership (PPP) policy recommendations for responding to future outbreaks and technical assistance to the GOL. However, USAID did not provide the evaluation team with documentation of any of these activities. Tetra Tech received Pillar II funds to rehabilitate the piped water systems in Lofa, Grand Cape Mount, and Nimba. The evaluation team received one document on this activity that provided targets, but little data on the activity’s 18. https://dhsprogram.com/pubs/pdf/FR291/FR291.pdf Liberia 2013 DHS collected information from the entire country and totals are country-wide. IBTCI’s 2017 HHS collected information in eight priority counties and totals refer to those counties only. County-level trend comparisons are shown in Annex L tables. performance. The activity generally exceeded its year-one targets related to non-subsidized WASH infrastructure operating expenses (Tetra Tech, 2017). USAID partnered with C-Squared, a partnership between Google, Convergence Partners, International Finance Corporation, and Mitsui Company Limited, to improve Liberia’s communications infrastructure and address critical weaknesses in communications systems exposed during the EVD crisis. Strong communications infrastructure facilitates disease surveillance by supporting rapid identification of disease outbreaks. This partnership was also designed with the secondary objective of supporting economic recovery. ICT for the health system: Pillar II ICT support in Liberia included MEASURE Evaluation’s embedment of experts in the MOH, who contributed the development of an HMIS strategic plan. The activity also provided six months of virtual technical support (Hart et al., 2017). There has been a lag in central￾level achievements manifesting at the subnational level. IBTCI’s 2017 HFS found that facilities reported still using paper-based mechanisms for data collection and reporting and all counties were assessed as suboptimal (less than 50 percent) in the use of electronic reporting systems. The evaluation team acknowledges that installing electronic reporting systems at the primary care level in Liberia is a challenging task. mSTAR partnered with Lonestar Cell MTN to adapt the mobile salary payment platform that it developed for the Ministry of Education (MOE) for use by the MOH. The service was operational in four of nine targeted counties by the third quarter of FY17 and had enrolled 119 MOH employees, out of a life-of-activity target of 3,600 (FHI 360, 2017). Liberia, where the burden of disease was highest, was often targeted for field applications of the Grand Challenge innovations. Seven of the 14 Fighting Ebola Grand Challenge innovations were field-tested in Liberia. These included: ■ TOMI™ Environmental Solutions field-tested its decontamination chambers in collaboration with PCI; ■ Highlight colorized bleach solution (Columbia University/ Kinnos tested its colorized bleach solution in Liberia and Guinea with PCI and IMC, respectively); ■ JHU field-tested its redesigned PPE suit in Liberia in partnership with Jhpiego; ■ Baylor’s Emergency Pod ■ Shift Lab’s DripAssist; ■ IntraHealth’s mHero platform; and ■ Dimagi’s CommCare platform. 34 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SIERRA LEONE In descending order of magnitude, USAID Pillar II investments in Sierra Leone focused on AFS, restoration of non-EVD essential health services, health systems recovery, economic crises mitigation, and information, technology, and partnerships. MCP was the most common type of intervention (eight activities), followed by social protection (six activities) and ICT (five activities). The details of the main activities funded with Pillar II resources are described in the Sierra Leone Desk Review (Annex M). Annex K includes the results of IBTCI’s 2017 HHS and HFS in Sierra Leone. Trend tables with baseline (pre-EVD, 2008–2013), midline (2014–2015), if available, and post-EVD (2016–2018) are included in Annex L. NON-EVD HEALTH SERVICES, HEALTH SYSTEMS RECOVERY AND SURVIVOR PROGRAMS Pillar II health activities included facility upgrades, including improvement of WASH infrastructure, policy development, improvements to the HIS and supply chain management, health worker training with a focus on addressing the needs of EVD survivors, IPC training and community engagement. Restoration of basic health services: IBTCI’s 2017 HFS shows fluctuation in outpatient attendance in surveyed HFs since 2014 (Figure 15), with a general pattern of increase. The mean number of outpatient clients in the sampled HFs increased by 35 percent between 2013 and 2017. The JSIR&T/APC Post-Ebola Recovery of Health Services (PERHS)activity found that ANC visits in 305 HFs across five districts (Bombali, Port Loko, Tonkolili, Western Area Rural (WAR), and Western Area Urban (WAU)) increased by 10 percent between 2014 and 2016. ANC4 increased by 25 percent and the number of deliveries grew by 19 percent over the same period. JSI R&T/APC trained 950 health facility staff (666 clinical) on RMCH, IMNCI, IPC, WASH, and HMIS and 1,491 community health workers (APC, 2017a). Strengthening of health infrastructure and management system: APC supported the development of a manual on WASH standards and guidelines for HFs and improved WASH infrastructure in 110 peripheral health units in five districts. This included improved electricity through installation or rehabilitation of solar panels, improved water supply, the installation of sinks, and the installation or rehabilitation of showers, toilets, and septic tanks. These efforts complemented the investments of organizations such as MSF and the UN Office for Project Services (UNOPS). Further, APC distributed minor medical equipment (such as delivery beds, delivery kits, weighing scales, steam sterilizers, and infant resuscitators) to 305 facilities. To support the establishment of a national emergency medical services system, APC procured at least four ambulances for each hospital it supported for patient transportation and support referral. To collate morbidity and context data at the facility level, the EPIC platform and dashboard, developed under the Fighting Ebola Grand Challenge, were piloted in one district. Five of the eight districts assessed in IBTCI’s 2017 HFS were reported to have above-average availability (80 percent or more) of basic amenities such as improved water sources and safe methods of waste disposal at HFs. Tonkolili and WAR were assessed as average on improved water source (50–79 percent), while Kailahun was assessed as average on appropriate sharp waste disposal. Solar energy is the primary mode of electricity in all districts except WAU. However, districts were scored below average (less than 50 percent) in availability of telephone, internet access, and backup sources of electricity at the HFs. The HFS further shows that surveyed HFs in Bombali, Kailahun, Kenema, and Porto Loko were assessed above average (80 percent or more) and had stocks of medicines, vaccines, and contraceptive commodities at the time of the survey. WAR and WAU districts were assessed as average (50–79 percent) and Koinadugu and Tonkolili were assessed as below average (less than 50 percent). However, all eight districts reported challenges with timely delivery of medicines and commodities to HFs. HC3 supported the Ministry of Health and Sanitation (MOHS) to develop and disseminate the Healthy Lives logo to promote health facility service utilization. JSI R&T/APC reactivated or established facility management committees (FMCs) in 214 peripheral health units (PHUs) and trained more than 2,539 FMC members (903 women and 1,627 men). The proportion of FMCs that reported holding quarterly meetings increased from 76 percent in early 2016 to 97 percent by mid-2017 (APC, 2017a). The KIIs reported that these FMCs raised funds from community members for health facility maintenance and rehabilitation. One KII with an IP said that some FMCs are Figure 15. Mean number of outpatient clients and the percentage change from previous year (Sierra Leone) Source: IBTCI 2017 HFS; n=128 2013 2014 2015 2016 2017 Percentage change from previous year Mean number of outpatient clients each year 1,261 ▲ 16% 1,459 ▼ 5% 1,380 ▲ 2% 1,414 ▲ 20% 1,702 Source: IBTCI 2017 HFS; n=128 EVALUATION FINDINGS : EQ 1– PERFORMANCE n 35 “…identifying and managing their own transportation solutions to facilitate the transport of complicated cases to HFs.” IBTCI’s 2017 HHS found that about 40 percent of HHs reported participating in activities associated with community-based health services. Increasing health worker capacity to deliver EVD and non￾EVD services: USAID funded the International Organization for Migration (IOM) to support integration of IPC into Sierra Leone’s pre-service training for health professionals. More than 30 master trainers have completed IPC training and are expected to reach 5,000 university students (USAID Presentation, October 2017). Pillar II activities in the health sector collaborated to avoid duplication of effort and maximize impact. For example, HC3 linked with the Management Sciences for Health (MSH)- led SIAPS activity to ensure all HC3-supported PHUs that received essential storage equipment and facility drug stores also received packing support. The PERHS activity led by JSI R&T trained 950 staff from 243 HFs in five districts on various topics, including IPC, RMNCH, and integrated management of childhood illnesses (APC, 2017a). Five of the 14 Fighting Ebola Grand Challenge innovations were field-tested in Sierra Leone. These included two innovative devices to enhance patient care and HCW safety (Shift Lab’s Drip Assist, and STAMPS remote monitor of patients’ vital signs), improved designs for emergency health posts (Baylor’s Emergency Pod), and two ICT innovations to improve HCW access to timely health information in outbreak and routine circumstances (IBM EPIC platform and Dimagi’s CommCare platform). IBTCI’s 2017 HFS found in all eight surveyed districts, health workers in at least 80 percent of facilities reported receiving IPC training within the last two years. Five districts were found to have guidelines for IPC in more than 80 percent of their HFs. Kenema, WAR and WAU were the exceptions. At least 80 percent or more of HFs reported receiving supervision from a higher level at least once in last three months in seven districts. This rate was 50–79 percent in WAR. Provision of specialty services to EVD survivors: In IBTCI’s 2017 HHS, respondents reported at least one person had been diagnosed with EVD in 1.3 percent of HHs surveyed. The Government of Sierra Leone (GOSL) designed the Comprehensive Program for EVD Survivors (CPES) to address the physical and mental health, livelihood, and reintegration of EVD survivors and mitigate the risk of EVD resurgence. Starting in 2015, JSI R&T and partners implemented community health dialogues targeting EVD survivors, facilitated by trained community health officers and social workers. Groups of 15 to 18 community members met once a week to discuss how to promote mental health, talk about social issues in their communities, and work toward community resilience (APC, 2017a). Gaps remain in the provision of services to EVD survivors. IBTCI’s 2017 HFS showed only one district (Bombali) scoring an acceptable coverage (50 percent or more) of HFs that offer care and support for EVD survivors. However, it must be noted that JSI R&T’s survivor project did not get started until 2017. Therefore, JSI R&T’s engagement with EVD survivors prior to 2017 was through the PERHS project. Health behavior gains: HC3 helped develop Sierra Leone’s National Health Promotion Strategy 2017–2021, which provides a framework for SBCC interventions. The activity built the capacity of 49 government staff to manage and coordinate SBCC activities and trained 499 individuals in interpersonal communication approaches. HC3 engaged communities in planning, implementing, and monitoring activities to foster ownership. The activity also produced a radio program and supported 93 community listening groups that discussed RMNCH issues emerging from the broadcasts. Four KIIs and two FGDs reported a decrease in EVD stigmatization, which may have also led to a reduction in the mistrust in health workers. “Confidence and trust between health care staff and patients has been restored. IPC is now practice. We see it when we visit the hospital. WASH facilities have been provided. The free health care is back and to crown it all, we have a fully operational ambulance and referral services. This never existed [before the outbreak].” —KII respondent IBTCI’s 2017 HHS found that 93 percent of respondents indicated that they would seek care at an HF if they had a fever. District-specific rates ranged between 84 percent (WAU) and 99 percent (Kailahun) (baseline not available). Among those who said they would not seek care, about half stated a lack of money for treatment as their major obstacle. Less than 25 percent indicated their choice was due to fear of being stigmatized for having EVD; none reported they believed the facility would be contaminated by EVD. IBTCI’s 2017 HHS found the reported level of ANC4 coverage in priority districts (72 percent) similar to the figure reported by the Sierra Leone 2013 DHS nationally (76 percent). The 2017 rate ranged from 62 percent (Kailahun) to 90 percent (Bombali). Reported skilled provider attendance during delivery increased from 60 percent nationally (Sierra Leone 2013 DHS) to 88 percent in priority districts (IBTCI’s 2017 HHS), with 2017 district-specific results ranging from 72 percent in Kenema to 98 percent in Kailahun. Reported facility￾based deliveries increased from 54 percent nationally (Sierra Leone 2013 DHS) to 85 percent in priority districts (IBTCI’s 2017 HHS) and range from 73 percent in Porto Loko to 97 percent in Kailahun. 36 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES AGRICULTURE AND FOOD SECURITY In Sierra Leone, USAID’s FFP supported UCTs, conditional cash transfers (CTs), provision of agricultural inputs, training in agricul￾tural production, health, nutrition, household resource manage￾ment, and other relevant topics, and support for Village Savings and Loan Associations (VSLAs). These interventions were implemented in six priority districts, Kailahun, Bombali, Tonkolili, Koinadugu, Kenema, and Port Loko. FFP also partnered with UNICEF to provide in-kind food assistance to HHs impacted by the EVD outbreak. Details of these activities are included in the Sierra Leone Desk Review (Annex M). Restoring and improving agriculture: Across all activities, more than 28,000 families received seed vouchers, 400 small traders (mostly women) received CCTs, and 9,600 VSLA members received support (Radice, 2017; ACDI/VOCA, 2018). CRS trained farmers on production and marketing of high-yield crops, such as squash. It also supported livestock and poultry production. An evaluation of the program found that: ■ Supplementary payments provided to mitigate the impact of inflation on participants’ purchasing power were mostly used to buy seeds (81 percent), pay for farm labor (62 percent), and purchase other agricultural inputs (38 percent). ■ Forty-eight percent of respondents reported that the amount of farmland they cultivate has increased (CRS, 2016a; CRS, 2016b). ACDI/VOCA implemented a program to provide recovery grants to small agribusiness enterprises heavily impacted by the EVD outbreak. Nine KIIs mentioned the importance of this support to stimulate or increase production and/or yield (ACDI/VOCA, 2018). “. . . our farmers are back on track. They have received inputs such as fertilizers, seeds, and seedlings and training on improved farming methods. A lot of activities to support the sector have been implemented by government and partners. Most of the farmers who stayed in their communities have gone back to work as they did before the outbreak. Some have even increased their farm size as a result of the recovery projects.” —KII respondent, Bombali IBTCI’s 2017 HHS survey found that of various types of social assistance, four percent of HHs reported receiving agricultural inputs during the past 12 months. Changes in household food security: IP performance reports and USAID documents indicate that over 67,000 HHs benefited from CTs by November 2017, meeting practically all targets (USAID, 2017). Further, 1,225 children suffering from severe malnutrition were referred a food supplementation program (ACDI/VOCA, 2018). An earlier assessment of CTs in Liberia and Sierra Leone conducted by the Cash Learning Partnership (CaLP) found that 75 percent of funds provided through unconditional CTs were spent on food (Radice, 2017). CRS conducted a midterm evaluation in June 2016, ten months after their first distribution of cash, that found that among supported HHs the percentage that reported moderate or severe hunger decreased from 68 percent at baseline to 28 percent at midterm and HDDS increased from 2.3 at baseline to 5.5 at midterm (CRS, 2016a; CRS, 2016b). IBTCI’s 2017 HHS reported an HDDS of 5.3 across all districts. The score ranged from 4.0 in Kailahun to 6.2 in WAU. The average reported household hunger score was 1.5, ranging from 1.1 in WAU to 2.0 in Kenema (the lower the score, the lower the hunger). Thirty-four percent of HHs reported receiving assistance in the last 12 months. Food and cash (19 percent and 18 percent, respectively) were reported to be the most often-received types. ECONOMIC CRISIS MITIGATION Sierra Leone’s GDP fell from 20.7 percent in 2013 (pre-EVD) to -20.5 percent in 2015. It is forecasted to rise to 6.1 percent in 2018 (see Desk Review, Annex M). Social protection for EVD survivors and other vulnerable groups: Five IPs implemented CT activities that reached over 67,000 HHs (USAID, 2017). Results from KIIs suggest that the CTs had positive ripple effects, especially when individuals invested in productive activities. “People were able to use some of the cash for longer-term livelihood activities such as farming; the farmers were able to save some of the seeds they received through seed vouchers for second planting.”—KII respondent, NGO in Freetown One of the KII respondents gave an example of a beneficiary: “After receiving her first payment of $90 in early 2016, she set up a small table in the center of her village and began selling condiments. …She used the next two cash payments to expand and diversify her inventory and built a small shop in the village where she sold staples (such as rice and flour), and other essential commodities such as gasoline. …this helped her increase her income and she and her family are eating better. She has joined a savings group with about 10 of her fellow villagers and is now able to send all her four children to school, including three to high school.” An evaluation report of the Rapid Ebola Social Safety Net and Economic Recovery activity revealed that 70 percent of female-headed HHs reported strong involvement of women in decision-making regarding the use of cash resources and expenditure, compared to 25 percent of male-headed HHs reporting similar involvement by women in decision-making regarding the use of cash resources and expenditure (IDRC 2017). Recipients of the cash transfers also received gender training aimed at increasing women’s involvement in household financial decisions. EVALUATION FINDINGS : EQ 1– PERFORMANCE n 37 IBTCI’s 2017 HHS shows that 34 percent of HHs reported receiving some form of social assistance in the past 12 months. Current employment rates are lowest in Bombali and Kenema (54 percent and 56 percent, respectively) compared to 66 percent across the eight priority districts. Friends and family were the most often cited source of assistance across all priority districts and were the sole source for at least 14 percent of HHs in five priority districts. Government, NGOs, CBOs, and community groups were mentioned as a source of social assistance by two percent of HHs in WAR and six percent in WAU. Household standard of living: According to IBTCI’s 2017 HHS, only 29 percent of all HHs reported having electricity, compared to 14 percent in 2013 (Sierra Leone 2013 DHS19). The rate is highest in WAU (78 percent) and lowest in Kailahun (3 percent). Seventy-two percent of HHs reported having a mobile phone, compared to 55 percent in 2013 (Sierra Leone 2013 DHS). Reported access to improved drinking water sources has improved from 60 percent as reported in the Sierra Leone 2013 DHS to 76 percent of HHs surveyed in IBTCI’s 2017 HHS. The latter found that 41 percent of HHs reported access to improved sanitation facilities, up from 10 percent in 2013 (Sierra Leone 2013 DHS). The proportion of HHs with observed handwashing stations that have water and soap, detergent or other cleansing agents has improved from 34 percent in 2013 (Sierra Leone 2013 DHS) to 54 percent. INNOVATION, TECHNOLOGY, AND PARTNERSHIPS Funded through Fighting Ebola Grand Challenge, the Scripps Translational Science Institute, Rhythm Diagnostic Systems, and PhysIQ tested the Sensor Technology and Analytics to Monitor, Predict and Protect Ebola (STAMP2) Wearable Patient Sensor in Sierra Leone in the Fall of 2015 in partnership with International Medical Corps (Steinhubl, SR 2015). This wearable patient sensor allows HCWs to remotely monitor patients’ vital signs, including heart rate, respiratory rate, temperature, and oxygen saturation. The USAID Global Development Lab also supported activities to improve health information system interoperability for example through support to MOHs across the Economic Community of West African States (ECOWAS) in building the local information systems capacity of the West Africa Health Informatics Team (WAHIT). 19. https://dhsprogram.com/pubs/pdf/FR297/FR297.pdf Sierra Leone 2013 DHS collected information from the entire country and totals are country-wide. IBTCI’s 2017 HHS collected information in eight priority districts and totals refer to those districts only. District-level trend comparisons are shown in Annex L tables. IntraHealth’s mHero innovation was piloted in Liberia and adapted and further developed in Guinea and Sierra Leone. Lack of awareness about the tool in Sierra Leone was a challenge (USAID Office of Inspector General, 2016b). UNICEF’s Innovations Lab developed an open source tool called Rapid Pro to support data collection efforts and share data in real time. Sierra Leone’s MOHS is using Rapid Pro to send information to health workers. Community-based health workers also use the tool to send data to HFs. UNICEF is working to create an enabling environment for financial innovation by developing a “sandbox” where financial services innovators can test their ideas in a safe digital environment before rolling them out to a mass market. The GOSL is strengthening its electronic payment system, through which $20 million every month is disbursed to 80,000 government workers (USAID, November 2017). Three KIIs with GOSL representatives mentioned that USAID’s support for training facilitated surveillance and the dissemination of health information. Key informants also noted new technology and infrastructure such as computers in PHUs, motorbikes for health workers, and solar panels received from USAID. Though key informants did not cite much in the way of formal PPPs, this is not surprising since the Global Development Lab tends to focus on higher-order PPPs where the bilateral Missions were more involved with specific recovery activities. There was engagement, however, between GOSL and the private sector during recovery. According to one CSO KII, the private sector stepped in to procure equipment and pharmaceutical products and gas stations provided fuel for supply chain work for MCP interventions. Two key informants mentioned GOSL work with private companies for institutional enhancements, such as rebuilding schools. With USAID support, the GOSL also worked closely with cellular phone providers Africel and Orange, and mobile money provider Splash to initiate mobile cash transfers and increase the number of access points from one per 20,000 people to one per 1,000 people (USAID, November 2017). 38 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Evaluation Question 2: Sustainability 20. The final analysis of the sustainability of Pillar II activities will be included in the PE2. EQ2 Sustainability: What results of Pillar II activities have endured after the activities have closed, and why? The evaluation team applied the intervention typology shown in Figure 5 to the analysis of EQ2.20 For PE1, under the domain of sustainability, the Team created a framework to demonstrate the inter-relationships between proximate and distal determinants and each of the intervention types. IBTCI described a sustainability pathway to examine the determinant(s) for both promising interventions and those with a lesser chance of positive outcomes. Which determinants are missing may help explain why an intervention may not achieve maximum sustainability. Akin to the need to aggregate similar components under one rubric, IBTCI submits that these sustainability determinants fall under one or more of these categories: policy, strategies, and plans, finance, accountability, institutional support, technical choices, human dimension, and/ or external threats and enabling environments. Based on a review of the literature as to the determinants of sustainability, the Team developed a framework for measuring it; this framework was vetted at the 2017 USAID Mini-University (Figure 16). Further development of the model led to its operationalization and it is currently implemented by the Team members to analyze various data sets—both quantitative and qualitative. In PE2, the Team will define case interventions as either likely sustainable or unsustainable using mostly qualitative data to identify and verify the existence of seven key determinants. To meet this end, the Team first defined each of the determinant categories. Key determinants that contribute to sustainability include policies, financial resources allocated from national or private sector accounts, the anticipation and mitigation of operational issues (such as supply and logistics, maintenance, and provision of power, internet, and water), accountability, institutional support, human resources (availability, capacity, and supportive supervision), and community support at the local and national levels. The Team assumes that there are both proximate and distal factors that influence results and that they may operate within the support of mediating factors. The results in question are organized around the three Pillar II objectives: prevent the loss of development gains, recovered and/or strengthened institutions and infrastructure, and sustained systems. IBTCI posits that results may be sustained over time provided that a number of enabling and influencing determinants are also supported during program efforts. The Team utilizes seven potential categories of distal and proximate determinants that enhance or impede sustainability: policy, finances, institutional Figure 16. Analytical framework to measure sustainability Sustained Results Outbreak Thematic Areas of Response Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology Institutional Enhancements Social & Behavior Change Communications Intervention Types Policies, Strategies, and Plans Finances External Threats and Enabling Environments Human Dimensions Determinants of Sustainability Accountability Institutional Support Technical Choices Immediate Results Ebola Pillar II Response Challenge: Mitigate second-order impacts of the outbreak at the individual, community, and institutional level Survivor Programs EVALUATION FINDINGS : EQ 2– SUSTAINABILITY n 39 support, technical choices, accountability, human dimension, external threats and enabling environment (see Box 1). Below are the definitions for each determinant. 1. Policies, Strategies, and Plans—Policies are predicated by procedures, rules, and allocation mechanisms to provide services and/or to set priorities and guide programs. Policies may be implemented through four types of instruments: laws, regulations, assignment of responsibilities to institutions operating in the sector, and economic incentives (i.e., subsidies) for a given sector to operate properly. Hence, a determinant labeled policy will indicate both how it is manifested and also how it is implemented. IBTCI submits a broad definition for policy: Policy is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. Policies with sound strategies are more likely to be adopted. Likewise, strategies need comprehensive and clear plans in order to be implemented. Thus, policies, strategies and plans are subsumed under one overarching determinant, as they should be inextricably linked in order for the effect of the policies to be realized. Therefore, the IBTCI definition for the first determinant is expanded: There is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. There are also extant strategies and plans for policy implementation. Policies, strategies, and plans create a framework for the implementation of any of the activities associated with post￾EVD recovery. An example might be a health policy or an agricultural development policy that was conceived either as the result of USAID funding and/or influenced or shaped by the Pillar II interventions that USAID supports. 2. Finances—As noted, policies and strategies are operationalized through plans. The implementation of plans, however, requires budget allocations and access to funds. In regard to finances, IBTCI defines this determinant as the: Allocation of financial resources for the implementation of plans is in place, with a determination of how much funding is needed, the source of those funds, and how the funds will be apportioned and used. 3. Accountability—Accountability describes the mechanisms and procedures through which decision-makers, policy makers, and implementers justify and assume responsibility for the actions, decisions, services, and products that they undertake. Accountability is best proven through written documentation. There are different types of accountability. Further defined, the salient aspects of accountability follow (UNDP Water Governance Facility/UNICEF, 2015): a. Financial accountability means that institutions and individuals must document the intended and actual use of financial resources allocated to them. A clear demonstration of how the funds were expended should be explicit and without ambiguity. b. Political accountability occurs when constituents request their representatives to justify their actions. If there is political accountability, there are no questions as to the legitimacy of the authority or position of a politician. There may be vertical channels of accountability that link citizens to their government. Horizontal accountability refers when partners of equal standing request their counterparts to account for their actions (i.e., when one local government requests another to justify and explain their actions. c. Social accountability are actions taken by civil society, the media and individuals to incite or implore decision-makers to account for decisions and actions undertaken. d. Administrative accountability is exemplified when one level of an administration asks another to account for their decisions and actions and is documented in an organized manner. Guidance and protocols need to be in situ to ensure accountability. Governmental and other institutional and organizational systems are put into place to decide who gets what, when, and how. These mechanisms should be clearly delineated, and expectations should be communicated to relevant stakeholders. These may include incentives for high performance or disincentives/sanctions for inadequate or irresponsible performance. In sum, the IBTCI accountability determinant is defined as: Systems and/or mechanisms are in place to ensure appropriate use of financial resources and rights holders’ and duty bearers’ access to information (e.g., on performance and results). These systems and/or mechanisms safeguard the responsible and constructive use of resources and have transparent processes that are manifested by the populace’s respect for the respective institution. 4. Institutional Support—The institutional support determinant refers to organizations that are responsible for implementing actions in support of a policy, strategy, plan, or program. Institutional support examines the actions that are implemented as part of that support and how it may be coordinated among the actors involved. Further down this determinate’s pathway to sustainability, exit and transition strategies are established. Institutional support is thus defined by IBTCI as: Organizational arrangements and/or mechanisms are in place that a) operationalize agreed-upon inputs, roles and responsibilities for implementing policies and interventions and b) produce exit and transition strategies when feasible and appropriate. 40 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 5. Technical Choices— Assuming resources are constant, a list of technical choices can be infinite. Hence, the Team will measure a technical choice based on the following criteria: a) appropriateness and relevance to the problem at hand and whether it will solve that problem in an efficient and effective fashion at best value; b) timeliness; and c) demand on resources (human, capital, physical). Thus, the IBTCI definition of the technical choices determinant is: The best technical approach has been selected to lead to implementation decisions determining whether interventions are “fit for purpose” (effective in addressing the identified problem), feasible given time and resource constraints, and technically relevant. 6. Human Dimension—The importance of the human dimension in the sustainability of development programs is multi-faceted. The human dimension might be as straight￾forward as having a person available whose job is to maintain a newly refurbished school or health facility, or a cadre of information technology experts to maintain a fiber optic cable around Monrovia. On the other hand, the human dimension might include more oblique characteristics, such as cultural or institutional norms. Norms present challenges to discussion or systematic analysis as their salient components may be difficult to define. Questions to ask might include, “Is there stigma and discrimination and if so, to what degree?” The Team is exploring the degree to which Pillar II funding ameliorated the consequences of negative norms and/or directed their activities to change them. Transparent and equitable decision￾making may be a harbinger for successful implementation of an activity or project, the uptake of services, or changes in social behaviors. This particular determinant stretches across all intervention typologies. Interventions need involvement from a human being to sustain it and serve as its champion. Arguably, this is a somewhat oblique determinant that is open to interpretation and further definition. Thus, for the purposes of evaluation, IBTCI defines human dimensions as follows: Any situation whereby a human being’s input is needed to sustain the intervention. Identify the human dimension— describe it and define it—and the level at which it is available in the appropriate quantity and quality. These nuances are then examined qualitatively to identify activities poised for success and sustainability. 7. External Threats and Enabling Environments—Complex environments, such as those areas in which EVD was most palpable, are influenced by external threats and/or buoyed by enabling environments. This category of determinants is focused on external threats and factors that influence the intervention outcome. They revolve around socio-political stability, geo-politics, climatic conditions, and population migration/internal displacement. For example, roads are needed for the transportation of goods and services to and from villages (e.g., the cold chain for vaccines). Torrential rains or natural disasters may affect the lifespan of roads, thus increasing the need for frequent maintenance or replacement. Climate then is a contextual factor that affects the sustainability of road access and use. Other threats may include internal conflicts between territories, elections, or natural disasters. External threats and enabling environments as a determinant are thus defined by IBTCI as: Those factors that have a direct or indirect influence on the ability of an intervention to be sustained, even when all other determinants are supportive of a sustainable outcome. Photo by Elie Gardner for Catholic Relief Services EVALUATION FINDINGS : EQ 2– SUSTAINABILITY n 41 CASE STUDY: HEALTH FINANCING IN GUINEA The EVD outbreak brought Guinea’s economy to a standstill, resulting in an estimated US$600 million loss and a reduction in real GDP growth to 3.5 percent in 2015 (World Bank, 2016). The country has proven its resiliency, however; fueled by an increase in the production of bauxite and gold and resurgence in the agriculture sector, GDP grew by a 6.6 percent in 2016 and the national deficit fell to 1.4 percent from 8.1 the previous year (World Bank, 2017). Real GDP growth was estimated at 6.4 percent in 2017 and projections for 2018–19 are similar (African Development Bank, 2018). To ensure that Guinea’s health sector benefits from the country’s financial resurgence, HFG has worked with the Legislative Assembly’s Health Commission to build the legislators’ capacity to develop a realistic budget for the health system and advocate for its funding. With the activity’s support, the commission was able to increase Guinea’s health budget to 8.2 percent of national spending in 2017 from 5.2 percent in 2016. The proportion of funding allocated to health dropped slightly in 2018 and the country is far from their goal of committing 15 percent of the national budget to health by 2020, but the country’s commitment to a stronger health system remains evident (MOH, 2017a). The MOH’s strategic plan and budget for 2018 includes post-EVD health system strengthening activities and funding for an additional 2,000 health workers (determinant: financial resources). The decrease in the MOH budget appears to be a reduction across ministries in an attempt to reduce the overall national budget deficit (LFI, 2017; MOH, 2017b). GUINEA While there are few primary data available to assess the sustainability of interventions, Pillar II activities contributed to outputs that could help health systems and practices to improve the population’s health over time, such as policy and strategy documents and standard operating procedures (SOPs), and set the groundwork for sustainability. MCP: As presented previously in the performance findings section, HFG supported the National Assembly’s Health Commission to increase the health budget to 15 percent of government expenditure by 2020. This may contribute to sustainability of Pillar II investments.21 However, the MOH’s 2018 budget decreased (determinant: financial resources) from the anticipated 8.2 percent to 7.2 percent (LFI, 2017; MOH, 2017a; MOH, 2017b). KIIs with service providers (5 out of 20 respondents) reported insufficient funding, short duration of activities, problems maintaining infrastructure and logistics, and stigma against survivors as challenges to sustainability. GOG informants (n=2) said that health sector coordination meetings are still held with IPs, but not as frequently as during the EVD response. Respondents also noted a lack of coordination among development partners, the lack of essential drugs, and weak health governance as ongoing challenges. “We note that after Ebola, the principle of handwashing is observed in our different communities, in our various services and even in some schools so far we force children to wash their hands before they start courses in school. There are changes in terms of health infrastructure but the worst thing is that, aids that partners sent to victims did not reach their destination and they were sold in stores here; Change is mostly seen in 21. KII HFG health. This change is most noticeable in big cities whereas, in villages so far, the health system is still struggling.” —FGD participant Frontline worker support: Respondents in an FGD with USAID and KIIs with GOG (8 out of 61 respondents) said that institutional support such as training and capacity building of prefecture health teams and facility staff will have long-lasting effects. Two GOG respondents believed that improvements such as IPC in HFs, facility rehabilitation, health worker training and capacity building, and disease surveillance will be sustained. “Training has had a lot of impact in our daily activities. First, the way we take care of patients has changed. We can see that even the way of dressing has changed. It is also important that focal points be trained and empowered in their activities so that they can take over. It’s a small group that is there that can transfer skills. Even if the institutions come back, they can continue to train the health staff so that we do not lose habits.”—FGD participant, service provider group Social protection: Pillar II funds three AFS activities in Guinea: EFSP, ILADP, and a WFP activity. The EFSP and WFP activities provide emergency food assistance only. ILADP’s objectives are to build human and social capacity in agricultural productivity, entrepreneurship (focused on women), food marketing, and vocational skills to increase employment opportunities and vibrancy in the agricultural market. IP KII respondents noted that there have been limited interventions for livelihood and agricultural training. It is too early to analyze sustainability of ILADP’s interventions. “In the field of agriculture, we have not benefited from trainings worthy of the name. But, by giving inputs or seeds, there were small trainings, advice to the peasants for a change of behavior.” —FGD participant 42 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SBCC: The goal of Pillar II SBCC health sector activities was to promote the adoption of improved behaviors such as handwashing, use of health services during pregnancy and childbirth, and personal hygiene. FGDs with beneficiaries (12 of 54 FGDs) said that IPC practices promoted at the height of the EVD outbreak are still being practiced. HC3 supported community leaders to facilitate community dialogues about the quality of services and perception of care, with the end goal of increasing use of health services and celebrated results by introducing a reward system (the Gold Star campaign). The ETP&SS activity promoted community ownership through community-based surveillance activities and support for the national association of EVD survivors, addressing the human dimension determinant of sustainability. KII respondents (n=4) from the administrative authorities and political parties reported facilitating peaceful elections by meeting with those campaigning to discuss the role of women and youth and national cohesion. Key informants (13 out of 28 respondents) said that social mobilization around the electoral process and conflict resolution may have long-lasting effects and USAID assistance should continue in this area. Respondents stressed the importance of being neutral, and as such in meet￾ing either all or none of the parties during an election campaign. KIIs (5 out of 28 respondents) appreciated SCG’s work in pro￾moting civic harmony and conflict resolution, especially in the Forest Region. Both SCG and CEPPS played critical roles leading up to the long-delayed municipal elections in February 2018. Institutional enhancements: CEPPS has worked with the national electoral commission (determinant: institutional support and accountability) and through local CSOs and mass media (community- and institutional-level support) has informed more than nine million Guineans about their civic and voting rights and responsibilities (determinant: human dimensions). “Yes. I participated in a training in N’Zérékoré with USAID. The contribution was to compose the exchange team, train them on the dialogue, how to make the elections, how to be interested in people and how the elections should happen. The aim was to raise awareness among young people and to consider their training in Guéckédou and N’Zérékoré. These young people, in turn, should return to their peers. The media also received training in this area.” —KII respondent ICT and institutional enhancements: The twin objectives of the Pillar II-funded ITP activities were to strengthen the MOH’s information systems and its data use. The MEASURE Evaluation Strengthening Data Availability and Use in Guinea activity improved Guinea’s HMIS, increased availability and quality of health service delivery data, and institutionalized a culture of data-driven decision making. HFG supported the MOH in the institutionalization of the human resources information system and SIAPS supported the logistics management information systems. To build an enabling environment for a strong HMIS, MEASURE Evaluation supported the establishment of a national HMIS technical working group and developed a strategic plan for implementation (determinant: policy). The activity also built the MOH’s capacity for data use through training and provision of a local server to support the transition to DHIS 2 (determinant: institutional support). “All the data from 2015 to September 2017 have been entered (in DHIS 2). We have a catalogue of 1st and 2nd order indicators and ready to start the system nationally in January 2018 with real-time data entry.”—KII respondent, MOH “Country is not yet autonomous to take care of it (DHIS 2) entirely or another activity should take over the continuation (of support).”—KII respondent, IP CASE STUDY: 2018 ELECTIONS IN GUINEA The EVD outbreak in Guinea illuminated and exacerbated the country’s weaknesses in governance. Breakdown in government systems led to extensive mistrust of government, which contributed to resistance to the response. Epidemic containment activities and health messaging campaigns reminded many Guineans of previous experiences of political and social repression (Cenciarelli et al, 2015). Ethnic minorities in the Forest Region (sometimes referred to as “Aboriginal”) and the Manding Savanna empire clashed with Conakry-centered political networks. In the months leading to the country’s February 2018 elections, Pillar II-supported governance activities worked to promote peace and reconciliation through dialogue and consensus building, strengthening the capacity and structure of political parties, expanding civic and voter education, enhancing citizen engagement and oversight of electoral systems, and strengthening the electoral administration (determinant: institutional support and accountability) through community, regional, and national dialogues (determinant: technical choices) addressing the root of community-based conflict (determinant: external threats and enabling environment). The USAID Guinea 2016 portfolio review reported that USG-assisted programs provided nearly 9.5 million people with civic and voter education in 2016, up from 2.5 million in 2015. About 5.9 million of Guinea’s 6.5 million people over the age of 18 were registered for the February 2018 elections (News Report 2018). Twenty-three percent of the 30,000 candidates were women, not far from the 30 percent quota set by the electoral commission (USAID Guinea, 2016). Pillar II activities played a critical role in raising awareness of electoral issues, especially among women and youth. The day of the municipal elections passed peacefully, though there have been incidences of post-election violence. EVALUATION FINDINGS : EQ 2– SUSTAINABILITY n 43 LIBERIA In Liberia, respondents in KIIs with three IPs and one education officer, as well as in three FGDs (one each with adult female EVD survivors, a student group, and a social club), noted that the short duration of Pillar II-funded activities limits the sustainability of the results and that it is too early to assess sustainability. It is also noteworthy that some Pillar II-funded activities are still under way. Thus, a perfect sustainability case study has not yet emerged from the Pillar II Liberia portfolio and it is premature to map Pillar II activities along a sustainability pathway. Nonetheless, activity experiences have yielded insights on some determinants. Liberia’s Pillar II activities worked with the GOL to develop crucial policies that either did not exist or needed revision (see Liberia Desk Review Report, Annex M). KIIs with two IPs and a GOL respondent highlighted that political will (determinant: human dimension), not just the mere existence of policies, is a factor of sustainability. Respondents in two youth FGDs (one student group, one group of young male EVD survivors) and nine KIIs (four with IPs, five with education officers) mentioned that the ability of the GOL to mobilize resources from different sources and via mechanisms is crucial, as is accountability at the community and institutional levels to ensure appropriate use of resources, effective targeting, collaboration, and information sharing. Key informants from FGDs with social group members and EVD survivors, two CSOs and one IP mentioned mobility, which affects access, the frequency of “exposure” (e.g., to SBCC messages), and continuity of support as factors for sustainability at the level of individual beneficiaries. This is a significant factor when implementing CT programs in urban and peri-urban settings (Radice, 2017). Social protection: There are mixed results with respect to social protection. JSI R&T’s APC/ETP&SS activity contributed to the development of Liberia’s National Ebola Survivors Care and Support Policy, which will inform future care and support efforts. EVD survivors in Bong and Nimba—two counties targeted by AFS social protection activities—mentioned that Pillar II-supported provision of food, money, and clothes met basic, immediate needs but that the benefits were not long￾lasting. Members of petty trader groups and social clubs, as well as five IP, CSO, and county AFS key informants, stated that training and skill-building (determinant: technical choices), with routine reinforcement, were essential to the sustainability of AFS efforts. While CT programs are generally implemented to meet short￾term needs in emergencies, Pillar II CT interventions in the AFS sector integrated several determinants of sustainability: ■ All AFS activities engaged the communities in which they worked. (determinant: human dimension) ■ Rather than disbursing the cash themselves, IPs involved the private sector (e.g., Lonestar for mobile money, Ecobank for direct cash), establishing a link that did not previously exist between those entities and rural beneficiaries. However, poor network connectivity in rural areas and other factors were challenges. As a result, direct cash, and not e-transfers, was the primary distribution mode for all AFS IPs (Radice, 2017). (determinant: technical choices) ■ IPs adopted measures to limit fraud, such as spot verification and complaint hotlines. (determinant: accountability) CASE STUDY: BUILDING HMIS IN GUINEA Guinea had identified the need to transition its HMIS to DHIS 2 before the EVD outbreak, but this work had not yet begun. Since a well-functioning HMIS is essential for rapid identification of disease outbreaks, USAID used Pillar II resources to fund MEASURE Evaluation, which was already working in Guinea, to build a new health information database using the DHIS 2 platform (MEASURE Evaluation 2017a). In addition to their role in disease surveillance, strong HMIS creates greater transparency and accountability by increasing access to information (MEASURE Evaluation, 2017b). MEASURE supported the development of Guinea’s Health Management Information System Strategy, an M&E plan, and a cost analysis (determinant: policy). The activity trained statistical officers, piloted and rolled out DHIS 2 in several regions, and supported the MOH’s dissemination of routinely collected data through quarterly bulletins (MEASURE Evaluation, 2017a). Systems integration and data warehousing was completed for public facilities (determinant: institutional support) and are still in progress for private health facilities. The new HMIS and the MOH’s increased capacity will improve the quality of the country’s health data and its use for planning. GOL CONTRIBUTIONS TO SUSTAINABILITY The GOL supports sustainability of social protection interventions in several ways: ■ The National Social Protection Policy and Strategy (2013) guides the country’s social protection efforts (determinant: policy); ■ The Ministry of Gender, Children and Social Protection has a Social Cash Transfer Unit (determinant: institutional support); and ■ Cash working groups facilitate coordination, information sharing, and learning (determinant: technical choices). 44 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES However, IP documents and KIIs with one IP and two CSOs, as well as five FGDs with AFS beneficiaries and EVD survivors, also mentioned concerns that would detract from sustainability, such as instances of fraud and lack of transparency in targeting that resulted in resources not always reaching the right people. For example, in an FGD with adult female EVD survivors in Voinjama, respondents alluded to cronyism in food assistance that prevented persons of greatest need receiving that assistance. Frontline worker support: Thirteen KIIs (IP, local CSOs, county key informants in the health, education and AFS sectors) and three FGD respondents (beneficiaries such as farmers, traders, and young female EVD survivors) said that health workers’ capacity had improved, which contributes to the health system’s resilience and preparedness to handle future emergencies. For instance, the MCSP/HRH activity strengthened midwifery and laboratory faculty’s teaching skills and provided equipment for skills labs in pre-service training institutions (MCSP/HRH Quarter 1 Report, FY16). However, stakeholders also underscored the need to redouble efforts in this area to improve resilience and performance in the health sector over the long term. MCP: Institutional support is the most prominent sustainability determinant for this intervention type. mHero (an ITP activity) worked with the GOL to institute improved HIS policies and guidelines. Five KIIs (three with CSOs, one with a county agriculture official, and one with a governance IP) identified the need for resources and capacities to trickle down to the lowest implementation level. PACS, a community-based health and WASH activity with a SBCC component, did this by building the capacity of general community health volunteers and CSOs to use proven health communication and social mobilization approaches, addressing some of the implementation requirements described in the GOL’s National Community Health Services Policy (2016) and Investment Plan for Building a Resilient Health System in Liberia, 2015–2021. Institutional enhancements: The Liberia Municipal Water Project, implemented in three urban municipalities by Tetra Tech, instituted mechanisms to generate community funds to maintain water infrastructure. Additionally, the activity supported the development and implementation of six action plans for water security, integrated water resource management, and water source protection. The water points exceeded their year-one targets for recovery of operating expenses through customer fees in two of the three target municipalities (Tetra Tech, 2017). Stakeholders (seven KIIs and 13 FGDs with EVD survivors, petty traders, farmers, IPs, CSO, county education officers) said that physical upgrades of HFs and WASH infrastructure improvements in schools and communities were intended to have a longer shelf life than other recovery activities. Two IPs identified a lack of accountability in the WASH sector, since no single line ministry has the legal authority for WASH coordination and oversight in Liberia. IPs have attempted to strengthen coordination mechanisms and have had some success between MOE units and NGOs and between the MOE and the Ministry of Public Works. At an institutional level, two IP key informants, one CSO key informant and two county education officers stated that transitional planning with formal mechanisms for handover to local implementers is important. The education sector also yields examples of CASE STUDY: BOLSTERING INSTITUTIONAL SUPPORT TO LEAVE A LEGACY FOR HEALTH WORKER TRAINING Liberia’s National Human Resources Policy and Plan for Health and Social Welfare 2011–2021 established a long￾term vision for strengthening Human Resources for Health (HRH). The EVD crisis, however, shone a light on HRH shortcomings. Pillar II resources are both addressing immediate HRH capacity gaps and strengthening training infrastructure (determinant: technical choices). FGD and KII respondents mentioned improvements in health worker capacity. In considering other sustainability determinants, it is apparent that requisite financing of HRH strengthening, functional accountability mechanisms, and possible review of existing policies through the lens of health-sector readiness to rapidly mobilize and deploy HRH (in the event of another crisis) are also necessary. Sources: KIIs with IP, CSO, and multiple central and district MOH officials; IP reports, National Health and Social Welfare Policy (2011–2021) CASE STUDY: COMMUNITY INVOLVEMENT FOR SUSTAINABLE FINANCING OF WASH INTERVENTIONS In Margibi and Grand Cape Mount counties, education officials are engaging parent-teacher associations to devise creative solutions for resupplying hygiene and sanitation materials such as hand sanitizer and toilet tissue in schools. (determinant: human dimension). Additionally, schools were given seed money to devise creative means of generating funds (determinant: financial resources) to sustain WASH interventions (source: two KIIs with county education officers). EVALUATION FINDINGS : EQ 2– SUSTAINABILITY n 45 how the human dimension, such as the use of parent-teacher associations, is being leveraged. Thus, both community- and institutional-level factors are relevant. ICT: mHero and mSTAR built momentum for improved ICT. KIIs (n=13) identified both poor ICT and road infrastructure as challenges to sustainability. SBCC: There is a widely held sentiment that SBCC activities effected long-term changes in knowledge and practices. Respondents (five GOL KIIs and two FGDs among social club group members and young male EVD survivors) mentioned that improvements in IPC and handwashing have been sustained. Meaningful community engagement and responsiveness to community-identified needs were identified by GOL key informants and beneficiaries as key to this success. SIERRA LEONE Social protection: Pillar II investments provided access to agricultural inputs, financial support, and markets and strengthened farmers’ technical skills, laying the groundwork for sustainability by stimulating agricultural production and productivity. “The direct beneficiaries were able to use some of the cash resources they received for short-term relief to pursue longer term sustainable activities such as farming, small business. . .” —KII respondent, IP “The program also provided participants with opportunities to learn new business skills and encouraged them to join a Village Savings and Loans Association group to ensure sustainability of their economic activities.” —KII respondent, IP There are, however, challenges to sustainability, such as insufficient staffing of agriculture of extension services. It is expected that this will be resolved in the future, as the reliance on volunteers may not be sustainable. “We also found out that the government could not maintain most of the [staffs], they sent people to early retirements and once the staff capacity was reduced, it was difficult for us to reach our farmers on a timely basis, you will see one staff would have to cover a whole chiefdom, which will make the work difficult for them to reach the people, so those were most of the setbacks we had as a ministry. Except now we are encouraging people to come as volunteers so we could be able to reach our farmers.” —KII respondent, public sector office As stated earlier in the report, CT programs are implemented in emergency situations and are not necessarily intended to have sustained results; however, linking CT beneficiaries to VSLAs and other investment opportunities were used to enhance sustainability of these programs’ results. CT programs could also contribute to the empowerment of women and youth. Institutional enhancements: The 24-month National Ebola Recovery Strategy, developed by GOSL, set the groundwork for sustainability in multiple sectors, including health (determinant: policy). The recovery strategy aimed to improve disease surveillance and retention of staff within CASE STUDY: EXAMINING DETERMINANTS OF SUSTAINABILITY IN AGRICULTURAL INTERVENTIONS Though Sierra Leone’s soil may contain huge deposits of high-value minerals, some argue that its true diamond is its vast, fertile, and often uncultivated land. Seventy (70) percent of Sierra Leone’s labor market was working in agriculture before the EVD outbreak. This percentage increased to 90 percent in rural areas (World Bank, 2014). During the crisis, however, farmers were unable to grow, harvest, or sell their crops. The price of agricultural inputs went up, fields went unharvested, buyers were reluctant to travel to rural areas, and agricultural trade across borders slowed. Once the outbreak was brought under control, an emergency funding jump started agricultural production. FFP interventions provided farmers with financial resources, access to seeds and tools, support to purchase inputs in bulk for lower costs, training in agricultural production and financial management, and rehabilitation of feeder roads to facilitate access to markets. In addition, new farming areas were brought into production. Fragile as it may be, recovery seems to be underway. The foundation for sustainability of activities in the sector has been built by: - facilitating access to seeds and agricultural inputs to restart agriculture production (determinant: technical choices); - building farmers’ technical capacity (determinant: human dimension); - strengthening farming associations (determinant: institutional support); - supporting VSLAs, which have begun to capture resources that can be invested in future agricultural activities (determinant: financial resources); and - rehabilitating feeder roads (determinant: external threats and enabling environment). Beneficiaries of CT programs have been women and vulnerable social groups, laying the groundwork for more equitable development. As illustrated, several determinants of sustainability have been introduced, but others, such as policy and accountability, were not targeted with Pillar II funding. 46 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES the health system (Government of Sierra Leone, 2015). The GOSL reviewed its community health worker policies and, in acknowledgment their importance within the health system, developed a policy to compensate and retain them. Pillar II activities supported the development of community health worker policy. Community health workers now receive a monthly compensation of Le100,000 per month plus an additional Le50,000 per month for transportation and cellular data expenses. The policy review also resulted in the establishment of the Free Health Clinic Initiative, which extends free services to EVD survivors, pregnant women, lactating mothers, and children under five years of age. Health-related activities, such as the “117” hotline established during the response to report all disease conditions and other health-related issues, continue despite the end of funding. Another illustrative example is disease surveillance (both in￾country and at cross-borders) that is implemented by MOH personnel and complemented with other donor funding. Frontline worker support: Key informants said that the skills the health workforce has acquired through training funded by Pillar II will last. Skills development begets service provider confidence and improved service quality, in turn generating client satisfaction and subsequent increased client load. “The knowledge and impacts of the trainings we have received will last. We have seen the value on the use of IPC in our work. I want to work and be safe from infections.”—Public sector health worker The APC PERHS activity trained PHUs, FMCs, and District Health Management Team (DHMT) staff on how to operate and maintain new facility-level tools and technology, including the management of new and upgraded WASH/IPC facilities (determinant: technical choices). “A lot of trainings and support have been put to strengthen IPC in the Hospital and PHUs. We carry out regular training and monitoring to support and ensure that our staff adheres to strict IPC standards at all times. IPC supplies are readily available for use. We have a focal person. A lot of resources have also been invested to provide and support WASH activities in our institutions.” —MOHS key informant “Sierra Leone is now better prepared to detect any disease outbreak and provide an effective response within 72 hours. … rapid response teams now exist and they are able to respond quickly.”—KII respondent, IP SBCC: FMCs are enforcing positive social norms, such as facili￾ty-based delivery. FMCs have developed operation and main￾tenance plans for facilities and, in some cases, gathered financial contributions from the community. FMCs have been recognized by the DHMTs, chiefdom leaders, PHU staff, and community members, which has provided them legitimacy, allowed them to fulfill their responsibility to promote quality improvements, and in turn, encouraged increased service utilization. “The improvements in infrastructure and maintenance of health facilities, combined with the health awareness raising activities conducted in the communities by the FMC members, had a positive impact on the quality and utilization of MCH services in the (our) region.”—KII respondent, IP Stakeholders’ recommendations to support sustainability of the FMCs include: ■ In FMC training materials, include guidance on how to create and sustain an FMC fund, which is critical for implementing action plans and ensuring that community health priorities are met; ■ Determine longer-term motivation for sustaining FMCs and identify how they can move forward after the activity support ends; and ■ Designate a district-level coordinating body and process to ensure continued FMC capacity building. CASE STUDY: LAYING THE FOUNDATION FOR SUSTAINABILITY IN THE HEALTH SYSTEM Pillar II investments in Sierra Leone met determinants of sustainability in five of the WHO’s health system building blocks: Leadership and governance: Development of regulations to make health facilities IPC compliant, revitalization of the Integrated Disease Surveillance and Report Technical Guidelines, the Free Health Care Initiative, and the National Community Health Workers Policy 2016-2020 (determinant: policy) Service delivery: Facility upgrades, such as improvements in WASH infrastructure, provision of IPC supplies and equipment (determinant: technical choices); community engagement for the care of EVD survivors (determinant: human dimension); strengthening of FMCs (determinant: Accountability) Health information systems: Strengthening of the HMIS (determinant: institutional support); Access to essential medicines: Support for the national supply chain management system (determinant: institutional support) Health workforce: Training in IPC and other technical skills (determinant: institutional support) The follow-up evaluation (PE2) will look for evidence of the presence of determinants of sustainability within the health financing building block. EVALUATION FINDINGS : EQ 2– SUSTAINABILITY n 47 The APC activity conducted qualitative research on the FMCs to identify actions needed to make them sustainable. Proposed interventions included refresher trainings and supportive supervision, development of trust between all actors at the community and facility levels, and creation of an FMC fund to finance facility maintenance and improvements (APC, 2017a). The KIIs identified the following ways to ensure sustainability in the health sector: ■ Continue developing HRH, including frontline workers (determinant: human dimension); ■ Grant access to WASH facilities at more service-delivery points and further promote related WASH behaviors in the population (determinant: technical choices); ■ Improve internet access to facilitate timely communication of surveillance information to the central level (determinant: institutional support); ■ Provide continuous care and support for EVD survivors (determinant: human dimension); and ■ Seek complementarity with other donors such as the WHO and CDC to reinforce sustainability of all efforts, including the operation of the expanded program for immunization and the distribution of bed nets (determinant: financial resources and institutional support). MCP: Key informants don’t believe Sierra Leone is fully prepared to address its recovery needs once Pillar II funding comes to an end, and that transition funding will still be required. The major obstacles perceived are financial in nature. “USAID’s funding has been very crucial to recovery efforts in Sierra Leone. With the recovery funding coming to an end in 2019, we only hope there will be a supplementary program to follow up. If they could come up with another community led developmental program, it will be much appreciated. We are not fully prepared, and yet, have the task to continue supporting communities, but it will be on a very low scale without their funding.”— KII respondent, IP “We want the USAID to help our government with drugs the essential ones for every three months. They should increase and the numbers of health workers and improve on the salary structure try to put volunteer nurses of pin code and put attention on the Ebola survivors and the orphan children.” —Health worker ICT: There is less information about how ICT solutions have operated in the ITP sector and what lessons have been drawn from this experience. The MOHS and the Ministry of Informa￾tion and Publication signed a memorandum of understanding to support collaboration on health communication issues and to coordinate health communication activities. There have been challenges in ICT efforts to improve communication between the district and central levels. Adequate internet connectivity and telephone coverage is lacking at the facility level. “We are facing challenges in maintaining a network commu￾nication system. An IT/ Communication system was being es￾tablished to support information and data management within the ministry improve service delivery for the recovery. The sole political decision of the then government minister terminated this promising project. At the moment we, struggle as a min￾istry with field staff all over the country to manage and share information on time.”—KII respondent Photo courtesy of the U.S. Department of Defense 48 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Evaluation Question 3: Gaps and Opportunities EQ3: Gaps and Opportunities: What gaps and opportunities arose in the course of Pillar II activities that influenced the achievement of Pillar II objectives? The evaluation team explored the contextual changes that affected Pillar II activities’ performance: which activities were most successful in adapting to change; what opportunities were seized to improve or amplify Pillar II results; and whether development issues or barriers were ignored or not adequately addressed. GUINEA Prior to the EVD outbreak in Guinea, the country suffered from weak health governance, poor health infrastructure, and poor health communications. When the outbreak occurred in 2014, those weaknesses contributed to Guineans’ misconceptions and distrust in the government and health system (Wilkinson et al, 2017). The outbreak’s impact was compounded by fear, which led people to reject use of health services. Guinea’s health system suffered disproportionate EVD infection rates among health workers. By May 2015, 0.02 percent of Guinea’s population had died due to EVD, compared with 1.45 percent of the country’s doctors, nurses, and midwives (Evans et al, 2015). Pillar II health sector activities closely aligned with the GOG’s health sector recovery strategy. IPs designed interventions to promote frontline health worker safety through IPC training, and rehabilitate HFs to acquire basic infrastructure and equipment. Better HFs also promoted use of services, as seen in the increase in outpatient clients. Key informants from IPs (4/8) noted that they had included social mobilization as a key element during the activity design in health sector and governance activities, after realizing that inadequate community engagement led to mistrust during the EVD outbreak. CSO (n=4) and service provider respondents (n=3) said that the greatest contextual change since the EVD outbreak is that people’s trust in health services and medical personnel has been gradually restored. Respondents from USAID (one FGD), GOG (three KIIs), and IPs (two) said that continuing close coordination between partners and government established during the response was a factor in recovery activities’ success. GOG (four KIIs) and IP (two KIIs) respondents mentioned the short duration of Pillar II activities as a challenge; for instance, HC3 closed after two years. The closure of the Strengthening Data Availability and Use in Guinea activity in 2017 was a missed opportunity to support the country’s continued implementation of DHIS 2 in 2018. “HSS takes a long time. It would be really ambitious that we can make a difference in two years. We are really just back to where we were pre-Ebola.”—USAID Political tensions between government and opposition parties in Guinea were on the rise in March-April 2015 over disagreements regarding the electoral calendar and the sequencing of elections. Pillar II governance activities were designed to address this gap. One identified gap was the controversy and misinterpretation surrounding the composition of the Independent National Electoral Commission and inadequate political will for electoral reforms. However, GOG KIIs (n=5), KIIs with CSO respondents (n=3), and FGDs with beneficiaries (n=2) indicated that the Pillar II governance activities found opportunities to mobilize CSOs and communities for open dialogue and communication about electoral issues, electoral reforms, and conflict resolution. As part of the activity design process in January 2015, EFSP conducted a livelihoods assessment to identify the local livelihood-related gaps in Macenta and N’Zérékoré so interventions could be tailored local livelihood practices. The activity then directed its interventions to provide local varieties of seeds and tools for agricultural production including gardening, processing (palm oil, rice, groundnut), petty trade, skilled labor, and casual labor. As the activity progressed, other gaps were found, such as poor access to capital, training, and technical support in vocational and business skills. To address these gaps, EFSP collaborated with the Department of Agriculture (DPA) to provide the technical training and counsel to activity beneficiaries (EFSP, 2017). Collaboration with the DPA ensured the quality of agricultural inputs per the Ministry of Agriculture’s standards. However, there were missed opportunities to create sustainable livelihood or employment-promotion activities. The livelihoods activity ended in December 2016. The final evaluation report found that beneficiaries were already starting to run out of stock of seeds by February 2017 (CRS, 2017). In addition, GOG respondents (2 KIIs) and beneficiary participants in FGDs (12/64 FGDs) felt that USAID could have supported local employment and EVD orphan and youth education rather than focusing primarily on health (and to some degree governance). “Need more resources to strengthen resilience beyond the health sector. Need longer project duration to achieve goals.”— GOG There is a paucity of health services for EVD survivors, as noted in the Performance section. In three KIIs with GOG and six FGDs with EVD survivors, respondents noted missed opportunities in addressing EVD-related stigma and discrimination experienced by survivors, which exist even EVALUATION FINDINGS : EQ 3 – GAPS AND OPPORTUNITIES n 49 after the outbreak is over. However, cause for optimism exists. JSI R&T’s ETP&SS activity continued their work closely collaborating with Guinea’s national network of EVD survivors, RENASEG, to scale up their sensitization program to fight stigma against survivors into 2018, which appears to have achieved measurable results in this area. LIBERIA AFS: A large majority of Liberians depend on agriculture for their livelihood. EVD struck at a critical point during the annual planting and harvesting cycle, and many HHs lost crops because people could not go to their farms or get to markets to sell or trade any surplus (FEWS NET, 2017a). Pillar II AFS activities (particularly social protection interventions) addressed the gap that emerged due to loss of income and food insecurity. CTs and food assistance yielded short￾term benefits (as discussed in the sustainability section) and, as mentioned in eight FGDs, contributed to families’ survival. African Indigenous Vegetable (AIV) interventions solved the main bottleneck of crop production by providing seeds and farm tools. Seed voucher interventions were designed to promote both income generation for farmers and increased household dietary diversity. Also, seeds were tested by the IP and the MOAg to ensure viability (SC, 2016), a crucial step for rebuilding trust in GOL services. Nonetheless, there were missed opportunities. In some instances, seeds and other agricultural inputs were received too late for planting in the first year and were saved for the next planting season (reported by participants in one FGD with farmers and in two KIIs with county agriculture officers). There is, however, evidence that Pillar II IPs adapted to implementation challenges. AFS activities attempted to distribute cash through mobile money solutions, but the limited availability of CT agents in rural areas and the agents’ limited liquidity created challenges. IPs then negotiated with two banks to develop creative solutions, including increasing incentives to cash-out agents (CaLP, 2016). Funding was insufficient to cover all families that met vulnerability criteria with all three interventions, so IPs covered more families with single interventions, rather than providing fewer HHs with all three interventions (CTs, AIV, and cash-for-work). PCI’s PEER activity introduced beneficiary feedback methods to improve transparency, accountability, and acceptability (e.g., beneficiary verification, a toll-free complaint hotline, regular post-distribution monitoring), and publicized and required staff to sign an Ethical Code of Conduct pledge (PCI, 2016). Despite this positive example, poor accountability and transparency was a recurring theme across many KIIs and FGDs. For example, an FGD with a petty traders’ group in Gbargna highlighted that there were instances when community leaders were entrusted to distribute recovery resources (e.g., food assistance) to community members in need, but instead distributed resources among themselves. A similar observation was made by a CSO key informant in Ganta. Education: WASH and IPC, which were addressed through institutional enhancements, frontline worker support and SBCC in the health and basic education thematic areas, responded to gaps in community and facility-based hygiene practices. For example, a respondent from an FGD with young male EVD survivors mentioned UNICEF EVD recovery efforts that entailed constructing toilets and hand pumps in schools. An MOE key informant in Grand Cape Mount noted that UNICEF and the MOE successfully handed over maintenance responsibilities of school WASH facilities to parent-teacher associations. Three KIIs with county education officials mentioned that Pillar II basic education activities implemented by UNICEF engaged community groups, such as parent-teacher organizations, to identify creative sustainable financing options. Key informants (11 KIIs with IPs and district/county education and health officials) and FGD participants (2 FGDs with students) highlighted that WASH activities in schools helped mainstream gender-sensitive planning within the education sector, and stimulated more open discussion of previously￾taboo topics, such as menstrual hygiene management. Governance: The CSML activity found that communities in six counties had concerns about care for children orphaned by EVD, poor school quality, and the need for memorialization for those lost in the outbreak (IREX, 2016). To address some of these concerns, the IP organized memorialization events in 18 communities, adapted to local cultural and religious traditions. Health: Though some Pillar II activities provided care and support to EVD survivors as noted in the performance section, there is a lack of those services within the health sector (IBTCI’s 2017 HHS). As mentioned prior, at the time of the data collection for this evaluation the JSI R&T/APC EVD survivor activity was just getting started. The survivor activity includes investments in mental health and stigma reduction. For the purposes of this evaluation it is too soon to expect significant results. In three KIIs and three FGDs with students and EVD survivors, respondents and participants noted a lack of interventions addressing the psychosocial aspects of recovery or EVD-related stigma and discrimination. One key informant commented that they expected EVD stigma to be highest in the most-affected counties, but in reality it was highest in less-exposed counties that had only benefited from interventions with a national scope, potentially indicating that stigma reduction training may have actually reduced stigma in the most-affected counties. Young male EVD survivors said they experienced stigma and discrimination from people such as coworkers, but not from family. Pillar II health activities saw gaps between central, county/ district, and community levels of the health system and 50 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES supported district level efforts to engage communities in health management. ICT investments in health sought to improve connections between MOH staff at the district/county level and central accountability and technical guidance mechanisms. Furthermore, 10 of the 14 innovations funded under the Fighting Ebola Grand Challenge, from improved PPE and ETUs to patient monitoring devices, aimed to improve frontline worker safety. SIERRA LEONE AFS: EVD struck Sierra Leone during the preparation of land for rice cultivation, and many farmers lost crops and seeds. Many petty traders lost their stock and had no capital with which to rebuild after the EVD crisis. As a result, food prices increased and HHs resorted to coping mechanisms, such as borrowing money to pay for food and/or skipping meals. It is estimated that, during the EVD outbreak, farmers in Sierra Leone lost about 35% of their produce between the harvesting stage and the time the rice was ready for consumption or marketing (Radice, 2017). Several key informants (one IP, one public sector official, four EVD survivors) said that the interventions implemented in the AFS sector benefited a relatively limited number of families, a small sample of those who required support, and that wider coverage could have addressed this key gap. “People in the community were saying that everything that is coming in the community is for survivors, they said they too are survivors, because they the too suffered because of the Ebola, they too were affected.” —EVD survivor “Only few of us benefited. It is still very hard for those who did not benefit to get back on track….” —CTP recipient “The livelihood intervention should have had broader coverage.” — KII respondent, IP Others said that the support was too limited in either quantity or scope. “…the money they do give us is not enough to do farming with . . .you will (still) have to pay workers.” —EVD survivor “I would want them to help me in rearing animals and treating them, so they will not die.” —EVD survivor “The allocated funding for support to the farming sector is either not available or it comes in very late when the farming season is over.” —KII respondent, IP Participation of potential beneficiaries in the planning process was noted as a missed opportunity. Three EVD survivors thought that a lack of consultation with beneficiaries may have led to inadequate support. “…they were not asking us, they will only come and give us what they want. [But] all of us our wants are different. So please, let them come and ask us what we want and [have them] support us on that.” —EVD survivor “There is a need to know first before projects are implemented. Most communities cried for fertilizers which the project did not cover so these are some of the problem it may pose for its lasting.” —EVD survivor “…the type of rice provided for the project is Erica rice. In some areas, it is doing great. It is not doing so well in others…” —EVD survivor “There is a need to know first [what communities need] before projects are implemented. Most communities cried for fertilizers, which the project did not cover so these are some of the problem it may pose for it lasting . . .” —EVD survivor For example, some KII respondents noted the lack of farming equipment as a key gap in agricultural investments (a missed opportunity), which may have been addressed through further consultation with beneficiaries. “…we have been using manpower to do our job…we don’t have tractors.” —KII respondent, farmer “…we lack the necessary equipment to do the work.” —KII respondent, farmer “We need support with post-harvest machines for use in processing and adding value to our produce. This will increase our earnings and motivate us to do more.” —KII respondent, farmer “There are also challenges faced by farmers in communities who depend on the government to help with machines for post￾harvest processes. The government is unable to provide them as an input to support increased or large-scale production by farmers.” —KII respondent, public sector Others noted an infrastructure issue in the need for better feeder roads. Five key informants representing IPs and the public sector identified the need for investing in feeder roads. “We would have loved to see some NGOs working on the recovery to take up programs on doing rural roads. Most of the activities they have worked on require good roads for access and sustainability. Health needs them for ambulance and referral, agriculture needs them for the transport and sale of farm produce.” —KII respondent, IP “…(the problem is) how to access the community, especially to convey materials and supplies. The very bad roads make the effort of partners to construct schools in that community very difficult.” —KII respondent, IP “There are still challenges with feeder roads to link farmers with the market and also the market orientation.”—KII respondent, public sector Health: When EVD hit the country in May 2014, Sierra Leone was on the path to recovery from a protracted civil war and the GOSL was engaged in rebuilding and rehabilitating its infrastructure. The EVD outbreak exposed prior weaknesses in the health care system and layered on an urgent need to improve these systems in both the short and long terms. EVALUATION FINDINGS : EQ 3 – GAPS AND OPPORTUNITIES n 51 The USAID Pillar II Ebola recovery activities in Sierra Leone were not without unique setbacks and challenges. Availability of water. The APC PERHS project faced challenges in finding sustainable water sources that would be available year-round. In some cases, geophysical survey readings indicated a very high chance of finding a sustainable water supply, but drilling yielded no water. Over time, some bore holes and hand-dug wells collapsed. At some sites, water quality tests indicated high iron content in the water after bore holes were drilled, while original water quality tests had shown no iron content (JSI R&T October 2016). Availability of staffing. Three key informants noted a missed opportunity to strengthen the training and retention of health workers: “The opportunity to train more staff was missed. Funding should be provided to train professional staff and support our national training institutions.” —KII respondent, public sector “There is the need to establish training institutions to scale up the training of professional staff for both the government and private institutions.”—KII respondent, public sector “…(we should) not forget one important aspect and put the volunteer nurses on salary.”—KII respondent, public sector However, it is worth noting that in all three countries, trained health workers may leave the HFs, bringing the facility back to square one. Further, “retention” is typically a long-term development effort, not a “recovery” one. Availability of pharmaceutical supplies. Eight KIIs mentioned remaining shortages of supplies, despite the activities working to strengthen the pharmaceutical supply chain. “The clinic does not have all necessary drugs for especially us Ebola survivors.” —EVD survivor “The health center has been built but there are no drugs. We are asking that they help us with drugs for the health center.” —CTP beneficiary “The free health care system is not effective. We are now buying our own drugs and paying for our own treatment. The drugs provided for the Ebola survivors were stolen by the medical people.” —EVD survivor “So when we go to hospital, they will check us, but at the end they will tell us that some medicines are not available unless we go and buy it at some pharmacy.” —EVD survivor “The drugs for the primary health care unit where we find ourselves are not enough to tackle our health complications.” —EVD survivor “We still face a lot of stock-outs in supplies and drugs.” —KII respondent, public sector “There have been a lot of challenges with the supply of drugs and other supplies essential for service delivery.” —KII respondent, public sector It must be noted, however, that the Pillar II activities can’t ensure the continuous supply of all essential medications and commodities in the country. This is a long-term aid effort that is usually supported by several donors and is often a challenge. It is therefore unfair to blame this on USAID’s recovery portfolio. Furthermore, at the time of this evaluation, drugs for EVD survivors hadn’t arrived in the country (distribution began in March/April 2018.). While in the AFS sector some beneficiaries had lamented the missed opportunity for consulting with them on their needs, in the health sector the opportunity to engage with the community was a strength. The APC PERHS project supported efforts to rebuild trust in the health care system by establishing the FMCs (or reactivating already existing ones) and supporting the development of the Community Engagement (CE) Strategy and Toolkit (JSI R&T, n.d.). The CE Strategy and toolkit were used to engage local communities through the FMCs in the operation and maintenance of PHUs, and in so doing, improve relationships between community leaders and HFs, and improve utilization of government health services (JSI R&T, n.d.). Respondents in all three countries and in Washington noted that the evaluation team’s process of vetting and refining the Pillar II theory of change (conducted by the evaluation team in Phase One) was useful, as they had generally focused on work and plans within their own sector. Neither the missions nor the Washington-based OUs used the USAID/AFR Pillar II M&E Results Framework (Figure 2) to create a more granular activity-specific M&E framework, as was the original intent of the AEU in USAID/AFR. The majority of support was core￾funded through USAID/GH and USAID/BFS and much of the USAID/GH funding was added on to existing projects/ activities. As such, from the perspective of funding OUs, it was more logical to follow existing M&E results frameworks rather than retrofit one to the AEU’s Results Framework. 52 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Evaluation Question 4: Management EQ4: What lessons have been learned regarding how best to use the strengths of USAID’s emergency and long-term development mechanisms in a complex emergency? This section discusses: 1) how the strengths of USAID’s emergency and long-term development mechanisms were used; 2) the perceived strengths and weaknesses in the Pillar II programming process; 3) coordination among stakeholders to maximize coverage and avoid duplication of effort; 4) lessons learned emerging from Pillar II about transitioning from emergency to development programming; and, 5) the actions needed to prepare missions and Pillar II beneficiaries for the end of Pillar II funding. This section begins with findings from interviews in Washington, and then follows with sections on each country’s experience. USAID/WASHINGTON In this section, the qualitative findings from 18 KIIs are presented. Respondents included program staff from GH CII, FFP, USAID Democracy and Governance (DG), the Global Development Lab and operations staff in the Department of State “F” Bureau, and USAID BRM and LPA. Feedback from members of the AEU and the Africa Bureau is also included here. PERCEIVED STRENGTH & WEAKNESSES Commitment: In the initial stages of Pillar II, the AEU attracted the best and brightest personnel because it was considered prestigious to be working on the EVD effort. Before the AEU, a secretariat based in the front office of USAID benefited from support from the administrator and indirectly, the National Security Council, Congress, and the Administration. Countless people worked 14-hour days, seven days a week, and trips to the missions lasted months at a time. Both Washington and Mission staff were pushing themselves to their limits. The Fighting Ebola Grand Challenge extended this sense of urgency to an array of public- and private-sector partners, achieving needed actions in “months, not years.” Yet with this massive human resource effort, there was felt a sense of camaraderie—that people were in it together. The volume of work surpassed the available people working on the problems. This was acknowledged and appreciated by all respondents. Technical foci: One member of a cross-cutting OU said that EVD money was primarily used to fund interventions that were known to be effective so they would face less risk. Conversely, a respondent familiar with the Challenge Grants said the opposite was true—that innovations aimed at emergency responses can only be tested during real emergency conditions. Technical staff said that the AEU structure imposed upon GH Bureau was illogical. The AEU Results Framework required activities to be sequentially implemented, first to restore basic health services and then, afterward, to focus on health system recovery. This seemed counterintuitive to one respondent, who stated that the AEU did not understand how global health programs are implemented. Nonetheless, the GH Bureau worked within the AEU Results Framework and designed activities to address each of the six health system building blocks, with a particular emphasis on HMIS, procurement, and HRH. Leadership: At least six or more key informants from across Washington OUs openly shared strong views about the AEU leadership and reported it to be autocratic and lacking in strategic vision. At least five respondents noted that though protocols and processes were present in the Africa Bureau, the AEU did not have clear structure or protocols to govern its work. The stated impression by these respondents was there did not appear to be any standard criteria on which to base decisions. For example, three respondents from different OUs reported a delay of more than a year in allocating governance funds to the field, because the AEU leadership was not convinced it was a priority. Economic growth was also deemed to be an afterthought, according to one respondent familiar with the early planning stages of Pillar II, who noted the original intent of the pillar was to focus on economic growth. A respondent from a technical bureau said s/he did not have a clear understanding of what the AEU required. Another respondent gave an example of an after-action review (AAR) that was never allowed to move forward, because an Office of the Inspector General audit would be done in its stead. However, this respondent did not think the audit was a replacement for an AAR. As there were no mandates in place within the AEU and the staff was made up of indirect hires, s/he could not move the AAR forward. At least two other indirect hires working in technical offices shared similar experiences. These experiences are noteworthy, especially given the agency’s interpretation of the emergency appropriation that funds could only be used for indirect hires. “We had a brutal meeting on the Hill where we had to justify every single person hired on OE [operating expenses] and explicitly make it clear they were working on Ebola.” “[The AEU leadership] had implicit control and veto power; there were no criteria for approvals.” “There were people giving directions at a tactical level who misunderstood or were not aware…this unfamiliarity and the tactical approach led to decisions and programmatic implementation that missed a bunch of marks.” Staffing: A resounding cry from the missions was for more staff to help move this large volume of money in a short time EVALUATION FINDINGS : EQ 4 – MANAGEMENT n 53 frame. Before the outbreak, the Sierra Leone Mission was staffed by one person in the Embassy. While staff increased afterward, there was a disproportionate amount of funding for the available staff (8–9 foreign service nationals and personal service contractors under one foreign service officer). Washington staff are less available to support Pillar II work now that the crisis has passed. Respondents also noted challenges in staffing due to delays in receiving funding; then, there were delays in getting approved staff on board. The Liberia Mission noted delays in adding staff due to the lack of office space. The Mission was unable to add staff because the Embassy controlled office spaces and could not add space for the Mission. Other staffing challenges mentioned were frequent turnaround of staff (mostly temporary duty staff from Washington), and difficulties associated with new staff, often indirect hires, representing the Mission’s interests without being oriented to the Mission’s mandate. Three people reported challenges finding staff qualified to manage funding in a high stress, complex setting. “One of the most important things is to figure out the right staff needed for the work we have now and not the work we had before….There are different levels of staffing and what they are prepared for. People were bidding on lower level posts for a quieter time, and not such a high profile mission. So the people you would select to lead a mission is based on what they can do for the Mission’s situation at the time of the posting [in this case, pre-Ebola]….Now they have four times the activities and funding and all the coordination that goes with it. And the Government Accountability Office audits...it’s not just the number of staff but who they are.” Many of the staff hired with EVD funds were young, indirect hires. An operations staff member expressed concern that most EVD staff were program staff, many of whom were not versed in procurement and activity management. “You got picked to work on Ebola if you were known to be hard working and would do whatever you were told to do…there wasn’t a tremendous amount of experience among them.” Monitoring progress: All Pillar II activities were expected to develop M&E plans and submit monthly, quarterly, and annual progress reports, but the reporting standards were diverse and scattered between the contractual agreements, IPs, sectors, and OUs. GHET reported that although IPs were reporting on activities on a monthly basis, the reporting did not include all indicators that fed into the GHET results framework. GHET did not begin receiving quarterly reports with reporting against all indicators until there was a switch to a quarterly reporting cycle (beginning with FY 2017 Q3). One key informant from USAID mentioned the need for a USAID/Washington web-based reporting system as a central system for the IPs to upload quarterly reports and 22. Heterogeneity in reporting and different reporting requirements especially affects information on financial obligations and expenditures. Annex J includes data on expenditures taken directly from USAID’s O&E’s spreadsheet, through IP documents, the Office of Inspector General June 2016, and information given from key informants may differ from these numbers. We have considered the O&E spreadsheet to be the most accurate source, as this is what is used to report to Congress. update progress against indicators. This would allow for USAID/AFR staff to more easily monitor activities from afar. USAID field staff would need to have access to corroborate information reported by IPs. Such a system would ideally have a georeferenced mapping component to allow for physical mapping of activities, prevalence, etc., and could create graphic reports for progress against indicators. Some multilateral IPs (for example, UNICEF and WFP) were not required to submit regular progress reports to USAID or even required to have MEL plans.22 “There was no performance progress reporting for the Ebola funds; it is outside the normal M&E work we do.” OFDA funding, incidentally, does not require that activities have M&E plans. One technical staff member was concerned about the lack of monitoring as s/he felt pressure to report out on results: “This is hugely scary that they haven’t been reporting out on indicators yet. I don’t think we spent enough time thinking through the M&E portion of the portfolio. We were just reporting narratives to Congress every month.” Intra-agency and partner coordination: The transition of power between the Ebola Secretariat, from a retired, well￾regarded foreign service officer who sat in the front office of the Agency’s administrator to a political appointee in a regional bureau, is notable. In addition to the robust inter-agency coordination that was in situ from the onset of the outbreak, recovery efforts also resulted in strong intra-agency coordination (i.e., among three missions, three OUs, a regional bureau, and the AEU). The team observed that the respective roles of the Agreement Officer Representatives (AORs) and the Mission were ill￾defined. There were distinct differences noted between the management and leadership of the Secretariat; for example, daily tasks for each OU were assigned in writing to OUs in contrast to the AEU which tasks were less prescribed and not delineated in writing. Under the Secretariat, the leadership followed up daily to assess progress against the daily tasks assigned in writing and ensured execution. While the AEU was recognized across the Agency as the coordinating body for the recovery efforts, the roles and responsibilities of the AEU were not clearly set out. Further, USAID staff on the ground did not consistently comprehend the role that the GHET played vis-à-vis the activity managers of Ebola Pillar II-funded activities; this despite efforts to make it clear that the GH Bureau was in charge of the scope and program direction of EVD activities and would provide their AORs. The varying procurement mechanisms and differing contractual arrangements across USAID OUs made it difficult to obtain 54 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES consistent information from quarterly financial reports on individual activities. However, the BRM provided IBTCI with obligation and expenditure information and the official list of Pillar-funded activities as of September 2017. Coordination between the OUs was inconsistent. According to a couple of USAID respondents, FFP reached out several times to coordinate with GH but met some resistance. From one respondent: “GH doesn’t seem to think [FFP] health programs are real, but they are real….We do everything, we do health, educations, agriculture, disaster risk mitigation…so we are a ‘jack-of-all￾trades’ and a master of none.” Other challenges encountered were differences in private￾sector approaches of Lab and CII versus public-sector approaches. An M&E Specialist in one OU asserted that there was no coordination among the various OUs regarding M&E plans. “When I first came in, I asked for all the RFs and M&E plans so we could think through and coordinate and support each other’s M&E plans. This needed to come from the AEU leadership as my OU has no authority over another OU…People are protective of their RFs; it’s a sensitive topic.” The AEU attempted to convene routine M&E Ebola Response Group meetings, but faced challenges with consistent attendance and having the right people with results at the table. IBTCI witnessed this first-hand when attending two of these meetings. Entire OUs were not present; those who were present did not openly share their M&E plans at the meetings or when requested for this study. Standard operating procedures: Three respondents discussed the problem of adhering to SOPs. One person recommended easing up on the procurement requirements by giving a waiver to the Agency during emergency responses, similar to the ease on restrictions for humanitarian assistance. Conversely, one Operations Officer argued that Pillar II was development funding and those processes should be adhered to. S/he said that there was a lot of back-and-forth between program and operations due to errors in or lack of reporting. This respondent corroborated other respondents who said the GH Bureau was called out as working independently and being difficult to approach. One operations respondent claimed that most of the errors in financial reporting were coming out of this Bureau. Another program respondent (not GH) provided feedback to back this perception—they noted that the only time their funding had issues with expenditure tracking was for the funds given to GH activities. As another example of poor compliance with reporting structures, the Congressional Notifications should have cover sheets that explain the nature of the activities the funding would be used for. If there were any activity cover sheets, IBTCI did not receive them. GUINEA Activity mapping shows that USAID provided funding to implementing mechanisms already on the ground having the resources and experience to work with target communities. In Guinea, Pillar II activities aligned with the GOG’s post-Ebola Recovery and Resilience Strategy. Three GOG respondents appreciated the flexibility of USAID programming. “The Ebola epidemic has led to greater flexibility in programming. Before the epidemic, all development activities had to focus on “peace-building”. Now, development is more focused on the needs of communities.” —Central GOG “USAID Guinea must be one of the best partners with often innovative approaches that allow attaining more than 60 percent of the objectives as compared to other partners.” — Prefectural GOG Most comments on the strengths and weaknesses of USAID programming came from IPs and GOG respondents. GOG stakeholders (n=4) stated that USAID was consultative and supportive with other non-USG development partners and government stakeholders and highlighted the supportive role of the Mission’s M&E and program-planning sections. Respondents from a subnational level CSO (n=2) stated that there had been inadequate consultation with government stakeholders, especially outside Conakry. AFS and governance IP respondents (n=2) commented that the Pillar II activities were too short in duration and GOG respondents (n=2) noted insufficient funding. “A good approach, but it’s too short… it takes 3-4 months just to get set up. We spend the first year planning.” —IP “Excellent coordination between USAID and MOH/ANSS.” — Central GOG. “USAID is consultative with partners. Local Mission hosts once￾in-three months meeting. Also, frequent technical meetings hosted by the Head of Mission. There is good sharing between sectors.” —IP Sixteen KII respondents noted that the post-EVD recovery efforts should continue and reported that the effects of most EVD recovery efforts will fade if activities are not sustained. “In preparing people towards the end of the project, a lot of efforts must be made in this direction. People still think it’s money from institutions and it’s always going to continue.” —KII respondent, CSO “The country is still fragile and is not strong enough to overcome large-scale epidemics such as Ebola.” —KII respondent, GOG Respondents suggested the following actions to prepare missions, IPs, and communities for the end of Pillar II funding: ■ A community study on the perception of populations on funding closure, with suggestions on how to prepare the communities; EVALUATION FINDINGS : EQ 4 – MANAGEMENT n 55 ■ Regional workshops (post-EVD recovery symposia); ■ USAID Washington must continue be active in financing development programs; ■ Education and awareness-building of the citizens must continue; ■ Recovery activities must be decentralized to the interior; ■ USAID offices should be opened in the regional capitals; ■ Funding for human capacity building activities should be continued; and ■ Income-generating activities for the affected communities should be implemented to enable them to support themselves after funding ends. LIBERIA In Liberia, several activities were designed and managed by the GHET in USAID/Washington’s Bureau of GH, USAID’s FFP, and the Washington-based DG team. The pre-EVD presence of AFS activities in Liberia was an important strength on which to build for both response and recovery efforts. Under the 2013 Annual Programming Statement, USAID/FFP had a non-competition memo that enabled them to engage potential partners about their objectives for the EVD response. Applications were received after the release of a new Programming Statement and an EVD-focused amendment. However, some activity applications took several months for approval, resulting in implementation delays (Radice, 2017). The Liberia Mission worked closely with the MOH to align Pillar II activities with the government’s Investment Plan for Building a Resilient Health System, Liberia 2015–2021(GOL, 2015). The principal mechanisms for maintaining coherence within USAID are the Mission’s Country Development and Cooperation Strategy (CDCS) and the universal commitment of USG-funded entities to align with and contribute to the GOL’s Agenda for Transformation and the MOH’s 2015–2021 investment plan. After an initial period of crisis, the GOL worked with the CDC to institute an Incident Management System (IMS), which the CDC adapted for public health emergencies to establish the following: 1) a defined chain of command and organizational structure, 2) effective resource management, and 3) advanced planning related to the country’s EVD response, positioning the MOH to lead and oversee efforts across partners. Ongoing coordination among partners occurred in technical working groups. The response framework provided by the IMS was a new concept to the MOH and the country as a whole; however, it now serves as a foundation to enable the country to more rapidly and effectively respond to future public health emergencies, should they arise. “USAID has multiple projects with different institutions, lots of resources, and duplicating efforts. Partners A, B, and C are doing almost the same thing. We had to restore some of the loss-- during Ebola, [it was] all hands [on deck], but after Ebola, we needed to have an agenda, and an integrated, not fragmented, approach.” —KII respondent, GOL Qualitative evaluation data reveal generally positive views on coordination. However, KIIs with three IPs, four CSOs, and one central-level GOL official noted that the commitment to GOL-led, multi-sectoral coordination was more palpable in the midst of the EVD crisis. These respondents felt that while some coordination is occurring in the recovery phase, it is less than when OFDA was present, leading to some duplication and cost-inefficiency. IP respondents reported that WASH efforts in schools and community-led total sanitation (CLTS) interventions are not coordinated. The perception is that if CLTS had been conducted in the same areas as school-based WASH interventions, for example, activities would have avoided having to construct fences around school latrines to prevent community members from using them. As reflected in KIIs with two WASH IPs, there is a broader observation that the governance and coordination of WASH-sector efforts is suboptimal. At the grassroots level, seven KII respondents from GOL health, education, and agriculture sectors, one CSO, and one IP, cited examples of multi-sectoral coordination, particularly between health and either agriculture or education. By-products have been cross-sectoral information sharing, diffusion of innovation, logistics support, and joint training. KII respondents from two IPs and two CSOs noted that USAID’s structured reporting, M&E processes, and tools established clear expectations of entities involved and an objective means of tracking progress. A USAID/Liberia-funded midterm evaluation of the Collaborative Support for Health (CSH) activity, noted that the “shifting financial and technical landscape” necessitates “continuous coordination with other partners, and frequent modification of the specific activities defined in the original SOW” (CSH, 2017). In KIIs with four IPs and two GOL stakeholders, as well as an FGD with members of a farming group, respondents raised the issue of the relative inflexibility of Pillar II support given the short duration of Pillar II activities. The Emergency Appropriation for Ebola Activities set parameters for the use of allocated Pillar II funds, restrictions about which on-the-ground stakeholders might not have been fully aware. A key lesson learned is that, even within the context of an emergency, on-the-ground validation must be done to ensure that efforts are needs-based, fit-for-purpose, and that implementers are aware of the parameters within which they must work. Respondents from one IP, four CSOs, and one county-level agriculture official raised concerns about suboptimal trickle￾down of resources to the beneficiaries with the greatest need. 56 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES In addition, one central-level GOL official, seven county GOL officials representing multiple sectors, and four respondents representing CSOs and service providers reported suboptimal transparency regarding the allocation of funding, the impact each activity is making/has made, and the level of quality with which activities have been implemented. One IP key informant noted that centralized decision-making (in Washington as opposed to at Mission level) contributes to the above. A county MOE key informant raised that USAID is inflexible in responding to on-the-ground realities; there is limited leverage to alter USAID’s plans, even when those plans are not fully aligned with local needs and implementation realities. SIERRA LEONE The President’s Delivery Team on Transition and Recovery (PDTTR) oversaw Sierra Leone’s recovery priorities. It provided technical support in monitoring and embedded its members in relevant ministries, departments, and agencies. International and national CSOs have collaborated with these groups to reach the recovery priorities. The level of collaboration with sector￾relevant ministries varied. More formal relationships existed with CSOs working with the MOHS, which required service-level agreements before the CSOs could operate (Dalan, 2016). A middle-of-the-road approach was adopted by the Ministry of Social Welfare Gender and Children’s Affairs, which relied heavily on CSOs to aid with social protection, while Ministry staff monitored the process. At the other end of the spectrum were the Ministry of Agriculture and Forestry and Food Security, where affiliated CSOs are reported to have implemented interventions without much collaboration with the Ministry at all. Numerous respondents from the GOSL (5), CSO (3), and development partners (1) stated their appreciation for the coordination that USAID provided. IPs said that USAID staff provided strong technical support and guidance in a timely manner. “Their [USAID] coordination of programs was good. They are very much supportive to partners. Their communication and information sharing with partners is good. They are able to set clear time lines for the implementation of their programs and they make sure the timelines are maintained. . .” —KII respondent, MOH “Programming of USAID Ebola Recovery resources has been an inclusive process. All the stakeholders (including the GOSL) come together and discuss and decide on priorities to be addressed within each sector.” —KII respondent, IP In 2011, Sierra Leone enacted the National Social Protection Policy, which covers CT programs. The CT program implemented with Pillar II funding was aligned with the National Standards and Policies on Social Safety Net. The National Commission for Social Action (NaCSA) is a semi￾autonomous agency in charge of the coordination and implementation of social protection interventions, and maintains a database of all CT beneficiary HHs in the country. NaCSA appeared to have operated in the same manner as the MOHS, as NaCSA worked closely with FFP activities in the identification potential CT beneficiaries, training of field staff on the beneficiary selection process, establishing the CT system, and monitoring activity implementation. “Working Groups were established, which allowed partners to work together and to coordinate their efforts. NaCSA was an active member of the WGs. Quarterly coordination meetings were held with all USAID IPs. . . NaCSA, the Anti￾Corruption Commission (ACC), and the Ministry of Social Welfare (MINSWGA) were engaged in the planning and implementation of Ebola Recovery activities, including the targeting of beneficiaries.” —KII respondent, IP “…(we) are involved directly as a government ministry by providing input and technical support to our farmers and institutions in affected communities and individuals. Seeds and fertilizers have been given to farmers by both the CSOs and our ministry. . . We are also involved with the coordination of CSOs working in our sector in the district. We collaborate with them to support activity implementation. We have collaborated with CSOs to transfer cash, seeds and inputs to farmers affected directly and in directly by the EVD in this district.” — KII respondent, public sector Productive coordination and interaction was reported between line ministries at the district level. “…after Ebola, we have created that collaboration and it is now held very strongly, between the Ministry of Agriculture and Ministry of Health.” —KII respondent, public sector On the downside, during the response to the outbreak new IPs were not always familiar with operating procedures in Sierra Leone. The DHMT had to provide guidance and coordinate the efforts of multiple IPs operating in the same geographic areas. This happened during the recovery efforts as well. Weekly coordinating meetings were organized at the district and were crucial to align and integrate activities. Despite these efforts, some respondents (n=2) pointed out that coordination was still challenging at the district level. “Most (CSO) partners have been going into communities to implement similar activities that (either) have or are being implemented in the same community by other CSOs. They ignore and bypass the national coordinating structures in the district.” —KII respondent, district-level public sector “In most cases, they [CSO] will listen to the community and follow their suggestions without cross checking with the DHMT. The concept of the activity was “Communities to choose what they want and do what they want”. In some cases, there was disagreement between the community response and the DHMT plan.” —KII respondent, district-level MOHS Respondents from the public sector and civil society (n=3) stated the importance of channeling recovery funds through EVALUATION FINDINGS : EQ 4 – MANAGEMENT n 57 CSOs for transparency reasons. “All the USAID resources were managed by partners, therefore, there was less mismanagement of USAID resources allocated to Ebola recovery program.” —KII respondent, public sector “…accountability activities have brought transparency in the management of the resources provided by external donors, made communities and schools more aware of the resources available, and reduced mismanagement of funds by school administrators.”—KII respondent, CSO Accountability is an important aspect of an activity implemen￾tation for EVD survivors. “There should be proper monitoring to ensure that the support reach us as planned. The activity should also be implemented by credible people because many support [sic] did not reach us.”—EVD survivor “We even need representation in any activity the government or the aid agencies is implementing. We should be included in decision making.”—EVD survivor Multiple respondents from the GOSL (2), IPs (4), development partners (2), and service providers (1) noted that post￾EVD recovery activities need to be continued to strengthen sustainable services. “The period of implementation of the recovery activities has been short. Ebola Recovery program should be continued after the recovery activities because the program has a good potential to demonstrate real impact.” —IP respondent A number of KII respondents said that USAID’s planning and implementation activities were sometimes slow and bureaucratic (GOSL (3)), and lacked a preparedness plan that resulted in funding not always matching needs or slowed start￾up (GOSL (1) and IP (1)). “They [USAID] were late in their intervention. I think they should not wait to support response to an emergency, but should have supported preparedness for the emergency.” —KII respondent, GOSL Six service providers were concerned about the lack of understanding of and preparation for the end of USAID￾supported activities at the community level and called upon IPs to discuss this with the communities in which they work. “The implementing agencies should meet with the beneficiaries to explain to them that the funding is ending. They also need to be provided with information on how to sustain the benefits of the activities they have benefited from. These can be done through meetings and trainings in the communities.”—KII respondent, service provider Photo courtesy of Michael Duff 58 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES CONCLUSIONS Performance Pillar II activities were designed to combine proven interventions for both short-term relief and long-term development and a limited research and development effort to produce needed innovations in six thematic areas: non-EVD essential health services, health system recovery and survivor services, education, AFS, governance, ECM, and ITP. Pillar II provided a rare opportunity to observe support from multiple sectors directed toward common outcomes. Also rare was the opportunity to pursue innovations using business approaches to research and development in the heat of a health emergency. Much of the deficit in the health sector in Guinea, Liberia, and Sierra Leone predated EVD. As such, Pillar II health sector activities were heavily concentrated on increasing basic capacity by strengthening multiple components of the health system and restoring basic health services. The aspects of the health system that were given the greatest focus were health worker capacity, MCP (including supply chain management), health facility infrastructure, health information systems, community engagement, and SBCC. Pillar II health activities have done more than merely prevent the loss of development gains; they have helped to elevate health behavior and inspired substantial gains in health-seeking behaviors. With the Pillar II investments in place, basic services are in place and IPC practices are institutionalized. The health promotion activities and community engagement have restored some trust in health services and health workers, and encouraged people to seek health services. These achievements cannot be attributed to one intervention. Rather, it was the combining of interventions addressing systems and supply-side issues with activities that stimulated demand that has led to positive health outcomes at the population level. Pillar II supply-side inputs involving institutional enhancement, frontline worker support, MCP, and ICT contributed to recovery for most primary health care services. It is unclear whether Pillar II efforts to increase health worker capacity are translating into improved quality of care. There is an unmet need for care and support services for EVD survivors and affected families, and stigma against survivors and their families persists. The impact of HMIS and supply-chain management strengthening activities is slower to be seen at the lowest implementation level. Even when powerful innovations such as mHero and the EPIC and CommCare platforms are introduced in the MOH, capacity development to utilize them takes considerable time to bring to scale. Improvements to some of the health system building blocks—for example, access to essential medicines—continue to be slow and timeline challenges still exist. AFS and health service activities mutually supported each other. CT programs allowed beneficiaries to purchase food and other essentials for health, and helped infuse money back into the markets. Assistance to market traders ensured that there were products to purchase. In locations where Pillar II AFS activities were implemented, standard measures of household food security improved considerably in a relatively short period of time and helped many to avert hunger and death. However, the extent to which nutritional status has improved is unknown. It is unclear whether agricultural investments have improved overall resilience. Stakeholders said that cash and food distributed by Pillar II implementers was essential in meeting short-term needs. Livelihood training and support are locally identified priorities for building long-term resilience and development. ECM and governance activities were few, in comparison to other thematic areas. Their funding was delayed and their achievements are more difficult to correlate directly to health and development outcomes at the population level. Pillar II investments have strengthened government agencies, CSOs, and the media. How much of that improved capacity has translated into increased trust is unclear. Despite noteworthy achievements in social accountability, functional accountability mechanisms between health providers and communities such as FMCs are nascent. Standards of living have improved somewhat, but not uniformly across the priority areas. Pillar II social protection activities are highly valued by stakeholders. Households in all three countries are largely reliant on informal sources of social assistance, making social capital, social cohesion, and VSLAs especially relevant for resilience to future shocks. ITP activities, though a small part of each country’s Pillar II funding allocation, were higher in number, implemented by multiple IPs, and common to all three countries. ICT-based interventions are a natural nexus of innovation, technology, and public-private partnerships. However, given local challenges SUMMARY CONCLUSIONS n 59 (e.g., cell phone coverage, network connectivity) and the time requirements to roll out ICT interventions, the trickle-down effects of those interventions are not widely documented. Women were prioritized in the food security, health, and governance programs. Several activities required gender balance in management committees, engaged female journalists to promote social accountability, and deliberately targeted women and/or female-headed HHs in social protection activities. Activities have also taken steps toward mobilizing women in the electoral process. Yet there is only limited evidence that this gender-specific programming addressed underlying reasons for women’s vulnerability or improved gender equity. Overall, Pillar II recovery activities were implemented over a short period, averaging 30 months or less. The actual implementation period was often shorter than the award period, due to time needed to introduce and start up activities. Nonetheless, activities for which the team has data report achieving their targeted outputs. Sustainability To assess interventions’ sustainability, IBTCI created a conceptual framework to define determinants of sustainability and their influence on the various intervention types. Elements of sustainability vis-à-vis these determinants were present in various thematic areas, including health services and health systems, AFS, governance, and ITP. Health sector activities have bolstered key building blocks of a functional health system (e.g., health workforce, health information), strengthened service quality, and restored trust in the health system. The CII team supported Grand Challenge innovators with market shaping and innovative financing strategies such that nearly half of the innovations are in use or available to purchase. Institutional enhancements in the health sector, such as collaborative development of guidelines and SOPs, were designed to have sustained effects, which the evaluation team observed. Activities supported a number of important policy initiatives in the areas of health information management, human resource management, infrastructure and financial management, and health communications. The activities worked closely with government counterparts to make that guidance available and often funded the development of manuals and job aids to roll out policies and incorporate them into training. Pillar II activities supported small steps towards transformative change, mostly notable in introducing or revitalizing frameworks and groups for communities to participate in decision making and creating government accountability. HC3 supported the MOHs to develop and disseminate the Healthy Lives logo, to signal that information and HFs are trustworthy. Misinformation was a key source of fear and conflict in the EVD crisis, and encouraging citizens to look for a government stamp of approval will help avert similar problems in the future. Sustainable outcomes are more likely to come from the existing longer-term activities that were expanded to address Pillar II objectives, and those that may continue after 2019. HFG’s mentorship of the National Assembly of Guinea’s Health Committee led to a substantial increase in the government health budget, which could be a transformative intervention. Community engagement was an important component of health services activities. Other kinds of mechanisms include the facility management committees or community-led total sanitation. Having such structures in place could potentially contribute to resilience, as they can provide pooled resources, know-how, and support. As noted earlier, the AFS activities addressed HHs’ immediate basic needs and the loss of agricultural productivity and markets. They also restored financial and social safety nets— helping communities to adapt—by restocking and revitalizing markets, linking people with sources of financial and social support such as VSLAs, and providing inputs to restart agricultural activity. Financial and nutritional training helped HHs establish a financial foundation and more robust nutritional baseline, enabling them to weather future shocks. All of these interventions contribute to resilience and have the potential to endure beyond Pillar II funding. The Grand Challenge innovations have spurred partnerships between innovators and private-sector entities to bring effective innovations to the global marketplace. For example, an Ebola Grand Challenge team from Johns Hopkins University (JHU) designed safer, simpler, and more comfortable PPE for health care workers. DuPont, one of the largest PPE manufacturers, saw the potential in the new PPE designs and signed a licensing agreement with JHU. This partnership between JHU and DuPont will ensure that the new suits are both designed to meet basic cost and manufacturability requirements and are capable of reaching the global market. Gaps and Opportunities The influx of funding for recovery activities has created opportunities to build stronger systems than existed prior to the outbreak. The Pillar II activities supported each country’s respective prosperity agendas, by: 1) improving and increasing 60 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES the use of health services by strengthening RMNCH services, facility infrastructure, and health worker trainings; 2) improving WASH infrastructure in HFs and schools and expanding community access to safe water and improved sanitation; 3) bolstering small-scale agricultural production; and finally, 4) strengthening information systems for better planning and targeting of resources. The experience with mobile money in the CT programs has shown that several obstacles must be overcome before e-payment works smoothly in rural areas. In the absence of an e-payment system, a well-deployed and monitored offline cash transfer process can adequately replace the technology-based cash transfers. However, the use of mobile technology would have increased the speed of transaction, transparency, and security and would have supported the business and employment markets. Cash-for-work activities may have considerable potential for continuation and scale￾up, and the tools and knowledge already shared could be leveraged for longer-term community-led maintenance of roads. Use of mobile tools and platforms for beneficiary registration, verification, and performance monitoring had many benefits, such as increasing the speed, flexibility, and reach of communications, and could be expanded as cellular connectivity improves. Management The short period of implementation of Pillar II activities and delays in start-up were challenging. For instance, though activities’ intention was to cover food insecurity during the lean season, CTs did not reach beneficiaries until after the most challenging time, due to procurement and contracting delays. The evaluation team observed that circumventing USAID rules and regulations in an effort to speed delivery (i.e., rushing through clearance processes) did not accelerate progress but rather stalled it because of the time consumed in correcting errors. It is likely that should the Mission and Washington systems and actors be able to work together in real time, they would increase speed without compromising USAID’s principles of sound procurement. For example, in Liberia, under the 2013 Annual Programming Statement, USAID/FFP had a non-competition memo in place that enabled them to engage potential partners about their objectives for the EVD response. USAID programming is constrained in many ways by funding earmarks and conditions; it is understandable that program management should be organized accordingly. However, the diversity of funding units complicated the planning and implementation of Pillar II activities. KII data suggest that some USAID Mission staff were not equipped with the skills to implement in an emergency recovery situation. Moreover, due to a lack of time and human resources, there were staff new to the agency and who had not yet been trained or benefited from peer-to-peer mentoring that is normally offered to new staff. The esprit de corps among the agency personnel, while intangible, was felt to be palpable as key informants told their stories. The agency’s strength as a team might have been an essential contributor to the successes realized. Photo courtesy of the U.S. Department of Defense RECOMMENDATIONS n 61 RECOMMENDATIONS The following recommendations are addressed to USAID/ Washington, USAID/Guinea, Liberia, and Sierra Leone. In any future outbreaks of similar nature and country context, recovery activities should lay the foundation for short-, medium- and long-term recovery while the medical emergency response is still active. The following recommendations are ranked by importance, not sector. 1. For USAID’s future emergency coordinating bodies, such as the AEU, the Agency should consider a base in the Front Office.Careful consideration should be given to identify the management approach that best serves the needs, attributes, and characteristics needed for successful leadership. This coordinating team should be comprised of senior staff experienced in humanitarian relief and development. The rules of engagement for the team, specific roles, and the extent of their authority should be clearly delineated. The team should be multi-sectoral, capable of drawing on the full range of the agency’s expertise in development. This central team should have the authority to solicit cooperation from OUs across the agency, drawing technical resources into an integrated emergency response. 2. The team should be charged with invoking special procurement and staffing regulations, redesigned by the agency for emergencies, including explicit criteria that specify the authorities of the Mission and Washington OUs for clearances and decision-making processes. OUs should identify ways to expedite access to funding while maintaining accountability and safeguarding the use of USG funds. Procedures need to be in place to avoid use of recovery funds without constraints, misappropriation of funds or mismanagement at the IP level, and inefficient use of funds. The Agency might consider waivers for routine activity design and implementation during emergency situations to mirror OFDA’s procedures. Waived actions (e.g., instituting an M&E plan) should be addressed within one to two years of implementation. 3. An emergency coordinating body or task force should oversee the design of a results framework to be used across the Agency and an M&E and reporting sub-team, which would hold the individual frameworks and M&E plans from each OU. This sub-team should have the authority to ensure results are reported out in a systematic fashion as consistently as possible with the OUs’ results frameworks— but also complies with the overarching results framework of the emergency response. This sub-team would be comprised of M&E experts in the respective sectors, reporting to their respective head of the OU. The sub-team would be then able to analyze data from across OUs and ensure IPs contribute to them. 4. USAID should create modules for SOPs to be available for immediate use in future emergencies. For example, “SOP: How to build a task force” might include guidance on 1) staffing up, including the necessary qualifications and locations of the team members, 2) identifying the authorities and rules of engagement and decision-making, 3) clearance processes, 4) how and where documents should be obtained and retained, 5) timing and protocols for after-action reviews, and 6) required communication and dissemination plans. These protocols would include variations when applied to short-, medium-, and/or long-term phases of recovery. These modules would be specific to the recovery activity phases as delineated in recommendation 2, above. 5. USAID should consider designing recovery activities by short-, medium-, and long-term phases of the recovery and adapting their procurement rules, strategy, and activities accordingly. For example, CT programs are short-term activities that need to be launched rapidly, while health systems strengthening activities may be ramped up more gradually. Short-term activities would look more like a humanitarian response and abide by more flexible regulations. Medium￾term activities would be a hybrid between humanitarian and development activities. Long-term activities should be designed and built for sustainability, with an exit strategy and M&E plan. 6. USAID should judiciously support with non-emergency funds limited follow-on activities at the national and district levels in each country, in order to prepare government agencies and communities for the transition to the end of Pillar II funding. Each country will struggle to sustain the gains and improvements this funding made possible. These struggles could devolve into partisan conflict and resurgence of mistrust in the government if not handled proactively. Communication and dialog activities, using the skills built during the response and recovery, can help communities understand that the time limit was a condition of the EVD response funding, that the resources have been used as planned, and that they have produced benefits for the people (as well as engaged stakeholders) in planning ways to sustain and extend activities as necessary. 7. In addition, USAID should establish a steering committee that convenes regularly (semi-annually) to develop a hand￾over/sustainability plan in partnership with the respective governments. 62 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 8. Every USAID Mission should review and reinforce its disaster contingency plans yearly in an appreciative, forward-looking manner. This may be done internally at first, but afterward in a safe, uncritical context, to gain input from other organizations. Missions’ disaster teams should include experts in all sector areas and should be aware of and prepared to work jointly with the agency’s emergency coordinating body (per recommendation 1). For example, in addition to the six sector areas mobilized for the EVD response, disaster teams could also include members with expertise in the environment, fisheries, and natural resource management. The selected areas must, of course, be guided by each country’s context. 9. USAID should propose a system to Congress that would allow the agency to rapidly respond in emergency situations during the period that it examines the need and decides on the resource envelope for the overall response. Further, USAID should design interventions that support local governments to develop their own contingency plans and funds, including identifying their own emergency management teams. 10. USAID should support establishment of a regional integrated disease surveillance network in West Africa and the continent. Efficiencies of scale and cross-case learning would benefit from a regional approach, and would enable Guinea, Liberia, and Sierra Leone to continue to solidify their health crisis response systems after Pillar II resources expire. 11. While it is understandable that M&E and reporting might get set aside during a health emergency, de-prioritizing M&E is more a result of staffing shortages than of difficulty in establishing M&E basics (clear objectives, program impact pathways/log frame, measures of success). Both USAID and IPs must include qualified M&E staff in all staffing plans, during every phase of the activity lifecycle. Their functions drive sound and rapid evidence-based decision-making in all aspects of development and emergency programming. USAID M&E staff should work closely with the IP M&E staff from the onset of an activity to agree upon indicators that can be realistically collected in a given location (e.g., to define the reliability of a country’s HMIS data before and during a crisis, to answer the “so what?” question?). USAID/Washington should operationalize a web-based reporting system as a one-stop￾shop for IPs to upload quarterly reports and update quarterly progress against indicators. 12. Even in emergency situations, IPs must establish early and accurate communication with communities. For example, it is quite possible that the extended impact EVD had on rural agricultural markets might have been reduced had the population been informed earlier that it was perfectly safe to buy and consume food produced in areas and by HHs affected by EVD. 13. Furthermore, it is recommended that IPs continue to invest in trust-building activities with government and civil society. Sensitivity to politics pervades civil society and government in the region, especially around the time of elections. IPs should improve and expand the depth and frequency of contacts between CSOs and community leaders. USAID and IPs should take every opportunity to highlight the value and mandate of government officials’ willingness to listen and respond to community needs. 14. At both the Mission and Washington levels, OUs should deepen the dialog around the intersection of governance, management, and health, to ensure the agency is maximizing synergies in these areas. Discussion Nexus between humanitarian response and development: Timing is everything. The four months between March and August 2014 was enough to enable EVD to spread throughout West Africa, and for rumors and fears to gain traction. These fears had a profound impact on the individual and organizational behaviors that constituted the main second￾order impacts of the EVD outbreak. Delays in release of funding for Pillar II activities, very short by normal development standards, led in some cases to major inefficiencies during implementation. While IBTCI has some ideas for smoothing the transition from an emergency response to the development context, there are sundry intervening variables with which the team is not sufficiently familiar in order to form concrete recommendations at this juncture. Rather, this is viewed as an internal discussion among USAID’s OUs, including Missions, to design approaches to expedite funding access during a disaster. At the same time, this approach should maintain accountability and necessary minimum due process to protect the appropriate use of USG funds. Procedures would need to be in place to avoid funding misappropriations, the use of funds without constraint, and/or mismanagement at the IP level, and the inefficient use of funds (or funding interventions that are remote to mitigating the second order effects). OUs might consider waivers for some procurement regulations and activity design and implementation during emergency situations, mirroring OFDA’s procedures, so that procurements could move through the bureaucracy more quickly. A clear strategy and results framework for the recovery effort writ large is clearly needed and feasible, as we have just witnessed through the AEU. However, does each OU need a distinct RF to manage the funds as the GHET endeavored during this past crisis? A time frame for rolling out the recovery effort could be done in a cascade-like fashion (in phases, as recommended RECOMMENDATIONS n 63 earlier) and could also take into consideration procurement regulation waivers and leniencies in the ADS guidance (e.g., all new activities for the short term might not need a RF or to institute an M&E Plan). Stigma against EVD survivors still persists. Stigma against EVD survivors (accentuated by the continued medical follow￾up), orphans, and dependents has slowed recovery for them, relative to rest of the population. More data are needed to identify the sociocultural context underlying the stigma, determine the survivors’ needs, and realize which influencers or change agents are best poised to support survivors. These findings would help tailor the activities accordingly. The evaluation team will follow up on stigma against EVD survivors during the second performance evaluation (PE2), after the Ebola survivor activities will have ended. Gender. Women were prioritized and targeted in the food security, health, and governance programs. However, to date there is only limited evidence of gender-specific programming that addressed women’s vulnerability and achieved substantial results in gender equity. What is not yet clear is whether Ebola Pillar II efforts did all that was possible to address gender, or if the larger issue was inadequate measurement of gender indicators. To best ensure some level of gender analysis is taken into consideration during the design phase of future outbreaks, perhaps a gender advisor could be a member of the central Secretariat. Similarly, IPs might be requested to have a qualified gender expert assigned as a point of contact, to provide support to their activities. During the immediate throes of the recovery period, a gender analysis might need to be abbreviated, but having clearly identified experts who are responsible for monitoring gender issues might go a long way toward ensuring equity. At a minimum, responses that are deemed more developmental than emergency in nature should require that some or all of the nine standard gender indicators are incorporated into their M&E Plans. n 64 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES REFERENCES 1. ACDI/VOCA 2016. Final Evaluation Report. Sustainable Nutrition and Agricultural Promotion (SNAP) Program. 2. ACDI/VOCA 2018. Sierra Leone Emergency Food Security Program. Supporting Vulnerable Households Recover from the Ebola Crisis. Available from: http://www.acdivoca.org/ projects/sustainable-nutrition-and-agriculture-promotion￾snap-program-2 (Accessed January 31, 2018). 3. Advancing Partners and Communities (APC) 2017a. APC Post Ebola Health Services Strengthening Sierra Leone Quarterly Report, Quarter 3. John Snow Inc., July 2017. 4. Advancing Partners and Communities (APC) 2017b. ETP&SS Monitoring and Evaluation Strategy Liberia: October 2016, Updated July 2017. John Snow, Inc. Research & Training Institute, July 2017. 5. African Development Bank 2018. Guinea Economic Outlook. Available from: https://www.afdb.org/en/countries/ west-africa/guinea/guinea-economic-outlook/ (Accessed February 13, 2018). 6. Brolin Ribacke KJ, Saulnier DD, Eriksson A and von Schreeb J 2016. Effects of the West Africa Ebola Virus Disease on Health-Care Utilization – A Systematic Review. Front. Public Health 4:222. 7. Cash Learning Partnership 2016. Learning event report. Cash Learning Partnership (CaLP). Liberia. 8. Catholic Relief Services (CRS) 2016a. Project Overview. Sierra Leone Country Program. Cash Based Food Security Assistance for Ebola Virus (EVD)-Affected Sierra Leoneans in Kenema District. 9. CRS. 2016b. EFSP Post Distribution Monitoring Survey. EFSP Cash Assistance to EVD-Affected Sierra Leoneans 2015–2017. 10. CRS 2017. Emergency Access to Food for EVD Affected Guineans (EFSP) Final Program Report - March 2017. 11. Cenciarelli O, Pietrpaoli S, Malizia A, Carestia M, D’Amico F, et al 2015. Ebola Virus Disease 2013-2014 Outbreak in West Africa: An Analysis of the Epidemic Spread and Response. International J of Microbiology. 2015; Article ID 769121, 12 pages. 12. CEPPS, 2017. Quarterly Report: January 1, 2017-March 31, 2017. Quarterly Report 4, 2017: July 1, 2017–September 30, 2017. 13. Dalan Development Consultants 2016. GOSL, 6–9 Month Early Recovery Priorities. Independent Evaluation and Assessment Report. http://docs.wixstatic.com/ ugd/767c32_00256f549ff943b8b7dfcbe7adfe60e1.pdf (Accessed March 30, 2018). 14. Evans David K, Goldstein M, Popova A 2015. “Health-care worker mortality and the legacy of the Ebola epidemic.” The Lancet Global Health 3 (8): e439-e440. http://www.thelancet. com/journals/langlo/article/PIIS2214-109X(15)00065-0/ fulltext. 15. FEWS NET 2017a. Assessment of Chronic Food Insecurity in Liberia. Famine Early Warning Systems Network (FEWS Net III) TO 3 and 4. Quarterly Performance Report. January - March 2017. Washington, DC: FEWS NET. 16. FHI360 2017. Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 3 (4/1/2017-6/30/2017). 17. Global Health Ebola Team, 2016: Guinea TDY Debrief. Conakry, July 14, 2016. 18. Government of Guinea 2015a. Government of the Republic of Guinea Summary note June 2015, Guinea: Post-Ebola Socioeconomic Recovery and Resilience Strategy 2015– 2017. 19. Government of Sierra Leone 2015. National Ebola Recovery Strategy for Sierra Leone. Available at: https://reliefweb. int/sites/reliefweb.int/files/resources/sierra_leone_ebola_ strategy_030715.pdf. 20. Guinea 2012 DHS. Available at: https://dhsprogram.com/ pubs/pdf/FR280/FR280.pdf. 21. Hart, L, Street, D, & Kulatilaka, H 2017. Building capacity for resilient health systems. Lessons learned from Sierra Leone, Guinea and Liberia in the time of Ebola. Washington, DC. 22. International Development Research Centre 2017. Inclusive Development and Research Consultancy (IDRC). Evaluation Report. Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone. October 2017. 23. IntraHealth 2018. Retrieved from website: https://www. intrahealth.org/news/intrahealth-receives-grand-challenge￾award-for-ebola-response (Accessed March 19, 2018). 24. IREX 2016. Trusted Voices in Challenging Times: Civil Society and Media Leadership (CSML) Program. Final Report. Washington DC. 25. Jhpiego 2017. MCSP/RHS Quarterly Report July-Sept 2017. 26. John Snow, Inc. (JSI) 2017. The Post-Ebola Recovery of Health Services. Program Overview September 2017. Advancing Partners & Communities. Retrieved from: https://www.advancingpartners.org/sites/default/files/ sites/default/files/resources/sl_infogrphic_endline_web_ tagged_1_0.pdf (Accessed February 13, 2018). REFERENCES n 65 27. JSI 2018. Ebola Transmission Prevention and Survivor Services. Retrieved from: https://www.advancingpartners. org/sites/default/files/sites/default/files/resources/apc_ guinea_one_pager_a4_web.pdf (Accessed February 13, 2018). 28. Johns Hopkins Center for Communication Programs 2017. HC3 Guinea Country Program Research Report. Findings from an Endline Quantitative Phone Survey. Baltimore. November 2017. 29. LFI 2017. Preparation of 2017 budget including hypotheses and objectives, execution plan, and key figures and characteristic ratios for the 2017 financial law project (2016, undated). 30. Liberia 2013 DHS. Available at: https://dhsprogram.com/ pubs/pdf/FR291/FR291.pdf 31. Mbevi Lydia 2016. Gender Impact Assessment of Cash Transfers Support to Vulnerable Households in Sierra Leone. ACDI/VOCA. November 2016. 32. MCSP/HRH Quarter 1 Report, FY 2016. 33. MEASURE Evaluation 2017a. Building Capacity for Resilient Health Systems Lessons Learned from Sierra Leone, Guinea, and Liberia in the Time of Ebola, March 2017. 34. MEASURE Evaluation 2017b. Guinea Phase IV M&E Plan, January–December 20. 35. Mercy Corps 2017. Economic Recovery from Ebola for Liberia (EREL) Program. Funded by USAID/Food for Peace (FFP). Fiscal Year (FY) 2017 Quarter 1 Report October 1 - December 31, 2016. 36. Ministry of Health (MOH), Guinea 2017a. “Presentation du projet de budget 2018” Presentation of budget project 2018 (dated 11/20/2017). 37. MOH Guinea 2017b “2017-343 Portant promulgation de la loi 059-AN du 12 decembre 2018”: Upholding declaration of the law 059-AN of December 12, 2017 (dated 12/29/2017). 38. News Report 2018. Retrieved from: https://m.news24. com/Africa/News/guinea-holds-first-local-elections-since￾military-rule-20180204 (Accessed March 19, 2018). 39. Office of the Inspector General (OIG) U.S. Agency for International Development (USAID). (January 24, 2018). Lessons From USAID’s Ebola Response Highlight the Need for a Public Health Emergency Policy Framework. Audit Report 9-000-18-001-P. 40. Parpia AS et al 2016. Effects of Response to 2014–2015 Ebola Outbreak on Deaths from Malaria, HIV/AIDS, and Tuberculosis, West Africa. Emerg Infect Dis. 2016;22(3):433- 441. 41. Project Concern International 2016. Emergency Food Security and Ebola Response Program. Liberia. Protect and Empower for Ebola Resilience (PEER). Final Report. 42. Radice WH 2017. Cash Transfers for Food Security in Epidemics: A Review of the USAID Food for Peace Response to the Ebola Crisis in Liberia and Sierra Leone. The Cash Learning Partnership. http://www.cashlearning.org/downloads/calp￾ebola-documentation-report-web.pdf. 43. SFG 2017. Search for Common Ground 2017. Rebuilding Together: Community-Driven Reconciliation and Enhanced Communication in Guinea Forestière. Final report. June 30, 2017. 44. Save the Children (SC) 2016. Emergency Food Assistance for Ebola Affected Families in Liberia - Margibi and Bong Counties. Fiscal Year 2016 Annual Results Report. October 1, 2015 – September 30, 2016. 45. Sierra Leone 2013 DHS. Available at: https://dhsprogram. com/publications/publication-FR297-DHS-Final-Reports.cfm 46. Steinhubl SR, 2015. Steinhubl SR, Marriott MP, Wegerich SW. Remote sensing of vital signs: a wearable, wireless ‘’band-aid’’ sensor with personalized analytics for improved Ebola patient care and worker safety. Glob Health Sci Pract. 2015;3(3):516-519. Available at: http://dx.doi.org/10.9745/ GHSP-D-15-00189 (Accessed January 31, 2018). 47. TetraTech 2017. Liberia Municipal Water Project (LMWP) II Combined Year 1 and Quarterly Report 4 (submitted October 31, 2017; revised November 20, 2017). 48. UNDP Water Governance Facility/UNICEF (2015) “WASH and Accountability: Explaining the Concept” Accountability for Sustainability Partnership: UNDP Water Governance Facility at SIWI and UNICEF. Stockholm and New York. Available from http://www.watergovernance.org/. 49. UNICEF 2017. ECRL Fifth Progress Report Submitted to USAID. June 2017. 50. UNICEF Liberia 2015. Contribution Summary: RUTF in￾kind support to Liberia. Retrieved from Monrovia, Liberia. 51. USAID 2014. USAID Development Innovation Accelerator Broad Agency Announcement for Fighting Ebola: A Grand Challenge for Development. November 2014. http://pdf. usaid.gov/pdf_docs/PBAAB273.pdf. 52. USAID 2015. Ebola Response, Recovery and Resilience in West Africa, Call for Partnership Concept Papers, Global Development Alliance (GDA) Annual Program Statement USAID 2015. 53. USAID 2016. West Africa - Ebola Outbreak Fact Sheet #12, September 30, 2016. 54. USAID/Liberia Collaborative Support for Health (CSH) 2017. Liberia Strategic Analysis Midterm performance evaluation report. 55. USAID November 2017. Learning Conference PowerPoint presentation, Freetown, November 2017. 56. USAID PowerPoint Presentation October 2017. Ebola 66 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Programs Team – Africa Bureau TDY Debrief + Strategic Program Review. October 3, 2017. 57. USAID Guinea 2016. Year-end Portfolio Reviews: Health. Conakry, November 22, 2016. 58. USAID Office of Inspector General 2016b. Quarterly Progress Report on U.S. Government International Ebola Response and Preparedness for Quarter 3 of FY2016. 59. Wilkinson A et al 2017. Comparison of social resistance to Ebola response in Sierra Leone and Guinea suggests explanations lie in political configurations not culture. Crit Public Health. 2017 Jan 1;27(1):14-27. 60. World Bank 2014. The World Bank Group. The economic impact of the 2014 Ebola Epidemic: Short and Medium Term Estimates for West Africa. October 7, 2014. 61. World Bank 2016. 2014 – 2016 West Africa Ebola Crisis: Impact Update. May 10, 2016. Available at: http://pubdocs. worldbank.org/en/297531463677588074/Ebola-Economic￾Impact-and-Lessons-Paper-short-version.pdf. 62. World Bank 2017.The World Bank in Guinea: Overview. October, 2017. http://www.worldbank.org/en/country/ guinea/overview. 63. World Health Organization 2016. Ebola Situation Report, WHO, 30 March 2016. At the request of the United States Agency for International Development (USAID), this publication was prepared independently by International Business and Technical Consultants, Inc. (IBTCI). © 2014 Peter Kilmarx, Courtesy of Photoshare ANNEXES FINAL REPORT ANNEX DOCUMENT PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES March 2019 Performance Evaluation of USAID Ebola Pillar II Activities: ANNEX DOCUMENT USAID CONTRACT # AID-OAA￾I-15-00022 Task Order # AID-OAA￾TO-16-00040 Authors: Zhuzhi Moore, Annette Bongiovanni, Swati Sadaphal, Orlando Hernandez, Donna Espeut, Carol Levin, and Michael Toole Submitted December 31, 2018 Updated March 5, 2019 Prepared for: United States Agency for International Development Ronald Reagan Building and International Trade Center 1300 Pennsylvania Ave NW Washington, DC 20004 Prepared by: International Business & Technical Consultants, Inc. (IBTCI) 8618 Westwood Center Drive Suite 400 Vienna, VA 22182 USA Tel: +1 (703) 749-0100 Citation: Moore Z et al. “Performance Evaluation of USAID Ebola Pillar II Activities: Final Report.” Evaluation Report to USAID, March 2019, International Business & Technical Consultants, Inc., Vienna, VA. Project Description: The Ebola Pillar II, Monitoring, Evaluation and Learning (MEL) activity is a three-year USAID-funded contract addressing USAID-coordinated efforts in mitigating the second-order impacts of the Ebola virus outbreak in Guinea, Liberia, and Sierra Leone. The activity focuses on four main components: evaluation, routine monitoring, data quality assurance, and improved knowledge management and learning. The activity is led and managed by International Business and Technical Consultants Inc. (IBTCI), with partners StatView International in Guinea, Global Research Insights, LTD (GRI) in Sierra Leone and Liberia, and Opinion Research Business (ORB) International in all three countries. Cover Photo: A girl smiles among vendors at a roadside Ebola checkpoint in Port Loko, Sierra Leone. © 2014 Peter Kilmarx, Courtesy of Photoshare. PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX DOCUMENT March 2019 This publication is made possible by the generous support of the American people through the United States Agency for International Development (USAID) under the terms of contract AID-OAA-I-15-00022/AID-OAA-TO-16-00040 managed by International Business and Technical Consultants Inc. (IBTCI) and does not necessarily reflect the views of USAID or the United States Government. ACKNOWLEDGMENTS The authors would like to acknowledge the diligent involvement of the full technical team: Richard Columbia, Barbara de Zalduondo, Salima Mutima, Elisabeth Nolan, Naomi Rutenberg, Suzanne Essama-Bibi, Andrew Reuter, Edward Allan, Robert Grossman-Vermaas, Kapil Ahmed, Roger Emmanuel Millimono, Richard Ngafuan, Aaron Kokolie, and Samuel Dilito Turay. The authors are grateful to Cara Carter, Yuliya Dudaronak, and Ellie Bailey of ORB International and the data collection teams for implementing household and health facility surveys in Guinea, Liberia, and Sierra Leone and for conducting key informant interviews and focus groups discussions in Liberia and Sierra Leone. In addition, we want to thank Aliou Barry of StatView International and the data collection teams for conducting key informant interviews and focus groups discussions in Guinea. The authors would like to thank the project coordination staff for their continued support and commitment, including Katherine Labombarde and Meredith Kerrigan. Special thanks to the editors Mary Burket and Charlotte Wilkins and the graphic designer, Erin Dowling. Amadou Bakayoko, Amanda Boachie, Latrisha Chappin and Nadine Ritcheson from USAID’s Africa Bureau (USAID/AFR) and representatives from the bureaus for Global Health, Global Development Lab, Food Security, Democracy, Conflict, and Humanitarian Assistance, and Economic Growth, Education, and Environment, the Guinea, Liberia, and Sierra Leone missions, and various implementing partners provided valuable support and constructive comments throughout the preparation of this report. Finally, our gratitude goes to the many individuals who participated in the key informant interviews, focus group discussions, and surveys. This evaluation would not have been possible without your contributions. n 05 ANNEXES ANNEX A. SCOPE OF WORK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A–1 ANNEX B. THEORY OF CHANGE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B–1 ANNEX C. USAID EBOLA RESPONSE AND RECOVERY STRATEGIC FRAMEWORK, 2015 . . . . . . . . . . . . . . . . . C–1 ANNEX D. EBOLA FUNDING FLOWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D–1 ANNEX E. DATA COLLECTION METHODOLOGY & IMPLEMENTATION, DATA QUALITY ANALYSIS & PLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–1 ANNEX F. DOCUMENTS CONSULTED FOR DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F–1 ANNEX G. NON-DISCLOSURE AGREEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . G–1 ANNEX H. DATA COLLECTION TOOLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–1 ANNEX I. LIST OF PERSONS INTERVIEWED, INTERVIEW AND FGD SITES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I–1 ANNEX J. ACTIVITY MAPPING TOOL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . J–1 ANNEX K. HOUSEHOLD AND HEALTH FACILITY DATA TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–1 ANNEX L. TREND TABLES, PRE- TO POST-EVD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–1 ANNEX M. DESK REVIEWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–1 06 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX A. SCOPE OF WORK n A–1 ANNEX A. SCOPE OF WORK Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document SECTION C – DESCRIPTION/SPECIFICATIONS/STATEMENT OF WORK C.1. TITLE Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone, & Liberia C.2. BACKGROUND Since the acceleration of the Ebola Virus Disease (EVD) outbreak began in Guinea, Liberia and Sierra Leone in mid-2014, the United States has mounted a whole-of-government response based on four pillars: I) Controlling the epidemic at its source in West Africa; II) Mitigating second-order impacts, including blunting the economic, social, and political consequences in the region; III) Engaging and coordinating with a broader global audience; and IV) Fortifying health security infrastructure in the region and beyond. This background section summarizes the US Government (USG) strategy for Pillar II: Mitigating Second￾Order Impacts. The 2014 Ebola virus disease (EVD) outbreak was first of that scale and the first to affect large urban populations. While historical precedents like HIV/AIDS and SARS are informative about channels of second-order impact, there are no precedents in recent experience for a large-scale epidemic that combines EVD’s deadliness, immediate and long-term impact, and rapid spread. The West African Ebola outbreak accelerated in mid-2014 and swiftly disrupted development progress in the most affected countries—Guinea, Liberia, and Sierra Leone. The epidemic has brought the human and institutional weaknesses in each of these countries to the fore, while creating extraordinary demands on political leaders and global responders. Public institutions in these countries were under acute stress, with risks of institutional failure and increased vulnerability to similar crises in the future. Due to fears associated with EVD, the health, agriculture, democracy and governance, trade and markets, and education sectors, as well as the overall economy, have been heavily impacted by government restrictions, reduced international investments and operational transitions and activity suspension and withdrawal, as well as border closures. In addition, the scientific community is now learning a great deal about the psychological and physical consequences of EVD, EVD viral persistence in survivors of the disease, risk of EVD relapse in survivors, and the potential for survivors to transmit the virus to others. There have been significant second-order impacts of the epidemic on household welfare, human development, and investment. While focusing primarily on containing the outbreak, governments and development partners sought from the outset to limit or mitigate these second-order impacts by avoiding excessive restrictions on movement and providing humanitarian relief to areas with high infection rates. However, with the global efforts focusing on containing EVD, many people have failed to receive treatment for other diseases, such as malaria and measles, and this has led to even more deaths. The main impacts of the crisis on private incomes and public revenues have resulted from policy-induced and voluntary reductions on normal economic activity in the form of transport restrictions, border closings and local quarantines, furloughs of non-essential public-sector workers, departure of expatriate personnel, and reduced engagement in production and trade by citizens. While economic activity has begun to rebound in the region, where many official restrictions have been lifted, the pace of recovery is weakened in all three most-affected countries by the depletion of household and business-sector assets (including in the financial sector) during the crisis, the interruption of large￾scale investment plans, and the ongoing caution of investors in a situation where there are elevated uncertainties and health risks. The USAID Ebola strategic framework laid out an EVD recovery approach that includes measures to revive economic activity, restore inclusive growth, and strengthen the capacities and resilience of public institutions. Pillar II programs targeted second￾order impacts of EVD with a focus on restoring development gains lost due to the effects of EVD including strengthening key institutions and infrastructure further weakened by the impact of the disease; addressing decreased levels of citizen trust in government; rebuilding a reduced willingness of people to accept social messaging on EVD; and ensuring that countries are recovering in a way that leaves them better prepared for and resilient to crises like EVD moving forward. A–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES C.3. RELATIONSHIP TO USG GLOBAL PRIORITIES USAID’s Ebola recovery efforts fall within the development priorities of the U.S. Government (USG), particularly in Africa, which focus on 1) strengthening democratic institutions, 2) spurring economic growth, trade, and investment, 3) advancing peace and security, and 4) promoting opportunity and development. USAID, as the USG’s premier development agency, leads the development and is leading the implementation of the USG’s strategy on Ebola. Pillar II of the Strategy focuses on recovery, post-Ebola, and ensuring that countries better prepare for and to respond to EVD and other future crises. There are five sub-components of Pillar II: food security, health systems recovery and non-Ebola health services, governance and economic crisis mitigation, basic education, and innovation, technology and partnership. C.3.1. PILLAR II SUB-COMPONENT M&E OBJECTIVES United States Missions in West Africa (e.g., USAID/Liberia, USAID/Guinea, USAID/Sierra Leone), and USAID Operating Units (OUs) are responsible for developing, implementing, monitoring, evaluating, and reporting on sector-specific activities such as food security, health, democracy and governance, etc. They are directly responsible for carrying out the field activities and are expected to achieve the intended activities-level results in one or more of the affected countries where they have program activities (see list below). Each Bureau or Office in Washington that is supporting Ebola recovery efforts under Pillar II of the USG Strategy adheres to its own M&E framework with established objectives and activities and quarterly updates to the Africa Ebola Unit (AEU). Below are illustrative objectives each Bureau or Office is working toward. Food Security Increased availability of, and accessibility to, food by vulnerable households reduced poverty and under nutrition, and trade and market systems restored: ■ Household food access increase: increase food access of the most vulnerable groups impacted directly and indirectly by Ebola in both urban and rural areas • Households received cash transfers/food vouchers for food • Children received hot meals in schools ■ Market and agricultural recovery: increase food availability through the recovery of local food production and market function • Households received agricultural input vouchers • Small-scale traders received cash grants Health Systems Recovery and non-Ebola Emergency Health Services USAID, working with host governments and other partners, will address the longer-term recovery of health systems with the goal of improving their ability to respond to the EVD outbreak; increase demand for health services within communities in EVD-affected countries; build capacity to manage health crises; and improve service delivery. The following objectives summarize the goals under this subcomponent: ■ Restoration of primary health care in public facilities: restart health services including immunizations, materials and child health, and family planning to pre-EVD levels ■ Social mobilization, behavior change risk communication ■ Supply chain and commodities: improve service delivery and logistics ■ Human resources for health: improve management of health crises through triage protocols; integrate capabilities created during the response into the health care system ■ Health governance and management: address healthcare workforce issues. Governance and Economic Crisis Mitigation Programs focused on recovery will target those individuals and communities hardest hit by EVD, supporting activities that empower individuals, including survivors, and community groups to foster greater accountability of government-provided services most critical to respond to and recover from EVD; improve social￾protection services for vulnerable populations; enhance water and sanitation services in EVD-affected communities; and provide healthy and safe learning environments for the return of students to more than 4,400 primary schools. The intended results for this subcomponent are: ■ Civil society: empower individuals and communities to improve accountability in public sector services ■ Elections process: peaceful and transparent elections ■ Governance ■ Water and sanitation: reestablish clean water provision in Voinjama, Liberia ■ Safe learning environments ■ Social protection systems ■ Private-sector engagement Innovation, Technology and Partnership Funds for this subcomponent will support collaborative efforts between the USG, local governments, and the private sector in the three countries and the West Africa region to improve the quality and sharing of health information and decision-making, and strengthen communications and digital financial systems necessary ANNEX A. SCOPE OF WORK n A–3 to prevent, detect, and respond to epidemics. These objectives will be attained by pursuing the following indicators: ■ Grand challenge: attract innovations, new technologies and partnerships to address response and recovery challenges ■ Information communications technology: improve health information sharing, quality, and decision making; strengthen communications and digital financial systems to improve prevention, detection and response to future epidemics Tailored to individual country context and needs, these activities will help strengthen critical institutions, including governance and infrastructure required to keep countries on a path toward long￾term development progress. USAID will also support efforts that address the limited capacity of government institutions to respond to this shock through initiatives to increase the effectiveness, trans￾parency, accountability, and responsiveness of governance struc￾tures, as well as maintain momentum on key governance reforms. Funding will support critical infrastructure rehabilitation and investments that address key barriers to trade and access to basic services including power, water supply, transportation, and telecommunications infrastructure. Funding will also support education programs aimed at re-opening and improving the conditions of schools in order to keep children in school and protect and advance educational gains in literacy and numeracy. Activities will focus on ensuring that schools are equipped to prevent EVD transmission—for example, through the installation of hygiene and handwashing stations—and that school personnel are trained in appropriate screening and response protocols. Funds will support programs to assist vulnerable populations in meeting their basic needs as well. To meet the challenges of lack of local capacity, infrastructure deterioration, and the government’s difficulty in carrying out the activities or provide adequate supervision, USAID entered into partnerships with contractors and subcontractors knowledgeable about and experienced in implementing activities in Guinea, Sierra Leone, and Liberia. Through these efforts, USAID worked collaboratively with their partners to develop activities implementation plans and M&E plans. Those plans delineated the roles and responsibilities of each party. Implementing partners are in charge of executing the tasks in the field and provide regular monthly and/or quarterly reports to USAID. Additionally, they conduct routine monitoring of their activities. USAID undertakes routine monitoring, usually on a quarterly basis, and procures an evaluation of their programs in the field. C.4. SCOPE The contractor will conduct performance evaluations of USAID’s Ebola recovery efforts in Guinea, Liberia, and Sierra Leone during the timeframe of March 2015 through December 2019. The purpose of these activities will involve capturing, aggregating, and reporting on all recovery activities’ performance and impact toward achieving established Pillar II macro-level objectives as stated in the AEU Strategic Framework and the monitoring and evaluation (M&E) Results Framework. More than a year since the start of the EVD outbreak, the affected countries have reduced the number of cases to zero by identifying and breaking new chains of transmission with the support of the international community. However, given the precarious state of the current situation in Guinea, Sierra Leone, and Liberia, the EVD response highlighted seven significant challenges to recovery (See Annex II: Challenge to Recovery). The evaluations shall examine the extent to which the activities undertaken in the field and the implementation approaches effectively mitigated second-order impacts of EVD and assess the sustainability of these interventions post-EVD outbreak. The evaluations will capture stories related to outputs- and outcomes￾level changes, as well as stories regarding challenges and successes regarding implementation of projects’ activities. These evaluations will be instrumental in determining how successful the USAID recovery strategy has been in meeting the objectives and providing recommendations for improving the USG design and coordination efforts to contain outbreaks around the world. In addition to analyzing component-specific information on indicators during this recovery phase, the evaluations team will also provide in-depth evaluations of and lessons learned from the activities. C.5. CONTRACT COMPONENTS The present contract contains four major components, as outlined below. Throughout all of these components, the Contractor shall ensure that Pillar II goals, objectives, and assumptions are in line with the USAID Country Development Cooperation Strategy and host country priorities: ■ Component I – Evaluation ■ Component II – Routine Monitoring ■ Component III – Data Quality Assurance ■ Component IV – Support Improved Knowledge Management and Learning A–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES C.5.1. COMPONENT 1: EVALUATION Under this component, the Contractor shall carry out two performance evaluations in each of the three countries during the period of performance of this contract. The Contractor shall design the performance/process evaluations that focuses on descriptive and normative questions, as defined in ADS 203.3.1.1, capturing what a particular activity/project or program has achieved; how it is being implemented on the ground; whether the activities are on track to attaining the expected results; how it is perceived and valued by host governments and benefiting target populations with regard to questions pertaining to program design, management, operational, and decision-making. Performance evaluations involve before and after comparisons but lack in general a rigorously defined counterfactual. The Contractor shall also design and implement each of the performance evaluations in close collaboration with USAID/Washington, based on the circumstances in each targeted country. The design will include variables measuring key outcomes, carrying out data collection and analysis, and advising each Mission on how to develop precise approaches to develop measurable indicators, key evaluation questions, and a learning plan/agenda. C.5.2. EVALUATIONS STRATEGY/DESIGN Given the nature and extent of the EVD outbreak as well as the illustrative evaluation questions, we anticipate that the design shall use a mixed method approach to collecting data, i.e., include a combination of rigorous qualitative data gathering as a main research approach with time-bound descriptive quantitative data to capture such quantitative information. In order to ensure the maximum value for learning and use, the proposed methodology should include: 1. A detailed evaluation work plan, including activities, timeline, and level of effort. 2. A detailed evaluation design, including a description of the methodology outlining the methodological strengths and limitations identified or anticipated – The contractor shall submit a detailed evaluation design and methodology. However, it is anticipated that the final methodology will be developed collaboratively by the proposed Contractor and USAID/AEU M&E team and the USAID M&E Reference Group (EMERG) in Washington, DC. Within 15 days of approval of the evaluation work plan, the Contractor must submit to the Task Order Contracting Officer’s Representative/Task Order Agreement Officer’s Representative (TOCOR/TOAOR) an evaluation design which will become an annex to the Evaluation report. The evaluation design will include: 1) a detailed evaluation design matrix that links the Evaluation Questions in the SOW to data sources, methods, and the data analysis plan; 2) the list of potential interviewees and sites to be visited and proposed selection criteria and/or sampling plan (must include calculations and a justification of sample size, plans on how the sampling frame will be developed, and the sampling methodology); 3) known limitations to the evaluation design; and 4) a dissemination plan. Note that the draft questionnaires and other data collection instruments or their main features will be prepared while USAID is reviewing the Evaluation Design and delivered at a later date to be mutually agreed. USAID Washington Bureaus, Mission Offices, and relevant stakeholders will be asked to take no more than 15 business days to evaluate the proposed design and consolidate comments through the (TOCOR/ TOAOR). Once the Contractor receives the consolidated comments on the initial evaluation design and work plan, it is expected to respond with a revised evaluation design and work plan within six business days. The final Evaluation Design will include the final evaluation questions; the methodology and data collection approaches and instruments; the involvement of Missions’ staff, local partners, and other national stakeholders; data analysis, data quality assurance, and dissemination plan as well as a plan for communication and effective use of evaluations results. The following elements shall be taken into consideration in planning, executing, and reporting on the task: a. Evaluation Briefings: The Contractor shall hold an in-briefing and out-briefing with the Mission in each country to discuss plans and present major and/or preliminary findings to USAID and other stakeholders. It is anticipated that the briefings shall take place in each Mission’s facilities. b. Draft Evaluation Report: The draft evaluation report should be consistent with the guidance provided in Section C.5.3 c. Final Evaluation Report: The final report shall address each of the questions identified in the SOW and any other issues the Contractor considers to have a bearing on the objectives of the evaluations. Any such issues may only be included in the report after consultation with USAID. The submission date for the draft evaluation report shall be determined in the evaluation work plan. Once the initial draft evaluation report is submitted, the AEU shall have 10 business days in which to evaluate and comment on the initial draft, after which the TOCOR/TOAOR shall submit the consolidated comments to the Contractor. The Contractor shall then be asked to submit a revised final draft report seven business days hence, and the AEU will evaluate and submit comments on this final draft report within five business days of its submission. The main body of the Final Evaluation Report shall not exceed 30 pages (exclusive of annexes) and must include a title page, an executive summary, introduction, background of the project being evaluated, the main evaluation questions, the methodology or methodologies, the limitations to the evaluation, findings, conclusions, and recommendations and lessons learned. The executive summary must be between three and five pages in length and summarize the purpose, background of the intervention being evaluated, main evaluation questions, methods, findings, ANNEX A. SCOPE OF WORK n A–5 conclusions, and recommendations and lessons learned. The executive summary shall not contain any information not included in the main body of the Report. The evaluation methodology shall be explained in the report in detail. Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (e.g., selection bias, recall bias, etc.). The report must include the evaluation design as an annex. All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides must be included as annexes. Sources of information must be properly identified in an annex. All quantitative data collected by the evaluation team must be provided in an electronic file in easily readable format per USAID data policy and requirements (e.g., MS Excel). The data must be organized and fully documented for use by those not familiar with the intervention or the evaluation, so that an independent evaluator could use it and arrive at the same findings and conclusions.The annexes to the report shall include: ■ The Evaluation SOW; ■ Any statements of difference regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; ■ All tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; ■ Sources of information, properly identified and listed; and ■ Disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of interest. In accordance with AIDAR 752.7005, the Contractor shall make the final evaluation reports publicly available through the Development Experience Clearinghouse within 30 calendar days of final approval of the formatted report. C.5.3. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per the USAID Evaluation Policy and USAID ADS 203, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. ■ The evaluation report should represent a thoughtful, well￾researched, and well-organized effort to objectively evaluate what worked in the project, what did not, and why. ■ Evaluation reports shall address all evaluation questions included in the SOW. ■ The evaluation report shall include the SOW as an annex. All modifications to the SOW—whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline—need to be agreed upon in writing by the (TOCOR/TOAOR). ■ The evaluation methodology shall be explained in detail. All tools used in conducting the evaluation—such as questionnaires, checklists, and discussion guides—will be included in an annex in the final report. ■ Evaluation findings will assess outcomes and their impact on males and females. ■ Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups if applicable, etc.). ■ Evaluation findings shall be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or the compilation of people’s opinions. Findings shall be specific, concise, and supported by strong quantitative and/or qualitative evidence. ■ Sources of information need to be properly identified and listed in an annex. ■ Recommendations need to be supported by a specific set of findings. ■ Recommendations shall be action-oriented, practical, and specific, with defined responsibility for the action. C.5.4. OTHER REQUIREMENTS All quantitative data collected by the evaluation team must be provided in machine- readable, non- proprietary formats as required by USAID’s Open Data policy (see ADS 579). The data shall be organized and fully documented for use by those not fully familiar with the project or the evaluation. USAID will retain ownership of the survey and all datasets developed. ANNEX B. THEORY OF CHANGE n B–1 ANNEX B. THEORY OF CHANGE Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document Ebola Pillar II Activities (in six interacting thematic areas) Activities & Input Types (Used across several thematic areas) Basic health service delivery restored Health worker capacity to deliver EVD and non-EVD services Comprehensive services developed and provided for EVD survivors Health behavior gains enhanced and reinforced Health infrastructure (including management systems) strengthened Health information system developed and supported Health financing & resource allocation improved School attendance restored and gender balance improved Literacy & numeracy improved Household food security increased Nutritional status improved Household standard of living improved Enabling gvt policies developed & harmful policies & practices revised Community service organizations strengthened to provide advocacy, transparency, accountability Management systems & accountability of ministries & stakeholders improved at all levels Agriculture & import market function restored & improved Social protection provided for EVD-affected groups ICT systems for health, gender equity, & civic participation improved Power, water, roads, & communica￾tion infrastructure restored & expanded Outcomes Ebola Pillar II Objectives Loss of development gains prevented Institutions and infrastructure recovered and strengthened Sustained systems built through partnerships, innovation, and capacity building Technical Assistance Training (leadership, infection protection & control, management, gender equity) Clinical Care M&E Systems Development Consultation with Community Leaders Education Refurbishing/Restocking Facilities Nutritional Support Income Support (cash transfers, vouchers, food for work) Agricultural Inputs Social and Behavior Change Communication Staffing Support Restoring/Improving Infrastructure (roads, power) Procurement, Logistics, & Distribution Support Partnership Development ICT Inputs Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery, & Survivor Programs ANNEX C. USAID EBOLA RESPONSE AND RECOVERY STRATEGIC FRAMEWORK, 2015 n C–1 ANNEX C. USAID EBOLA RESPONSE AND RECOVERY STRATEGIC FRAMEWORK, 2015 Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document GOAL: Rapidly respond to the Ebola crisis, mitigate Ebola’s key negative impacts, and prepare for, prevent, and respond to future outbreaks Household Food Access: Increase food access of the most vulnerable groups impacted directly and indirectly by Ebola, in both urban and rural areas Market and Agricultural Recovery: Increase food availability through the recovery of local food production and market function Restoration of Primary Health Care in Public Facilities: Restart health services including immunizations, maternal and child health, and family planning to pre-Ebola levels Social Mobilization: Behavior Change and Risk Communication Supply Chain & Commodities: Improve service delivery and logistics Human Resources for Health: Improve management of health crises through revised triage protocols, integrate capabilities created during the response into the health care system Health Governance and Management: Address health care workforce issues Civil Society: Empower individuals and communities to improve accountability in public sector services, governance, private sector engagement, elections process (Guinea) Water and Sanitation: Liberia Safe Learning Environments: Liberia Social Protection Systems: Liberia Grand Challenge: Attract innovations, new technologies, and partnerships to address response and recovery challenges Information Communication Techology: Improve health information systems PILLAR II RECOVERY: Mitigate second-order impacts of Ebola; prevent loss of development gains; strengthen key institutions and infrastructure whose weaknesses enabled the rapid spread of Ebola or slowed the response PILLAR I: RESPONSE Control the outbreak PILLAR VI: GLOBAL HEALTH SECURITY Food Security Emergeny Non-Ebola Health Svcs Health Systems Recovery Governance & Econ. Crisis Mitigation Innovation, Technology & Partnership Command & Control of Response Surveillance and Epidemiology Restoration of Essential Health Services Social Mobilization and Communications Recovery, Resilience and Transition Case Management Logistics Prevent avoidable epidemics Detect threats Respond rapidly and effectively Preparedness PILLAR III RECOVERY: Build coherent leadership and operations (cross-cutting) C–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX D. EBOLA FUNDING FLOWS n D–1 ANNEX D. EBOLA FUNDING FLOWS Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document CONGRESS After appropriation, Congress approves a joint State and USAID budget. State has a coordination unit and USAID an Ebola Secretariat. F State and USAID BRM pulled together the budget request. Ebola funds are allocated to different accounts, such as ESF, IDA, INCLE, PKO, that are then distributed by USAID or State. USAID STATE USG operating units include GHET, FFP, CII, DG, AFR Bureau, LAB, BRM, country missions, and regional offices. USG Agency HQ/country mission or regional office USG agencies and missions develop funding arrangements with Prime IP. They may use a new or existing mechanism (grant, contract, or cooperative agreement). Prime implementing partners Sub-partners Beneficiaries vary depending upon activity. BENEFICIARIES D–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–1 ANNEX E. DATA COLLECTION METHODOLOGY & IMPLEMENTATION, DATA QUALITY ANALYSIS & PLAN Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document ANNEX CONTENTS ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–3 PART 1. DATA COLLECTION METHODOLOGY AND IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . E–4 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–4 SAMPLE DESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–4 Qualitative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–5 DATA COLLECTION IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–12 Household Surveys (HHSs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–12 Health Facility Surveys (HFSs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–14 Qualitative Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–16 PART 2. DATA ANALYSIS AND DATA COLLECTION QUALITY ASSURANCE PLANS . . . . . . . . . . . . . . . . . . E–18 DATA ANALYSIS PLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–18 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–18 Description of the Intervention Typology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–18 General Approach for Quantitative Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–20 General Approach for Qualitative Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–20 Detailed Analysis Plan by Evaluation Question . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–22 DATA COLLECTION QUALITY ASSURANCE PLAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–34 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–34 Description of DCQA Phases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .E–34 SUB-ANNEX A: BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–39 SUB-ANNEX B: GUIDELINES FOR DATA COLLECTION AND SUBMISSION OF QUALITATIVE TRANSCRIPTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–39 Note: This annex was originally submitted in March 2018 as two separate reports. It was updated in December 2018 and is presented here as one annex in two parts. E–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Part 1: Table E1–1. Sample size calculations by guiding outcome indicators: Guinea (IBTCI’s HHS 2017) . . . . . . . . . . . . . . . . E–6 Table E1–2. Sample size calculations by guiding outcome indicators: Liberia (IBTCI’s HHS 2017) . . . . . . . . . . . . . . . . . E–6 Table E1–3. Sample size calculations by guiding outcome indicators: Sierra Leone (IBTCI’s HHS 2017) . . . . . . . . . . . . E–6 Table E1–4. Number of completed and desired household interviews, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . . . . E–6 Table E1–5. Sampling frames used for IBTCI’s 2017 HHS (Sample frames used were the latest population and housing census obtained by the national statistics bureau of each country) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–7 Table E1–6. Population exclusions by country, IBTCI’s HHSs 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–7 Table E1–7a. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and prefecture, Guinea, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–8 Table E1–7b. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and district, Liberia, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–8 Table E1–7c. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and prefecture, Sierra Leone, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–9 Table E1–8. Sampling errors per country, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–9 Table E1–9. Number of completed and desired health facility interviews, IBTCI’s HHS 2017 . . . . . . . . . . . . . . . . . . . E–10 Table E1–10. Number of health facility substitutions by country and reason, IBTCI’s HFSs 2017 . . . . . . . . . . . . . . . . E–11 Table E1–11. Field Questionnaires: Written and Verbal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–12 Table E1–12. Training dates by country, IBTCI’s HHSs 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–13 Table E1–13. Training dates by country, IBTCI’s HFSs 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–15 Part 2: Table E2–1. Key outcome indicators by thematic area . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–19 Table E2–2a. Data Analysis Plan for Evaluation Question One: Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–23 Table E2–2b. Detailed Outcome Indicators for EQ1.4 (How have Pillar II activities contributed to the achievement of Pillar II’s three objectives?) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–25 Table E2–3. Data Analysis Plan for Evaluation Question Two: Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–29 Table E2–4. Data Analysis Plan for Evaluation Question Three: Gaps & Opportunities . . . . . . . . . . . . . . . . . . . . . . . . E–30 Table E2–5. Data Analysis Plan for Evaluation Question Four: Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–32 Table E2–6. Roles and responsibilities of evaluation team members in DCQA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–37 Figures Part 2: Figure E2–1. Overview of the six intervention types for analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–19 Figure E2–2. Linkage between the Intervention Types/Themes & the AEU Outcomes . . . . . . . . . . . . . . . . . . . . . . . . E–21 Figure E2–3. Analytical framework to measure sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E–22 ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–3 AEU Africa Ebola Unit CAPI Computer-assisted Personal Interviewing CBOs Community-based Organizations CDCS Country Development Cooperation Strategy C-HMIS Country-Health Management Information System COP Chief of Party CSO Civil Society Organizations CHVs Community Health Volunteers DAP Data Analysis Plan DHS Demographic and Health Survey DHIS 2 District Health Information Software 2 DCQA Data Collection Quality Assurance EMONC Emergency Obstetrics and Neonatal Care EQ Evaluation Question EVD Ebola Virus Disease EUV End Use Verification FCs Field Coordinators FGD Focus Group Discussion FP Family Planning GBV Gender-Based Violence GOG Government of Guinea GOL Government of Liberia GOSL Government of Sierra Leone GPS Global Positioning System HDDS Household Dietary Diversity Score HF Health Facility HH Household HMIS Health Management Information System HO Home Office ID Identification IMNCI Integrated Management of Newborn and Childhood Illnesses IP Implementing Partner IPC Infection Prevention and Control KII Key Informant Interview LLIN Long-lasting Insecticidal Nets LMIS Logistics Management Information System LSMS Living Standards Measurement Survey M&E Monitoring and Evaluation MEL Monitoring, Evaluation and Learning MICS Multiple Indicator Cluster Survey MSF Médecins Sans Frontières NER Net Enrollment Rate ODF Open Defecation Free OU Operating Unit PPP Public-Private Partnership PSU Primary Sampling Unit RMNCH Reproductive, Maternal, Newborn and Child Health SBA Skilled Birth Attendant ToC Theory of Change UNESCO United Nations Educational, Scientific and Cultural Organization USAID United States Agency for International Development USG United States Government ACRONYMS E–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES PART 1. DATA COLLECTION METHODOLOGY AND IMPLEMENTATION INTRODUCTION Phase 2 of the Ebola Pillar II Performance Evaluation 1 (PE1) was the data collection phase. Defining and explaining Pillar II results in each thematic area, according to the evaluation questions and the projects’ theories of change, required a mix of qualitative and quantitative methods, and differing levels of detail. The methods included: ■ Systematic Desk Review of Pillar II program documents and technical literature were reviewed in the six thematic areas. From March to September 2017, IBTCI created points of contact at the U.S. Agency for International Development/Africa Bureau (USAID/AFR), the three missions in Liberia, Sierra Leone, and Guinea, and implementing partners (IPs) (through their respective Contracting or Agreement Officer Representative (C/ AOR) to request contract or grant agreements, quarterly and annual reports, monitoring and evaluation (M&E) plans and reports, baseline and endline studies, and other reports for all Pillar II-funded activities. ■ Key Informant Interviews (KIIs) with individuals in each major stakeholder group and thematic area across the USAID priority areas (the prefectures/counties/districts selected by USAID for Pillar II focus) were conducted to allow for perspectives from beneficiaries and partners at the national, prefecture/county/ district, and community levels. Respondents were selected based on their knowledge of the subject matter and their capacity to reflect and discuss said material with an outside interviewer. Every effort was made to include female as well as male respondents, but in some stakeholder sub-groups the universe of female respondents was very small. KIIs took place between September 2017 and January 2018 at the central level and in the Pillar II priority areas (13 prefectures in Guinea, eight counties in Liberia, and eight districts in Sierra Leone; see next page for the selection criteria). ■ Focus Group Discussions (FGDs) with representatives from various stakeholder groups were held using a methodology tailored to the evaluation questions and the type of the participant group. FGDs took place between September 2017 and January 2018 at the central level and in the Pillar II priority areas. ■ A Health Facility Survey (HFS) in a sample of primary health care settings was required to assess the performance and results of many of the health-focused Pillar II activities in all three countries, with special focus on the priority areas (i.e., prefectures/counties/districts). HFS included direct observations and record reviews at the facility level. The HFS data collection activities took place between December 2017 and February 2018. ■ A Structured Household Survey (HHS) of a multi-stage stratified random sample of households, proportionate to population size, was designed to be representative of the priority area level (i.e., prefecture/county/district). The HHS was designed to collect information about knowledge, attitudes, and practices relative to EVD recovery and other relevant information to address the evaluation questions. HHS data collection activities took place between December 2017 and January 2018. SAMPLE DESIGN Three principles guided the design of every procedure in Phase 2 of the evaluation: ■ Samples for quantitative methods were large and diverse enough to produce credible conclusions and stable estimates of key indicators at the prefecture/county/district level; ■ Gender issues were considered in every stage and methods will permit disaggregation of findings by gender; and ■ Methods were designed to minimize the burden on stakeholders (USAID missions, government, IPs, beneficiaries) and avoid duplication of effort. In all cases, the evaluation team developed data collection tools, sampling strategies, and field procedures in consultation with the local evaluation experts and the mission’s M&E officers. The design carefully balanced the advantages of local tailoring/adaptation with those of standardization to permit data pooling and comparison. The tools, strategies, and procedures were designed and implemented in an ethically sound and sensitive manner. The Pillar II priority areas were selected in consultation with USAID, based on four criteria: 1) geographical areas prioritized in USAID’s country development strategy, 2) level of EVD burden, 3) presence of USAID Ebola Pillar II activities, and 4) importance of geographic areas for understanding contextual factors that have potentially affected recovery. In Guinea, IBTCI collected data in 13 prefectures: Boffa, Boké, Faranah, Kankan, Forécariah, Kindia, Labé, Beyla, Guéckédou, Lola, Macenta, N’Zérékoré, and Conakry. In Liberia, data collection took place in eight counties: Bomi, Grand Bassa, Grand Cape Mount, Bong, Lofa, Margibi, Montserrado, and Nimba. Bomi and Grand Cape Mount were added to USAID/Liberia’s six ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–5 focus counties based on activity mapping and local experts’ advice that both counties were important settings for understanding the Ebola recovery efforts in Liberia. In Sierra Leone, there were eight Pillar II priority districts: Kailahun, Kenema, Bombali, Koinadugu, Port Loko, Tonkolili, Western Area Rural, and Western Area Urban. QUALITATIVE METHODS Key Informant Interviews and Focus Group Discussions To gather qualitative data, the team selected respondents purposively, based on their stakeholder group and their capacity to speak knowledgeably about the topic at hand. As Pillar II activities vary greatly in geographical coverage, and time periods/ duration, and they involve such a wide range of stakeholders (from world class policy-makers in the capital cities to non-literate EVD survivors in hard-to-reach rural villages), it was necessary to prioritize and recruit respondents from the stakeholder groups relevant for each evaluation question, and the list differs for each country. A quasi-quota sampling method1 was used in each country, with strata based on: ■ Geographic location (prefecture/county/district, urban/rural); ■ Relevant stakeholder group according to the thematic area (e.g., questions on agricultural markets for farmers and food-sellers; questions about health services for health facility managers and clients); and ■ Organizational level: Central/national; prefecture/county/ district; and ultimate beneficiaries. The aim of the stratification was to capture insights and lessons learned from as broad a range of knowledgeable and experienced people as possible, to maximize practical insights and lessons learned. KII and other time-intensive interview participants were selected from the prioritized respondent groups that were identified in Phase 1, keeping in mind that granularity increases both the cost and time required for each procedure. Recruiters were advised to tap at least three different sources or entry points (e.g., local women’s groups; association of market traders, and the county superintendent) when identifying potential respondents. This was done to reduce clustering and skewing recruitment by political or other biases. A quota sampling matrix for the key informant interviews was provided to each country team, assisting each team to recruit respondents from all priority areas, distributed as evenly as possible across the strata, and including as many women and girls as possible. FGDs were conducted separately with adult males, 1. Proper quota sampling requires recruiting individuals in numbers proportional to an estimate of the fraction of the total sampling that their segment represents. Such estimates cannot be made in these cases. 2. HHSs measured a number of indicators. These two indicators were used as proxy for basic health services and their values/trends over time were used for the sample size calculations. 3. Full vaccination coverage guided sample size calculations and it was taken into consideration in the final sample size determination. However, due to the (1) variation of the vaccination calendar across countries, (2) the complexity of measurement, (3) the controversial nature of this indicator, and (4) the fact that it is measured in other ongoing/ upcoming surveys, we excluded full vaccination coverage questions in the final HHS questionnaire. adult females, adolescent males, adolescent females, and adult male and adult female service implementers. A full list of the KII respondents and FGD stakeholder groups and location is included in Annex I. QUANTITATIVE METHODS Household Surveys Sample Size Calculations Household surveys (HHSs) were conducted in the Pillar II priority areas (13 prefectures in Guinea, eight counties in Liberia and eight districts in Sierra Leone. When calculating the sample size for each country, we considered two key health outcome indicators estimated by previous household surveys. These indicators were: (1) Percentage of live births in the three years preceding the survey delivered at a health facility; and (2) Percentage of children aged 12-23 months who had received full vaccinations (8 WHO￾recommended vaccinations).2 , 3 For the two outcome key indicators, we calculated with a desired 95% confidence interval (5% margin of error/precision). For non-response, we used the household response rates from the latest Demographic and Health Surveys (DHS) in the three countries (0.5% non-response in the 2012 Guinea DHS, 1.0% non-response in the 2013 Liberia DHS, and 1.0% non-response in the 2013 Sierra Leone DHS). The following standard formula was used for calculating the minimum sample size required for the estimates: (Z2 P(1-P)) d 2 Where: n= minimum sample size Z= Z statistic for a desired level of confidence P= proportion in the population with the characteristic of interest d= margin of error The final sample size is determined based on the indicator that requires the highest desired number of households. Results are shown in Tables 1 (Guinea), 2 (Liberia) and 3 (Sierra Leone). The desired sample size was 4,860 households for Guinea (representative at the prefecture level), 3,079 households for Liberia (representative at the county level), and 3,038 households for Sierra Leone (representative at the district level). n = E–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E1–1. Sample size calculations by guiding outcome indicators: Guinea (IBTCI’s HHS 2017) Outcome Key Indicators Percentage (P) GDHS 2012 Sample size (number of HHs per prefecture) Considering non-response Total desired sample size (number of HHs for the 13 Pillar II prefectures) Percentage of live births in the three years preceding the survey delivered at a health facility 41.0% 372 374 4,860 Percentage of children 12–23 months who had received full vaccination (8 WHO￾recommended vaccinations) 36.5% 357 359 4,664 Assumption: (P) is the same in all prefectures. Table E1–2. Sample size calculations by guiding outcome indicators: Liberia (IBTCI’s HHS 2017) Outcome Key Indicators Percentage (P) LDHS 2013 Sample size (number of HHs per county) Considering non-response Total desired sample size (number of HHs for the 8 Pillar II counties) Percentage of live births in the three years preceding the survey delivered at a health facility 59.4% 371 375 2,998 Percentage of children 12–23 months who had received full vaccination (8 WHO￾recommended vaccinations) 54.8% 381 385 3,079 Assumption: (P) is the same in all counties. Table E1–3. Sample size calculations by guiding outcome indicators: Sierra Leone (IBTCI’s HHS 2017) Outcome Key Indicators Percentage (P) SLDHS 2013 Sample size (number of HHs per district) Considering non-response Total desired sample size (number of HHs for the 8 Pillar II districts) Percentage of live births in the three years preceding the survey delivered at a health facility 57.5% 376 380 3,038 Percentage of children 12–23 months who had received full vaccination (8 WHO￾recommended vaccinations) 68.0% 335 338 2,707 Assumption: (P) is the same in all districts. Table E1–4. Number of completed and desired household interviews, IBTCI’s HHS 2017 Country Number of interviewed households Number of desired households Guinea 5,189 4,860 Liberia 3,297 3,079 Sierra Leone 3,235 3,038 Total 11,721 10,977 ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–7 Table E1–5. Sampling frames used for IBTCI’s 2017 HHS (Sample frames used were the latest population and housing census obtained by the national statistics bureau of each country) Country Source Year Guinea Recensement Général de la Population et de l’Habitation 2014 Liberia 2008 Liberia Population and Housing Census 2008 Sierra Leone Sierra Leone 2015 Population and Housing Census 2015 Table E1–6. Population exclusions by country, IBTCI’s HHSs 2017 Country Area Population by area (in number of households) % of regional population % of total population Liberia Select towns Nimba County 602 <1% <1% Guinea Select locality Kankan 71 <1% <1% Sampling Approach IBTCI employed a multi-stage sampling approach. Household samples were drawn up by the national statistics bureau of each country. The sample was prepared as follows: 1st stage: Sample distribution. Locations were chosen based on the most recent census and distributed across the 13 Pillar II priority prefectures in Guinea, eight priority counties in Liberia, and eight districts in Sierra Leone. 2nd stage: Selection of starting points. In each of the selected primary sampling units (PSUs), a permanent and conspicuous structure was identified from which interviewers could start the random selection process. Examples of starting points include mosques, churches, schools, markets, and major intersections. 3rd stage: Selection of the households. The household selection process was carried out according to a procedure described below. Interview teams gather at their assigned starting points. They calculate the day code (adding the digits of the current day [date] together, until arriving at a single digit) to determine the initial skip pattern. Each interviewer then heads in a different direction (e.g., north, south, east, and west, although this may vary somewhat depending on the layout of streets and roads in the vicinity of a particular starting point), skipping the appropriate number of dwellings as determined by the day code, and starts his/her assignment at the next house, counting from the left. The interviewer then seeks agreement from an adult for the household to participate in the survey. 4th stage: Selection of the respondent within the household. Once a household was successfully contacted for interview, the interviewer asked to speak to a responsible adult and member of the household, preferable the head of the household. The selected responsible adult could answer Sections A-E of the questionnaire (see Annex H: Data Collection Tools). Section F was restricted to responsible adults who prepared food for the household, and Section G was restricted to women, aged 15-49, who had given birth since October 2014. Households located in “gated” or “reserved” communities such as military barracks or hospitals were excluded from the sample. In addition, select PSUs were replaced due to inaccessibility (Table 6). Sample Representativeness From the national population statistics frame, a cross-tab of region by urban/rural for the universe (N) was used. A similar cross-tab for the sample (n) using the data set was also developed. The sample cross-tab was compared with the universe cross-tab; this comparison was used to calculate the weighting factors. Tables 7a– 7c (pages 8 and 9) show the unweighted and weighted numbers and percent distributions, according to urban-rural residence and area for the three countries. E–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E1–7a. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and prefecture, Guinea, IBTCI’s HHS 2017 Urban/Rural Residence/Prefecture Unweighted percent Unweighted number Weighted percent Weighted number Boffa Urban 0.52% 27 0.13% 7 Boffa Rural 6.94% 360 3.43% 178 Boke Urban 4.51% 234 3.34% 173 Boke Rural 3.04% 158 4.53% 235 Conakry Urban 9.10% 472 30.52% 1583 Faranah Urban 1.60% 83 0.87% 45 Faranah Rural 5.59% 290 3.57% 185 Kankan Urban 3.26% 169 2.56% 133 Kankan Rural 4.95% 257 3.47% 180 Forécariah Urban 0.56% 29 0.33% 17 Forécariah Rural 7.44% 386 3.88% 201 Kindia Urban 2.10% 109 2.39% 124 Kindia Rural 5.28% 274 5.71% 296 Labe Urban 2.14% 111 1.91% 99 Labe Rural 5.45% 283 5.38% 279 Beyla Urban 0.50% 26 0.31% 16 Beyla Rural 7.23% 375 4.61% 239 Guéckédou Urban 1.56% 81 1.06% 55 Guéckédou Rural 5.72% 297 4.78% 248 Lola Urban 1.12% 58 0.52% 27 Lola Rural 6.67% 346 3.12% 162 Macenta Urban 1.43% 74 0.98% 51 Macenta Rural 6.01% 312 4.72% 245 N’Zérékoré Urban 3.51% 182 3.36% 174 N’Zérékoré Rural 3.78% 196 4.51% 234 Table E1–7b. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and county, Liberia, IBTCI’s HHS 2017 Urban/Rural Residence/County Unweighted percent Unweighted number Weighted percent Weighted number Grand Bassa Urban 3.09% 101 2.08% 68 Grand Bassa Rural 9.82% 321 5.75% 188 Bomi Urban 2.75% 90 0.61% 20 Bomi Rural 9.24% 302 2.36% 77 Bong Urban 4.07% 133 3.61% 118 Bong Rural 7.62% 249 8.14% 266 Grand Cape Mount Urban 1.62% 53 0.28% 9 Grand Cape Mount Rural 11.38% 372 4.19% 137 Lofa Urban 4.41% 144 2.94% 96 ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–9 Table E1–7b. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and county, Liberia, IBTCI’s HHS 2017 Lofa Rural 8.08% 264 6.82% 223 Margibi Urban 5.63% 184 3.15% 103 Margibi Rural 7.49% 245 4.28% 140 Montserrado Urban 10.95% 358 36.56% 1195 Table E1–7c. Unweighted and weighted number and percent distribution of selected HHs, according to urban-rural residence and district, Sierra Leone, IBTCI’s HHS 2017 Urban/Rural Residence/District Unweighted percent Unweighted number Weighted percent Weighted number Kailahun Urban 4.17% 135 3.18% 103 Kailahun Rural 7.88% 255 7.76% 251 Kenema Urban 5.44% 176 5.66% 183 Kenema Rural 6.28% 203 7.05% 228 Bombali Urban 3.49% 113 3.62% 117 Bombali Rural 9.49% 307 9.03% 292 Koinadugu Urban 2.35% 76 1.55% 50 Koinadugu Rural 9.55% 309 6.99% 226 Port Loko Urban 3.25% 105 3.34% 108 Port Loko Rural 9.03% 292 9.49% 307 Tonkolili Urban 2.38% 77 2.23% 72 Tonkolili Rural 9.95% 322 8.84% 286 Western Area Rural – Urban 13.48% 436 8.35% 270 Western Area Rural – Rural 0.99% 32 0.90% 29 Western Area Urban 12.27% 397 22.02% 712 Sampling errors Table E1–8 shows the approximate sampling errors at a 95% confidence level for the three countries. Table E1–8. Sampling errors per country, IBTCI’s HHS 2017 Country Sample size Sample error +/- at 95% confidence level Guinea 5,189 1.4 Liberia 3,297 1.7 Sierra Leone 3,235 1.7 E–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Response rate and call-backs Since the design of this survey called for the respondent to be any responsible adult in the household rather than a randomly selected respondent, the response rate was very high. Non-response after the required number of call-backs was <1%. The number of call-backs recorded was also <1%. This can be attributed to two factors. First, as with the response rate, the interviewers were more successful in getting respondents because they could speak to any responsible adult, not only one. Second, while interviewers were more likely to have to make a call-back to interview the household members responsible for preparing food or the woman with a child, this would result in the interview being paused and resumed later, not in being coded as a call-back. Health Facility Surveys Sample Size Calculations We have followed the Lot Quality Assurance Sampling (LQAS) methodology for determining the sample sizes of the health facility surveys in the three countries. The selection criteria for the health facility surveys are shown below: 1. We selected health facilities only in the Ebola Pillar II priority areas: 13 prefectures in Guinea, eight counties in Liberia, and eight districts in Sierra Leone; 2. In the priority prefectures/counties/districts, we have selected a sample that is representative for each type4 of health facility (i.e., hospital, clinic, health center, and health post—excluding pharmacies, laboratories, and veterinarians); 3. We over-sampled by 15% to account for non-response, lack of staff, or non-functioning facilities at the time of the data collection; and 4. Regarding ownership (i.e., public, private for-profit, private not￾for-profit, etc.), we collected data from any kind of facilities that are included in the sample, but the sample is not representative of a single type of ownership. 4. Country-specific. At right is a summary of the sample size calculations for all three countries (different shading for each country). Sample design Table E1–9 shows number of completed and desired health facility interviews for all three countries. The LQAS methodology used a combination of randomly and purposefully selected facilities. In cases where facilities had to be replaced, ORB randomly selected a facility in the same prefecture/county/district. Substitutions Health facilities were substituted for five reasons (see details in Table E1–10). The main reasons are listed below: 1. Facility refused to be interviewed; 2. Facility could not be located; 3. Facility was closed; 4. Interview was conducted at facility to some extent but was removed for quality control. When interviews were deleted, interviewers were encouraged to return and re-interview the same facility, unless it would cause significant inconvenience for the facility; and 5. Facility was a duplicate of another in the sample. Table E1–9. Number of completed and desired health facility interviews, IBTCI’s HHS 2017 Country Number of completed interviews Number of desired interviews Guinea 248 245 Liberia 153 152 Sierra Leone 128 128 Total 529 525 ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–11 Health Facility Sample Distribution - Guinea Sample Size Calculation_LQAS Sl. Guinea (Prefecture) HF Sample HF H HC CL HP H HC CL HP 19 Decision rule=13 1 Boffa 55 19 1 10 2 42 1 3 1 14 22 22 (with 15% non-response) 2 Boké Prefecture 94 19 2 15 41 36 1 3 7 8 22 × 13 Actual Alpha= 0.0676 3 Faranah Prefecture 48 19 1 13 0 34 1 6 0 12 22 Total= 247 Actual Beta= 0.0835 4 Kankan Prefecture 62 19 1 18 0 43 1 6 0 12 22 5 Forécariah 56 19 1 10 5 41 1 3 2 13 22 6 Kindia Prefecture 67 19 3 14 6 44 1 4 2 12 22 7 Labé Prefecture 57 19 1 18 4 34 1 6 2 10 22 8 Beyla 48 19 2 15 0 31 1 6 0 12 22 9 Guéckédou 72 19 2 13 4 53 1 3 1 14 22 10 Lola 49 19 1 9 0 39 1 4 0 14 22 11 Macenta 86 19 2 17 0 67 1 4 0 14 22 12 Nzérékoré Prefecture 90 19 2 18 11 59 1 4 3 11 22 13 Conakry 337 19 26 42 244 25 2 2 14 1 22 Sub-total 1121 247 45 212 317 548 13 54 31 148 176 Type of HF Sample (without NS) *With 15% NS H=Hospital, HC=Health Center, Cl=Clinic, HP=Health Post Health Facility Sample Distribution - Liberia Sl. Liberia (County) HF Sample HF Sample Size Calculation_LQAS H HC CL HP H HC CL HP 1 Bomi 24 19 1 0 23 0 1 0 18 0 22 19 Decision rule=13 2 Bong 52 19 3 0 49 0 1 0 18 0 22 22 (with 15% non-response) 3 Grand Bassa 31 19 3 1 27 0 1 1 17 0 22 × 8 Actual Alpha= 0.0676 4 Grand Cape Mount 33 19 1 2 30 0 1 1 17 0 22 Total= 152 Actual Beta= 0.0835 5 Lofa 59 19 4 2 53 0 1 1 17 0 22 6 Margibi 44 19 1 11 32 0 1 3 15 0 22 7 Montserrado 315 19 10 21 284 0 2 4 13 0 22 8 Nimba 75 19 4 6 65 0 1 1 17 0 22 Sub-total 633 152 27 43 563 0 9 11 132 0 176 Type of HF Sample (without NS) *With 15% NS H=Hospital, HC=Health Center, Cl=Clinic, HP=Health Post Health Facility Sample Distribution - Sierra Leone Sl. Sierra Leone (Districts) HF Sample HF H HC CL HP H HC CL HP Sample Size Calculation_LQAS 1 Kailahun 81 19 3 19 2 57 1 5 0 13 22 2 Kenema 145 19 6 34 6 99 1 4 1 13 22 19 Decision rule=13 3 Bombali 128 19 8 23 5 92 1 3 1 14 22 22 (with 15% non-response) 4 Koinadugu 101 19 3 15 9 74 1 3 1 14 22 × 8 Actual Alpha= 0.0676 5 Port Loko 129 19 8 24 3 94 1 4 1 13 22 Total= 152 Actual Beta= 0.0835 6 Tonkolili 133 19 3 16 5 109 1 2 1 14 22 7 Western Rural 58 19 5 10 10 33 2 3 3 11 22 8 Western Urban 176 19 54 39 57 26 6 4 6 3 22 Sub-total 951 128 90 180 97 584 13 29 14 95 176 Type of HF Sample (without NS) *With 15% NS H=Hospital, HC=Health Center, Cl=Clinic, HP=Health Post Table E1–10. Number of health facility substitutions by country and reason, IBTCI’s HFSs 2017 Reason for substitution Sierra Leone Liberia Guinea Total Refusal 3 5 11 19 Unable to locate 5 3 9 17 Closed 5 1 18 24 Quality Control 1 23 24 48 Duplicate 2 0 0 2 Total 18 32 62 112 Table E1–9. Number of completed and desired health facility interviews, IBTCI’s HHS 2017 Country Number of completed interviews Number of desired interviews Guinea 248 245 Liberia 153 152 Sierra Leone 128 128 Total 529 525 E–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES DATA COLLECTION IMPLEMENTATION IBTCI subcontracted ORB International (ORB) and StatView International (SVI) to carry out the data collection activities. ORB conducted the qualitative data collection in Liberia and Sierra Leone and the household and health facility surveys in Guinea, Liberia, and Sierra Leone. IBTCI staff held a training of trainers in the Home Office for representatives of ORB on both quantitative and qualitative instruments and methodology in October 2017. Overall project management for the survey was executed by Yuliya Dudaronak, Associate Director at ORB. She was assisted by Elizabeth Bailey and Christine Delmeiren. SVI conducted the qualitative data collection in Guinea. The project was overseen by Aliou Barry, Senior Evaluation Advisor. This section describes details of the sampling design, questionnaire pre-testing, data collection and data processing. HOUSEHOLD SURVEYS (HHSs) Face-to-face interviews were conducted from December 14, 2017 to January 26, 2018. A total of 11,721 interviews were achieved across the three countries, resulting in margins of error of between +/- 1.4–1.7% calculated at a 95% confidence interval. As mentioned above, households were selected from the Pillar II priority areas. The data have been weighted based upon area (prefecture/county/ district) and urban/rural residence, to accurately depict the population distribution across the sample universe. Data Collection Teams The local data collection teams were selected from a pool of experienced supervisors and interviewers. All interviewers and supervisors received training on the questionnaire, survey methods, and tablets and conducted two pilot interviews before beginning fieldwork. The responsibilities of the interviewers were as follows: ■ Actively participate in interviewer training; ■ Implement fieldwork as directed, employing correct household and respondent selection procedures, and accurately and fully completing each questionnaire; and ■ Ensure that proper tablet procedures were followed. The responsibilities of the supervisors and/or team leaders were as follows: ■ Actively participate in interviewer training; ■ Guide and motivate the interviewing team; ■ Check, select, and accompany field teams to assigned starting points each day; ■ Randomly back-check interviews; ■ Accompany interviews, ensuring that the interviewers adhered to respondent selection and interviewing procedures; ■ Debrief team and report issues to the Project Manager; and ■ Ensure data from tablets were uploaded at the end of each day. Questionnaire development and translation One challenging part of conducting IBTCI’s 2017 HHSs was the issue of dealing with a diverse set of languages in each country, while the survey questions need to be asked in a uniform manner. As is standard practice, a “master” questionnaire (comprehensive for all three countries) was first created in English. This was then translated into a French master version, as French is the official language of Guinea. The English master was translated into Liberian English and Krio, while the French master was translated into Sousou, Poular, and Malinke. In Liberia and Sierra Leone, large sectors of the population speak vernacular languages. While ORB worked extensively with the translated written surveys during training, interviewers remained uncomfortable working with Liberian English and Krio written translations. As these languages are often spoken but not written, ORB prepared interviewers to conduct a simultaneous translation from English into the vernacular while in the field. In order to ensure accurate language comprehension and standardization among interviewers, ORB devoted training time for interviewers to debate and agree upon the most appropriate terms in the vernacular. Table E1–11 shows where a written version was prepared and where verbal translation was conducted in certain languages. Table E1–11. Field Questionnaires: Written and Verbal Country Written field questionnaires Verbal simultaneous translations Sierra Leone English Krio Liberia English Liberian English Guinea French, Sousou, Malinke, Poular ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–13 Samsung Galaxy Tab A Android tablets were supplied to the local team for data collection. The SurveyToGo Android application was used for data collection in all countries. The ORB project manager oversaw the scripting process from the scripting of the English questionnaire to the translations. Each iteration of the script was tested multiple times to ensure no errors were present. The final script was downloaded onto all tablets in all countries so that all teams were using the same version of the questionnaire. Training and supervision Interviewers were recruited by ensuring that they have the appropriate educational level and experience. Training was provided for interviewers and supervisors in each of the countries in late November to early December 2017. The training sessions served to re-familiarize the interviewers with quantitative research and to explain the scope of this study and the specific requirements. They also provided ORB with the opportunity to assess each interviewer and ensure that each local team was sufficiently briefed and competent to complete the work required of them. At least one ORB executive was present at each interviewer training session. The training sessions were conducted in English (Liberia and Sierra Leone) and French (Guinea). Table E1–12 shows the training dates by country along with the ORB trainer who oversaw the training. Training venues were located in the capital cities of each country, in private classroom-style rooms. Each team was provided with copies of all training materials, including printed copies of the questionnaire. Each day generally included one morning break and a lunch period, as well as prayer times where customary. The training sought to accomplish a number of objectives, including but not limited to: ■ Introduction to the project and recap on the essence of quantitative research; ■ Question-by-question review on paper and on tablets; ■ Proper tablet procedures including uploading data, protecting the tablets from damage, etc.; ■ Explanation of the respondent selection procedures, including skip patterns and substitution procedures; ■ Explanation of the quality control procedures; and ■ Discussion on how to deal with any foreseeable challenges in the field. Once training was completed, interviewers conducted the pilot survey in the surrounding area; the pilot served to identify any potential problems in the questionnaire or translation, as well as indicate an interviewer’s skill level in accurately completing the questionnaire. Overall, the training consisted of several parts, including the following. Introduction to the Survey and Question-by-Question Review Training typically began with self-introductions and an introduction to the study itself—its objectives and context—as well as an explanation of the role and responsibilities of the interviewer in the research process. Following this, teams were introduced to the questionnaire and proceeded to spend most of the day in a thorough review, walk￾through, and rehearsal of the instrument. As a group, the team reviewed the questionnaire, question by question, and answered any interviewer queries. The questionnaire often prompted debate amongst the interviewing teams, which implied they were engaging fully with the survey and had a real interest in it. While going through the questionnaire, any small problems or routing issues that arose were addressed and noted for other countries. The intent of certain questions, ways to probe for answers, and routing/ filtering were all addressed during this exercise. Mock Interviews Once the question-by-question review of the survey was complete, interviewers paired up and practiced the questionnaire with each other. This was a time-consuming process, as each individual played the alternating role of interviewer/interviewee, but it helped to further familiarize them with the questionnaire and its implementation. The mock interviewing also highlighted areas where interviewers needed to pay greater attention, such as whether a question required a single or multi-coded response, something that was checked as the ORB staff walked around the room monitoring the progress of interviewers. Table E1–12. Training dates by country, IBTCI’s HHSs 2017 Country Training Dates Pilot Date ORB Trainer Liberia November 27 – December 1 November 29 Yuliya Dudaronak / Elizabeth Bailey / Christine Delmeiren Sierra Leone December 4 – December 7 December 6 Yuliya Dudaronak / Elizabeth Bailey Guinea December 4 – December 7 December 6 Christine Delmeiren E–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Review of Household Selection Procedure Once a full review and practice of the survey instrument was complete, time was devoted in training to household and respondent selection techniques and procedures. General household selection was covered, such as the use of the day code to determine a starting point, use of proper skip patterns in urban and rural locations, and procedures to follow in multi-household dwellings and apartment buildings. Interviewing Technique and Quality Control Training on interview techniques included a reminder of the local etiquette required to secure an interview, the need to follow interviewer instructions exactly as they are prescribed, the need to remain neutral during the interview process, and the skills required to probe effectively for answers during an interview. This training session included a discussion of the ideal interview environment by providing good, bad, and ideal examples of the environment and conditions in which an interview would be conducted. In reviewing quality control procedures, specific requirements for both interviewers and supervisors were covered. Although the specific supervisory requirements were reviewed with the supervisors separately as well, reviewing them with the group as a whole helped the interviewers become cognizant of just how closely they would be watched. Tablet Procedures A segment of the training was dedicated to practicing implementing the survey on tablets. The SurveyToGo software was used in all countries. Proper tablet procedures were covered, including how to use SurveyToGo, how to upload data, and protecting the tablets from damage. Pilot Testing Once training in survey methodology and the questionnaire was complete, teams of interviewers were grouped together and sent to different districts of the city where interview training had taken place. Each interviewer was required to complete two pilot interviews. Once all pilot interviews were complete, the field teams returned to the training facility for an evaluation and debrief. The data was then uploaded to the SurveyToGo servers, which was immediately accessible by ORB executives to download. The ORB executive present at each training downloaded and offered feedback on the spot to the field team. This included displaying the GPS coordinates of each interview and listening to all audio recordings, as well as playing back examples of good and bad audios to the team. ORB submitted a pilot review report for Liberia to IBTCI on December 6, and for Sierra Leone and Guinea on December 15. Composition of Field Teams Each interviewer team had a ratio of one supervisor to four or five interviewers. The supervisor’s role was to be a point of reference for the interviewers. They were present and actively involved in the full training sessions along with the interviewers and it was their responsibility to be completely familiar with the questionnaire and the sampling plan. HEALTH FACILITY SURVEYS (HFSs) Face-to-face interviews were conducted from December 18th, 2017 to February 8th, 2018. A total of 529 interviews were achieved across the 3 countries. All interviewers were fully trained, local citizens, with experience in the medical field or in administering medical surveys. Interviewer and fieldwork team selection Local teams for this project were selected from a pool of experienced supervisors and interviewers. All interviewers and supervisors received training on the questionnaire, survey methods, and tablets and conducted two pilot interviews before beginning fieldwork. Interviewers and supervisors in Liberia were selected from those who worked on the medical portion of the Ebola Pillar I evaluation; a large number of these interviewers went through training, from which the best interviewers were selected. In Sierra Leone and Guinea, interviewers with experience in the medical field were recruited. The responsibilities of the interviewers were as follows: ■ Actively participate in interviewer training; ■ Implement fieldwork as directed, employ correct health facility and respondent selection procedures, and accurately and fully complete each questionnaire; and ■ Ensure that proper tablet procedures were followed. The responsibilities of the supervisors and/or team leaders were as follows ■ Actively participate in interviewer training; ■ Guide and motivate the interviewing team; ■ Check, select, and accompany field teams to assigned starting points each day; ■ Randomly back-check interviews; ■ Accompany interviews, ensuring that the interviewers adhered to respondent selection and interviewing procedures; ■ Debrief team and report issues to the Project Manager; and ■ Ensure data from tablets were uploaded at the end of each day. Questionnaire development and translation One challenging part of conducting this particular survey is the issue of dealing with a diverse set of languages in each country where the survey questions need to be asked in a uniform manner. Again, as is standard practice, first, a “master” questionnaire ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–15 (comprehensive for all three countries) was created in English. This was then translated into a French Master version, as French is the official language of Guinea. Samsung Galaxy Tab A Android tablets were supplied to the local team for data collection. The SurveyToGo Android application was used for data collection in all countries. The ORB project manager oversaw the entire scripting process from the scripting of the English questionnaire to the translations. Each iteration of the script was tested multiple times to ensure no errors were present. The final script was downloaded onto all tablets in all countries so that all teams were using the same version of the questionnaire. Training and supervision Interviewers were recruited by ensuring that they have the appropriate educational level and experience. Training was provided for interviewers and supervisors in each of the countries in late November to early December 2017. The training sessions served to re-familiarize the interviewers with quantitative research and to explain the scope of this study and the specific requirements. They also provided ORB with the opportunity to assess each interviewer and ensure that each local team was sufficiently briefed and competent to complete the work required of them. At least one ORB executive was present at each interviewer training session. The training sessions were conducted in English (Liberia and Sierra Leone) and French (Guinea). Table E1–13 above shows the training dates by country along with the ORB trainer who oversaw the training. Training venues were located in the capital cities of each country, in private classroom-style rooms. Each team was provided with copies of all training materials, including printed copies of the questionnaire. Each day generally included one morning break and a lunch period, as well as prayer times where customary. The training sought to accomplish a number of objectives, including but not limited to: ■ Introduction to the project and recap on the essence of quantitative research; ■ Question by question review on paper and on tablets; ■ Proper tablet procedures including uploading data, protecting the tablets from damage, etc.; ■ Explanation of the respondent selection procedures, including skip patterns and substitution procedures; ■ Explanation of the quality control procedures; and ■ Discussion on how to deal with any foreseeable challenges in the field. After the completion of training, interviewers conducted the pilot survey in the surrounding area. The pilot served to identify any potential problems in the questionnaire or translation as well as indicate an interviewer’s skill level in accurately completing the questionnaire. Overall, the training consisted of several parts, including the following. Introduction to the Survey and Question-by-Question Review Training typically began with self-introductions and an introduction to the study itself – its objectives and context as well as an explanation of the role and responsibilities of the interviewer in the research process. Following this, our teams were introduced to the questionnaire and proceeded to spend most of the day in a thorough review, walk-through, and rehearsal of the instrument. As a group we reviewed the questionnaire, question-by-question, and answered any interviewer queries. The questionnaire often prompted debate amongst the interviewing teams that implied they were engaging fully with the survey and had a real interest in it. While going through the questionnaire, any small problems or routing issues that arose were addressed and noted for other countries. The intent of certain questions, ways to probe for answers, and routing/ filtering were all addressed during this exercise. Mock Interviews Once the question-by-question review of the survey was complete, interviewers paired up and practiced the questionnaire with each other. This was a time-consuming process, as each individual played the alternating role of interviewer/interviewee, but it helped to further familiarize them with the questionnaire and its implementation. The mock interviewing also highlighted Table E1–13. Training dates by country, IBTCI’s HFSs 2017 Country Training Dates Pilot Date ORB Trainer Liberia November 30 – December 2 December 2 Yuliya Dudaronak / Elizabeth Bailey / Christine Delmeiren Sierra Leone December 7 – December 9 December 11 Yuliya Dudaronak / Elizabeth Bailey Guinea December 7 – December 12 December 11 Christine Delmeiren E–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES areas where interviewers needed to pay greater attention, such as whether a question required a single or multi-coded response, as mentioned on page 13. Review of Medical Terms and Equipment We devoted significant time in training to a review of the medical equipment covered in the survey and to proper pronunciation of medical terms. This was to ensure that interviewers are comfortable distinguishing the equipment they would be asked to observe and that they would be well-understood by respondents. A review of the meaning of important terms was also done to allow for accurate explanation and oral translation. Interviewing Technique and Quality Control Training on interview techniques included a reminder of the local etiquette required to secure an interview, the need to follow interviewer instructions exactly as they are prescribed, the need to remain neutral during the interview process, and the skills required to probe effectively for answers during an interview. This training session included a discussion of the ideal interview environment by providing good, bad, and ideal examples of the environment and conditions in which an interview would be conducted. In reviewing quality control procedures, specific requirements for both interviewers and supervisors was covered. Although the supervisor specific requirements were reviewed with the supervisors separately as well, reviewing them with the group as a whole ensured the interviewers were cognizant of just how closely they would be watched. Tablet Procedures A segment of the training was dedicated to practicing implementing the survey on tablets. The SurveyToGo software was used in all countries. Proper tablet procedures were covered including how to use SurveyToGo, how to upload data and protecting the tablets from damage. Pilot Testing Once training in survey methodology and the questionnaire was complete, teams of interviewers were grouped together and sent to different facilities in the city in which interview training had taken place. Each group conducted one pilot interview. The purpose of this pilot was to observe any issues with the questionnaire and the preparedness of interviewers. Once all pilot interviews were complete, the field teams returned to the training facility for an evaluation and debrief. The data was then uploaded to the SurveyToGo servers which was immediately accessible by ORB executives to download. The ORB executive present at each training downloaded and offered feedback on the spot to the field team. This included displaying the GPS coordinates of each interview and listening to all audio recordings, as well as playing back examples of good and bad audios to the team. ORB submitted a pilot review report for Liberia to IBTCI on December 6, 2017 and for Sierra Leone and Guinea on December 15, 2017. Field Team Composition Each interviewer team had a ratio of one supervisor to four or five interviewers. The supervisor’s role was to be a point of reference for the interviewers. They were present and actively involved in the full training sessions along with the interviewers and it was their responsibility to be completely familiar with the questionnaire. QUALITATIVE DATA COLLECTION The qualitative data collection was carried out between September 2017 and early February 2018. Each interview was conducted by a moderator with previous interviewing experience and trained by ORB (Liberia and Sierra Leone) or StatView (Guinea). A full list of the KIIs and FGDs and their locations can be found in Annex I. Interviewer selection Local moderators for this project were selected from a pool of experienced supervisors and interviewers. All interviewers and supervisors received training on the instruments, moderation techniques, and qualitative methods. The responsibilities of the moderators were as follows: ■ Actively participate in training; ■ Implement fieldwork as directed, employ interviewing methods and procedures, and accurately and fully complete each interview; and ■ Ensure that notes and transcripts are accurate and complete. Questionnaire development and translation As mentioned prior, master instruments were created in English (comprehensive for both Liberia and Sierra Leone). Then these instruments were translated to French to create the French Master for Guinea. In the three countries, large sectors of the population speak vernacular languages. As these languages are often spoken but not written, the evaluation team prepared interviewers to conduct a simultaneous translation from English or French into the vernacular while in the field. In order to ensure accurate language comprehension and standardization among interviewers, we devoted training time for interviewers to debate and agree upon the most appropriate terms in the vernacular. Training and supervision Interviewers were recruited by ensuring that they have the appropriate educational level and experience. Training was provided for moderators in each of the countries from early October to early December 2017. The training sessions served to re-familiarize the interviewers with qualitative research and to ANNEX E. DATA COLLECTION METHODOLOGY: PART 1 n E–17 explain the scope of this study and the specific requirements. They also provided ORB and StatView with the opportunity to assess each interviewer and ensure that each local team was sufficiently briefed and competent to complete the work required of them. At least one ORB or StatView executive was present at each interviewer training session. The training sessions were conducted in English in Liberia and Sierra Leone and in French in Guinea. Training venues were located in the capital cities of each country, in private classroom-style rooms. Each team was provided with copies of all training materials, including printed copies of all interview guides. Each day generally included one morning break and a lunch period. The training sought to accomplish a number of objectives, including but not limited to: ■ Introduction to the project and recap on the essence of qualitative research; ■ Question by question review of guides; and ■ Discussion on how to deal with any foreseeable challenges in the field. Introduction to the Instruments and Question-by￾Question Review Training typically began with self-introductions and an introduction to the study itself—its objectives and context—as well as an explanation of the role and responsibilities of the interviewer in the research process. After this, teams were introduced to the instruments and proceeded to spend most of the day in a thorough review, walk￾through, and rehearsal of the instrument. As a group teams reviewed all guides, question-by-question, and answered any interviewer queries. The guide often prompted debate amongst the interviewing teams, which implied they were engaging fully with the survey and had a real interest in it. The intent of certain questions, ways to probe for answers, and routing/filtering were all addressed during this exercise. Mock Interviews Once the question-by-question review of the guides was complete, interviewers paired up and practiced the questionnaire with each other. This was a time-consuming process, as each individual played the alternating role of interviewer/interviewee, but it helped to further familiarize them with the questionnaire and its implementation. The mock interviewing also highlighted areas where interviewers needed to pay greater attention—something that the ORB/StatView staff observed as they walked around the room checking on the progress of interviewers. Interviewing Technique and Quality Control Training on interview techniques included a reminder of the local etiquette required to secure an interview, the need to follow interviewer instructions exactly as they are prescribed, the need to remain neutral during the interview process, and the skills required to probe effectively for answers during an interview. This training session included a discussion of the ideal interview environment by providing good, bad, and ideal examples of the environment and conditions in which an interview would be conducted. E–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES PART 2. DATA ANALYSIS AND DATA COLLECTION QUALITY ASSURANCE PLANS DATA ANALYSIS PLAN INTRODUCTION This Data Analysis Plan (DAP) has been developed for a macro￾level performance evaluation of USAID’s surge support for Pillar II activities in the three Ebola-affected countries, to be conducted in 2017 and 2019. Performance Evaluation (Component 1) is one of four components of the Ebola Pillar II Monitoring, Evaluation, and Learning (MEL) project. The evaluation component of the USAID/ Pillar II MEL project aims to bring diverse Pillar II activities and in￾dicators into a single evaluation frame with multiple phases (phase 1 and phase 2) and, where possible, measure their joint contributions to Pillar II’s three over-arching objectives: 1) preventing the loss of development gains, 2) recovering and strengthening existing institu￾tions and structures, and 3) building more resilient and sustainable systems to avert and handle future health crises, while defining progress in terms of the outcomes listed in Pillar II M&E frame￾work (final report, Figure 2, page 11). Details of the evaluation’s purpose, phases, and methodology are described in the inception report. This plan describes IBTCI’s approach for evaluating Pillar II activities under four analytic domains: Performance; Sustainability; Gaps and Opportunities; and Management, which correspond to the four evaluation questions. This DAP aims to be a “living” document, meaning that it will be updated as additional data analysis tasks are identified based on: a) the four major domains; b) relevant interest among key stakeholders; c) unanticipated findings emerging from initial data analysis; and d) a desk review of IP documents and literature. The data analysis approach, statistical methods or tasks could be modified or changed as data collection progresses and the evaluation team gains more insight after reviewing primary and secondary data. Purpose of the DAP The DAP’s primary audience are the evaluation team members responsible for data collection, data analysis, triangulation and report writing. This plan should be considered as an expanded version of the evaluation data analysis plan laid out in the methodology of the Inception Report (IR) (latest version, dated September 29, 2017) for this evaluation. The DAP and inception report are meant to be read together. The plan focuses on the analysis of the quantitative and qualitative data obtained from various data sources. The following sub-sections describe in detail the analysis plan for each evaluation question, sub-questions, and their corresponding analytical domains. The general approaches are described below. They primarily focus on quantitative and qualitative data analysis obtained from the quantitative surveys and qualitative interviews. DESCRIPTION OF THE INTERVENTION TYPOLOGY The Intervention Typology (IT) encompasses all the interventions described on activity documents and categorizes them according to thematic area (or sector), and also by an intervention type (or theme). All the interventions under the rubric of the intervention typology will be analyzed to answer evaluation question 1 on performance: How are USAID’s Pillar II Ebola Recover activities in Guinea, Liberia and Sierra Leone contributing to the achievement of the Ebola Pillar II Strategic Framework’s three objectives: to halt the loss of development gains; recover and strengthen key institutions and infrastructures; and build sustained systems through PPPs, innovation and capacity building? The thematic areas determined by the Pillar II Theory of Change (ToC) (final report, Figure 3, page 13) and outcome indicators are listed Table E2–1. See Figure E2–1 for an overview and brief description of the intervention types. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–19 Table E2–1. Key outcome indicators by thematic area Pillar II Thematic Area Pillar II Outcome Indicators Health: Non-EVD Health Services, Health Systems Recovery, and Survivor Programs Basic health service delivery restored Healthcare worker (HCW) capacity to deliver EVD and non-EVD health services increased Comprehensive services developed and provided for EVD survivors Health behavior gains enhanced and reinforced Health infrastructure (including management systems) strengthened Health information system developed and supported Health financing and resource allocation improved Basic Education School attendance restored and gender balance improved Literacy and numeracy improved Agriculture and Food Security Household food security increased Nutritional status improved Household standards of living improved Governance Enabling government policies developed, and harmful policies and practices revised Civil Society Organizations (CSOs) strengthened to conduct effective advocacy, transparency and accountability activities Management systems and accountability of ministries and stakeholders improved at all levels Economic Crisis Mitigation Agriculture and import markets’ function restored and improved Social protection provided for EVD-affected groups Innovation, Technology, and Partnerships* Information and Communication Technology (ICT) for health, gender equity and civic participation improved Power, water, roads and communication infrastructure restored and expanded *Please note that Information Communication Technology (ICT) and public-private partnerships (PPPs) are components of ITP. Gender-sensitive economic strengthening for vulnerable groups and non-economic strengthening support Social Protection Interventions to enhance worker motivation, safety, and performance Frontline Worker Support Policy development, leadership and governance interventions, including organization development as well as partnerships and coordination among stakeholders Management, Coordination, and Partnerships Strengthening information systems and applying technological innovations to promote the use of data for decision-making Information, Communication, and Technology Use of diverse communication methods (education, media, community dialogue, etc.) to change knowledge, attitudes, and norms toward the intervention’s objectives Social and Behavior Change Communication Upgrading or refurbishment of physical infrastructure and provision of commodities, equipment, and supplies Institutional Enhancements Figure E2–1. Overview of the six intervention types for analysis E–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES GENERAL APPROACH FOR QUANTITATIVE DATA ANALYSIS Approach for Quantitative Surveys: Household, Health Facility and School Surveys IBTCI will conduct two types of analyses: (1) Descriptive Analysis and (2) Test of Significance Analysis. These analyses will logically proceed from simple (uni-variate) analyses to complex (multi￾variate) analyses. This will allow us to understand the data better, and to make decisions about the next steps for each type of analysis. ■ Descriptive Analysis: We will conduct uni-variate and initial bi-variate analyses to describe, or summarize data in such a way that, for example, patterns might emerge from the data. Some examples of descriptive statistics include percentages, frequencies, ranges, modes, medians, and means. We will use the results of uni- and bi-variate analyses to select methods and variables to further examine in the multivariable analysis(es). ■ Test of Significance Analysis: After describing the characteristics of a study population and its sub-groups, the next step is to consider whether characteristics differ by a statistically significant margin between groups. The bi-variate and multi-variate analysis methods will be used to assess these characteristics according to the type of variable analyzed and the hypothesis tested. Statistical tests will be used to assess associations between variables of interest, and to examine a statistically significant association between outcome and independent variables. Statistical tests include t-tests for continuous data, chi-square (X²) tests for categorical data, ANOVA for assessing a continuous variable within categories (linear and logistic), least square regressions to examine relationships between continuous independent variables and categorical dependent variables, and a correlation coefficient to assess the association between two continuous variables. We will create table shells to help prepare for the following analysis activities: ■ conducting descriptive analysis ■ calculating confidence intervals ■ conducting statistical testing ■ assessing potential confounding ■ conducting multivariable analysis IBTCI will use the table shells described in the analysis plan to analyze the data. However, we will remain flexible in case we decide to group categories differently or pursue interesting, unanticipated findings that help address the hypotheses being tested. GENERAL APPROACH FOR QUALITATIVE DATA ANALYSIS Approach for Qualitative Data: Interviews, discussions and document review From the Phase II data collection, IBTCI has a good understanding of the Pillar II activities and therefore have already created very broad headings for organizing the qualitative data transcripts into different domains. This understanding has helped to create KIIs and FGDs/round-table guides. The team has incorporated definitions of the domains and the independent, dependent, and mediating variables that factor in each activity￾level ToC and beneficiary group into the KII/FGD/round-table guides. A coding plan has been developed to provide a framework that will structure, label, and define the data. The framework is explanatory and is guided by the evaluation questions. Atlas.ti™ software will be used to organize the qualitative data, assigning the coding and analysis. Data coding based on the coding plan will identify patterns, themes, identify relationships between themes, and attempt to find explanations from the data. After themes, patterns, connections, and relationships have been identified, the team will conduct meetings with evaluation team members to discuss data triangulation and attach meaning and significance to the findings. During this process, it will develop lists of key ideas, create diagrams, and/or use models to explain the findings. IBTCI will use a variety of approaches for this process, including: ■ Thematic Analysis: Involves the search for and identification of common threads that extend across an entire interview or set of interviews. ■ Content Analysis: Used to analyze and interpret document content. Content can be analyzed descriptively or interpretatively. It is a systematic approach used to code and categorize large amounts of textual information unobtrusively so as to determine trends and patterns of words or phrases used, their frequency, their relationships, and the structures and discourses of communication. ■ Narrative Analysis: Used to analyze text that may come from a variety of sources, including transcripts from interviews, field notes, and other written documents. Narrative analysis often involves reformulating stories presented by people in different contexts, and based on their different experiences. ■ Grounded Theory: (Also called analytic induction.) This is a method that attempts to develop ‘most likely association’ explanation of a phenomenon from one or more cases being studied. Explanations are altered as additional cases are studied until the researcher arrives at a statement that fits all cases. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–21 ■ Contribution Analysis: An analytic approach used for determining a complex, multifaceted program’s effectiveness in a complex setting (i.e., one with multiple activity components, multiple levels of funders (from global to local), a wide array of actors and providers, and varying socio-political contextual factors). Contribution analysis will be used to mitigate the limited ability to attribute outcomes to individual activities due to presence of multiple actors and programs. More details are provided under Evaluation Question (EQ) 1.4. Analytic Frameworks Linkage between Intervention Types/Themes and the Achievement of Ebola Pillar II Outcomes To conduct the thematic analysis for evaluation question 1 (performance), we have plotted the 19 outcomes as delineated in the Ebola Pillar II ToC. Note that this theory of change, while constructed retrospectively, is an iteration of the AEU Results Framework in situ as of March 2015. In Figure E1–2 below, the outcomes are plotted according to the intervention themes and color-coded according to the sector covered (food security, etc., per the legend.) Analytical framework to measure the sustainability of Ebola Pillar II interventions This intervention typology will also be applied to the analysis for evaluation question 2, which is focused on sustainability: What results of Pillar II activities have endured after the activities have closed, and why? The initial presentation of the findings to answer this sustainability question will be included in the PE-1 report, but the final analysis will be conducted under PE-2 in 2019, once the majority of the activities have closed out. For the second evaluation question focused on sustainability, IBTCI has created a conceptual framework to demonstrate the interrelationships between proximate and distal determinants, and each of the intervention types. The evaluation team will describe a sustainability pathway for promising intervention types, as well as those with less of a chance of making a mark to examine the missing elements (in other words, which determinants were not in place, which might explain why an intervention is not being continued or addressed in some way after the activity was closed out). Just as there is a need to aggregate similar components under one rubric, these determinants are submitted under one or more of these categories: policy, finance, accountability, institutional support, technical choices, human dimension, external threats and enabling environments. Figure E2–2. Linkage between the Intervention Types/Themes & the AEU Outcomes The top row of this figure describes the six intervention types in gray. Below each intervention type is the list of corresponding outcome indicators. IBTCI plotted the 19 Pillar II outcome indicators identified in the USAID Pillar II M&E framework and Ebola Pillar II theory of change according to the relevant intervention type, and color coded them according to the thematic area (legend appears lower right). Social Protection Frontline Worker Support MCP ICT SBCC Institutional Enhancements Social protection provided Health workers’ capacity to deliver EVD and non-EVD services increased Enabling governance policies developed & harmful policies and practices revised Health information system developed & supported Health behavior gains enhanced and reinforced School attendance restored & gender balance improved Specialty services provided to EVD survivors in specific health facilities Basic health service delivery restored HH standard of living improved Management sys￾tems & accountabil￾ity levels, ministries & stakeholders improved ICT for health sys￾tems developed Health infrastructure and community involvement strengthened HH food security increased Power, roads and communication infra￾structure expanded Nutritional status improved Health financing & resource allocation improved Literacy & numeracy improved Ag & import market function restored & improved CSOs strength￾ened to provide effective advocacy, transparency, and accountability Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Non-EVD Health Services, Basic Education Health Systems Recovery & Survivor Programs LEGEND E–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Based on the overarching sustainability question and a review of the literature as to the determinants of sustainability, IBTCI first developed a conceptual framework for measuring sustainability. This framework was vetted at a validation workshop at the 2017 USAID Mini-University (see Figure E1–3). Further development of the model led to its operationalization and its current implementation as the evaluation team members analyze the various data sets—both quantitative and qualitative. As noted, IBTCI will depict case studies of sustainable intervention types using the intervention typology as a basis for inputs on the outcomes and using mostly qualitative data to formulate the existence (or lack) of determinants. This analysis will include the level of the systems where the intervention(s) have been focused—institutional, community, or individual. To achieve this goal, we first needed to define each of the determinant categories so that all data are analyzed according to the same rubric. DETAILED ANALYSIS PLAN BY EVALUATION QUESTION See Tables E2–2a, 2b, 3, 4 and 5 on the following pages. Figure E2–3. Analytical framework to measure sustainability Sustained Results Outbreak Thematic Areas of Response Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology Institutional Enhancements Social & Behavior Change Communications Intervention Types Policies, Strategies, and Plans Finances External Threats and Enabling Environments Human Dimensions Determinants of Sustainability Accountability Institutional Support Technical Choices Immediate Results Ebola Pillar II Response Challenge: Mitigate second-order impacts of the outbreak at the individual, community, and institutional level Survivor Programs ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–23 Table E2–2a. Data Analysis Plan for Evaluation Question One: Performance Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 1.1: What activities have been implemented with Pillar II funds in each country at the end of the period of performance? What are the activities used in each thematic area? Type of activities supported by Pillar II funds, ended on or before Sept 2017 (for 2017 report), by thematic areas and country. IBTCI will disaggregate the activities according to implementation maturity: those that have already closed out, those ongoing now, and those which have not yet started as of September 2017. IBTCI will group activities by development sectors and intervention types (for example￾Social Protection, SBCC etc.) Type of activities supported by Pillar II funds, ended on or before Sept 2019 (for 2019 report), by thematic areas, by country. IP Statement of Work, IP reports and USAID Pillar II strategy, Country Strategies (CDCS) document review, KII IP staff (Question (Q) 100 Performance (P) 1, 101-107. P1); KII United States Government (USG) (Q100 P1, 101-107. P1); KII Govt. (Q100 P1, 101-107. P1). Narrative and graphical descriptive analysis with comparisons with respect to timing, scale (in terms of number, types of activity, target population, or geographic location), and funding in each country (e.g., number and type of activities supported by USAID, by IPs, by themes, location, population or area coverage, timeline, start and end dates, and duration of support). Data will be disaggregated by prefecture/ county/district of implementation and type of activity (six thematic areas: Agriculture and Food Security; Health; Basic Education; Governance, Economic Crisis Mitigation; and Innovation, Technology and Partnerships, and six intervention types: Social Protection; Frontline Worker Support; Management, Coordination and Partnerships; ICT; SBCC; Institutional Enhancements). EQ 1.2: To what extent are these Pillar II activities on track to achieve their objectives and targets? Number of IPs who achieved objectives and targets/Number of IPs whose activities ended on or before Sept 2017 (for 2017 report) (%); Number of IPs on track to achieve objectives and targets/ Number of IPs whose activities will continue beyond September 2017 (for 2017 report) (%); Number of IPs who achieved objectives and targets/Number of IPs whose activities ended on or before Sept 2019 (for 2019 report) (%). IP routine monitoring data, IP report review, KII among IP staff (Q114-115. P1), KII USG- (Q114- 115. P1) KII Govt- (Q114-115. P1). Narrative and tabular analysis of the extent of achievements of M&E outcome indicators for IPs in each country. Outcome of each of Pillar II IP’s “main activities” in relation to ToC described in each IP’s reports and documents or reported by IP KII. Proposed criteria for defining “main activities” are: 1) geographic coverage includes four or more USAID priority areas or national scope; 2) implementation spanned at least 18 months (not including close out); 3) budget across all IPs represented at least 20% of the allocation of Pillar II funds for the thematic area. N.B.: A major limitation to the feasibility of a “triangulated” analysis of data collected by the data sources mentioned (including documents from the IPs) is the limited availability, accessibility, and the lack of consistent, comprehensive reporting of such data. Extent of achievement of outcomes indicators categorized as achieved, partially achieved, not achieved compared with baseline values, yearly targets, and endline targets (to the extent where all these data points are available); Data will be disaggregated by prefecture/ county/district of implementation and type of activity (six thematic areas described above). Data display according to coverage: Green (>79%=good coverage) Yellow (50–79% =improving, but not good coverage) Red (<50% =not good coverage). E–24 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–2a. Data Analysis Plan for Evaluation Question One: Performance Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 1.3: How do Pillar II stakeholders (including direct and indirect beneficiaries) perceive the value of the Pillar II activities that they are aware of? Description: Pillar II stakeholders with awareness of Pillar II activities that reported any value and explained why they considered the activities to be of value to them or others Description: Pillar II stakeholders with awareness of Pillar II activities that reported no value and explained why they considered the activities to be of no value to them or others KII among direct beneficiaries￾recipients: KII among indirect beneficiaries￾national government, other partners; KII USG (Q501.P3) KII Govt (Q501.P3). Thematic analysis of rating and ranking of perceived value (high, low, not at all) by thematic area, stakeholder group, and country. EQ 1.4: How have Pillar II activities contributed to the achievement of Pillar II’s three objectives? 19 Outcome Indicators (provided in Table 2b, Detailed Outcome Indicators for EQ1.4); Types of Pillar II stakeholders (beneficiaries – population or organizations) who reported the contribution of Pillar II activities to the achievement of Pillar II’s three objectives and explained why they were or were not of value. HFS data will be disaggregated by the level of USAID support (e.g., HFs with substantial support in equipment, training, and infrastructure, HF receiving some support, and HF receiving no USAID support and/or receiving other donor support. All health facility data are disaggregated by “supported” and “not-supported” by USAID Ebola Pillar II funds. List of USAID Ebola Pillar II￾supported HFs were provided by the Global Health Ebola Team (GHET) for Guinea; Liberia mission for Liberia; and JSI for Sierra Leone. For Sierra Leone, only Tier 1 HFs were viewed as “supported by USAID Ebola Pillar II funds.” USG KIIs (Q116. P4, 140. P4, 141. P4.1); Round Table discussions; Reporting by IPs; Household (HH) surveys in each country; Peer￾reviewed literature; Reports from independent expert groups and other international aid organizations. Contribution analysis of Pillar II-supported activities to Ebola Pillar II Strategic Framework’s three objectives: to halt the loss of development gains; recover and strengthen key institutions and infrastructures; and build sustained systems through Public-Private Partnerships (PPPs), innovation and capacity building. The conclusions of “how activities or combinations of activities made the contribution” will be derived from the analysis of multiple data sources and data triangulation discussions among the evaluation team members. The following will be considered: 1. Whether (extent to which) the activity is aligned with the ToC; 2. Whether the activities were implemented; 3. Confirmation by primary and/or secondary data evidence that the expected results occurred; and 4. Assessment of whether there were other contextual factors that are known to affect the desired outcomes. Comparison of outcome indicators (Table 1) will be done with the indicators from available baselines (when available) to assess improvements or achievements. The process of data triangulation will not seek to provide definitive proof of cause-effect, but instead to reduce uncertainty about the ways in which an activity, or combination of activities, contributed toward achieving the Pillar II strategic objectives. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–25 Table E2–2a. Data Analysis Plan for Evaluation Question One: Performance Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 1.5: What are the successes, challenges, and lessons learned that the evaluators recommend be disseminated throughout USAID to advance resilience and recovery in health emergencies? 1.Types and description of success stories identified; 2. Types and description of most common challenges; 3. Types and description of most common lessons learned. IP routine monitoring data, IP report review, KII among IP staff (Q116, P4, Q117 P4, 146 P5), KII USG- (Q116, P4, Q117 P4, Q142 P4.2, Q143. P4.3, 146 P5); KII Govt (Q116, P4, Q117 P4, Q142 P4.2, Q143. P4.3, Q146 P5 FGD beneficiary. Content analysis of desk review of IP reports and primary qualitative data sources to identify factors mentioned. 1. Identification of success stories: What didn’t work? 2. Identification of most common challenges; 3. Identification of most common lessons learned. What gaps still remain? EQ 1.6: To what extent did Pillar II Ebola recovery activities ensure the involvement of women and assist men and women equally? What steps might USAID undertake to increase gender equity in Ebola recovery efforts? Number and percentage of Pillar II activities designed to promote gender equity. Percentage of Pillar II activities designed to promote gender equity that achieved the objective of gender equity. IP KIIs-Q 144. P6, Q145; USG KIIs - Q145. P6.1; Govt KIIs- Q145. P6.1; Beneficiary-and Recipient FGDs - Q8. Narrative and descriptive analysis with comparisons with respect to gender considerations in designing and implementing program activities across the six thematic sectors and three countries. Tabular analysis of IP M&E data and disaggregated by priority areas in which USAID-funded IPs were implementing one or more activities (per ToC), with attention to outputs and outcomes by gender. One major anticipated constraint is that the gender-specific data may not be available in all priority areas. Table E2–2b. Detailed Outcome Indicators for EQ1.4 (How have Pillar II activities contributed to the achievement of Pillar II’s three objectives?) Pillar II Outcomes by Thematic Area 2017 results compared with available or constructed baselines Indicator definition Data source Health Basic health service delivery restored Restored or improved vis-à-vis 2013 levels (pre-Ebola levels) % of health facilities offering the complete package of integrated essential Reproductive, Maternal Neonatal and Child Health (RMNCH) care (safe delivery, Emergency Obstetric and Neonatal Care (EmONC), essential newborn care, Integrated Management of Newborn and Childhood Illnesses (IMNCI), family planning) % of health facilities with minimum infrastructure (per Government of Guinea/Government of Sierra Leone/Government of Liberia (GOG/ GOSL/GOL) standards) to remain functional Health Facility Survey, 2017 % of health workers trained in new Infection, Prevention and Control (IPC) protocols Health Facility Survey, 2017 % of pregnant women who received antenatal care for the most recent pregnancy since October 2014 Household Survey, 2017 Health care worker capacity to deliver EVD and non-EVD services increased Restored or improved vis-à￾vis 2013 levels; compare with DHS, Multiple Indicator Cluster Survey (MICS) % of births attended by a skilled birth attendant (SBA) in USG-supported health facilities Household Survey, 2017 E–26 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–2b. Detailed Outcome Indicators for EQ1.4 (How have Pillar II activities contributed to the achievement of Pillar II’s three objectives?) Pillar II Outcomes by Thematic Area 2017 results compared with available or constructed baselines Indicator definition Data source Compare with District Health Information Software 2 (DHIS2) % of deliveries in health facilities assisted by skilled birth attendants Health Facility Survey, 2017 % of HF workforce trained on the Basic Package of Essential Health Services Health Facility Survey, 2017 Comprehensive services developed and provided for EVD survivors Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines % HF offering comprehensive health services for EVD survivors Health Facility Survey, 2017 Health behavior gains enhanced and reinforced Restored or improved vis-à￾vis 2013 levels; compare with DHIS2, DHS Proportion of women who have attended at least four antenatal care visits for the last live births since October 2014 Household Survey, 2017 Trust/health-seeking behaviors regarding health services Household Survey, 2017; Desk Review Compare with Country’s Health Management Information System (C-HMIS), health promotion surveys. Baseline=25% % of households in priority areas practicing correct use of recommended household water treatment technologies Household Survey, 2017 Compare with C-HMIS, health promotion surveys. Targets=385 in Y1, 393 in Y2, 198 in Y3, Relative to USAID targets # of communities certified as “open defecation free” (ODF) as a result of USG assistance Desk Review Health infrastructure (including management systems and community involvement) strengthened Restored or improved vis-à-vis 2013 levels # of laws, policies, or procedures updated or adopted to restore services Desk Review; KII/ FDGs/RT Baseline: 2014/15 EUV report: 70%. 5/10% increase per year due to improved drug storage, drug management, and drug compliance. Target is no stock￾outs reported by 2019/2020 % of USG-assisted service delivery points that do not experience a stockout at any time during the previous reporting period of a contraceptive method; malaria medicine/commodities; delivery kits and tracer commodities Health Facility Survey, 2017; Desk Review Average stock-out rate of contraceptive commodities at family planning (FP) service delivery points Health Facility Survey, 2017; Desk Review % of health facilities meeting country-specific quality standards Health Facility Survey, 2017; Desk Review Compare with C-HMIS, health promotion surveys. Targets=75% at Year 5 (baseline = 25%) % of population in USAID Pillar II priority areas using an improved drinking water source Household Survey, 2017 Compare with C-HMIS, health promotion surveys. (Baseline = 20%) % of population in Pillar II priority areas using an improved sanitation facility Household Survey, 2017 ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–27 Table E2–2b. Detailed Outcome Indicators for EQ1.4 (How have Pillar II activities contributed to the achievement of Pillar II’s three objectives?) Pillar II Outcomes by Thematic Area 2017 results compared with available or constructed baselines Indicator definition Data source Health Information System developed and supported Baseline in Liberia= 68% Lofa, 53% Nimba, 71% Bong) % health facilities that receive a quarterly supervisory visit from the Community Health Volunteers (CHT) (Joint Integrated Supportive Supervision) Health Facility Survey, 2017 % of health facilities submitting timely Logistics Management Information System (LMIS) reports Health Facility Survey, 2017 % of health facilities submitting timely and complete reports to the district level Health Facility Survey, 2017 Health financing and resource allocation improved Improved from 2013 levels Liberia: 15% improvement in 2016 with 5% increases in following years % improvement in quarterly budget execution rate in counties compared to the same period in the previous year Desk Review Basic Education School attendance restored and gender balance improved Restored or improved vis-à-vis 2013 levels, disaggregated by gender In Liberia: Compare with third￾party sources such as United Nations Educational, Scientific, and Cultural Organization (UNESCO). Institute for Statistics 2017-18 target = 35% (FY 2015 target was 23% and result was 27%) (Note that Net Enrollment Rate (NER) dropped from 2011–2015) Primary Net Enrollment Rate and percentage change Percentage difference in school attendance between: 1. Intervention group (schools with full package of WASH services) vs. 2. Control group (schools with no or partial package of WASH services) Desk Review; HH survey, 2017 School Survey, Liberia (2019 report) Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines % of KIIs reporting improved school attendance; gender balance KIIs CSO, Govt, Service Providers Literacy and numeracy improved Improved from 2013 levels (sources: Ministry, DHS, literature), disaggregated by gender Proportion of students who, by the end of two grades of primary school, demonstrate that they can read and understanding the meaning of grade level text Desk Review Agriculture and Food Security Household food security increased Restored or improved vis￾à-vis 2013 levels, available information IP award document target and baselines % EVD-affected households (in priority areas) that received cash transfers/vouchers to increase household access to food Household Survey, 2017 Compare with Feed the Future (FTF) activity reports Prevalence of households with moderate or severe hunger HHS in priority areas Household Survey, 2017 Nutritional status improved Improved from 2013 levels Household Dietary Diversity Score (HDDS): Mean # of food groups consumed by household members Household Survey, 2017; Desk review Household standard of living improved Restored or improved vis-à￾vis 2013 levels; compare with Living Standards Measurement Survey (LSMS) FTF 4.5.9 Daily per capita expenditures (as a proxy for income) in USG-assisted areas Desk review E–28 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–2b. Detailed Outcome Indicators for EQ1.4 (How have Pillar II activities contributed to the achievement of Pillar II’s three objectives?) Pillar II Outcomes by Thematic Area 2017 results compared with available or constructed baselines Indicator definition Data source Governance Enabling government policies developed and harmful policies and practices revised Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines # of public policies introduced, adopted, repealed, changed, or implemented as a result of CSOs receiving USG assistance engaged in advocacy activities Desk review; qualitative interviews CSOs strengthened to provide effective advocacy, transparency, and accountability TBD from project norms and beneficiary expectations # of CSOs/CBOs staff trained in developing and implementing effective Ebola health-related advocacy activities KIIs CSOs Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines # of public-private partnerships harnessed to address second-order of Ebola impacts Desk review; qualitative interviews Management systems and accountability of ministries and stakeholders improved at all levels Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines Civil society's satisfaction with their voices being heard in decision-making and monitoring processes KIIs CSOs Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines # of USG-assisted political parties implementing initiatives to increase the # of candidates and/ or members who are women, youth, and from marginalized groups (F) Desk review Economic Crisis Mitigation Agriculture and import markets’ function restored and improved Restored or improved vis-à-vis 2013 levels # laws and policies implemented to support private enterprise growth through USG assistance Desk review, KII Number of community market organizations reporting improved agriculture and import markets FGDs with CSOs Social protection provided for EVD￾affected groups Relative to USAID targets￾available information in country CDCS documents or IP award document target and baselines % of EVD-affected communities benefitting from social protection services as a result of USG support Household Survey, 2017; Desk review Information, Communication, Technology ICT for health and other systems developed Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines Number of ICT tools rolled out with USG assistance that are in use % of facilities that provide timely, accurate, and complete HMIS reports in USAID-focus counties Health Facility Survey, 2017 Power, water, roads and communication infrastructure restored and expanded Relative to USAID targets, available information in country CDCS documents or IP award document target and baselines # beneficiaries with improved energy services due to USG assistance Desk review FGDs with Beneficiaries (Recipients) ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–29 Table E2–3. Data Analysis Plan for Evaluation Question Two: Sustainability Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 2.1: Which Pillar II activities in each thematic area were designed to have sustained effects on beneficiaries and/or institutions and systems? For those that were intended to have lasting effects, what program elements and strategies did they use to promote sustainability? For each activity, whether exit strategy or sustainability objectives described or reported by IP - yes/no. If yes, type of activities, supported by Pillar II funds, designed to have sustained effects on beneficiaries and/or institutions and systems, ended on or before Sept 2017 (for 2017 report). If yes, type of activities, supported by Pillar II funds, designed to have sustained effects on beneficiaries and/or institutions and systems, ended on or before Sept 2019 (for 2019 report). IP award, IP reports and USAID document review; KII IPs (Q201, Sustainability (S) 1); KII USG (Q201. S1). Narrative and descriptive analysis with comparisons with respect to timing, scale, and funding in each country. Data will be disaggregated by prefecture/ county/district of implementation and type of activity (six thematic areas- Agriculture and Food Security; Health; Education; Governance, Economic Crisis Mitigation; and Innovation, Technology, and Partnerships, six intervention types: Social Protection; Frontline Worker Support; Management, Coordination and Partnerships; ICT; SBCC; Institutional Enhancements). EQ 2.2: What types and levels of intentional (planned) and unintentional (unplanned) results are detectable after the activities ended, and why? Type and number of activities with detectable effects after the activities ended for IPs whose activities (planned activities) ended on or before Sept 2017. % of IPs modified activities due to needs assessment/gap analysis during the course of the implementation to promote sustainability/Number of IPs whose activities ended on or before Sept 2017 (for 2017 report). % of IPs modified activities due to needs assessment/gap analysis during the course of the implementation to promote sustainability/Number of IPs whose activities ended on or before Sept 2019 (for 2019 report). IP routine monitoring data, IP report review, KII IPs (Q202, S2), KII USG (Q202, S2); KII Govt. (Q202b S2), beneficiary. Narrative and descriptive analysis of the extent of achievements of proxy sustainability indicators in each country. Data will be disaggregated by 6 thematic areas, described above. An anticipated constraint for this analysis is that some Pillar II activities were deliberately short-lived. IBTCI will analyze whether these activities created the foundation or enabling environment for subsequent programs to build on with yes/no, examples, if any. EQ 2.3: To what extent did Pillar II investments accelerate development and resilience in the beneficiary populations/ organizations? Types of activities and examples reported by Pillar II stakeholders (beneficiaries – population or organizations) on the extent of accelerating development and resilience due to Pillar II activities, with explanations as to why or why not. FGD among beneficiaries- - Q6, 7, 9; KII among IPs (Q206, S4; Q207 S4); KII among IPs (Q204, S3; Q208 P5); KII Govt. (Q203 S3, Q204b S3). Thematic analysis of rating and ranking of perceived extent in sustainability: high, low, not at all; by thematic area, stakeholder group, and country. The three dimensions of resilience will be examined: absorptive capacity to mitigate shocks and stresses; adaptive capacity to restore services and rebuild systems, and transformative capacity that brings about fundamental changes in the society that reduce vulnerability and strengthen it to withstand shocks. E–30 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–3. Data Analysis Plan for Evaluation Question Two: Sustainability Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 2.4: What individual, organizational and socio￾economic factors, within and across sectors, are associated with sustained program effects? Factors or proximal and distal determinants associated with those activities which showed sustained program effects. (see Figure 3) 1. Types and description of individual factors; 2. Types and description of organizational factors; 3. Types and description of socio￾economic factors Surveys, interviews, roundtable discussion, reporting by IPs (Q208 S5, Q209 S5). KII USG (Q206, S4; Q207 S4; Q209 P5); KII Govt. (Q206, S4; Q208 P5; Q209 P5). Analysis of outcome indicators (described in Table 1.1) in each thematic area that shows detectable sustainability (identified through analysis of EQ2.1 and EQ2.2). Sustainability is defined as: Results reported at the end of an activity should be evident on an ongoing basis, months and ideally years later, in geographic areas where investments were made. 1. Identification of individual factors; organizational factors, and socio-economic factors associated with those activities which showed sustained program effects. 2. Examination of association using test of significance analysis for quantitative data to report level of association of outcome variable with individual (age, gender, education), organizational (skilled staff, supervision, resources, presence/ absence of organization structure, Policies/ Guidelines/Laws) and socio-economic factors (geographic location, literacy rate, per capita income, family size). Content analysis of IP reports, and primary qualitative data to identify factors mentioned. Table E2–4. Data Analysis Plan for Evaluation Question Three: Gaps & Opportunities Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 3.1: What gaps in Pillar II activities’ resources and conditions for success were identified by stakeholders in advance? Were they addressed? How, and/or why not? 1. Types and description of gaps in resources; 2. Types and description of gaps in conditions other than resources, reported by interviewed respondents. IP award, IP reports and USAID document review; KII IPs (Q302, Opportunity/Gaps (OG) 2); KII USG (Q305. OG5, Q306 OG6); KII Govt (Q302, OG1, Q305 OG1). Content analysis of IP reports, and primary qualitative data to identify gaps in resources mentioned. Identification of conditions of success through content analysis of success (and failure) stories. Content and thematic analysis for identification of most common gaps/needs, and measures taken to address those gaps/ needs. Data triangulation with HH and HF survey data- to understand changes in outcome indicators or relevant contextual indicators obtained from the surveys. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–31 Table E2–4. Data Analysis Plan for Evaluation Question Three: Gaps & Opportunities Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 3.2: As Ebola recovery efforts evolved, what changes in the situation or context affected Pillar II activities’ performance, and what programming responses were most successful in adapting to change? Types of changes in programming of Pillar II stakeholders who reported a need for the adjustment or adapting/changes in programming, and the nature of contextual factors reported and most successful in adapting reported. IP routine monitoring data, IP report review, KII IPs (Q304 OG4), KII USG (Q304 OG4) KII Govt (Q303 OG2) beneficiary. Content analysis of IP reports, and primary qualitative data to identify contextual factors, constraints, limitations affecting activities performance. Descriptive analysis and examples of any modifications to activities and evidence of its success after activity modifications. Data triangulation with HH and HF survey data to understand changes in outcome indicators or relevant contextual indicators obtained from the surveys. EQ 3.3: What opportunities, within the interventions and in the intervention context, were seized to improve and/or amplify Pillar II results? 1. Types and description of opportunities within activities and context; 2. Types and description of opportunities and how they were utilized to amplify Pillar II results, reported by interviewed respondents; 3. Behaviors, practices at the household level that have improved from baseline and/ or newly adopted that can be associated with activity; 4. Improved and/or new services that were not previously offered at facilities. FGD with Beneficiaries - Q6, 7, 9; KII among IPs (Q206, S4; Q207 S4); KII USG (Q303 OG3b); KII Govt (Q303b OG3); indirect beneficiaries￾national government, other donors and household and facility surveys. Content and thematic analysis for identification of available opportunities and relevant context. Descriptive analysis and examples of any measures taken to improve Pillar II results utilizing available opportunities and conducive context. Data triangulation with HH and HF survey data to understand changes in outcome indicators or relevant contextual indicators obtained from the surveys. EQ 3.4: What were the missed opportunities? What development issues or barriers were ignored or not adequately addressed? 1. Types and description and reason of missed opportunities; 2. Types and description of challenges/barriers, reported by interviewed respondents; 3. Any negative or no change in behaviors, practices at the household level - (compared with available baseline information); 4. Basic services that are lacking at the facility level (compared with available baseline information). FGD with Beneficiaries - Q6, 7, 9; KII IPs (Q303 OG3 or Q306 OG4) KII USG (Q303 OG3b), KII Govt (Q303b OG3, Q306 OG4) Round Table discussion, household and facility surveys. Content and thematic analysis for identification of challenges/barriers/missed opportunities, and measures taken to address those challenges. Identification of challenges/barriers/missed opportunities, for which measures were not taken to address those challenges/barriers; possible reasons for no action taken. Data triangulation with HH and HF survey data to understand changes in outcome indicators or relevant contextual indicators obtained from the surveys. E–32 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–5. Data Analysis Plan for Evaluation Question Four: Management Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 4.1: What were the perceived strengths and weaknesses in the processes for programming Pillar II resources, from the mission, USAID Washington, government, IP, and civil society/ beneficiary perspectives? Examples reported by Pillar II stakeholders (by type of stakeholder- IPs/Govt/ USAID mission/USAID Washington/ CSOs) on strengths of USAID’s Pillar II programming in a complex emergency. Examples reported by Pillar II stakeholders (by type of stakeholder-IPs/Govt/ USAID mission/USAID Washington/ CSOs) on weakness of USAID’s Pillar II programming in a complex emergency. Examples of strengths and weaknesses and their effects on management. KII IPs (Q404 Management (M) 1, Q 405 M1, Q406 M1); KII USG (Q404 M1, Q 405 M1, Q406 M1); KII Govt (Q404 M1, Q 407 M1). Content analysis of IP reports, and primary qualitative data to: 1. Identify strengths in the management process mentioned. 2. Identify weaknesses in the management process mentioned. Identify conditions of success through content analysis of success stories. Content and thematic analysis for identification of most common gaps/needs, and measures taken to address those gaps/ needs. Data triangulation with other sources. Comparison of results across different types of stakeholders. EQ 4.2: How well were the Pillar II Ebola Recovery activities in Guinea, Liberia, and Sierra Leone coordinated with the government, among all OUs, and among development partners, to maximize coverage of needs and to avoid duplication of efforts? Examples reported by Pillar II stakeholders on coordination with other (IPs, /govt. / USG/non￾USG partner) for Pillar II efforts to maximize resources and avoid duplication of efforts. Examples of: - Coordination (explanation why or why not; - Communication patterns; - Common results measurement frameworks; - Duplication of efforts; - Delays; - Activity design (alignment with country needs/ national strategies) Examples of leveraged activities within USAID and across USAID and other development partners. IP report review, KII among IP staff (Q402.M2 Q403b. M2), KII USG (Q402. M2 Q403.M2); KII Govt (Q402.M2, Q 403.M2) beneficiary. Content analysis of IP reports, and primary qualitative data to 1. Identify degree of coordination among partners. Ranking – well-coordinated, poorly coordinated, no coordination at all in maximizing the program coverage. 2. Any evidence of duplication of efforts – yes/no. 3. Any evidence of partner coordinating body – yes/no. EQ 4.3: What lessons learned emerge from Pillar II about transitioning from emergency to development modes of action in USAID? Examples reported by stakeholders on important lessons learned during Pillar II implementation. Examples of lessons learned and explanations on how they affected transitioning from emergency to development. IP report review, KII among IP staff, KII USG (Q408 M3); KII Govt (Q408 M3). Thematic analysis of qualitative data and data triangulation from various data sources to identify lessons learned. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–33 Table E2–5. Data Analysis Plan for Evaluation Question Four: Management Sub-questions Indicator and Analytical Approach Data Sources Data Analysis Method EQ 4.4: What actions are needed to prepare missions and Pillar II beneficiary organizations and communities for the end of Pillar II funding, and when? What sources and channels are likely to be most effective for which audiences? Examples of recommendations cited and explanations on what resources and channels will be most effective for target audience. KII IPs (Q409 M4, Q410 M4), KII USG (Q409 M4, Q410 M4), KII Govt (Q409 M4, Q410 M4); roundtable discussion. Thematic analysis of qualitative data and data triangulation from various data sources to identify actions needed and sources and channels most likely to effectively formulate recommendations tailored to the evaluation audience. DATA COLLECTION QUALITY ASSURANCE PLAN, INTRODUCTION This Data Collection Quality Assurance (DCQA) plan presents the proposed procedures for ensuring high-quality evaluation data for a macro-level performance evaluation study of USAID’s surge support for Pillar II activities in the three countries, fielded in 2017. Evaluation is one of four components of the Pillar II MEL Activity. Data collection methods will include further desk review, key informant interviews, field observations, group interviews (FGDs, expert round-tables), a structured HHS of a stratified random sample of Pillar II beneficiaries, and an HFS. In Liberia there will also be a school survey. This DCQA plan covers the methods for collecting KIIs, FGDs, and the quantitative surveys (household, health facility and school survey) data. It does not cover the methods for collecting the partner monitoring data. The DCQA plan will be implemented in three phases, namely pre-data collection, active data collection and post-data collection. DESCRIPTION OF DCQA PHASES A. The pre-data collection phase includes a series of tasks that must be completed in order to prepare for the data collection. A1: Survey questionnaire design, interview/FGD guide design and Review Process; Version Control: July–October, 2017 The process of questionnaire design has been completed using various successive steps: from the development of a conceptual framework (theory of change) based on the evaluation questions and the Africa Ebola Unit (AEU) strategic and results frameworks, to writing and sequencing the questions in survey questionnaires and KII/FGD guides. The questionnaire and guides were shared with various USAID operating units to obtain feedback and comments. The final versions are being pre-tested in the field. A2: Pre-testing and Questionnaire or KII/FGD Guides finalization: November–early December, 2017 All data collection instruments will undergo pre-testing so that the wording, structure, and layout of all questionnaires provide valid and reliable results, (i.e., make sure that survey questions are understood and answered correctly by the respondents) and can be administered consistently among all the interviewers. Once all instruments are pre-tested, piloted, and finalized, the final questionnaires will be translated from English into French, Liberian English, Creole, and main local languages, and adapted to the context of each country. The translations will be independently checked by a separate entity against the originals. A3: Guidelines for data submission: October, 2017 All KII and FGD facilitators will adhere to guidelines for qualitative data collection and submission of qualitative transcripts (see Sub-Annex B). All team members will transmit KII and FGD notes in a timely manner (within 48–72 hours of data collection) and send notes to our Home Office from an IBTCI email only. All correspondence will occur through IBTCI email. Personal non￾IBTCI email will not be copied or used. Before sending emails with data, team members will ensure correct email addresses. Personal/ work laptops, especially in the field, should be password-protected. A4: Preparation of Master Data Analysis File: December, 2017–January, 2018 Our DAP identifies a list of variables specifying the expected output of the quantitative surveys, qualitative interviews, discussions and secondary data analysis. Master Atlas.ti database with codes for qualitative analysis of KII and FGD data A pre-designed codebook framework with definitions has been created in a Master Atlas.ti Project file. The codes created address E–34 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES the key themes of the evaluation. These themes include: 1. The four evaluation questions and sub-questions; 2. A domain analysis of the Pillar II Response, namely Performance, Sustainability, Gaps and Opportunities and Management; 3. The ToC terms (activities, outputs, objectives) described in the evaluation inception report. Three types of codes will be used, as follows: ■ Attribute codes: Examples are gender, location, type of respondents; ■ Evaluation question codes: Examples are EQ1.1, EQ1.2; ■ Analytic codes per evaluation question: Examples are pattern codes (improved, declined, increased, decreased, persistent); thematic codes (achievement, failure, success, lessons, recommendation, barrier, challenge, adjustment, strengths, weakness); magnitude coding (positive, negative, no change, ratings scales), quotations. IBTCI coders will have weekly coordination meetings. IBTCI will oversee the data collector subcontractor and monitor the progress of qualitative and quantitative data collection. STATA do-files and dummy tables for quantitative analysis for household and health facility surveys As soon as preliminary datasets are received, following completion of pilot testing for all surveys, STATA do-files and dummy tables for quantitative analysis will be prepared. A do-file is a text—also called a “batch”—a file with a series of commands to be executed in order by STATA. The do-files will be used for creating variables, composing, revising, and saving data analytic commands. A5: Training of data collectors: mid-November to early December, 2017 Survey teams, including survey supervisors and data collectors, will undergo a week-long practical training and pilot testing on survey instruments and data collection procedures. An assigned Sr. Evaluation Advisor will oversee the data collector training and pilot testing in each of the trainings led by the Country Project Director. All field workers will be trained on the Household Survey instrument, given that this is the data collection component of the evaluation. This will ensure that data collectors and supervisors who might not do well during the training or data collection can be replaced by other trained data collectors. Training manuals have been prepared by IBTCI and will be used during the main training of field workers. A survey subcontractor team attended a workshop at IBTCI Home Office on the evaluation methodology and instruments. This Training of Trainers (ToT) was designed to bring the survey subcontractor team up to date on all aspects of the evaluation, and to ensure consistency in training across the three countries. B. The active data collection phase includes the data quality processes and procedures at the field and home office levels during data collection and management. B1: Field Data Collection: December, 2017–January, 2018 The survey data collection will use the Computer-assisted Personal Interviewing (CAPI) technique where data is collected using electronic questionnaires stored in a tablet (handheld electronic device). The electronic questionnaire allows the management of more complex interviews, helps to prevent data capturing errors, and makes data available in a short time (a data entry phase is not necessary). Set-up and maintenance of procedures for creating electronic questionnaires, CAPI, daily supervision of data collectors, and daily data transfer after quality control will be done by the IBTCI subcontractor. The same subcontractor will be used to train and collect data in all three countries to minimize bias. IBTCI’s approach to data collection for the activities is summarized below: ■ The survey subcontractor will liaise with IBTCI to plan all aspects of the project, including developing a timeline, finalizing all instruments, and sampling strategies, including substitutions, and troubleshooting. This includes the IBTCI ToT at the IBTCI Vienna office, as well as in-person meetings between the subcontractor and IBTCI staff in the three countries. ■ Recruitment of appropriate numbers of enumerators and moderators, to ensure recruitment of enumerators with experience in previous rounds of Ebola or similar studies. ■ Coding all quantitative survey instruments into SurveyToGo for use on Android smartphones/tablets. ■ Overseeing training of enumerators and moderators. The subcontractor will recruit 60 enumerators and 20 field supervisors for the quantitative surveys and 25 interviewers for the qualitative data collection. ■ Collect Global Positioning System (GPS) data for each quantitative survey site at the enumeration level, not at the household level. ■ Conduct extensive quality control on both the quantitative and qualitative data (see Section B3). ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–35 B2: Monitoring Data Transfer: December, 2017–January, 2018 The KII and FGD data will arrive in a continuous stream and be stored in its raw form as it is received from three types of sources: the subcontractor, Sr. Evaluation Advisors, and Field Coordinators (FCs). If data types in any category are collected by handwritten notes, the first line of action is to advise the interviewer to type and submit the entry in MS Word® format. The second line of action will be to upload digital images of the data daily (or as often as internet is accessible) and send it to Vienna, Virginia for transcription into digitized format. The Data Administrator will be responsible for tracking the data collection by FCs and Sr. Evaluation Advisors—date of interview, date of transcript received, data of uploading to Atlas.ti project library—ensuring that if there is data loss during transmission, the field-based teams are contacted to resend data. Survey data will arrive on a bi-weekly basis from subcontractors and be stored in MS Excel® format. Excel format will be converted to STATA to perform data quality checks and analysis. B3: Monitoring Data Quality in the Field: December, 2017-January, 2018 IBTCI’s Survey Specialist/Data Collection Lead will oversee and monitor the progress of all three countries. The Senior Program Associate and Senior Evaluation Advisors from IBTCI will travel to respective countries to monitor the data collection activities. The country FCs recruited by IBTCI will also be monitoring data collection activities on a frequent basis. IBTCI will follow a multi-layered approach to ensure data quality in the field for the quantitative and qualitative data collection. These approaches are described below. Quality control for quantitative surveys Field supervision Enumerators will be divided into teams of four or five, each under the direction of a field supervisor. The supervisor will be responsible for ensuring enumerators are implementing the correct sampling procedures, administering the questionnaire correctly, supervising interviews, and performing back-checks. Real-time data quality monitoring IBTCI will use Android smartphones/tablets for the survey data collection, which will allow data review for quality control on a daily basis and give feedback to the field teams, as well as to IBTCI. Daily data quality control checks will include: ■ Length of interview ■ Audio recordings ■ GPS ■ Sample monitoring To accomplish this, IBTCI has set up a team devoted to real-time quality control (QC) monitoring. The QC team downloads the data daily and listens to audio recordings as well as verifying survey locations via GPS and the length of interviews. Post-fieldwork data quality After fieldwork is completed, the data will be further analyzed through an internal data quality control program, Forensic. The program identifies duplicate interviews, overlapping times and unusually high rates of specified codes by enumerators. Analyzing the data from this perspective supplements our traditional data quality control checks, as it facilitates our ability to easily identify any patterns among the field team. Quality control for qualitative data collection methods Translation and Back-Translation of Instrument: mid-November to early-December, 2017 ■ Once qualitative instruments are designed, they will be translated into the local languages. IBTCI will then employ a third party who has not seen the original translations to independently verify the translations. Training of Moderators/Interviewers ■ Even though the subcontractor will be working with competent and experienced moderators/interviewers, it is crucial to conduct a face-to-face training with the moderators. ■ The subcontract Project Director will travel to the respective countries to train the moderators/interviewers on the study’s aims and objectives. ■ Training involves a full walk-through of the guide, question by question, exploring the meaning of each question, the best ways in which to approach this topic, and the objectives behind the pattern of questioning. ■ Subcontractor staff will also conduct a refresher session just before fieldwork starts on how best to ensure respondents are assured of their confidentiality and security, how to use a discussion guide to best effect, how to ask open questions, how to probe effectively, how to elicit views that may be more buried, and how to employ techniques that can be used to minimize “group-speak” or particular participants dominating the discussion. Accurate and Representative Recruitment: November, 2017 ■ Recruitment is supervised by a quality control officer to ensure that recruiters adhere to the recruitment specification and any quotas. The local project manager will personally call all respondents before the focus group takes place to ensure that they have been correctly recruited, and that they fit the agreed recruitment specifications. Any respondent who is ineligible will be replaced. E–36 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Observation of the KIIs/FGDs: November-December, 2017 ■ A representative from the subcontractor or their local partners will oversee management of the project and observe the data collection teams during the KIIs/FGDs as a measure of quality control. ■ The observer ensures transcripts are sent to IBTCI within 48–72 hours after the KII/FGD. If necessary, the observer will step in to guide the moderator/interviewer to ensure that the required information is being collected, that the moderator/interviewer is probing deeply enough, or to prevent the discussion from taking an unforeseen turn. ■ The local partner representative will debrief the moderators/ interviewers after each observation, giving feedback on the direction of the discussion, the style of moderating/interviewing, and any other ways for improvement. High-quality recording ■ Moderators/interviewers use their own video and audio recording devices to record both the discussion group and the simultaneous translator, and to avoid poor quality sound or inappropriate formats. IBTCI knows from prior experience that this is the most reliable way to procure good-quality audio files. ■ Upon receipt of the transcripts, these are compared against the notes taken by the observer at the time of the group, as well as against the local-language transcripts to ensure that all dialogue has been captured and translated correctly. The subcontractor will employ an independent third-party transcriber to listen to each of the audio files and verify that all dialogue has been captured within the transcripts. ■ If any problems or omissions are noted, the transcript is returned to the transcriber, who is asked to listen again to the entire audio file and provide further information. If necessary, a different transcriber will be selected until the team is satisfied with the quality. As mentioned in the inception report, both quantitative and qualitative instruments include an informed consent that ensures respondents/participants of the confidentiality of the data and that participation is voluntary. B4: Data review and spot checks at IBTCI HO: December, 2017–January, 2018 During the data collection period, IBTCI will review and manage the data in the following increments: 1. Daily Tasks: a. Original data creation b. Spot-checks for data quality by subcontractor in the field c. Transfer of qualitative data transcripts files to IBTCI HO d. Data Management Logs e. Translation of files as needed f. Chain-of-data security procedures, anonymization/ confidentiality g. Conversion of files to single format h. Upload of files to qualitative and quantitative data analysis programs i. Backup 2. Bi-Weekly tasks (2x/month) a. Data Administrator verifies completed transfer of files sent by field teams using Google Spreadsheet b. Subcontractor sends full updated survey dataset to IBTCI HO c. Data Quality Assurance (DQA) director spot checks and create backups DQA director will perform a review of completed questionnaires to identify potential problems, patterns or inconsistent data. Some specific issues to explore include: ■ Item non-response: Identify questions with high rates of “Don’t know” or “Refused” responses. ■ Frequencies: Examine frequency distributions of key questions to determine if data look appropriate. ■ Allowable ranges: Review appropriateness of the allowable ranges programmed into the handhelds and adjust for full survey implementation as needed. ■ Response categories: Determine if country-specific questionnaire adaptations (e.g., modifying question item lists and response categories) were appropriate. Any problems will be communicated to the subcontractor. If any problems are revealed, additional/enhanced data quality-check procedures will be developed and implemented until the problem is resolved. B5: Procedures for routine data management, data back￾ups and storage The data will be stored on a single machine (IBTCI HO server, “Primary Data’’ subfolder under Pillar II Folder) with incremental backup locations and periodically reviewed to check for authenticity, integrity, and quality. In addition, IBTCI will use regular incremental backup of the data at two other physical secondary locations (a laptop/hard drive in HO) to reduce vulnerability to any large-scale loss or corruption of the data. Given the heterogeneous sources of our qualitative data, all data in text files will be kept in a backup location to maximize usability of the data over platforms and time. IBTCI has a system of backing up the data in the field and at their headquarters. ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–37 C. The post-data collection phase (January￾February, 2018) includes a series of tasks that need to be completed in order to prepare an analytic data file for conducting data analysis and refers to a stage once all the survey data have been collected and aggregated. C1: Cleaning and Preparing Data: January–February, 2018 Procedures for qualitative data: After data collection, prior to data analysis, data must be converted to a single format in each of these categories (e.g., for text files, all documents converted to .txt or .doc) for upload to Atlas.ti to assure uniformity of data for storage and export. All qualitative data will be integrated into a single database and logged into a common data management log to ensure version control and chain-of-security. Once transcribed notes have been uploaded into the Atlas.ti database, analyses of qualitative data will be divided among a group of coders based in the IBTCI Home Office by the Data Administrator. Data Coders will code transcripts according to a pre-designed codebook template that has been populated according to pre-selected themes, questions, domains, and theories of change indicated above, as well as a select sub-group of specific themes that are designed to elaborate key questions with greater specificity. Efforts will be made to avoid creating sub-codes in order to streamline the analysis. Procedures for quantitative data: A set of analyses of the quantitative data will be conducted to evaluate the quality of specific questions to look for non-response, outliers and missing/ ambiguous information. The questions that present the highest non-response rates will be investigated for the possible causes. C2: Procedures for preparing data for submission to the client: End of March, 2018 By the end of contract, after final approval of the reports, all data will be prepared for the submission to USAID in electronic format after removing personal identifiers (ID). Any paper-based data with personal ID will be shredded. IBTCI will submit quantitative data in MS Excel format and Atlas.ti coded qualitative data in MS Word. IBTCI will not submit the entire interview/discussion transcript to protect a respondent’s identity. Personal identifications (IDs) of all respondents (surveys/ interviews) will be removed prior to data submission. See Table 6. Table E2–6. Roles and responsibilities of evaluation team members in DCQA DCQA Tasks Responsible Staff Role Technical oversight Annette Bongiovanni Project Director and Technical Oversight Management and quality control Richard Columbia Chief of Party (COP) KII and FGD questionnaire design, pre￾testing review and version control Barbara de Zalduondo, Donna Espeut, Mike Toole & Suzanne Essama-Bibi Sr. Evaluation Advisors HH Survey questionnaire design, pre￾testing review, version control Zhuzhi Moore Survey Specialist/Data Collection Lead HF Survey questionnaire design, design, pre-testing review, version control Salima Mutima Sr. Evaluation Specialist KII and FGD Facilitation, production and submission of written transcripts Sr. Evaluation Advisors, Subcontractor staff, Field Coordinators HH survey data collection, supervision, management and submission Subcontractor staff HF survey data collection, supervision, management and submission Subcontractor staff Qualitative and quantitative data collection activities Zhuzhi Moore Survey Specialist/Data Collection Lead Training and quality assurance of qualitative data management and analysis Swati Sadaphal Data Analyst/DQA Lead Quantitative surveys data review for quality and data cleaning Kapil Ahmed Statistician Quantitative surveys data analysis Swati Sadaphal with Kapil Ahmed Data Analyst/DQA Lead Atlas.ti Data Administrator Betsy Nolan and Katherine Labombarde Data Administrators E–38 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table E2–6. Roles and responsibilities of evaluation team members in DCQA DCQA Tasks Responsible Staff Role Desk Review Manager Betsy Nolan Data Administrator Atlas.ti Data Coders Betsy Nolan (lead), Katherine Labombarde, Meredith Kerrigan (Tyler Wallace, IBTCI staff trained to provide supplemental support as needed) Data Coders Data Management and Analysis oversight Swati Sadaphal Data Analyst/DQA Lead ANNEX E. DATA ANALYSIS PLAN: PART 2 n E–39 SUB-ANNEX A: BIBLIOGRAPHY UNESCO Institute for Statistics 2017–18. http://uis.unesco.org/ SUB-ANNEX B: GUIDELINES FOR DATA COLLECTION AND SUBMISSION OF QUALITATIVE TRANSCRIPTS The interviewer is responsible for fulfilling the following roles and obligations before, during, and after the interview. Before the interview: ■ Study the interview guide. All types of interview guides by the respondent categories are provided as a separate attachment. ■ Study the informed consent document. A written consent format is provided as a separate attachment. ■ Confirm appointment with the participant. Day of the interview: ■ Ensure that you have all the necessary forms and equipment: two printed copies of consent forms, interview guide, note￾taking form, (a sample note-taking form is included on page 3) notebook, and a pen. Be sure you have the appropriate interview guide for that participant. Conducting the interview: ■ Greet the participant in a friendly manner to begin establishing positive rapport. ■ Briefly describe the steps of the interview process (informed consent, question and answer, their questions). ■ Obtain informed consent. Assure the participant that confidentiality will be maintained. Give a copy of signed informant consent to the participant. ■ Conduct the interview according to the interview guide. Address all questions or topics listed in the interview guide. Take notes that are as extensive as possible by using note-taking forms /notebook or by writing notes directly in the question guide under the relevant question. If it is not possible to record direct quotations, write down key words and phrases. Plan to expand those notes immediately after the interview. ■ At the end of the interview, give the participant the opportunity to ask questions and thank the participant. After the interview: ■ Expand your notes in MS WORD format using the interview guide format as soon as possible after each interview, preferably within 24 hours, while your memory is still fresh. This involves transforming your raw notes into a narrative with complete sentences, a task most conveniently done using a computer. If no computer is available within a day or so, you should expand your notes by hand. Transcribe handwritten notes into WORD format using the interview guide format before sending to IBTCI. Only Word format is an acceptable submission. No other format is acceptable. ■ Complete Section A and Section B. Type notes directly in the appropriate interview guide under the relevant question. ■ If you asked any additional interview questions which were not in the interview guide, write down the question as well as the participant response at the end of the interview guide. Write down questions about participant responses that need further consideration or follow-up, issues to pursue, new information, etc. This continual adjustment of the interview questions and techniques is part of the iterative nature of qualitative research. Be sure that you create separate, clearly labeled sections to report additional questions/your observations/additional comments. ■ Name the MS Word file using the format in Box 1: Each file should be labeled based on the type of data, followed by information useful in tracking its source. ■ Double-check that you have completed all sections of the interview notes and that they are appropriately labeled. ■ Submit your typed interview notes via email within 48 hours, using your IBTCI email only. Submit to Elisabeth Nolan enolan@ibtci.com and Katherine Labombarde klabombarde@ ibtci.com. When sending an email with data, you must ensure correct email addresses before sending the email. All correspondence must occur through IBTCI email; personal email must not be copied or used. ■ Assemble all signed consent forms into one envelope and submit them after returning from the field by mailing it to IBTCI HO. E–40 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES BOX E2-2. NOTE-TAKING FORM Date: Time: Name of Respondent: Organization: Interview Guide Number (ID#): Location: Interview Q# Responses/observations/comments BOX E2-1. FILE-NAMING CONVENTION [KII/FGD_OrganizationName_Location_LastNameInterviewer_ InterviewDate_1] Example of a KII conducted with respondent John Doe from MSF in Monrovia on 16 November 2017 by interviewer Sadaphal is presented as: KII_MSF_Monrovia_Sadaphal_11162017_1. If multiple interviews with the same organization are conducted on the same day, additional file names would end with _2, _3, etc. A N N E X F. D O C U M E N T S C O N S U LT E D n F–1 ANNEX F. DOCUMENTS CONSULTED FOR DESK REVIEW Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document ANNEX F CONTENTS Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F–2 Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F–3 Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F–7 Regional . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F–11 GUINEA LIBERIA SIERRA LEONE F–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES GUINEA Catholic Relief Services (2017). Emergency Access to Food for EVD Affected Guineans (EFSP): Final Program Report – March 2017. Cenciarelli O., Pietrpaoli, S., Malizia, Andrea., Carestia, M., D’Amico, F., et al. (2015). Ebola Virus Disease 2013-2014 Outbreak in West Africa: An Analysis of the Epidemic Spread and Response. International J of Microbiology. 2015; Article ID 769121, 12 pages. Emrey, B. (2016). Preparing for future shocks: Building resilient health systems. In J. Charles, T. Ifafore, & K. Greene (Eds.). USAID Global Health Mini-University. Evans. D.K., Goldstein, M., & Popova, A. (2015). “Health-care worker mortality and the legacy of the Ebola epidemic.” The Lancet Global Health 3 (8): e439–e440. https://www.thelancet. com/journals/langlo/article/PIIS2214-109X(15)00065-0/fulltext Global Health Ebola Team. (2016, July 14). Guinea TDY Debrief. Guinea, Conakry. Government of Guinea. (2015). Post-Ebola Socioeconomic Recovery and Resilience Strategy 2015- 2017: Unofficial translation. Retrieved from https://papersmart.unmeetings.org/ media2/7650686/guinea-recovery-strategy-summary-non￾official-translation-en-3-.pdf HC3: Guinea. Retrieved 11/3/0/2017 from https:// healthcommcapacity.org/where-we-work/guinea-2/ HFG. (2016, August). Guinea M&E Plan. Conakry, August 2016. International Federation of Red Cross and Red Crescent Societies. “Red Cross Red Crescent denounces continued violence against volunteers working to stop the spread of Ebola.” February 2015. Retrieved from http://www.ifrc.org/en/news-and-media/press￾releases/africa/guinea/red-cross-red-crescent-denounces￾continued-violence-against-volunteers-working-to-stop-the￾spread-of-ebola/ IntraHealth (2018). Retrieved from IntraHealth website: https:// www.intrahealth.org/news/intrahealth-receives-grand￾challenge-award-for-ebola-response. Accessed March 19, 2018. Nfor, E, and C. Bahati, L. Camara. 2017. Guinea National Pharmaceutical Management System Assessment. Parpia AS, Ndeffo-Mbah ML, Wenzel NS, Galvani AP. (2016). Effects of Response to 2014–2015 Ebola Outbreak on Deaths from Malaria, HIV/AIDS, and Tuberculosis, West Africa. Emerg Infect Dis. 2016; 22(3):433-441. https://dx.doi.org/10.3201/ eid2203.150977 Search for Common Ground. (2017, June 30). “Search for Common Ground Rebuilding Together”: Community-Driven Reconciliation and Enhanced Communication in Guinea Forestière. Final report. USAID (2014). USAID Development Innovation Accelerator Broad Agency Announcement for Fighting Ebola: A Grand Challenge for Development. Nov 2014 http://pdf.usaid.gov/ pdf_docs/PBAAB273.pdf. USAID. (2015). Pillar II: Mitigating Second-Order Impacts of the Ebola Virus Disease (EVD) Epidemic. A Strategic Framework for Liberia, Guinea, Sierra Leone. Washington, DC. USAID Guinea. (2015). Country Development Cooperation Strategy (2015–2020). Conakry, Guinea. Retrieved from https://www. usaid.gov/sites/default/files/documents/1860/Guinea_CDCS_ May_2020.pdf USAID Guinea. (2016a). 2016 Year-end Portfolio Reviews: Health. Conakry, November 22, 2016. USAID Guinea. (2016b). 2016 Year-end Portfolio Reviews: Governance. Conakry, November, 2017. World Health Organization. (2016.). Ebola Situation Report, January 20, 2016. http://apps.who.int/ebola/current-situation/ ebola-situation-report-20-january-2016 World Health Organization. (2014). “Ground zero in Guinea: the Ebola outbreak smoulders – undetected – for more than 3 months: A retrospective on the first cases of the outbreak. September 2014.” Retrieved 12/18/2017 from http://www.who. int/csr/disease/ebola/ebola-6-months/guinea/en/ World Bank. (2016). 2014 – 2016 West Africa Ebola Crisis: Impact Update. May 10, 2016. http://pubdocs.worldbank. org/en/297531463677588074/Ebola-Economic-Impact-and￾Lessons-Paper-short-version.pdf World Bank. (2015, April 16). Summary of the Ebola Recovery Plan. Retrieved from http://www.worldbank.org/en/topic/ ebola/brief/summary-on-the-ebola-recovery-plan-guinea World Bank. (2017, October). The World Bank in Guinea: Overview. Retrieved from http://www.worldbank.org/en/ country/guinea/overview A N N E X F. D O C U M E N T S C O N S U LT E D n F–3 LIBERIA 1. Date of publication is not in the document; to be located elsewhere. ACAPS. (2015). Sierra Leone Multi-sector Needs Assessment Report. ACAPS. (2016). Beyond a public health emergency: Potential secondary humanitarian impacts of a large-scale ebola outbreak. Retrieved from https://www.acaps.org/special-report/beyond￾public-health-emergency-potential-secondary-humanitarian￾impacts-large-scale ACDI/VOCA. (2016). Quarterly Program Performance Report. Ebola Recovery and Resilience Program (ERRP). FY2016/ Quarter 3/April 1 - June 30, 2016. Action Against Hunger - US. (2016). Annual Results Report - FFP - G - 16-00054. UNDG West and Central Africa. (2015). Socio-Economic Impact of Ebola Virus Disease in West African Countries. A call for national and regional containment, recovery and prevention. Retrieved from https://undg.org/wp-content/uploads/2015/07/ Ebola.png Ansumana, R., Keitell, S., Roberts, G. M., Ntoumi, F., Petersen, E., Ippolito, G., & Zumla, A. (2017). Impact of infectious disease epidemics on tuberculosis diagnostic, management, and prevention services: experiences and lessons from the 2014-2015 Ebola virus disease outbreak in West Africa. Int J Infect Dis, 56, 101-104. doi:10.1016/j.ijid.2016.10.010 Barry, A., Bardon O’Fallon, J., Hazerjian, J., & Brodish, P. (2015). A rapid assessment of Ebola-related implications for RMNCH service delivery and utilization in Guinea. Brolin Ribacke, K. J., Saulnier, D. D., Eriksson, A., & von Schreeb, J. (2016). Effects of the West Africa Ebola Virus Disease on Health-Care Utilization - A Systematic Review. Front Public Health, 4, 222. doi:10.3389/fpubh.2016.00222 Cash Learning Partnership (CaLP). (2016a). Learning event report. CaLP. Liberia. Cash Learning Partnership (CaLP). (2016b). Scope of Work for Food for Peace (FFP) Ebola Research Documentation Report [Scope of Work]. (Action Against Hunger). Cash Learning Partnership (CaLP). (2017). USAID/FFP Quarterly Report, Q2 2017. Dahlmia, S. (2014, October 24, 2014). How foreign aid screwed up Liberia’s ability to fight Ebola. The Week. Ellis, S. (2005). The Mask of Anarchy. The Destruction of Liberia and the Religious Dimension of an African Civil War. Second Edition. New York: New York University Press. Emrey, B. (2016). Preparing for future shocks: Building resilient health systems. In J. Charles, T. Ifafore, & K. Greene (Eds.). USAID Global Health Mini-University. Fast, L., & Waugaman, A. (2016). Fighting Ebola with Information: Learning from the Use of Data, Information, and Digital Technologies in the West Africa Ebola Outbreak Response. Retrieved from www.globalinnovationexchange. org/fighting-ebola-information FEWS Net. (2017a). Ebola and Livelihoods. The effect of the Ebola virus on household livelihoods in Guinea, Liberia and Sierra Leone. FEWS Net. (2017b). Famine Early Warning Systems Network (FEWS Net III) to 3 and 4. Quarterly Performance Report. January - March 2017. FHI360. (2016). Understanding the Potential Role of Mobile Money in Civil Servant Payments in Liberia: A Contextual Analysis. Folayan, M. O., Haire, B. G., & Brown, B. (2016). Critical role of ethics in clinical management and public health response to the West Africa Ebola epidemic. Risk Manag Health Policy, 9, 55-65. doi:10.2147/RMHP.S83907 Gillespie, A. M., Obregon, R., El Asawi, R., Richey, C., Manoncourt, E., Joshi, K. … Quereshi, S. (2016). Social Mobilization and Community Engagement Central to the Ebola Response in West Africa: Lessons for Future Public Health Emergencies. Glob Health Sci Pract, 4(4), 626-646. doi:10.9745/ GHSP-D-16-00226 Government of Liberia. (2014). Liberia Economic Stabilization and Recovery Plan. Monrovia, Liberia. Government of Liberia. (2015). WASH in schools. Liberia’s first step to recovery from Ebola. Monrovia, Liberia: Government of Liberia. (2012). Liberia Vision 2030. Monrovia: Government of Liberia. Government of Liberia. (2015a). Agenda for Transformation. Steps toward Liberia rising 2030. Monrovia: Government of Liberia. Government of Liberia, Ministry of Health. (n.d.).1 The National Health Policy (2011–2021). Monrovia, Liberia: Government of Liberia. Government of Liberia, Ministry of Health. (2015). Investment Plan for Building a Resilient Health System. Liberia 2015–2021. Monrovia, Liberia: Government of Liberia. Government of Liberia, Ministry of Health. (2016). EVD survivors care and support national policy. Monrovia, Liberia: Government of Liberia. F–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Government of Liberia, Ministry of Health and Social Work. (2010). National health and social welfare policy; National health and social welfare plan 2011-2022. Monrovia: Government of Liberia Retrieved from http://www.nationalplanningcycles.org/ sites/default/files/country_docs/Liberia/ndp_liberia.pdf. Halsey, E. S. (2016). An Outbreak of Fearsome Photos and Headlines: Ebola and Local Newspapers in West Africa. Am J Trop Med Hyg, 95(5), 988-992. doi:10.4269/ajtmh.16-0245 Hart, L., Street, D., & Kulatilaka, H. (2017). Building capacity for resilient health systems. Lessons learned from Sierra Leone, Guinea and Liberia in the time of Ebola. Washington, DC: Hira, S., & Piot, P. (2016). The counter effects of the Ebola epidemic on control and treatment of HIV/AIDS, tuberculosis, and malaria in West Africa. AIDS, 30(16), 2555-2559. doi:10.1097/QAD.0000000000001231 IBI International. (2016). USAID/Liberia Governance and Economic Management Support (USAID-GEMS) Project. Final Report: July 2011–June 2016. Washington DC: ICT Works. (2017). “How We Fought Ebola with Information.” International Business & Technical Consultants, Inc. (IBTCI). (2017a). USAID Africa Bureau Ebola Pillar II Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone, and Liberia. Sierra Leone Desk Review. Washington, DC. International Business & Technical Consultants, Inc. (IBTCI). (2017b). Ebola Pillar II Monitoring, Evaluation, and Learning Project: Performance Evaluation Inception Report. Washington, DC. International Rescue Committee (IRC). (2015). Partnership for advancing community-based services, Liberia. Knowledge, attitudes and practices (KAP) survey among caregivers of children under five years of age in the counties of Bong, Lofa, and Nimba in Liberia. Report. Liberia: International Rescue Committee (IRC). (2016a). Partnership for Advancing Community-Based Services (PACS) Quarterly report. April 1, 2016 to June 30, 2016. International Rescue Committee (IRC). (2016b). Indicator matrix: Targets Achieved (PACS) [table]. IntraHealth. (2015). USAID’s Fighting Ebola Grand Challenge: mHero. Quarterly Report. June 6, 2015 - September 5, 2015. IREX. (2016a). Liberia Civil Society and Media Leadership Program (CSML) (Vol. 2017, pp. Project overview, covering broad range of activities, from developing a CSO media plan to strengthening the Traditional Council, to creating model radio stations.): IREX. IREX. (2016b). Trusted Voices in Challenging Times: Civil Society and Media Leadership (CSML) Program. Final Report. Washington DC: Jeans, H., Castillo, G. E., & Thomas, S. (2017). The Future is a Choice. Absorb, adapt, transform. Resilience Capacities. Retrieved from Oxfam Policy and Practice website: https:// policy-practice.oxfam.org.uk/publications/absorb-adapt￾transform-resilience-capacities-620178 JHPEIGO. (2016). Maternal and Child Survival Program (MCSP): Human Resources for Health in Liberia. Quarterly Report. Q1 July 1, 2016-September 30, 2016 (Fiscal Year 2016). JHPEIGO. (2017a). Non GHET Indicators [table]. JHPEIGO. (2017b). MCSP Restoration of Health Services (RHS) in Liberia. GHET Quarterly Report. July-September 2017. Washington DC: JHPEIGO. (ND). MCSP Liberia: Human Resources for Health. Strengthening pre-service education. Registered midwives (RMs) and medical lab technicians (MLTs). Timeframe April 2016-2018. (pp. 18): JHPEIGO. John Snow International (JSI). (2017a). ETP&SS Monitoring and evaluation strategy: October 2017. Updated July 2017. John Snow International (JSI). (2017b). Advancing Partners and Communities baseline Ebola survivor assessment. John Snow International (JSI). (n.d.). USAID | DELIVER project. Final country report. Liberia. Johns Hopkins University Center for Communication Programs (JHU/CCP). (2016). Pillar II Liberia JHU Indicator Report, Q2. Baltimore, MD Johns Hopkins University Center for Communication Programs (JHU/CCP). (ND). PII Liberia JHU Indicator Report [HC3 Project]. Kaplan, Z. A., Kyle, P., Shugart, C., & Moody, A. (2012). Developing Public Private Partnerships in Liberia. Retrieved from http://dx.doi.org/10.1596/978-0-8213-9480-9 Kobayashi, M., Beer, K. D., Bjork, A., Chatham-Stephens, K., Cherry, C. C., Arzoaquoi, S, Nyenswah, T. G. (2015). Community Knowledge, Attitudes, and Practices Regarding Ebola Virus Disease - Five Counties, Liberia, September￾October, 2014. MMWR Morb Mortal Wkly Rep, 64(26), 714- 718. Koch, T. (2016). Ebola in West Africa: lessons we may have learned. Int J Epidemiol, 45(1), 5-12. doi:10.1093/ije/dyv324 Liberia Ministry of Finance and Development Planning (MoFDP), 2015. MoFDP Strategic Plan 2015–2018. A N N E X F. D O C U M E N T S C O N S U LT E D n F–5 Liberia Strategic Analysis (LSA). (2017). USAID/Liberia Collaborative Support for Health (CSH): Midterm performance evaluation report. Retrieved from Liu, S. (2017). Wartime educational loss and attitudes toward democratic institutions. Working Paper No. 175. Liu, S. Q., Rayner, S., & Zhang, B. (2015). How Ebola has been evolving in West Africa. Trends Microbiol, 23(7), 387-388. doi:10.1016/j.tim.2015.05.003 Management Sciences for Health (MSH). (2016). Liberia. Collaborative Support for Health (CSH) program. Annual summary report for fiscal year 2016 (October 1, 2015 - September 30, 2016). Manguvo, A., & Mafuvadze, B. (2015). The impact of traditional and religious practices on the spread of Ebola in West Africa: time for a strategic shift. Pan Afr Med J, 22 Suppl 1, 9. doi:10.11694/pamj.supp.2015.22.1.6190 Mercy Corps. (2016). Economic Recovery from Ebola for Liberia (EREL) Program. Funded by USAID/Food for Peace (FFP). Fiscal Year (FY) 2017 Quarter 1 Report October 1 - December 31, 2016. Retrieved from Mohammed, E., Bedri, A., & Demeke, F. (2016). Liberia. Emergency Food Security and Ebola Response Program. Final Evaluation Report (Draft). Final Results Report, draft. (PCI). Navarro, R. (2014). Pillar II: Mitigating second-order impacts of the Ebola Virus Disease (EVD) Epidemic. A strategic framework for Liberia, Guinea and Sierra Leone. USAID/Africa Bureau, Africa Ebola Unit. [Unpublished]. Nyanfore, D. K., II. (2016). History of Ebola in Liberia: A Case of Under-Development. The Patriotic Vanguard. Parpia, A. S. M., Ndeffo-Mbah, M. L., Wenzel, N. S., & Galvani, A. P. (2016). Impact of the 2014-2015 Ebola Outbreak on Malaria, HIV, and Tuberculosis in West Africa. Emerging Infectious Diseases (in press). Emerging Infectious Diseases, 22(3), 433-441. doi:10.3201/eid2203.150977 Parshley, L. (2016). The Aftermath of Ebola in Liberia. The Atlantic (July/August 2016 issue), 17. PCI. (2016). Emergency Food Security and Ebola Response Program. Liberia. Protect and Empower for Ebola Resilience (PEER). Final Report. PCI. (2015). Protect and Empower for Ebola Resilience (PEER). Bomi and Grand Cape Mount Counties. Baseline Evaluation Report. Risso-Gill, I., & Finnegan, L. (2015). Children’s Ebola Recovery Assessment (CERA) in Sierra Leone. Santermans, E., Robesyn, E., Ganyani, T., Sudre, B., Faes, C., Quinten, C., … Plachouras, D. (2016). Spatiotemporal Evolution of Ebola Virus Disease at Sub-National Level during the 2014 West Africa Epidemic: Model Scrutiny and Data Meagerness. PLoS One, 11(1), e0147172. doi:10.1371/journal. pone.0147172 Save the Children (SC). (2016). Emergency Food Assistance for Ebola Affected Families in Liberia - Margibi and Bong Counties. Fiscal Year 2016 Annual Results Report. October 1, 2015 - September 30, 2016. Shultz, J. M., Cooper, J. L., Baingana, F., Oquendo, M. A., Espinel, Z., Althouse, B. M. … Rechkemmer, A. (2016). The Role of Fear-Related Behaviors in the 2013-2016 West Africa Ebola Virus Disease Outbreak. Curr Psychiatry Rep, 18(11), 104. doi:10.1007/s11920-016-0741-y Sridhar, S., Brouqui, P., Fontaine, J., Perivier, I., Ruscassier, P., Gautret, P., & Regner, I. (2016). Risk perceptions of MSF healthcare workers on the recent Ebola epidemic in West Africa. New Microbes New Infect, 12, 61-68. doi:10.1016/j. nmni.2016.04.010 STEPS Centre. (2015). The social life of infectious diseases. STEPS Impact Stories. Retrieved from https://medium.com/ steps-impact-stories/the-social-life-of-infectious-diseases￾d694d99114d4#.lib3e958g Strong, A., & Schwartz, D. A. (2016). Sociocultural aspects of risk to pregnant women during the 2013-2015 multinational Ebola virus outbreak in West Africa. Health Care Women Int, 37(8), 922-942. doi:10.1080/07399332.2016.1167896 The Lancet Infectious Diseases. (2016). Infectious disease emergencies: taking the long-term view. Lancet Infect Dis, 16(12), 1305. doi:10.1016/S1473-3099(16)30483-2 The Mitchell Group. (2014). Liberia monitoring and evaluation program (L-MEP) year four annual report (final). Retrieved from Monrovia, Liberia. The World Bank. (2015, April 16). Summary on the Ebola Recovery Plan: Liberia - Economic Stabilization and Recovery Plan (ESRP). Understanding Poverty. Retrieved from http:// www.worldbank.org/en/topic/ebola/brief/summary-on-the￾ebola-recovery-plan-liberia-economic-stabilization-and￾recovery-plan-esrp The World Bank. (2015a). West African Health Ministers and Development Partners Exchange Best Practices on Ebola Prevention (October 20, 2016). Retrieved from News (http://web.worldbank.org/WBSITE/EXTERNAL/ NEWS/0,pagePK:34382~piPK:34439~theSitePK:4607,00. html) F–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES The World Bank. (2015b). Net ODA received per capita (current US$). Retrieved from https://data.worldbank.org/indicator/ DT.ODA.ODAT.PC.ZS?locations=LR The World Bank. (2017, October 16). “The World Bank in Liberia.” Retrieved from http://www.worldbank.org/en/ country/liberia/overview Frontier Research U & UNDP. (2016). Liberia: Financial Diagnostic Update and Feasibility Study for the Possible Entry of a Third-Party Cash Management Partner or Agent Aggregator. Retrieved from Monrovia, Liberia. UNDG West and Central Africa. (2015). Socio-Economic Impact of Ebola Virus Disease in West African Countries. A call for national and regional containment, recovery and prevention. Retrieved from https://undg.org/wp-content/uploads/2015/07/Ebola.png UNECA. (2015). Socio-economic Impacts of Ebola on Africa. Revised Edition Retrieved from Addis Ababa, Ethiopia. UNICEF Liberia. (2017). ECRL Fifth Progress Report. Submitted to USAID, June 2017. UNICEF Liberia. (2016). Monthly Updates - August 2016. UNICEF/Liberia. (2015). Contribution Summary: RUTF in-kind support to Liberia. Retrieved from Monrovia, Liberia. United Nations Economic Commission for Africa. (2015). Socio￾economic Impacts of Ebola on Africa. Retrieved from Addis Ababa, Ethiopia: United Nations High Commission for Refugees (UNHCR). (2015, 18 December, 2015). Liberia, Cote d’Ivoire, UNHCR facilitate voluntary repatriation of Ivorian refugees after interruption by Ebola outbreak. Retrieved from https://unmil. unmissions.org/liberia-cote-d’ivoire-unhcr-facilitate-voluntary￾repatriation-ivorian-refugees-after-interruption USAID. (2015a). mHero: Leveraging HIS and Mobile Phones to Support Frontline Health Workers AMID the Ebola Outbreak in West Africa (BAA-EBOLA-2014 Addendum 1). Washington, DC. USAID. (2015b). (SBU) Modification of Assistance. PACS project. Modification to PACS award USAID. (2015c). Strategic Program Review Health, Pillar II Health. Washington, DC. USAID. (2016a). Feed the Future Global Performance Evaluation. Final Evaluation Report Annexes. Retrieved from Washington, DC. USAID. (2016b). USAID Ebola Fact Sheet 12. Retrieved from west_africa_ebola_fs12_09-30-2016. USAID. (2016c). MCSP Liberia: Restoration of health services. Midline assessment findings - in brief. USAID. (2016d). Strategic Program Review Health. Pillar IV – Global Health Security Agenda & Ebola Preparedness USAID Africa Bureau. (2015). SBU AEU Strategic Program Review 10-28-2015. USAID Bureau of Food for Peace (FFP). (2016, December). Ebola Response in Liberia. USAID Bureau of Global Health. (2015). SBU GHB slides Strategic Program Review July 2015_6.30_FINAL. USAID Center for Accelerating Innovation and Impact (CII). (2015). SBU Fighting Ebola. A grand challenge for development. (Internal Strategic Review 10.1). USAID Innovation Laboratory. (2015). The Lab’s Contribution to USAID’s Ebola Efforts. Strategic Program Review. Washington, DC. USAID Innovation Laboratory. (ND). Lab activities in Liberia. Internal Mission portfolio review. FY 2016/2017. Monrovia, Liberia. USAID Office of Inspector General. (2016). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, Second Quarter. Washington, DC. USAID/Burea of Global Health. (ND). MCSP GHET Indicators 1a, 1b and 2. USAID/Liberia. (2012). USAID Liberia Country Development Cooperation Strategy, 2013-2017. USAID. USAID/Liberia. (2015). Program Description. Mobile Money Roll-Out in the Liberian Health Sector. USAID/Liberia draft for Ebola Supplemental Funding under mSTAR. Retrieved from USAID/Liberia. (2016a). USAID Liberia Country Development Cooperation Strategy, 2013-2018. Revision 2. (USAID). USAID/Liberia. (2016b). USAID/Liberia Performance Management Plan (FY 2013 - FY 2017). 26. USAID/Liberia. (2016c). Liberia: Recovering from Ebola and Preparing for the Future. Retrieved from Washington, DC: USAID/Liberia. (2017). Lab Activities in Liberia. Internal Mission Portfolio Review. FY2016/17. Monrovia, Liberia. USAID/Liberia. (ND). MCSP Liberia: Human Resources for Health. Strengthening pre-service education. Registered midwives (RMs) and medical lab technicians (MLTs). Timeframe: April 2016-June 2018. World Food Program (WFP). (2016). APS USAID FFP/202(e). Final results report form. PRRO 200550. Retrieved from World Health Organization. (2000). World Health Report. Geneva. A N N E X F. D O C U M E N T S C O N S U LT E D n F–7 World Health Organization. (2008). International Health Partnership and related Initiatives (IHP+) Harmonization of Health in Africa (HHA). Retrieved from Lusaka: http://www. who.int/healthsystems/FINAL_IHP_LusakaWayForward.pdf World Health Organization. (2015). Ebola Situation Report. 11 November, 2015. http://apps.who.int/ebola/current-situation/ ebola-situation-report-11-november-2015 World Health Organization. (2016a). Latest Ebola outbreak over in Liberia; West Africa is at zero, but new flare-ups are likely to occur. Retrieved from http://www.who.int/mediacentre/news/ releases/2016/ebola-zero-liberia/en/ World Health Organization. (2016b, June 6). End of the most recent Ebola virus disease outbreak in Liberia. Retrieved from http://www.who.int/mediacentre/news/releases/2016/ebola￾liberia/en/ World Health Organization. (2016c, June 6). Ebola Outbreak 2014-2015. Retrieved from http://www.who.int/csr/disease/ ebola/en/ Zielinski, C., Kebede, D., Mbondji, P. E., Sanou, I., Kouvividila, W., & Lusamba-Dikassa, P.-S. (2014). Knowledge systems in health in sub-Saharan Africa: results of a questionnaire-based survey. Journal of the Royal Society of Medicine, 107(1S), 6. doi:10.1177/0141076813518525 SIERRA LEONE ACDI/VOCA. (2016a). Annex A: Scope of Work. Final Project Performance Evaluation. (Sierra Leone Sustainable Nutrition and Agriculture Promotion (SNAP) Project). ACDI/VOCA. (2016b). SNAP Final Evaluation Summary Report. Annex D: Summary of Persons Interviewed for Qualitative Data Collection. ACDI/VOCA. (2016c). Annex F: Final IPTT [table]. ACDI/VOCA. (2016d). Annex G: SNAP Theory of Change. ACDI/VOCA. (2016e). Attachment D: Indicator List. (SNAP+). ACDI/VOCA. (2016, October 14). Final Evaluation Plan (Final Draft). (SNAP). ACDI/VOCA & Nestbuilders International (NBI). (2016). FY16 Annual Results Survey on Agricultural Technology Adoption. (SNAP). ACDI/VOCA & Nestbuilders International (NBI). (2016). Program Endline Survey. (SNAP). ACDI/VOCA. (2016f). SNAP End-Line Follow-Up Survey. Follow￾up survey of selected agricultural indicators. ACDI/VOCA. (2016g). Sustainable Nutrition and Agricultural Program Plus (SNAP+): FY16 Annual Report, October 2015- September 2016. ACDI/VOCA. (2017a). Emergency Food Security Program (EFSP): FY17 Quarterly Report: January- March 2017. ACDI/VOCA. (2017b). Annex A: Scope of Work. Final Project Performance Evaluation. (SNAP). ACDI/VOCA. (2017c). Final Evaluation Report. (SNAP). ACDI/VOCA. (2017d). Final Evaluation Report (No Annexes). (SNAP). ACDI/VOCA. (2017e). Revised Indicator Performance Tracking Table: Sustainable Nutrition and Agriculture Promotion Project (SNAP+). January 1, 2017- December 31, 2017 [table]. ACDI/VOCA. (2017f). Sustainable Nutrition and Agricultural Program Plus (SNAP+): FY17 Quarterly Report 1: October￾December 2016. ACDI/VOCA & International Solutions Group. (2017). Final Evaluation FFP Debriefing. Sustainable Nutrition and Agriculture Promotion (SNAP) Program: June 1, 2017 [PowerPoint slides]. ACDI/VOCA. (n.d.). Selected Cash Transfer Survey Results [PowerPoint slides]. ACDI/VOCA. (n.d.). Summary of Key Variables Assessed During EFSP (SNAP+) Cash Transfer Exit Surveys. Alva, S., Cunningham, M. & Davis, N. (2016). Post-Ebola Health Services Strengthening: Baseline Facility Assessment Report. (Arlington, VA: Advancing Partners & Communities). CARE. (2015). Monitoring & Evaluation Plan for Rapid Ebola Social Safety Network and Economic Recovery Program (RESSNER). CARE. (2017). FY17 Q2 Program Report. (RESSNER) Sierra Leone). CARE. (n.d.). Annex A: RESSNER II Logframe. CARE. (n.d.). Logical Framework. (RESSNER). CRS. (2015a). Annex E: Monitoring and Evaluation Plan: August 15, 2015–December 30, 2016. (Cash-Based Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District). F–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES CRS. (2015b). Attachment G: AAR Technical Sectors Tracking Table [table]. (Cash Based Food Security Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District). CRS. (n.d.). Field Agent Home Visit Report Form. (Emergency Food Security Program (EFSP) Sierra Leone). CRS. (n.d.). Annex F: Logframe and Performance Monitoring Plan [table]. (Cash-Based Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District). CRS. (n.d.). Payment Voucher Form [table]. CRS. (2016a). Annual Results Report. (Cash Based Food Security Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District). CRS. (2016b). Attachment N: Actuals Table [table]. (Sierra Leone EFSP). CRS. (2016c). Baseline Survey Report. (USAID/FFP Emergency Food Security Program in Kenema District, Sierra Leone, Agreement N AID-FFP-G-15-00080). CRS. (2016d). CRS Sierra Leone Indicator List [table]. (Cash￾Based Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District). CRS. (2016e). CRS Sierra Leone Program Success Stories: September 2016. CRS. (2016f). EFSP Post Distribution Monitoring Survey. (EFSP Cash Assistance to EVD-Affected Sierra Leonean 2015-2017). CRS. (2016g). Market Price Data Collection form [table]. CRS. (2016h). Project Overview: Sierra Leone Country Program. (Cash Based Food Security Assistance for Ebola Virus (EVD)- Affected Sierra Leoneans in Kenema District). CRS. (2016i). Sample Payment Data Base [table]. CRS. (2017, July). Scope of Work (SoW). Final Evaluation of the Cash based Emergency Food Security Project (EFSP). Davis, N., Alva, S. & Son, Y. (2017). Comprehensive Program for Ebola Survivors: Baseline Report. Arlington, VA: Advancing Partners & Communities. Diop, M., & Crookes, Y. (2015). The Socio-Economic Impacts of Ebola in Sierra Leone: Results from a High Frequency Cell Phone Survey, Round 1. World Bank Group, IPA & Statistics Sierra Leone. Evans, D.K., et al. (2015). Health-care worker mortality and the legacy of the Ebola epidemic, The Lancet Global Health, 3(8), e439 - e440 Fast, L., & Waugaman, A. (2016). Fighting Ebola with Information: Learning From Data and Information Flows in the West Africa Ebola Response. Washington, DC: USAID. Government of Sierra Leone. (2013). Agenda for Prosperity, Road to Middle Income Status; Sierra Leone’s Third Generation Poverty Reduction Strategy (2013-2018). Sierra Leone. Government of Sierra Leone (n.d.). The President’s Recovery Priorities. Retrieved 10/13/2017 from http://www. presidentsrecoverypriorities.gov.sl/resources Government of Sierra Leone, Ministry of Social Welfare, Gender and Children’s Affairs. (2014). Report of the Multisector Impact Assessment of Gender Dimensions of the Ebola Virus Disease (EVD) in Sierra Leone. Sierra Leone. Government of Sierra Leone. (2017a). Sierra Leone Project Overview. Sierra Leone. Government of Sierra Leone. (2017b). Sierra Leone Project Update: A Preliminary Response to Controlled Disposal of Expired Products. Sierra Leone. HC3. (2016). Work Plan: Sierra Leone Monthly Report [table]. HC3-JHU. (2016a). Ebola Responses and use of Health Services in Sierra Leone. Qualitative Study: Final Version December 2016. HC3-JHU. (2016b). Health Communication Capacity Collaborative, Sierra Leone: Factors influencing health-seeking behaviors and access to use of RMNCH services. HC3-JHU. (2016c). Health Communication Capacity Collaborative, Sierra Leone: Factors influencing health-seeking behaviors access and use of RMNCH services. Study Report. HC3-JHU. (2016d). PII Sierra Leone JHU Indicator Report. HC3. (2017aa). HC3’s Project Journey, Using Social and Behavior Change Communication to Increase Trust and Address RMNCH in Sierra Leone July 2015-April 2017 [PowerPoint slides]. HC3-JHU. (2017b). HC3 Sierra Leone Country Program End of Project Report. Using Social and Behavior Change Communication to Increase Trust and Address Reproductive, Maternal, Newborn and Child Health in Sierra Leone. HC3-JHU. (2017c). HC3 Sierra Leone Facilities Participating in Makeover Process [table]. HC3-JHU. (2017d). HC3 Sierra Leone: List of Contacts [table]. HC3-JHU. (2017e). HC3 Sierra Leone Project Highlights. HC3-JHU. (2017f). Health Communication Capacity Collaborative, Sierra Leone: Outcomes Monitoring Assessment, Rapid Assessment and Client Exit Interviews, WAVE 1: Report. A N N E X F. D O C U M E N T S C O N S U LT E D n F–9 HC3-JHU. (2017g). Health Communication Capacity Collaborative, Sierra Leone: Outcomes Monitoring Assessment, Rapid Assessment and Client Exit Interviews, WAVE 2: Report. HC3-JHU. (2017h). Health Communication Capacity Collaborative, Sierra Leone: Trust in the Health System and Barriers to Healthcare Utilization in Sierra Leone: Literature Review. HC3-JHU. (2017i). PII Sierra Leone JHU Indicator Report. Implementing Partners, Sierra Leone, Portfolio Review [Powerpoint Presentation]. June 2017. Innovations for Poverty Action. (2015). The Socio-Economic Impacts of Ebola on Households in Sierra Leone. International Organization for Migration (IOM). (2017). IOM National Ebola Training Academy-Phase II. Establishment of Infection Prevention Control Short Course Departments and Mobile Training in Sierra Leone. Statement of Work and Project Logframe. AID-GH-IO-16-00001. Amendment I. Jackson-Garrett, D. (2016). “Sierra Leone’s Girls in the Aftermath of Ebola.” Council of Foreign Relations. JHU. (2017). Work Plan: PII Sierra Leone [table]. JSI. (2016a). APC Services Monthly Report Sierra Leone [table]. Advancing Partners & Communities. JSI. (2016b). Survivor Monthly Report Sierra Leone [table]. Advancing Partners & Communities. JSI. (2017a). A Data Quality Review of HMIS Data for Advancing Partners and Communities Project, Sierra Leone: DRAFT. May 24, 2017. Advancing Partners & Communities. JSI. (2017, March). Sierra Leone-Overarching Indicators. Advancing Partners & Communities. JSI. (2017, February). Ebola Transmission Prevention and Survivor Services Program Overview February 2017. JSI. (2017, June). Ebola Transmission Prevention and Survivor Services Program Overview June 2017. JSI. (2017b). Improving Community Health Services in the Post￾Ebola Context [infographic fact sheet]. Advancing Partners & Communities, Sierra Leone. JSI. (2017c). Post Ebola Health Services Strengthening: Sierra Leone Quarterly Report, Quarter 3. April 1-July 2017. Advancing Partners & Communities. JSI. (2017d). Post-Ebola Recovery: Strengthening Primary and Community Health Services. Arlington, VA. Advancing Partners & Communities. JSI. (2017, September). The Post-Ebola Recovery of Health Services Program Overview September 2017. Morlai, T.A. (2016). Baseline Assessment Report. RESSNER Project in Bombali and Tonkolili Districts in Sierra Leone. (Care International Sierra Leone). MSD Consulting Limited-Sierra Leone. (2016). Mid-Term Evaluation Report. Cash Based Food Security Assistance for Ebola Virus Disease (EVD)-Affected Sierra Leoneans in Kenema District. (CRS). MSH. September 28, 2016. Continuous Results Monitoring and Support System (CRMS) Review Meeting [PowerPoint slides]. (SIAPS). MSH. (2016). SIAPS Monthly Report [table]. MSH. (2017a). Monitoring and Evaluation Plan. (SIAPS Sierra Leone Pillar II). MSH. (2017b). SIAPS Sierra Leone Pillar II GHET Quarterly Report. MSH. (n.d.). Continuous Results Monitoring and Support System Tracks Post-Ebola Recovery in Sierra Leone. SIAPS Technical Highlight. MSH. Strengthening Post-Ebola Recovery and Resiliency in Four Countries. SIAPS Technical Highlight. Murphy, E.; Oot, L.; Sethuraman, K. (2016). USAID Office of Food for Peace Food Security Desk Review for Sierra Leone, 2018–2022. Washington, DC: FHI 360/FANTA. Ombogo, T. (2016). Emergency Food Assistance to Port Loko District Project Baseline Survey Report. World Vision Sierra Leone Emergency Food Security Program. Ombogo, T. Cash and Vouchers Beyond Ebola. World Vision Sierra Leone Emergency Food Assistance in Port Loko. Ombogo, T. “Vouchers and training, we now have more to eat!” World Vision Sierra Leone Emergency Food Assistance in Port Loko. Radice, H.W. (2017). Cash Transfers for Food Security in epidemics: A review of the Food for Peace Response to the Ebola crisis in Liberia and Sierra Leone. Final Version. Washington, D.C, USA. The Cash Learning Partnership (CaLP)/USAID. Save the Children. (2015, August). Baseline Report. Kailahun Food for Emergency for Ebola Virus Disease Support (FEEDS). Save the Children. (2016). Fiscal Year 2016 Annual Results Report. October 1, 2015-September 30, 2016. (FEEDS). Save the Children. (2017). Quarterly Programmatic Report. January 1, 2017-March 31, 2017. Washington, D.C, USA. (FEEDS). Save the Children. FEEDS Project Logframe and MEAL Plan [table]. F–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Save the Children. Tab 1: Indicators Performance Tracking Table (IPTT), Sierra Leone FEEDS [table]. Save the Children. Draft Meal Plan. (Food for Emergency Ebola Virus Disease Support (FEEDS)). Sesay, M. (2017). Systems for Improved Access to Pharmaceuticals and Services: Post-Ebola Recovery Sierra Leone Newsletter. Volume 1, Issue 2. SIAPS. (2017, April). Sierra Leone Post-Ebola Pharmaceutical Management Systems Strengthening Project Progress Update. Statistics Sierra Leone (SSL) & ICF International. (2014). Sierra Leone Demographic and Health Survey 2013. Freetown, Sierra Leone and Rockville, Maryland, USA: SSL and ICF International. Stulman, M. (2016, June, 28). “Ebola’s Second Punch: Hunger.” CRS, 1-5 Streifel, C. (2015). “How did Ebola Impact Maternal and Child Health in Liberia and Sierra Leone?” Washington DC: Center for Strategic and International Studies, Global Health Policy Center. Turay, A., Turay, S., Margolis, D., Rosas, N., & Vidarte, R. (2015). Sierra Leone 2014 Labor Force Survey Report. World Bank Group, IPA & Statistics Sierra Leone. UNICEF. (2014, September 11). Sierra Leone Country Programme Document: 2015-2018. Revised draft submitted September 11, 2014. UNICEF. (2017a). Briefing Note-Update on UNICEF’s USAID Supported Work, May 2017. UNICEF. (2017b). Health Information Systems and mHealth Support in Sierra Leone-UNICEF. UNICEF. (2017c). Measure and Connect Logframe [table]. UNICEF. (2017d). Proposal: Strengthening Post- Ebola Health Governance in Sierra Leone: Project Period April 1, 2017- September 30, 2018. UNICEF, Sierra Leone. (2017). Progress Report: Supported Health Information Systems and mHealth Programme Project: September 27, 2016 to March 15, 2017. USAID. (2017). Health Programming in Sierra Leone (Non-Ebola). June 2017. USAID, (2015), “Strategic Program Review Health,” Pillar II Heath, October 1, 2015 USAID. (2017). Ebola Pillar II Learning Conference in Sierra Leone [PowerPoint presentation]. Freetown, Sierra Leone: November 16, 2017. USAID Africa Bureau. (2015). SBU AEU Strategic Program Review 10-28-2015. USAID Office of Inspector General. (2015). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, First Quarter. Washington, DC. UNDP. (2014). Assessing the socio-economic impacts of Ebola Virus Disease in Guinea, Liberia and Sierra Leone: The Road to Recovery. Wittels, A., & Maybanks, N. (2016). Communication in Sierra Leone: An Analysis of Media and Mobile Audiences. Research Report. BBC Media Action. World Bank. [data tables]. Retrieved 10/15/2017 from https:// data.worldbank.org/indicator/SH.STA.MMRT.NE?locations=SL World Bank, (2015, April 16). Summary on the Ebola Recovery in Sierra Leone. Retrieved 9/2/2017 from http://www.worldbank. org/en/topic/ebola/brief/summary-on-the-ebola-recovery￾plan-sierra-leone World Health Organization, (2016, March 30). Ebola Situation Reports. Retrieved 9/20/2017 from http://apps.who.int/ebola/ ebola-situation-reports World Health Organization. (2017). WHO Sierra Leone 2016- 2017: From Ebola to Health. World Vision. (2015). Annex F: Logframe. Sierra Leone EFSP Logframe. World Vision. (2016). Logframe and Timeline [table]. (Sierra Leone Emergency Food Security Program). World Vision. (2017a). Annex 1: Modality Specific Reporting. Quarter 1. (Sierra Leone Port Loko Emergency Food Security Program). World Vision. (2017b). Annex 1: Modality Specific Reporting. Quarter 3: April-June 2017. (Sierra Leone Port Loko Emergency Food Security Program). World Vision. (2017c). “Success Stories. Port Loko District, Sierra Leone.” Sierra Leone Emergency Food Assistance in Port Loko. World Vision/USAID. (2017, July 19). Close out Plan: EFSP￾Emergency Food Security Program in Port Loko. USAID Bureau of Democracy Conflict and Humanitarian Assistance: Office of Food for Peace. World Vision/USAID. (2017, January 31). Office of Food for Peace 2017 Quarterly Report. Quarter 1. (Sierra Leone-Emergency Food Assistance to Loko District). World Vision/USAID. (2017, April 30). Office of Food for Peace 2017 Quarterly Report. Quarter 2. (Sierra Leone-Emergency Food Assistance to Loko District). A N N E X F. D O C U M E N T S C O N S U LT E D n F–11 World Vision/ USAID. (2017, July 30). Office of Food for Peace 2017 Quarterly Report. Quarter 3. (Sierra Leone-Emergency Food Assistance to Loko District). Zayid, J., Sichei, M., & Korseh-Hindowa, M. (2016). “Sierra Leone”. Retrieved from www.africaeconomic.outlook.org REGIONAL USAID (2015, May 22). Ebola Response and Preparedness Congressional Notification. USAID (2015, December 2). Ebola Response and Preparedness Congressional Notification. USAID (2015, December 2). Ebola Response and Preparedness Congressional Notification. USAID (2016, June 24). Ebola Response and Preparedness Congressional Notification. USAID (2016, June 24). Ebola Response and Preparedness Congressional Notification. USAID (2015, March 20). Ebola Response and Preparedness Congressional Notification. USAID (2016, March 24). Ebola Response and Preparedness Congressional Notification. USAID (2016, April 8). Congressional Notification and Inter￾Agency Transfer Notification. USAID (2016, May 18). Ebola Response and Preparedness Congressional Notification. USAID (2016, September 9). Ebola Response and Preparedness Congressional Notification. U.S. Department of State (2016, August 10). Congressional Notification Transmittal sheet. U.S. Department of State (2016, August 31). Congressional Notification Transmittal sheet. U.S. Department of State (2016, July 27). Congressional Notification Transmittal sheet. U.S. Department of State (2017, September). Action Memo: Ebola Report for Congress. U.S. Department of State/USAID (2015, April). Ebola Emergency Response Global Health Security Obligations and Disbursement-April Report. U.S. Department of State/USAID (2017, April). FY 2017 Ebola Response and Preparedness: Section 9004 Report. U.S. Department of State/USAID (2015, August). Ebola Response and Preparedness: Section 9004 Report. U.S. Department of State/USAID (2016, February). Ebola Emergency Response Global Health Security Obligations and Disbursement-January Report. U.S. Department of State/USAID (2016, January). Ebola Response and Preparedness: Section 9004 Report. U.S. Department of State/USAID (2015, January). FY 2015 Ebola Emergency Funding: Summary of Commitments, Obligations and Disbursements. U.S. Department of State/USAID (2015, July). Ebola Emergency Response Global Health Security Obligations and Disbursement-July Report. U.S. Department of State/USAID (2017, March). FY 2016 Ebola Emergency Response Global Health Security Obligations and Disbursement-March Report. U.S. Department of State/USAID (2015, October). Ebola Emergency Response Global Health Security Obligations and Disbursement-September Report. U.S. Department of State/USAID (2017, September). Ebola Emergency Response Global Health Security Obligations and Disbursement-September Report. U.S. Department of State/USAID (2017, September). FY 2017 Ebola Response and Preparedness: Section 9004 Report. F–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table F–1. Sources of Evidence on the Fighting Ebola Grand Challenge Innovations, According to Aspect of the Innovation Aspect of the Innovation Primary Source(s) of Evidence General Description and Classification Main website for “Fighting Ebola: A Grand Challenge for Development” (http://www. ebolagrandchallenge.net/) USAID. USAID Development Innovation Accelerator Broad Agency Announcement for Fighting Ebola: A Grand Challenge for Development. Level of Funding Office of Inspector General (OIG) Quarterly Progress Report on U.S. Government International Ebola Response and Preparedness Activities for Quarter 1 of FY 2016, Quarter 2 of FY 2016, and Quarter 3 of FY 2016. Spreadsheet of Obligations and Expenditures (O&E) dated March 2017 Period of Performance (Start and End Dates) OIG. Progress Report on U.S. Government International Ebola Response and Preparedness Activities for Quarter 1, FY 2016. Field-testing and Status Updates Press releases, Center for Accelerating Innovation and Impact/USAID. 2017 Impact Brief; OIG Quarterly Progress Reports ANNEX G. NON-DISCLOSURE AGREEMENTS n G–1 ANNEX G. NON-DISCLOSURE AGREEMENTS Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document Zhuzhi Moore, Survey Specialist Swati Sadaphal, Data Analyst G–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Donna Espeut , Senior Evaluation Advisor, Liberia Mike Toole , Senior Evaluation Advisor, Guinea ANNEX G. NON-DISCLOSURE AGREEMENTS n G–3 Orlando Hernandez , Senior Evaluation Advisor, Sierra Leone Carol Levin , Development Economist Annette Bongiovanni, Project Director G–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table G–1. Other Team Members (NDAs available upon request) Name Position IBTCI Employees Richard Columbia Chief of Party Esther Kaggwa Program Monitoring Director Andrew Reuter Knowledge Management Director Salima Mutima Senior Evaluation Advisor Juan Carlos Alegre Project Director, acting Robert Grossman-Vermaas Subject Matter Expert: Democracy and Governance Edward Allan Subject Matter Expert: Education Elisabeth Nolan Research Associate Katherine Labombarde Senior Project Associate Meredith Kerrigan Project Associate Consultants and Subcontractors Barbara de Zalduondo Senior Evaluation Specialist Suzanne Essama-Bibi Senior Evaluation Advisor Naomi Rutenberg Senior Evaluation Advisor Kapil Ahmed Statistician/Sampling Specialist Richard Ngafuan Evaluation Specialist, Liberia Aaron Kokolie Field Coordinator, Liberia Roger Emmanuel Millimono Field Coordinator, Guinea Samuel Turay Field Coordinator, Sierra Leone Mary Burket Knowledge Management & Communications Advisor Erin Dowling Graphic Designer Aliou Barry Director General, StatView International Yuliya Dudaronak Senior Research Executive, ORB International Elizabeth Bailey Research Executive, ORB International ANNEX H. DATA COLLECTION TOOLS n H–1 ANNEX H. DATA COLLECTION TOOLS Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document TOOLS LISTING Oral Informed Consent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–2 KIIS— Beneficiaries: Service Recipients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–3 KIIS— Beneficiaries: Service Providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–12 KIIS— Civil Society Organizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–22 KIIS— Government Stakeholders (regional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–32 KIIS— Implementing Partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–42 KIIS— Partner Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–51 KIIS— U.S. Government Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–62 SURVEY— Health Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–69 SURVEY— Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H–170 H–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1 USAID /Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone, and Liberia Performance Evaluation Phase 2 Oral Informed Consent Good morning/Good afternoon/Good evening: My name is __________________, [and my colleague is __________________.] We are part of the IBTCI team contracted by USAID Africa Bureau to conduct a performance evaluation of USAID’s activities in support of [Country’s] recovery from the Ebola outbreak in 2014-2016. USAID has programmed resources to support the country’s recovery from the indirect effects of the Ebola crisis beyond the actual illness, such as the effects of the Ebola crisis on health services for conditions other than Ebola, or the effects on the economic, agricultural, or education sectors. USAID will use the following two criteria to measure the performance of the resources programmed to limit these indirect effects of the crisis: - The achievement of planned objectives and targets, as specified in activity plans and modifications; - Stakeholders’ perceptions of the value of the activities, especially their effectiveness, efficiency, transparency, accountability and coordination. We are conducting in-person key informant interviews (KIIs) with stakeholders such as yourself, whose views are essential for answering the key questions that frame the evaluation. The Evaluation Questions (EQs) address four domains: performance, sustainability, opportunities and gaps, and management. If you agree to the interview, I will ask your views on a number of questions and issues, and I will take notes of our discussion. You are very welcome to stop me to ask questions during the interview. This interview will take approximately one hour. The information you provide will remain confidential. To protect your privacy, we will detach your name and organization from your responses. We might include quotes to emphasize a point made by many respondents; however, any quote used in the report will be identified only by the participant’s gender and stakeholder group. It will not be linked directly to you or to the organization where you work. If you do not wish to have anything you mention quoted, please let us know now. Your participation in this discussion is completely voluntary. You may choose not to participate, or not to answer some questions, or to stop the interview at any time. If you chose not to participate, there will be no negative consequences. You will not be receiving any payment or allowances for your participation. Do you have any objection to participating in this interview, or do you have any questions before you can decide? 25. Are you willing to be interviewed? [InfConsent] Yes / No I would like to tape record the interview so that we can be sure that we captured your views correctly. 26. May I record the interview? [OKtoRecord] Yes / No [INSERT NAME AND CONTACT OF LOCAL POINT OF CONTACT INCLUDING IRB IN CASE OF QUESTIONS AS APPLICABLE TO COUNTRY] Interviewer’s Signature and date: ______________________________________________________ Participant ID: ______________________________ Questionnaire type: _________________________ ORAL INFORMED CONSENT ANNEX H. DATA COLLECTION TOOLS n H–3 2 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for Beneficiaries – Service Recipients (Revised 11/17/17) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members (interviewers): 5. ___________________________________________________________________ 6. ___________________________________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): ______________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18 19 20. 21. 22. 23. 24. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A recorder • A “cheat sheet” that lists the USAID-funded activities that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. KIIS— BENEFICIARIES: SERVICE RECIPIENTS H–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 3 WARM UP AND INDIVIDUAL INFORMATION 49 As you know, the Ebola crisis had many economic and social effects on families, communities, businesses, organizations, and the government in [Country/the three countries] – beyond the actual illness. Schools were closed; in quarantine areas farming was interrupted; commerce and trade were affected, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect effects of the Ebola crisis. Are you aware of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refus ed 50 Did you experience any of these indirect effects yourself? [Exper2ndOrder] Yes No 51 What indirect effects have you heard about or experienced? Narrative: If respondent is NOT employed or volunteering in an organization (governmental or non￾governmental) skip to question 100. 60 Please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? 60 [OrgFocus] Narrative: 61 1.4. Does your work focus on a specific target population or geographic catchment area in [COUNTRY], or is your work at national and/or global levels? ? [Probe for geographic location/catchment areas ; selected beneficiaries (e.g. EVD survivors; USAID IPs), or other focus] 61 [OrgPop] Narrative: 62 1.5 How was your organization’s work here affected by the Ebola outbreak in 2014-2015? 62 [OrgEVDimpact] Narrative: 62b In what ways was the work affected? 62b. Narrative: ANNEX H. DATA COLLECTION TOOLS n H–5 4 63 Tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis? [Probe for specifics. If efforts are mentioned in general terms, or by type, ask Can you give me an example?] 63 [PPevdinv] Narrative 64 1.7 These days, do you, or does your organization, work with other people and organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the other organizations; If no, ask: when was the last time you or your organization/unit worked directly on Ebola recovery?] 64 [PPevdOrg] Narrative: Now we would like to focus in on USAID’s support for recovery from the Ebola crisis, from about April 2015 to the present, since that is the subject of our evaluation. EQ 1 – Performance [P] Work toward recovery from the Ebola crisis involved many organizations -- government and non￾governmental (NGOs) organizations, and civil society organizations (CSOs). Some of the activities funded by USAID, included: [see Cheat Sheet and insert the types of interventions funded by USAID in this country]. We are interested in your views about these diverse recovery efforts. We want to know, in your opinion, whether they were useful. 100.c. P1. In your opinion, what has been done, or is being done by government and non-governmental projects or activities to help your [country/county/district/community] recover from the economic, social or political impacts of the Ebola outbreak in this country? Narrative: 101-107. P1. Please tell me/us if you have had contact with, or received support from activities/projects in any of these areas in the past 2 years1. I will read out the areas and projects, and then ask if you did or didn’t have experience with each one. [Interviewers should be able to cite the specific project names and types of activities in each thematic area that were implemented in the respondent’s location; Respondents have been selected because they were or are beneficiaries of the resources USAID dedicated to limiting the indirect effects of the crisis, which are the effects beyond the actual illness, such as the effects of the crisis on health services for conditions other than Ebola, or its effects on the economic, agricultural, or education sectors] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communi￾cation, technology Y/N/DK 1 In this case, we are not restricting the question to Ebola related activities. We want the interviewer to learn more about the KI’s range of knowledge, to shape later questioning. H–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 5 Interviewers: For each activity or intervention cited, ask: 108a. P3.1a. How much did [Project 1] help you or your family recover from the Ebola crisis? Would you say it was no help at all? Was somewhat helpful? Was very helpful? Yes No No help at all Somewhat helpful Very helpful Don’t Know or Refused 109a. [#Proj1ValueN] P3.1 Please tell us more about that. In what ways did it help, or is it helping you or your family? Narrative: 110a. P3.1a. What about [Project 2]: How much did it help you or your family recover from the Ebola crisis? Would you say it was no help at all? Was somewhat helpful? Was very helpful? Yes No No help at all Somewhat helpful Very helpful Don’t Know or Refused 111a. [#Proj1ValueN] P3.1 Please tell us more about that. In what ways did it help, or is it helping you or your family? Narrative: 144. P6. Do you believe the concerns of both women and men have been addressed in the Ebola recovery efforts that you just mentioned? Yes No DK 144.P6b If so, in what ways? How have these Ebola recovery efforts involved women and girls or responded to the needs of women and girls? ANNEX H. DATA COLLECTION TOOLS n H–7 6 145. P6.1. Have women and men benefited equally from Ebola recovery projects? [Probe]: Not at all Somewhat/ in some cases Yes Don't Know/Refused 145.b. Please explain your views. What are the ways that women and girls benefited? Probe: What were the challenges in involving women and girls? 148. (custom) Did the Ebola outbreak affect everyone in your community in the same way? Or were some people and places more vulnerable than others? Yes No DK 148b If so, why do you think the Ebola epidemic affected some people more than others? [Probe for risk factors; language used to describe differences between people; language used regarding vulnerability] EQ 2 – Sustainability [S] 202.S2 What changes that were made in 2015 and 2016 due to Ebola are still evident today in your community? What kinds of effects or changes have actually lasted, as far as you know? [Probe for examples; locations}. 204.S3. We hope that no community will ever have to go through a crisis like the Ebola crisis. But what do you think your community needs in order to be more able to withstand and recover from that kind of extreme shock in the future? How could your community be more prepared to cope with health emergencies in the future? Narrative: H–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 7 208.S5 (Custom) EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable so far. We really appreciate your opinions and suggestions. Now we would like to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 301.OG3. Are you aware of any changes that took place after the Ebola epidemic was brought under control – economic or social or health changes in your community, or changes in the environment, -- that affected the recovery from the Ebola crisis? Yes No DK 301. OG3b If so, what types of changes occurred? Can you tell me more about these changes? 306.OG4. Do you think that Ebola recovery efforts have missed or have not paid sufficient attention to some things that are essential to success? Yes No DK 306.OG4b What gaps do you see in the ways projects to help communities and government to get back on track are planned and carried-out? 307.OG (custom) EQ 4 – Management [M] The Ebola crisis in this region caught everyone – government, communities, organizations, international development partners - by surprise. It was an emergency, and action was very urgent. This was understood not only here, in [country] but also in the halls of government of [country’s] international development partners. To save lives, the bulk of the international support was devoted to the urgent medical response. However, the Government of [Country] and development partners recognized that the crisis affected its long-term development goals, so recovery support was needed in multiple sectors. ANNEX H. DATA COLLECTION TOOLS n H–9 8 For the United States, this challenge brought the need to join up efforts through several agencies of the USG, including the Office of U.S. Foreign Disaster Assistance, the Centers for Disease Control and Prevention, USAID’s Offices and Bureaus in Washington, and USAID Missions and Embassies in Guinea, Liberia, and Sierra Leone. These joint USG efforts needed to be negotiated with local government and coordinated with national and international partners. We would appreciate hearing your opinions and advice regarding the ways in which the USG’s mechanisms were used to help the [country] manage and recover from the Ebola crisis of 2014-2015. Our focus is on the resources the USG provided through USAID to limit the indirect effects of the crisis. 401.M. Before turning to the Ebola crisis and recovery, have you experienced other humanitarian emergencies in the past? [Pause to give the KI time to think] How did [it/they] compare with the challenges of recovery from the Ebola crisis? 402.M2. In [Country] the American government’s Ebola recovery efforts focused on [list the thematic areas]. Do you feel that programs from government and from NGOs such as [mention specific programs that have been funded by the USG to limit the indirect effects of the crisis. NGOs that have worked in the area] have communicated and worked together well? In what ways have they worked together well? In what ways has coordination been limited? 402b.M2. How have these programs recognized and worked together with traditional authorities and leaders in the community? What are people saying about the involvement of religious leaders? What about traditional medical practitioners? 411.M (Custom) Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask these final questions? H–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 9 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] Narrative: 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/ Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola ANNEX H. DATA COLLECTION TOOLS n H–11 10 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID with a balanced performance evaluation of its Ebola recovery activities, and recommendations for the remaining 2 years. [Include information on the Learning Summits, and or where to access the report in 2018?] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. H–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 11 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for Beneficiaries – Service Providers (Revised 11/17/17) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members (interviewers): 5. _____________________________________________ 6. ____________________________________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): _______________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18 19 20. 21. 22. 23. 24. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A recorder • A “cheat sheet” that lists the USAID-funded projects that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. KIIS— BENEFICIARIES: SERVICE PROVIDERS ANNEX H. DATA COLLECTION TOOLS n H–13 12 WARM UP AND INDIVIDUAL INFORMATION 49 As you know, the Ebola crisis had many economic and social effects on families, communities, businesses, organizations, and the government in [Country/the three countries] – beyond the actual illness. Schools were closed; in quarantine areas farming was interrupted; commerce and trade were affected, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect effects of the Ebola crisis. Are you aware of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refu sed 50 Did you experience any of these indirect effects yourself? [Exper2ndOrder] Yes No 51 What indirect effects have you heard about or experienced? Narrative: 52 When was the last time someone or some project interviewed you, or asked you to participate in a study, regarding Ebola or any other health topic? Last 30 days 1-3 months ago 4 months to a year ago More than a year ago Never, this is the first time 60 Now, please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? 60 [OrgFocus] Narrative: 61 Does your work focus on a specific target population or geographic catchment area in [COUNTRY], or is your work at national and/or global levels? ? [Probe for geographic location/catchment areas ; selected beneficiaries (e.g. EVD survivors; USAID IPs), or other focus] 61 [OrgPop] Narrative: 62 Was your organization’s work affected by the Ebola outbreak in 2014-2015? H–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 13 T62 [OrgEVDimpact] Narrative: Yes No 62b In what ways was the work of your organization affected? 62b. Narrative: 63 1.6 Please tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis. [Probe for specifics. If efforts are mentioned only by type of effort, or in general terms, ask “Can you give me an example?”] 63 [PPevdinv] Narrative 64 1.7 These days, do you, or does your organization, work with other people and organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the other organizations; If no, ask when was the last time you or your organization/unit worked directly on Ebola recovery?] 64 [PPevdOrg] Narrative: Thank you very much for this background information. Now we would like to focus on USAID’s support for recovery from the Ebola crisis, from about April 2015 to the present, since that is the subject of our evaluation. EQ 1 – Performance [P] Work toward recovery from the Ebola crisis involved many kinds of specialists and organizations, both government and non-governmental (NGOs). Some of them were funded by USAID. In [country], the work included various activities to restore non-Ebola health services, strengthen health systems and provide support for Ebola survivors; efforts to restart and strengthen education; interventions to promote agriculture and food security; interventions to strengthen governance; and efforts to restore investment and trade and to improve information, communication, and technology in the three most affected countries. We are interested in your views about these diverse recovery efforts. We want to know, in your opinion, whether they were useful. ANNEX H. DATA COLLECTION TOOLS n H–15 14 100d.P1. What were you told about the objectives of the training and support that your unit received after the Ebola crisis? 101-107. P1. Please tell me/us if you have had any involvement with activities/projects in any of these areas in the past 2 years (either on the job, or in your community). I will read out the areas and projects, and then ask if you did or didn’t have experience with each one. [Interviewers should be able to cite the specific project names and types of activities in each thematic area that were implemented in the respondent’s location] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communi￾cation, technology Y/N/DK 114. P4. How would you rate the country’s progress in achieving its recovery objectives and goals, [specifically for your sector of interest]? Would you say it has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused 115. Please explain your rating [Narrative]: 116. P4. Where have you [where has the country] achieved more success? And why? [Probe: we are interested in types of activities that worked well, and also places where you had most success] 117. P4.Where have you [where has the country] faced more challenges? Why? H–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 15 140. P4. The Ebola crisis slowed down the progress [country] was making in health, education and infrastructure and economic development. Over all, do you think that the “recovery” programs that have been implemented after the Ebola crisis, have helped to reverse these losses? In what ways did they help the country get back on track in terms of health, education, and economic development? Narrative: 145. P6.1. Do you believe the concerns of both women and men have been addressed in the recovery programs that have been implemented after the Ebola crisis? Yes No DK 144.P6b If so, in what ways? How have these Ebola recovery efforts involved women and girls or responded to the needs of women and girls? 145.P6.I b Have women and men benefited equally from Ebola recovery projects? [Probe]: Not at all Somewhat/ in some cases Yes Don't Know/Refused 145.b. Please explain your views. What are the ways that women and girls benefited? [Probe: What were the challenges in involving women and girls? 148. (custom) Thinking back over the training and support you have received in Ebola recovery projects, How did the training and support help you? 149. (custom) Some of the Ebola recovery projects funded by USAID aimed to increase transparency and accountability of government services. [If necessary, explain: That is, they aimed to increase men’s and women’s awareness and involvement in development planning and they promoted information sharing and citizen engagement in oversight of public services. What improvements have you noticed in these areas? ANNEX H. DATA COLLECTION TOOLS n H–17 16 EQ 2 – Sustainability [S] 202. S2 Please think back to the key messages and results that the government and partners promoted during and after the Ebola crisis. What kinds of effects or changes have actually lasted, as far as you know? [Probe for examples; locations}. What changes that were made in 2015 and 2016 due to Ebola are still evident today? 204.S3. What do you think service providers like you need in order to be more able to withstand and recover from health crises such as disease outbreaks or droughts in the future? How could your organization, and your community, be more prepared to cope with health emergencies in the future? Narrative: 208.S5 (Custom) EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about recovery efforts have been very valuable so far. We really appreciate your time. Now we would like to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 301.OG1. Clearly, every development project is a joint effort: success of an intervention depends in large part on the people, the environment, and the traditions or systems that are in place. Upon reflection, can you name a place or places where It was most difficult for [Ebola recovery efforts in the KI’s thematic area] to take hold? Why do you think it was difficult in those places/situations? What kinds of social, economic, or political problems/obstacles did you encounter, or hear about? 302b.OG2. Please think for a moment about a time or community where it seemed that the Ebola recovery project that you were involved in went smoothly or where [interventions in the KI’s thematic area/s] had the greatest success. Why do you think it was easier for the [efforts/interventions] to succeed in those times or situations? What factors or conditions supported success? [Probe for types of factors] H–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 17 303c.OG3b. Looking back at the training or technical support that you and your colleagues received under Ebola recovery programs, what do you think was missed, or underemphasized? What could have been done better? 307.OG (custom) EQ 4 – Management [M] The Ebola crisis in this region caught everyone – government, communities, organizations, international development partners - by surprise. It was an emergency, and action was very urgent. For the United States, this challenge brought the need to join up planning and programming through several agencies, and as fast as possible to negotiate and coordinate the US government contribution with local government and with national and international partners. We would appreciate hearing your observations regarding the ways the US government allocated funds to help [the country] recover from the Ebola crisis of 2014-2015. Our focus is on the effectiveness of the resources USAID dedicated to limiting the indirect effects of the crisis, which are the impacts beyond the actual illness, such as the effects of the crisis on health services for conditions other than Ebola, or its effects on the economic, agricultural, or education sectors. 401.M. Have you worked on or experienced other humanitarian emergencies? [Pause to give the KI time to think] How did [it/they] compare with the challenges of recovery from the Ebola crisis? 402.M2. Now, turning to the Ebola crisis of 2014-2015 and the recovery efforts from 2015 to the present, what are people in your [team/organization] saying about the US Government’s coordination and management of its Ebola Recovery efforts? In [Country and thematic area] the resources that the USG allocated to limit the indirect effects of the crisis focused on [list the thematic areas]. Have you had any issues with the coordination of these activities? If so, what happened? What advice would you give to improve the situation? ANNEX H. DATA COLLECTION TOOLS n H–19 18 409b.M4. From the outset, the US Government has been clear with [country] that the Ebola funding will come to an end in 2019. Has your training and/or technical support project prepared for the end of the USG funding? 410.M4. What steps should USAID take to prepare recipients of Ebola recovery training or technical support for the end of this funding source? 410.M4. What steps should USAID take to prepare beneficiaries of the Ebola recovery program for the end of Ebola funding? What should USAID/Washington do? What should the USAID Missions do? 411.M (Custom) Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask two final questions? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] H–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 19 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID with a balanced performance evaluation of the resources programmed to limit the second order impacts of the crisis, and recommendations for the remaining 2 years. [Include information on the Learning Summits, and/or where to access the report in 2018?] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/ Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola ANNEX H. DATA COLLECTION TOOLS n H–21 20 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. H–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES KIIS— CIVIL SOCIETY ORGANIZATIONS 21 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide for Civil Society Organizations (Revised 11/17/2017) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members: 5. Interviewer____________________________________________ 6. Note-take______________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): ___________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18. 19 20. 21. 22. 23. 24.. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A “cheat sheet” that lists the USAID-funded activities that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. • A recorder ANNEX H. DATA COLLECTION TOOLS n H–23 22 WARM UP AND INDIVIDUAL INFORMATION 49 As you know, the Ebola crisis had many economic and social effects on families, communities, businesses, organizations, and the government in [Country/the three countries] – beyond the actual illness. Schools were closed; in quarantine areas farming was interrupted; commerce and trade were affected, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect effects of the Ebola crisis. Are you aware of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refus ed 50 Did you experience any of these indirect effects yourself? [Exper2ndOrder] Yes No 51 What indirect effects have you heard about or experienced? Narrative: If respondent is NOT employed or volunteering in an organization (governmental or non-governmental) skip to question 100. 60 Now, please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? [OrgFocus] Narrative: 61 Does your organization serve all people from all over [COUNTRY], or do you focus on people from a particular area? Or a particular community or age group for example? [Probe for geographic location/catchment area of beneficiaries; gender balance; any specific requirements such as residence, or occupation, or ethnicity] [OrgPop] Narrative: 62 Was your organization’s work affected by the Ebola outbreak in 2014-2015? [OrgEVDimpact] Narrative: Yes / No 62b In what ways was the work affected? 62b. Narrative: 63 Please tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis. [Probe for specifics. If efforts are mentioned only by type of effort, or in general terms, ask “Can you give me an example?”] [PPevdinv] Narrative: H–24 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 23 64 These days, does your organization, work with other people or organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the others; If no, ask when was the last time you or your organization/unit worked directly on Ebola recovery?] [PPevdOrg] Narrative: Now we would like to focus on USAID’s support for recovery from the Ebola crisis, from about April 2015 to the present, since that is the subject of our evaluation. EQ 1 – Performance [P] Work toward recovery from the Ebola crisis in 2014-2015 involved many kinds of specialists and organizations. For example, USAID funded activities to restore health services, and activities to enable children to go back in school. Other Ebola recovery projects funded by USAID were in agriculture and food security, governance, investment and trade, and communications and technology. We are interested in all these sectors so please feel free to talk about only those activities that you know about. 100.c. P1. In your opinion, what has been done, or is being done by government, non-governmental projects or activities, and civil society organizations to help your [country/county/district/community] recover from the economic, social or political impacts of the Ebola outbreak in this country? Narrative: 101. P1. Please tell us if your organization has had any involvement with projects in any of the following sectors or areas in the past 2 years. I will read out the areas and projects, and then ask if your organization did or didn’t have experience with each one. [Interviewers refer to “cheat sheet” with the specific project names and intervention types of activities in each thematic area that were implemented in the respondent’s area of responsibility] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communi￾cation, technology Y/N 114. P4. How would you rate the country’s progress in achieving its recovery objectives and goals, [specifically for your sector of interest]? Would you say it has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused ANNEX H. DATA COLLECTION TOOLS n H–25 24 115. Please explain your rating [Narrative]: 116. P4. Where have you [has the country] achieved more success? And why? [Probe: we are interested in types of activities that worked well, and also places where you had most success] 146. P5. Even in the places that were hit the hardest, by Ebola and its aftermath, we have heard of success stories about people and projects helping [communities/organizations] get back on track. What kinds of success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more success stories] Narrative: 117. P4.Where have you [has the country] faced more challenges? Why? 146. P5. We know that some places had a very hard time with Ebola, while other places were less affected. But everywhere, even in the places that were hit the hardest, we have heard there were people and projects that succeeded in helping people and organizations to get back on track. Some encountered challenges in trying to make things better than they were before Ebola. What challenges stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of challenges stories] 140. P4. The Ebola crisis threatened the progress [country] was making in health, education and infrastructure and economic development. Overall, do you think that the “recovery” programs that USAID funded and implemented after the Ebola crisis, have helped to reverse these losses? In what ways did they help the country get back on track in terms of health, education, and economic development? Narrative: H–26 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 25 141. P4.1. Which programs were most important, or made the greatest difference, toward helping the country get back on track? What was it about them that made them most helpful? Narrative: 142. P4.2. Were private sector, commercial groups, and/or private, not-for-profit groups involved in Ebola recovery? What would be a couple of examples of government working with non-governmental and private sector groups to promote [country’s] development? Narrative: 143. P4.3. What about infrastructure – water, waste removal, power, telecommunications, roads and transport: are these back on track? Are there examples from different parts of the country where infrastructure is in better shape now than before the Ebola crisis? Narrative: 145. P6.1. In your opinion, have women and men benefited equally from Ebola recovery projects? [Probe]: Not at all Somewhat/ in some cases Yes Don't Know/Refused What are the ways that women and girls benefited from the Ebola recovery projects? Probe: What were the challenges in involving women and girls? 148. (custom) Did the EVD outbreak affect everyone in your community in the same way? Or were some people and places more vulnerable than others? If so, why do you think the EVD epidemic affected some people more than others? [Probe for risk factors; language used to describe differences between people; language used regarding vulnerability;] ANNEX H. DATA COLLECTION TOOLS n H–27 26 149. (custom) Some of the Ebola recovery projects funded by USAID aimed to increase transparency and accountability of government services. [If necessary, explain: That is, they aimed to increase men’s and women’s involvement in development planning and they promoted information sharing and citizen engagement in oversight of public services]. What improvements have you noticed in these areas? EQ 2 – Sustainability [S] 202. S2 What changes that were made in 2015 and 2016 in response to the Ebola crisis are still evident today? [Probe for examples; locations]. Narrative: 203. S3 Of all the Ebola recovery efforts you have seen, which specific ones (if any) were especially effective in helping the community or the country get back on track? What did they accomplish? Why were those results important? Narrative: 203. S3b Why do you think it was easier for the [efforts/interventions] to succeed in those places/situations? What factors or conditions supported success? [Probe for types of factors] Narrative: 204b.S3. How can communities in [Country] become more able to withstand and recover from disasters or shocks such as disease outbreaks or droughts in the future? [Aiming for advice on building resilience] Narrative: 205.S4. What advice can you give us about ways to make the results/benefits continue after Ebola recovery projects end? What works to improve sustainability? And what does not work? Narrative: H–28 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 27 208.P5. Ebola Pillar II funding is ending, which activities are the most important for which funding should continue? [Probe: Which activities? Why are they so important?] Narrative: 409.M4. From the outset, the US Government has made it clear to [country] that the Ebola funding will come to an end in 2019. Are the civil society organizations that are funded for Ebola recovery work prepared for the end of the USG funding? EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable so far. We really appreciate your point of view. Now we would like to go a bit deeper to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 302.OG2. Please think for a moment about a place or community where it seemed that Ebola recovery efforts went smoothly or where [interventions in the KI’s thematic area/s] had the greatest success. Why do you think it was easier for the [efforts/interventions] to succeed in those places/situations? What factors or conditions supported success? [Probe for types of factors] Narrative: 303b.OG3b. Looking back at the USAID Ebola Recovery programming, what do you think was missed, or underemphasized? 305.OG1. Do you think the national and local resources, and conditions for success, were understood [by USAID program planners] at the time that the Ebola recovery efforts were planned? What was not so well understood? Narrative: 306.OG4. What are the gaps in the overall Ebola recovery programming? Do you think that USAID’s efforts missed or under-emphasized anything that is essential to recovery? Narrative: ANNEX H. DATA COLLECTION TOOLS n H–29 28 EQ 4 – Management [M] The Ebola crisis in this region caught everyone – government, communities, organizations, international development partners - by surprise. It was an emergency, and action was very urgent. For the United States, this challenge brought the need to join up planning and programming through several agencies, and as fast as possible to negotiate and coordinate the US government contribution with local government and with national and international partners. We would appreciate hearing your observations regarding the ways the US government provided and programmed funds to help the three countries to recover from the Ebola crisis of 2014-2015. Our focus is on the resources programmed to limit the indirect effects of the crisis. 401.M. Before turning to the Ebola crisis and recovery, please tell us – have you had experience with other humanitarian emergencies? Yes No 401. M b When you think about the Ebola crisis, to what do you compare it? Narrative: 402.M2. Now, turning to the Ebola crisis of 2014-4015 and the recovery efforts from 2015 to the present, what are people in your [team/organization] saying about the US Government’s coordination and management of its Ebola Recovery efforts? In [Country] the USG’s Ebola Recovery resources focused on [list the thematic areas and projects from the cheat sheet]. How, and how well, were these Ebola Recovery efforts coordinated with the Government? Narrative: 403.M2. Would you please describe what coordination goes on among your other partners in Ebola recovery efforts at national level, and at other levels (County/Préfecture/District; sub-district; village)? Narrative: 405.M1. Can you tell us about a particular strategy that your organization/OU/level used to overcome bottlenecks and/or to access USAID’s expertise and resources in support of the recovery? Narrative: 406.M1. What weaknesses have you noted in the way USAID managed and coordinated its Ebola Recovery Activities/ or in the way USAID managed the resources programmed to limit the indirect effects of the Ebola crisis? Narrative: H–30 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 29 410.M4. What steps should USAID take to prepare recipients of Ebola recovery funding for the end of this funding source? What should USAID/Washington do? What should the USAID Missions do? 411.M4 What steps is your organization taking to plan for the end of special Ebola funding, and to prepare your members or beneficiaries for the change? Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask two final questions? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] Narrative: 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/ Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola ANNEX H. DATA COLLECTION TOOLS n H–31 30 Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? Narrative: 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID with a balanced performance evaluation of its Ebola recovery activities, and recommendations for the remaining 2 years. [Include information on the Learning Summits, and or where to access the report in 2018] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. H–32 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 31 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for Government of Guinea, Liberia or Sierra Leone Stakeholders (Revised 11/17/2017) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members: 5. Interviewer_____________________________________________ 6. Note-take______________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer 13. Privacy of the setting DK Public Family, within earshot Private 14. Comfort of the Interviewee DK Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): _____________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18. 19 20. 21. 22. 23. 24.. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS WILL BE ASSOCIATED WITH SOCIODEMOG INFO ONLY BY THE PE1 ID CODE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors • A copy of the letter of introduction from the relevant body or official • A “cheat sheet” that lists the USAID-funded projects that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. • A recorder KIIS— GOVERNMENT STAKEHOLDERS (REGIONAL) ANNEX H. DATA COLLECTION TOOLS n H–33 32 WARM UP AND INDIVIDUAL INFORMATION 49 When we prepared for this evaluation, we learned that the Ebola crisis had many economic and social effects on families, communities, businesses, and organizations in [Country/the three countries] – beyond the actual illness. Schools were closed; farming was interrupted; commerce and trade and social relationships were affected, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect, effects of the Ebola crisis. Did you hear about any of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refused 51 What indirect effects have you heard about or experienced? Narrative: 602 Now, please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? [OrgFocus] Narrative: 61 Does your unit/department serve all people from all over [COUNTRY], or do you focus on people in a particular area? Or a particular sub-group? [Probe for geographic location/catchment area of beneficiaries; gender balance; any specific requirements such as residence, or occupation, or ethnicity] [OrgPop] Narrative: 63 Tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis? [Probe for specifics. If efforts are mentioned in general terms or by type, ask Can you give me an example?] [PPevdinv] Narrative: 64 These days, does your department and unit, work with other people or organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the others; If no, ask when was the last time you or your organization/unit worked directly on Ebola recovery?] [PPevdOrg] Narrative: 2 Questions 60-67 are matched by questions in Phase 1 key informant interviews H–34 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 33 EQ 1 – Performance [P] Work toward recovery from the Ebola crisis involved many kinds of specialists and organizations, both government and non-governmental (NGOs). Some of these efforts were funded by your government, and other development partners. Others were funded all or in part by USAID, including, various activities to restore non-Ebola health services and health systems, efforts to restart and strengthen education, projects to promote agriculture and food security, projects to strengthen governance, efforts to restore investment and trade, and projects to improve the information communication and technology in the country. We are interested in your views about these diverse recovery efforts. In this section I will be asking you for your opinions. 100b.P1. In your opinion, what has been done, or is being done by government and non-governmental projects or activities to help your [country/county/district/community] recover from the economic, social or political impacts of the Ebola outbreak? Narrative: 101 - 107. P1b. Please tell me/us if you have had any involvement with Ebola recovery activities/projects supported by USAID in the past 2 years. I will read out the areas and projects that have ended, and those that are ongoing, and I will ask if you did or didn’t have experience with each one. [Interviewers will have a “cheat sheet” with the specific project names and types of activities in each thematic area that have been implemented in the respondent’s location] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communi￾cation, technology Y/N/DK 114. P4. How would you rate your country’s progress in achieving its recovery objectives and goals, [specifically for your sector of interest]? Would you say it has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused 115. Please explain your rating [Narrative]: ANNEX H. DATA COLLECTION TOOLS n H–35 34 116. P4.a Where has the country achieved more success? And why? We are interested in types of activities that worked well, and your thoughts on the factors that contributed to success. [Probe: can you give me some examples?] 116. P4. b Where have you or your organization achieved more success? And why? We are interested in types of activities that worked well, and your thoughts on the factors that contributed to your success. [Probe: can you give me some examples?] 146. P5. Even in the places that were hit the hardest by Ebola and its aftermath, we have heard there were people and projects that succeeded in helping affected communities and organizations to get back on track. We have heard some even succeeded in making things better than they were before Ebola. What kinds of success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of success stories] 117. P4a. Where has the country faced more challenges? Why? [Probe for types of activities as well as locations where recovery efforts faced challenges] 117. P4.b Where have you or your organization faced more challenges? Why? [Probe for types of activities as well as locations where recovery efforts faced challenges] 146. P5. We know that some places had a very hard time with Ebola, while other places were less affected. But everywhere, even in the places that were hit the hardest, we have heard there were people and projects that succeeded in helping people and organizations to get back on track. Some encountered challenges in trying to make things better than they were before Ebola. What challenges stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of challenges stories] 140. P4. The Ebola crisis stalled the progress [country] was making in health, education and infrastructure and economic development. Over all, do you think that the “recovery” programs that USAID funded and implemented after the Ebola crisis, have helped to reverse these losses? In what ways did they help the country get back on track in terms of health, education, agriculture, and economic development? H–36 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 35 Narrative: 141. P4.1. Which programs were the most important, or made the greatest difference, toward helping the country get back on track? What was it about them that made them most helpful? Narrative: 145. P6.1. Have women and men benefited equally from Ebola recovery projects? [Probe]: Not at all Somewhat/ in some cases Yes Don't Know/Refused 145c. What are the ways that women and girls benefited from the Ebola recovery projects? Probe: What were the difficulties in involving women and girls? 148. (Custom – related to the Department, Sector or County): 403.M2. Would you please describe the coordination among partners that you observed at national level, and at other levels (County/Préfecture/District; sub-district; village) in Ebola recovery efforts? How has your development partners’ support for Ebola recovery been coordinated? We are interested in hearing your opinions about the strengths and the limitations of coordination efforts so far. Narrative: ANNEX H. DATA COLLECTION TOOLS n H–37 36 148. (custom) EQ 2 – Sustainability [S] 202b. S2 When you reflect on the various investments and programs that have been implemented since 2015 to help people, government services, and the economy to recover from Ebola, what results or changes have endured/lasted? [Probe for examples; locations}. What changes that were made in 2015 and 2016 in response to the Ebola crisis are still evident today? Narrative: 206.S4. Thinking about the Ebola recovery efforts that you saw or know most about [in respondent’s thematic area], what factors promoted their sustainability, in your opinion? [Probes: why did the changes mentioned in Q202 keep going? What caused them to last? What “works” to promote sustainability?] Narrative: 208.P5. Ebola Pillar II funding is ending, which activities are the most important for which funding should continue? [Probe: Which activities? Why are they so important?] Narrative: 409.M4. From the outset, it has been clear to the Government of [Country], to USAID and to its implementing partners that the Ebola funding will come to an end in 2019. How prepared are USAID’s partners and implementers for the end of the USG Ebola surge funding? What kinds of challenges are they facing? EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable so far. We really appreciate your point of view. Now we would like to go a bit deeper to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. H–38 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 37 303.OG2. If you think back to 2015, and then to 2016 when recovery efforts were under way, what were the main changes in the social, economic, political, or health situation in the country that affected efforts to help your sector get back on track after the crisis? Narrative: 303b.OG3. Looking back at [Country’s] overall Ebola Recovery programming, what do you think was missed, or underemphasized? 306.OG4. What are the gaps in USAID’s Ebola recovery programming? Do you think that USAID’s efforts missed or under-emphasized anything that is essential to recovery? Narrative: 307.OG7 (custom) EQ 4 – Management [M] The Ebola crisis in this region caught everyone – government, communities, organizations, international development partners - by surprise. It was an emergency, and action was very urgent. This was understood not only here, in [country] but also in the halls of government of [country’s] international development partners. To save lives, the bulk of the international support was devoted to the urgent medical response. However, the Government of [Country] and development partners recognized that the crisis affected its long-term development goals, so recovery support was needed in multiple sectors. We would appreciate hearing your opinions and advice regarding the ways in which the USG’s mechanisms were used to help [Country] – manage and recover from the Ebola crisis of 2014-2015. Our focus is on the resources USAID provided to limit the indirect effects of the crisis, that is the effects of the Ebola crisis beyond the actual illness, such as its effects on the provision of health services for conditions other than Ebola, or its effects on the economic, agricultural, or education sectors. 402.M2. So, considering the outbreak in 2014 and the recovery efforts from 2015 to the present, what are people in your [team/organization] saying about the US Government’s coordination and management of its Ebola Recovery efforts? In [Country] the USG’s Ebola Recovery resources focused on [list the thematic areas and projects from the cheat sheet]. How, and how well, were these Ebola Recovery efforts coordinated with the Government? Narrative: ANNEX H. DATA COLLECTION TOOLS n H–39 38 404.M1b. Still focusing on the Ebola crisis of 2014-4015 and the recovery 2015 to the present, what would you say are the strengths and weaknesses in the processes USAID used to plan, implement, monitor and manage their Ebola recovery efforts? Narrative: 407.M1. Were there things that complicated or impeded [your Department’s/Unit’s] efforts to support recovery from the Ebola crisis? Narrative: 410.M4. What steps should USAID take to prepare its Ebola recovery stakeholders for the end of Ebola surge funding? What should USAID/Washington do? What should the USAID Missions do? 411.M (Custom) Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask two final questions? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? [Probe: If you have anything to say to USAID, what would you say?] Narrative: 401.M. Have you worked on or experienced other humanitarian emergencies in the past? [Pause to give the KI time to think] How did [it/they] compare with the challenges of recovery from the Ebola crisis? H–40 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 39 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? Narrative: 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID with a balanced performance evaluation of the resources programmed to limit the second order impacts of the crisis and recommendations for the remaining 2 years. [Include information on the Learning Summits, and or where to access the report in 2018] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/ Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola ANNEX H. DATA COLLECTION TOOLS n H–41 40 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. H–42 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 41 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for Implementing Partners (revised 11/17/2017) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members (interviewers): 5. _____________________________________________ 6. ____________________________________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): ______________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18 19 20. 21. 22. 23. 24. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A “cheat sheet” that lists the USAID-funded projects that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. • A recorder KIIS— IMPLEMENTING PARTNERS ANNEX H. DATA COLLECTION TOOLS n H–43 42 WARM UP AND INDIVIDUAL INFORMATION 49 As you know, the Ebola crisis had many economic and social effects on families, communities, businesses, organizations, and the government in [Country/the three countries] – beyond the actual illness. Schools were closed; in quarantine areas farming was interrupted; commerce and trade were affected, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect, effects of the Ebola crisis. Are you aware of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refus ed 50 Did you experience any of these indirect effects yourself? [Exper2ndOrder] Yes No 51 What indirect effects have you heard about or experienced? Narrative: 60 Now, please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? 60 [OrgFocus] Narrative: 61 1.4. Does your work focus on a specific target population or geographic catchment area in [COUNTRY], or is your work at national and/or global levels? [Probe for geographic location/catchment areas . Probe for selected beneficiaries (e.g. EVD survivors; USAID IPs), or other focus] 61 [OrgPop] Narrative: 63 1.6 Tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis? [Probe for specifics. If efforts are mentioned by activity type or in general term, ask Can you give me an example?] 63 [PPevdinv] Narrative 64 1.7 These days, do you, or does your organization, work with other people and organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the other organizations; If no, ask when was the last time you or your organization/unit worked directly on Ebola recovery?] 64 [PPevdOrg] Narrative: H–44 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 43 Now let us focus on USAID’s support for recovery from the Ebola crisis, from about April 2015 to the present, since that is the subject of our evaluation. EQ 1 – Performance [P] Work toward recovery from the Ebola crisis involved many different specialists and organizations, both government and non-governmental (NGOs). These efforts covered six sectors: Agriculture and Food security; Health; Education; Governance; Economic crisis mitigation; and Information, Communication and Public-Private Partnerships. Some of these efforts were funded by USAID. We are interested in your views about these diverse recovery efforts, and specifically, about the ones that took place in [respondent’s country of responsibility]. 100.P1. What can you tell us about USAID’s Ebola recovery strategy in [Country] , that is, the efforts programmed to limit the indirect effects of the Ebola crisis? How would you describe the objectives and events that shaped the programming of resources to limit the second order impacts of the crisis? [Probe for knowledge of central Washington/HQ programming, as well as Mission-level programming] Narrative: 101-107. P1. Please tell me/us if you have had any involvement with Ebola recovery activities/projects in any of the following areas in the past 2 years. I will read out the areas and projects that have ended, and those that are ongoing, and I will ask if you did or didn’t have experience with each one. [Interviewers will have a “cheat sheet” with the specific project names and types of activities in each thematic area that have been implemented in the respondent’s location] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communications, Technologies Y/N/DK 114. P4. How would you rate this country’s progress in achieving its Ebola recovery objectives and goals, [specifically for your sector of interest]? Would you say the country has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused 115. Please explain your rating [Narrative]: ANNEX H. DATA COLLECTION TOOLS n H–45 44 116. P4. Where have you [has it] achieved more success? And why? [Probe: we are interested in types of activities that worked well, and also places where you had most success] 146. P5. We know that some places had a very hard time with Ebola, while other places were less affected. But everywhere, even in the places that were hit the hardest, we have heard success stories about people and projects that help people and organizations to get back on track. Some even succeeded in making things better than they were before Ebola. What success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of success stories] 117. P4.Where have you [has it] faced more challenges? Why? 146. P5. We know that some places had a very hard time with Ebola, while other places were less affected. But everywhere, even in the places that were hit the hardest, we have heard success stories about people and projects that help people and organizations to get back on track. Some encountered challenges in trying to make things better than they were before Ebola. What challenges stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of challenges stories] 144. P6. How have gender issues been addressed in Ebola recovery programming? Have women and girls been involved in the planning, and monitoring and evaluation, of the efforts to help communities recover from Ebola? H–46 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 45 145b. Please explain. What are the ways that women and girls benefited? EQ 2 – Sustainability [S] 202. S2 What kinds of effects or changes that were made in 2015 and 2016 due to Ebola are still evident today? [Probe for examples, locations] 206.S4. Thinking about the Ebola recovery efforts that you saw or know most about [in respondent’s technical area], what factors promoted their sustainability? [Probe: why did the change mentioned in Q202 keep going? What caused them to last?] What “works” to promote sustainability, in your opinion? 208.P5. Ebola Pillar II funding is ending, which activities are the most important for which funding should continue? [Probe: Which activities? Why are they so important?] 409.M4. From the outset, USAID has made it clear to the Government of Guinea and to USAID’s implementing partners, that the Ebola funding will come to an end. How prepared are USAID’s partners and implementers for the end of the USG funding in 2019? What kinds of challenges are they facing? ANNEX H. DATA COLLECTION TOOLS n H–47 46 EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable so far. We really appreciate your opinions and suggestions. Now we would like to go a bit deeper to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 303.OG3. Looking back at USAID support to Ebola Recovery efforts, what do you think was missed, or underemphasized? [Probe: Were those things ignored altogether, or were they left for handling by government or other donors?] 304.OG4. What policies or actions in [the respondent’s organization] increased the ability of Ebola recovery project’s to adapt to change? Please share with us your thoughts about how the situation evolved during the crisis, and how some projects were to adapt to change successfully, while others failed to adapt. EQ 4 – Management [M] We would appreciate hearing your opinions regarding the USG’s mechanisms that were used to help the three countries – Guinea, Liberia and Sierra Leone, particularly [country]– manage and recover from the Ebola crisis of 2014-2015. Our focus is on USAID’s “Pillar II” resources – that is, those that were dedicated to recovery from indirect effects of the epidemic. 402.M2. Regarding the Ebola crisis of 2014-2015 and the recovery efforts from 2015 to the present, what do you think about the US Government’s coordination and management of its Ebola Recovery efforts, and how the resources it provided to limit the indirect effects of the crisis were used? In [Country] the USG’s Ebola Recovery resources focused on [list the thematic areas and projects from the cheat sheet]. How, and how well, were these Ebola Recovery efforts coordinated with the Government of [country]? 403b.M2. Would you please describe the coordination among USAID’s implementing partners, and with development partners that you observed at national level, and at other levels ( County/Préfecture/District; sub-district; village)? H–48 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 47 404.M1. Still focusing on the Ebola crisis of 2014-2015 and the recovery 2015 to the present, what is good about the way USAID (overall) programmed its Ebola Recovery resources? What are the strengths of the approaches or processes USAID used to plan, implement, monitor, and manage USAID’s efforts to help communities and organizations to deal with the indirect effects of the Ebola crisis? 406.M1. What weaknesses have you noted in the way USAID programmed its resources to support Ebola Recovery efforts? What problems have you observed in the processes that USAID used to plan, implement, monitor and manage USAID’s Ebola recovery efforts? Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask a few closing questions? 401.M. Have you worked on or experienced other humanitarian emergencies in the past? [Pause to give the KI time to think] How did [it/they] compare with the challenges of recovery from the Ebola crisis? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s/ [country’s] Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] ANNEX H. DATA COLLECTION TOOLS n H–49 48 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola H–50 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 49 Thank you very much for your time and insights. In addition to the key informant interviews, the team is conducting a household survey in USAID’s priority [Counties/Préfectures/Districts], an online survey of Pillar II stakeholders and a survey in selected health care facilities. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID with a balanced performance evaluation of the resources used to limit the second order impacts of the crisis, and recommendations for the remaining 2 years. [Include information on the Learning Summits, and or where to access the report in 2018?] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. 52 And just out of curiosity, when was the last time someone or some project interviewed you, or asked you to participate in a study, regarding Ebola or any other health topic? Last 30 days 1-6 months ago 6 months to a year ago More than a year Never, this is the first time ANNEX H. DATA COLLECTION TOOLS n H–51 50 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for Partner Stakeholders (Revised 11/17/2017) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members: 5. Interviewer_____________________________________________ 6. Note-take______________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents, specify): _______________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17. 18. 19 20. 21. 22. 23. 24.. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A “cheat sheet” that lists the USAID-funded projects that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. • A recorder KIIS— PARTNER STAKEHOLDERS H–52 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 51 WARM UP AND INDIVIDUAL INFORMATION 49 When we prepared for this evaluation, we learned that the Ebola crisis had many economic and social effects on families, communities, businesses, and organizations, as well as on the government in [Country/the three countries] These were effects beyond the actual illness. For example, markets collapsed, and health services for conditions other than Ebola were disrupted. These disruptions and problems have been called indirect effects of the Ebola crisis. Did you hear about any of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refused 51 What indirect effects have you heard about or experienced yourself? Narrative: 60 Now, please tell us more about your organization and your role. What are the main responsibilities of [insert Organization from Q17] and specifically, your unit? [OrgFocus] Narrative: 61 Does your organization/department serve all people from all over [COUNTRY], or do you focus on people in a particular area? Or a particular sub-group? [Probe for geographic location/catchment area of beneficiaries; gender balance; any specific requirements such as residence, or occupation, or ethnicity] [OrgPop] Narrative: 62a Was your organization’s work affected by the Ebola outbreak in 2014-2015? Yes / No 62b In what ways was the work of your organization affected? 62b [OrgEVDimpact] Narrative: 63 Tell us about any efforts that you have been involved in since March 2015 that are dedicated to recovery from the Ebola crisis. [Probe for specifics. If efforts are mentioned in general terms or by type, ask: Can you give me an example?] [PPevdinv] Narrative: 64 Does your department/organization currently work with other people or organizations on Ebola (or Ebola recovery)? [Probe – if yes, probe for the nature and name of the others; If no, probe for when was the last time you or your organization/unit worked directly on Ebola recovery?] [PPevdOrg] Narrative: Now let us focus on USAID’s support for recovery from the Ebola crisis, from about April 2015 to the present, since that is the subject of our evaluation. ANNEX H. DATA COLLECTION TOOLS n H–53 52 EQ 1 – Performance [P] Work toward recovery from the Ebola crisis involved many kinds of specialists and organizations, both government and non-governmental (NGOs). Some of them were funded by the [Guinea/Liberia/Sierra Leone] government. Others were supported by development partners like [insert name of the Partner organization]. Others were funded all or in part by USAID. USAID funded various activities, including projects to restore non-Ebola health services and health systems, efforts to restart and strengthen education, projects to promote agriculture and food security, projects to strengthen governance, efforts to restore investment and trade, and projects to improve the information communication and technology in the country. We are interested in your views about these diverse recovery efforts. We want to know, in your opinion, how useful they were. 100.P1. In your opinion, what has been done, or is being done by government and non-governmental projects or activities to help the [country/county/district/community] recover from the economic, social or political impacts of the Ebola outbreak. [Warm-up and overview] Narrative: 101. P1. 101. P1. Please tell me/us if you have had any involvement with Ebola recovery activities/projects in any of these areas in the past 2 years. I will read out the areas and projects that have ended, and those that are ongoing, and I will ask if you did or didn’t have experience with each one. [Interviewers: refer to “cheat sheet” with the specific project names and intervention types of activities in each thematic area that were implemented in the respondent’s area of responsibility] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 104. Education 105. Governance 106. Economics, trade, investment 107. Information, Communicatio ns, technologies Y/ N/ DK 114. P4. How would you rate [your country’s] [Guinea’s/Liberia’s/Sierra Leone’s] progress in achieving its recovery objectives and goals, [specifically in your sector of interest]? Would you say it has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused 115. Please explain your rating [Narrative]: H–54 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 53 116. P4. Where have you [where has the country] achieved more success? And why? [Probe: we are interested in types of activities that worked well, and also places where you had the most success] 146. P5. Even in the places that were hit the hardest, we have heard about success stories of people and projects helping [communities/organizations] get back on track. What success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of success stories] Narrative: 117. P4.Where have you [where has the country] faced challenges? Why 146. P5. We know that some places had a very hard time with Ebola, while other places were less affected. But everywhere, even in the places that were hit the hardest, we have heard there were people and projects that succeeded in helping people and organizations to get back on track. Some encountered challenges in trying to make things better than they were before Ebola. What challenges stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of challenges stories] 140. P4. The Ebola crisis slowed down the progress [country] was making in health, education and infrastructure and economic development. Over all, do you think that the “recovery” programs that were implemented after the Ebola crisis (those we have been talking about), have helped to reverse these losses? Did these recovery programs help the country get back on track in terms of health, education, and economic development? Yes No DK 140.P4b If yes, how did they help? 140.P4c If no, how did they not help? Narrative: ANNEX H. DATA COLLECTION TOOLS n H–55 54 141. P4.1. Which programs were most important, or made the greatest difference in helping the country get back on track? What was it about them that made them most helpful? Narrative: 142. P4.2. Did you see evidence of government working with commercial groups, and/or with private, not￾for-profit groups, on Ebola recovery3? Yes No 142.P4.2b Can you give us a couple of examples of government working with non-governmental organizations and the private-for-profit sector to promote [country’s] development? Narrative: 143. P4.3. Now, let’s turn to infrastructure – water, waste removal, power, telecommunications, roads and transport: Are activities in these sectors back on track? Can you give us some examples where they are better now than before the Ebola crisis? Narrative: 145. P6.1. Have women and men benefited equally from Ebola recovery projects? Not at all Somewhat/ in some cases Yes Don't Know/Refused 145b. What are the ways that women and girls benefited? Probe: clarify if the respondent is talking about USAID Pillar II efforts, or projects funded by others. Narrative: 145c. What have been the challenges in involving women and girls in Ebola recovery efforts? 3 This question is aimed to elicit examples of public-private partnerships, that is, collaboration between governmental and non￾governmental organizations. H–56 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 55 146. P5. Even in the places that were hit the hardest, we have heard about success stories of people and projects helping [communities/organizations] get back on track. What success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of success stories] Narrative: 144. P6. How well have gender issues been addressed in USAID’s efforts to support recovery from the Ebola crisis? Are there strategies that you have observed that aimed to promote gender equity? 403.M2. Would you please describe the coordination among partners in Ebola recovery efforts that you observed at national level, and at sub-national levels (County/Préfecture/District; sub-district; village)? Narrative: 403.M2b How has your organization’s support for Ebola recovery been coordinated with the Government and other partners? We are interested in hearing your opinions about the strengths and the limitations of coordination efforts so far. 148. (custom) EQ 2 – Sustainability [S] 201. S1 What kinds of lasting results of EBOLA recovery efforts do you or did you expect to see in [country] after the epidemic was under control? [If appropriate, MENTION RESPONDENT’S technical area] [Interviewers should focus on efforts after 2015] Narrative: ANNEX H. DATA COLLECTION TOOLS n H–57 56 202. S2 What changes that were made in 2015 and 2016 in response to the Ebola crisis are still evident today? [Probe for examples; locations]. Narrative: 204.S3. What do you think the government, and its partners should do to help the [Guinean/Liberian/Sierra Leonean] people to be more able to withstand and recover from health crises such as disease outbreaks or droughts in the future? What kind of changes or activities can build “resilience”? Narrative: 207.S4. In your experience, why did the effects of some Ebola recovery efforts stop, or fade away, after an externally funded project ended? Narrative: 208.P5. Ebola Pillar II funding is ending, which activities are the most important for which funding should continue? [Country]’s development, that they should be funded from other sources? Which activities? Why? Narrative: 409.M4. From the outset, USAID has made it clear to the Government of [Guinea/Liberia/Sierra Leone], and to USAID’s implementing partners, that the Ebola funding will come to an end in 2019. How prepared are USAID’s partners and implementers for the end of the USG funding for Ebola recovery? What kinds of challenges are they facing? 209.P5. We are interested in your views about activities or projects that have run their course, and the ones that did not work well. If you had to prioritize all the [Ebola recovery/technical area] interventions that are being implemented in your [country/District/Department], which one, or ones, would you phase out first? And why would you discontinue them? Narrative: 208 (Custom) H–58 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 57 EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable. We really appreciate your views and opinions. Now we would like to go a bit deeper to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 302.OG1. Please think for a moment of a place or places where Ebola recovery efforts seem to have gone smoothly or where [interventions in the KI’s thematic area/s] have had the greatest success. [pause to allow the KI to think] Why do you think it was easier for the [efforts/interventions] to succeed in those places/situations? What factors or conditions supported success? [Probe for types of factors] Narrative: 306.OG4. In your opinion, what gaps (if any) have you observed in the way Ebola recovery programs were planned and funded? Do you think that Pillar II missed or under-emphasized anything that is essential to success? Narrative: 307.OG7 (custom) EQ 4 – Management [M] The Ebola crisis in this region caught everyone – government, communities, organizations, international development partners - by surprise. It was an emergency, and action was very urgent. This was understood not only here, in [country] but also in the halls of government of [country’s] international development partners. To save lives, the bulk of the international support was devoted to the urgent medical response. However, the Government of [Country] and development partners recognized that the crisis affected its long-term development goals, so recovery support was needed in multiple sectors. For the United States, this challenge brought the need to join up efforts through several USG agencies, including USAID’s Offices, regional and technical Bureaus in Washington, USAID’s missions and Embassies in Liberia, Guinea and Sierra Leone, the US Centers for Disease Control and Prevention, the Department of Defense, and other relevant institutions. These joint USG efforts also needed to be negotiated with local government and coordinated with national and international partners. ANNEX H. DATA COLLECTION TOOLS n H–59 58 We would appreciate hearing your opinions and advice regarding the ways in which the USG’s mechanisms were used to help the three countries – Guinea, Liberia and Sierra Leone – manage and recover from the Ebola crisis of 2014-2015. Our focus is on USAID’s “Pillar II” resources. 401.M. First, please tell me/us: have you experienced other humanitarian emergencies? If yes: How did it/they compare with the challenges of recovery from the Ebola crisis? Narrative [specify year and place of the emergencies cited]: 402.M2. Now, turning to the Ebola crisis of 2014-2015 and the recovery efforts from 2015 to the present, what are people in your [team/organization] saying about the US Government’s coordination and management of its Ebola Recovery efforts (“Pillar II”)? In [Country] the USG’s Ebola Recovery resources focused on [list the thematic areas and projects from the cheat sheet]. How, and how well, were these Ebola Recovery efforts coordinated with the Government? With other development partners? Narrative: 403.M2. Now looking more broadly (beyond America’s efforts), how was the coordination among other development partners that you observed? We are interested in your thoughts on coordination at national level, and also at other levels (County/Préfecture/District; sub-district; village)? 408.M3. If you compare the way your [Organization/Agency] works on long-term development goals, and the way it responds to an emergency, like a disaster, what lessons can USAID take from the recent Ebola recovery experience on how to support countries more rapidly and efficiently in humanitarian emergency situations? 410.M4. What steps should USAID take to prepare its recipients of Ebola recovery funds for the end of Ebola surge funding? 411.M (Custom) H–60 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 59 Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask a few more questions? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] Narrative: 47. May I ask a sensitive question? Did you yourself lose anyone to Ebola (did someone close to you die of Ebola?)? If Yes: Who was it? Record all that are mentioned) [LostToEVD] If yes: I am very sorry for your loss. If no: Thank you for telling me. Both: We believe that the information and views that you shared with us today will help to improve the US Government’s Ebola recovery efforts in [country/ies]. Interviewer: check off all the categories of people that the respondent lost. None Mother Father Sibling Child Spouse Other, (specify) Refused 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? Narrative: 45 May I ask, where did you live in 2014- 2015, that is during the Ebola crisis? (Record all that apply) [ResidDuring] Here (this county/District/Prefecture & country) Other town affected by Ebola Other town not affected by Ebola Other country affected by Ebola Other African country not affected USA Other country not affected by Ebola ANNEX H. DATA COLLECTION TOOLS n H–61 60 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to provide USAID and the countries a balanced performance evaluation of Ebola recovery activities, and recommendations for the remaining 2 years. [Include information on the Learning Summits, and or where to access the report in 2018?] End: Record the time: ______________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided on Question 503 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. 52 And just out of curiosity, when was the last time someone or some project interviewed you, or asked you to participate in a study, regarding Ebola or any other health topic? Last 30 days 1-6 months ago 6 months to a year ago More than a year Never, this is the first time H–62 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 61 USAID/Pillar II Ebola Recovery Efforts to Mitigate Second Order Impacts of the Epidemic in Guinea, Sierra Leone & Liberia Performance Evaluation Phase 2 Core Key Informant Interview Guide Template for US Government Stakeholders (revised 11/17/2017) BACKGROUND & INFORMED CONSENT 1. Date: 2. Country: 3. Time Start: 4. Time Finish: Names of KII team members (interviewers): 5. _____________________________________________ 6____________________________________________________________________ 7.County/Préfecture: 8. District: 9. Town/village . 10. Setting 11. Recording file ID # 12. Recording Ok? Circle Best Answer I don’t know 0 1 2 3 13. Privacy of the setting Public Family, within earshot Private 14. Comfort of the Interviewee Anxious Ok At Ease 15. Other people present (if any, not respondents):________________________________ Respondent Profile: Name Formal or Informal Position Organization Contact information (telephone?) PE1 ID Code 16. 17.. 18. 19. 20.. 21. 22.. 23. 24.. 25. THIS PAGE MUST BE REMOVED AND STORED SECURELY. THE INTERVIEW CONTENTS TO BE ASSOCIATED WITH THE SOCIODEMOGRAPHIC BACKGROUND INFORMATION ONLY THROUGH THE PE1 ID CODE (Q20) ON EVERY PAGE. Interviewers will bring to the interview: • A briefing document, outlining the scope of work, methods and timeline of the Performance Evaluation (PE) and identifying the HO key personnel and sub-contractors, with contact information. • A copy of the letter of introduction from the relevant body or official. • A copy of the Introduction/Consent Form. • A “cheat sheet” that lists the USAID-funded projects that were or are being carried out in the respondent’s geographic area of responsibility, grouped by thematic area. • A recorder KIIS— U.S. GOVERNMENT STAKEHOLDERS ANNEX H. DATA COLLECTION TOOLS n H–63 62 WARM UP AND INDIVIDUAL INFORMATION 49 As you know, the Ebola crisis had many economic and social effects on families, communities, businesses, organizations, and the government in [Country/the three countries] – beyond the actual illness. Schools were closed; in quarantine areas farming was interrupted; commerce and trade, and social relationships were affected. Also, health services for conditions other than Ebola were disrupted These disruptions and problems have been called indirect effects of the Ebola crisis. Are you aware of these indirect effects of the Ebola crisis? [Heard2ndOrder] Yes No Refused 51 What indirect effects have you heard about or experienced? Narrative: 61 1.4. Does your work focus on a specific target population or geographic catchment area in [COUNTRY], or is your work at national and/or global levels? [Probe for geographic location/catchment areas ; selected beneficiaries (e.g. EVD survivors; USAID IPs), or other focus] 61 [OrgPop] Narrative: EQ 1 – Performance [P] USAID funded various activities in the region to mitigate indirect effects of the Ebola epidemic. In the region as a whole, these efforts covered six sectors: Agriculture and food security; Health; Education; Governance; Economic crisis mitigation; and Information, communication and public-private partnerships. We are interested in your views about these diverse recovery efforts, and specifically, about the ones that took place in [Country]. We want to know, in your opinion, whether they were useful. 100. P1. What can you tell us about how USAID’s Ebola recovery programs and activities are prioritized and funded in [country]? How would you describe the objectives and events that shaped the allocation of resources by USAID to limit the indirect effects of the Ebola crisis – that is, the effects of the crisis beyond the actual illness, such as its effects on the provision of health services for conditions other than Ebola, or its effects on the economic, agricultural, or education sectors? [Probe for knowledge of Washington Operating Units (OU) programming as well as Mission-level programming] H–64 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 63 101-107. P1. Please tell me/us if you have had any involvement with Ebola recovery activities/projects in any of the following areas in the past 2 years. I will read out the areas and projects that have ended, and those that are ongoing, and I will ask if you did or didn’t have experience with each one. [Interviewers will have a “cheat sheet” with the specific project names and types of activities in each thematic area that have been implemented in the respondent’s location] 101. Agriculture; Food Security 102. Health Services & Systems 103. EVD survivor support 105. Governance 106. Economics, trade, investment 107. Information, Communications , Technologies Y/N/ DK Y Y Y Y 114. P4. How would you rate [country’s] progress in achieving its recovery objectives and goals, [specifically in your sector of interest]? Would you say it has made: [read out the options, and check the appropriate box] No progress Limited progress Good progress (met expectations) Significant progress (exceeded expectations) Don’t Know or Refused 115. Please explain your rating [Narrative]: 116. P4b. Where have you achieved success? And why? We are very interested in your views on the types of activities that worked well, and on the factors contributed to success. [Appreciate those given, and probe for different types of success stories] 146. P5b. What success stories can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of success stories] ANNEX H. DATA COLLECTION TOOLS n H–65 64 117. P4.Where have you faced challenges? Why? 146. P5b. What challenges can you tell us about, from your own experience? [Appreciate those given, and probe for more kinds of challenges] 143. P4.3. Now let’s talk about infrastructure – water, waste removal, power, telecommunications, roads and transport: is this sector back on track? Can you give some examples of areas in this sector that are doing better now than before the Ebola crisis? 144. P6. How have gender issues been addressed in Ebola recovery programs? Have women and girls been involved in the planning, monitoring, and evaluation of efforts designed to help communities recover from Ebola? 145b. Please explain. What are the ways that women and girls benefited? EQ 2 – Sustainability [S] 202. S2b. What kinds of effects or changes as a result of EVD recovery efforts have actually lasted after the project was over, as far as you know? [Probe for examples; locations: What changes that were made in 2015 and 2016 due to Ebola are still evident today? Where have you observed those lasting changes?] 206.S4. Thinking about the Ebola recovery efforts that you saw or know most about [in respondent’s technical area], what factors promoted their sustainability, in your opinion? [Probes: Why were they sustained? What caused them to last?] 207.S4. In your experience, why did the effects of some Ebola recovery efforts stop, or fade away, after an externally funded project ended? H–66 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 65 208.P5. Ebola Pillar II funding is ending, which activities are the most important for which funding should continue? [Probe: Which activities? Why are they so important?] 409.M4. From the outset, it has been made clear to the Government of [Liberia/Guinea/Sierra Leone] and USAID’s implementing partners that the Ebola funding will come to an end in 2019. How prepared are USAID’s partners and implementers for the end of the USG funding for Ebola? What kinds of challenges are they facing? EQ 3 – Opportunities and Gaps [OG] Your ideas and advice about Recovery efforts have been very valuable so far. We really appreciate your opinions and suggestions. Now we would like to ask you about the conditions surrounding the Ebola recovery efforts, so that we can understand these efforts in their real contexts. 303.OG3b. Looking back at the ways USAID-funded Ebola Recovery programs and projects were planned and funded, what were the gaps? What do you think was missed, or not given adequate consideration? [Probe: Were some aspects missed altogether? Or were they supposed to be addressed by another organization/partner?] 304.OG4. From your experience, what USAID policies or actions (if any) enabled Ebola recovery projects to adapt and respond to changing circumstances? We want to get your thoughts on how the situation evolved during the crisis, and the successful ways and unsuccessful attempts of projects to adapt to change. EQ 4 – Management [M] We would appreciate hearing your opinions regarding ways in which the USG’s mechanisms were used to help [country] manage and recover from the Ebola crisis of 2014-2015. Our focus is on the resources USAID provided to projects and interventions aimed at limiting the indirect effects of the crisis, and building countries’ resilience to withstand future shocks. 402. M2. In [Country], the USG’s Ebola Recovery resources focused on [INSERT THE SECTORS RELEVANT TO PILLAR II IN THIS COUNTRY; E.G. restoring non-Ebola health services, governance, economic recovery, and agriculture and food security]. How, and how well, were these Ebola Recovery efforts coordinated with the Government? [Probe both for “how” (the coordination processes/platforms), and how well (evaluation of their usefulness). 403.M2. Now looking more broadly (beyond America’s efforts), and based on your observations, how would you describe the coordination among other development partners? We are interested in your thoughts on coordination at national level, as well as at other levels. ANNEX H. DATA COLLECTION TOOLS n H–67 66 405.M1. Can you tell us about a particular strategy that you used to overcome bottlenecks and/or to mobilize USAID’s expertise and resources in support of the recovery? 406.M1. What problems did you observe in the processes that USAID used to plan, implement, monitor and manage USAID’s Ebola recovery efforts? Summary and Closing You have been so patient and generous with your time. We are very grateful! In closing, may I ask a few closing questions? 501.P3. What is the strongest message that the US Government should hear about USAID’s role in the region’s Ebola response and recovery? What do you think is the most important message to give USAID’s stakeholders? [Probe: If you have anything to say to USAID, what would you say?] 401.M. Have you worked on or experienced other humanitarian emergencies in the past? [Pause to give the KI time to think] How did [it/they] compare with the challenges of recovery from the Ebola crisis? 502. Is there a topic that we did not cover, or a question or issue that you would like to raise? 503. How old are you? If age is not provided, ask in which age category do you belong? Interviewer: check the age category accordingly. <30 30-45 46-60 >60 Refused Thank you very much for your time and insights. We will combine your responses with those of many other respondents from throughout [Country] as well as those from [other 2 countries], to present USAID and the countries with a balanced performance evaluation of the resources USAID provided to projects and interventions aimed at limiting the indirect effects of the crisis., including recommendations for the remaining 2 years. End: Record the time: _____________ INTERVIEWERS AND SUPERVISORS: AFTER INTERVIEW: PLEASE ADD THE PE1 ID CODE TO EVERY PAGE OF THE INTERVIEW TRANSCRIPT. Note the Gender and age group of the interviewee only when the age is not provided for Question 503 H–68 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 67 For the respondent identified in Gender Age Group Ado; Young adult; Mature adult; Elder Comment (if any) Question 16 26. 27. Question 21 28. 29. THEN DETATCH THE PAGES WITH PERSONAL IDENTIFICATION FROM THE REST OF THE TRANSCRIPT STORE THE TRANSCRIPT SEPARATELY FROM THE IDENTIFYING INFORMATION AND INFORMED CONSENT FORMS. ANNEX H. DATA COLLECTION TOOLS n H–69 68 Health Facility Survey Adapted Version of the Service Availability and Readiness (SARA) Survey SECTION A: IDENTIFICATION A1 Facility number SCRIPTER: WILL BE IMPORTED FROM SAMPLING FILE MEASUREMENT WAVE A2 Is this visit conducted for the Performance Evaluation in 2017 or in 2018? SCRIPTER: AUTOCODE 1 Performance Evaluation 2017 Performance Evaluation 2019 1 2 INTERVIEWER VISITS A3 Is this a visit made by an enumerator or by the supervisor of an enumerator? Enumerator …………………………………………… Supervisor of enumerator …………………………… 1 2 A4 Number of Visits Paid to Health Facility to Complete the Interview FACILITY IDENTIFICATION [THIS PART WILL BE PRECODED FROM THE SAMPLING FILE] A5 Name of facility ____________________________________ A6 Location of facility ____________________________________ A7 Region/Province A8 District A9 Chiefdom A10 Locality A11 Type of facility* * To be adapted at country level prior to implementation* NATIONAL TERTIARY REFERRAL HOSPITAL ........... PROVINCIAL SECONDARY HOSPITAL ....................... DISTRICT SECONDARY HOSPITAL ……………………………… COMMUNITY HEALTH CENTRE (CHC) ..................... COMMUNITY HEALTH POST (CHP)............................ MATERNAL/CHILD HEALTH POST (CHP).................. OTHER (SPECIFY) _________________________ 1 2 3 4 5 6 96 A12 Managing Authority GOVERNMENT/PUBLIC.................................................... NGO/NOT-FOR-PROFIT .................................................. PRIVATE-FOR-PROFIT ....................................................... MISSION/FAITH-BASED..................................................... OTHER (SPECIFY) _________________________ 1 2 3 4 96 A13 Urban/Rural URBAN .................................................................................. RURAL.................................................................................... 1 2 A14 Outpatient only YES .......................................................................................... NO.......................................................................................... 1 2 GEOGRAPHIC COORDINATES AUTOCAPTURE GPS SURVEY— HEALTH FACILITIES H–70 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 69 Date Interviewer Name Result Visit # 1 _______________ _______________ Visit # 2 _________________ _______________ Visit # 3 _______________ _______________ FINAL VISIT DAY MONTH YEAR INT. NUMBER RESULT CODE: Interim outcome code 1. Contact made with HEALTH FACILITY (--> Put Code 1 for the visit and go to introduction and A13) 2. No reply/No one at the facility (--> Put Code 2 for the visit, and plan re-visit ) 3. Ineligible address/Health Facility abandoned/does not exist (--> Put Code 3 for the visit, need to replace) 4. Refusal to cooperate – No permission to interview (→ Put Code 4 for the visit, code 3 in the I1 FinalOutcome) ANNEX H. DATA COLLECTION TOOLS n H–71 Number Question Result Skip GENERAL INFORMATION INTERVIEWER CODE FIND THE MANAGER, THE PERSON IN-CHARGE OF THE FACILITY, OR MOST SENIOR HEALTH WORKER RESPONSIBLE FOR OUTPATIENT SERVICES WHO IS PRESENT AT THE FACILITY. READ THE FOLLOWING CONSENT: Good day! My name is _____________________. We are here on behalf of [IMPLEMENTING AGENCY] conducting a survey of health facilities to assist the government in knowing more about health services in [COUNTRY]. Now I will read a statement explaining the study. Your facility was selected to participate in this study. We will be asking you questions about various health services. Information about your facility may be used by the [MOH], organizations supporting services in your facility, and researchers, for planning service improvement or for conducting further studies of health services. Neither your name nor that of any other health worker respondents participating in this study will be included in the dataset or in any report; however, there is a small chance that any of these respondents may be identified later. Still, we are asking for your help to ensure that the information we collect is accurate. You may refuse to answer any question or choose to stop the interview at any time. However, we hope you will answer the questions, which will benefit the services you provide and the nation. If there are questions for which someone else is the most appropriate person to provide the information, we would appreciate if you introduce us to that person to help us collect that information. At this point, do you have any questions about the study? Do I have your agreement to proceed? A15 May I begin the interview? YES............................................................ 1 NO ........................................................... 2 ➔I0 A16 INTERVIEW START TIME (use the 24 hour-clock system) : H–72 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip SECTION B: STAFFING B1 I have a few questions on staffing for this facility. Please tell me how many staff with each of the following qualifications are currently assigned to, employed by, or seconded to this facility. Please count each staff member only once, on the basis of the highest technical or professional qualification. For doctors, I would also like to know, of the total number, how many are part-time in this facility. INTERVIEWER: CODE 998 FOR DON’T KNOW, 999 FOR REFUSE. To be adapted at country level prior to implementation A) ASSIGNED/ EMPLOYED/ SECONDED (INCLUDING PART TIME) CODE EACH QUESTION AS Q#A B) PART TIME CODE EACH QUESTION AS Q#B S4 B2 Generalist (non-specialist) medical doctors S4 B3 Specialist medical doctors S4 B4 B5 Community health officers B6 Community health assistants S4 B7 State enrolled community health nurses B8 State registered Nurses B9 Maternal child health aides B10 Nursing officers S4 B11 Midwifery professionals B12 Pharmacists B13 Pharmacy technicians B14 Nutritionists B15 Laboratory technicians/officers B16 Environmental and public health officers B17 Non-clinical support staff. (Please let me know how many you have by category below) B18 Monitoring and Evaluation officers B19 Data Clerks B20 Administrative and finance officers B21 Equipment and facility maintenance officers B22 Drivers ANNEX H. DATA COLLECTION TOOLS n H–73 B23 Does this health facility have sufficient human resources to provide the services that are offered at this facility? YES............................................................ 1 NO............................................................ 2 DON’T KNOW…………………………98 REFUSE…………………………………..99 B24 What professions this health facility is lacking? Please indicate the number needed for each profession and specify if it is for full time or part time. The list should be adapted at country level prior to implementation. FULL TIME PART TIME B25 Generalist (non-specialist) medical doctors B26 Specialist medical doctors B27 Community health officers B28 Community health assistants B29 State enrolled community health nurses B30 State registered Nurses B31 Nursing officers B32 Laboratory technicians/officers B33 Etc ACCOMMODATION B34 Please tell me if the health facility provides accommodation (has quarters) for the staff. YES, OBSERVED……………………1 YES, REPORTED NOT SEEN………2 NO……………………………………3 DON’T KNOW…………………….98 REFUSE……………………………...99 TRAINING B35 Have you or any health providers in this health facility received an IPC protocol training within the past 2 years? YES............................................................ 1 NO............................................................ 2 DON’T KNOW……………………..98 REFUSE………………………………99 2, 98, 99➔ C1 B36 How many health workers have received the IPC protocol training within the past 2 years. PROBE TO GET THE ESTIMATES Number of HW DOES NOT KNOW…………………………….1 H–74 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SECTION C: INPATIENT AND OBSERVATION BEDS S2 C1 Excluding any delivery beds, how many overnight/inpatient beds in total does this facility have, both for adults and children? # OF OVERNIGHT/ INPATIENT BEDS. . . . . 98 DK 99 REF S3 C2 Of the overnight/inpatient beds in this facility, how many are dedicated maternity beds? THIS DOES NOT INCLUDE DELIVERY BEDS # OF DEDICATED MATERNITY BEDS. . . . . 98 DK 99 REF Indicator code Number Question Result Skip/D on’t Know SECTION D: INFRASTRUCTURE This section will focus on questions related to infrastructure. COMMUNICATIONS I5 D0 Does this facility have a functioning telephone or short wave radio that is available to call outside at all times client services are offered? CLARIFY THAT IF FACILITY OFFERS 24-HOUR EMERGENCY SERVICES, THEN THIS REFERS TO 24-HOUR AVAILABILITY. YES................................................................... 1 NO................................................................... 2 DON’T KNOW…………………………………98 REFUSE…………………………………99 I6 D1 Is there access to email or internet within the facility today? YES................................................................... 1 NO................................................................... 2 DON’T KNOW…………………………………98 REFUSE…………………………………99 AMBULANCE/TRANSPORT FOR EMERGENCIES I7 D2 Does this facility have a functional ambulance or other vehicle for emergency transportation for clients that is stationed at this facility or operates from this facility? YES................................................................... 1 NO................................................................... 2 DON’T KNOW…………………………………98 REFUSE…………………………………99 ➔D4 I7 D3 Does this facility have access to an ambulance or other vehicle for emergency transport for clients that is stationed at another facility or that operates from another facility in near proximity? YES................................................................... 1 NO................................................................... 2 DON’T KNOW…………………………………98 REFUSE…………………………………99 1, 2, 98, 99➔D5 I7 D4 Is fuel for the ambulance or other emergency vehicle available today? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE………………………………….99 POWER SUPPLY I1 D5 Does your facility have electricity from any source (e.g. electricity grid, generator, solar, or other) including for stand-alone devices (EPI cold chain)? YES................................................................... 1 NO................................................................... 2 DON’T KNOW…………………………………98 REFUSE…………………………………99 2, 98, 99➔D 14 ANNEX H. DATA COLLECTION TOOLS n H–75 Indicator code Number Question Result Skip/D on’t Know I1 D6 What is the electricity used for in the facility? ONLY STAND-ALONE ELECTRIC MEDICAL DEVICES/APPLIANCES (e.g. EPI cold room, refrigerator, suction apparatus, etc.).................................................................. 1 ELECTRIC LIGHTING (EXCLUDING FLASHLIGHTS) AND COMMUNICATIONS ................................. 2 ELECTRIC LIGHTING, COMMUNICATIONS, AND 1 TO 2 ELECTRIC MEDICAL DEVICES/ APPLIANCES ................................................. 3 ALL ELECTRICAL NEEDS OF FACILITY 4 DON’T KNOW…………………………………98 REFUSE…………………………………99 D7 What is the facility’s main source of electricity? CENTRAL SUPPLY OF ELECTRICITY (e.g. national or community grid) ....................... 1 GENERATOR (FUEL OR BATTERY OPERATED GENERATOR) ....................... 2 SOLAR SYSTEM............................................. 3 OTHER _______________________ 96 (SPECIFY) DON’T KNOW…………………………………98 REFUSE…………………………………99 D8 Other than the main or primary source, does the facility have a secondary or backup source of electricity? IF YES: What is the secondary source of electricity? NO SECONDARY SOURCE ..................... 0 CENTRAL SUPPLY OF ELECTRICITY (e.g. national or community grid) ....................... 1 GENERATOR (FUEL OR BATTERY OPERATED GENERATOR) ....................... 2 SOLAR SYSTEM............................................. 3 OTHER _______________________ 96 (SPECIFY) DON’T KNOW…………………………………98 REFUSE…………………………………99 I1 D9 During the past 7 days, was electricity available at all times from the main or any backup source when the facility was open for services? ALWAYS AVAILABLE (NO INTERRUPTIONS) ....................................... 1 OFTEN AVAILABLE (INTERRUPTIONS OF LESS THAN 2 HOURS PER DAY) ..... 2 SOMETIMES AVAILABLE (FREQUENT OR PROLONGED INTERRUPTIONS OF MORE THAN 2 HOURS PER DAY) ......... 3 DON’T KNOW…………………………………98 REFUSE…………………………………99 CHECK QD7 AND QD8: FACILITY HAS A GENERATOR ( “2” SELECTED FOR EITHER QUESTION) FACILITY DOES NOT HAVE A GENERATOR (“2” NOT SELECTED FOR BOTH QUESTIONS) H–76 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/D on’t Know ➔D12 D10 Is the generator functional? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE…………..........................................99 2, 98, 99➔D 12 D11 Is there fuel or a charged battery available today? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 D12 CHECK QD7 AND QD8: FACILITY HAS A SOLAR SYSTEM ( “3” SELECTED FOR EITHER QUESTION) FACILITY DOES NOT HAVE A SOLAR SYSTEM (“3” NOT SELECTED FOR BOTH QUESTIONS) ➔D14 D13 Is the solar system functional? YES, FUNCTIONING.................................. 1 PARTIALLY, BATTERY NEEDS SERVICING/REPLACEMENT ..................... 2 NO, NOT FUNCTIONAL.......................... 3 DON’T KNOW.......................................... 98 REFUSE.....................................................99 BASIC CLIENT AMENITIES D14 On average, how many hours per day is this facility open? 4 HOURS OR LESS ...................................... 1 5 TO 8 HOURS............................................. 2 9 TO 16 HOURS .......................................... 3 17 TO 23 HOURS ........................................ 4 24 HOURS ..................................................... 5 DON’T KNOW………………………...98 REFUSE………………………………….99 I2 D15 What is the most commonly used source of water for the facility at this time? OBSERVE THAT WATER IS AVAILABLE FROM THE SOURCE OR IN THE FACILITY ON THE DAY OF THE VISIT. E.G. CHECK THAT THE PIPE IS FUNCTIONING. PIPED INTO FACILITY ............................... 1 PIPED ONTO FACILITY GROUNDS ...... 2 PUBLIC TAP/STANDPIPE........................... 3 TUBEWELL/BOREHOLE............................. 4 PROTECTED DUG WELL.......................... 5 UNPROTECTED DUG WELL................... 6 PROTECTED SPRING................................. 7 UNPROTECTED SPRING .......................... 8 RAINWATER COLLECTION .................... 9 BOTTLED WATER.................................... 10 CART W/SMALL TANK/DRUM ............. 11 TANKER TRUCK....................................... 12 SURFACE WATER ..................................... 13 OTHER _______________________ 96 (SPECIFY) DON'T KNOW .......................................... 98 ➔D17 ➔D17 ➔D17 ➔D17 ➔D17 ➔D17 ANNEX H. DATA COLLECTION TOOLS n H–77 Indicator code Number Question Result Skip/D on’t Know NO WATER SOURCE .............................. 00 REFUSE............................................................99 ➔D17 ➔D17 I2 D16 Is water available from this source on facility premises? YES, INSIDE THE FACILITY....................... 1 YES, WITHIN THE GROUND OF THE FACILITY ........................................................ 2 NO, OUTSIDE THE FACILITY GROUNDS .................................................... 3 DON’T KNOW.......................................... 98 REFUSE.....................................................99 I3 D17 Is there a room with auditory and visual privacy available for patient consultations? AUDITORY PRIVACY ONLY .................... 1 VISUAL PRIVACY ONLY............................ 2 BOTH AUDITORY AND VISUAL PRIVACY ........................................................ 3 NO PRIVACY................................................ 4 DON’T KNOW.......................................... 98 REFUSE............................................................99 I4 D18 Is there a toilet (latrine) on premises in functioning condition that is accessible for general outpatient client use? IF YES: What type of toilet? IF MULTIPLE TOILETS ARE AVAILABLE, CONSIDER THE MOST MODERN TYPE OBSERVE THAT THE TOILET (LATRINE) IS ACCESSIBLE (UNLOCKED OR KEY AVAILABLE) AND FUNCTIONING FLUSH TOILET ............................................. 1 VENTILATED IMPROVED PIT LATRINE (VIP) ................................................................. 2 PIT LATRINE WITH SLAB ......................... 3 PIT LATRINE WITHOUT SLAB/OPEN PIT.................................................................... 4 COMPOSTING TOILET ............................. 5 BUCKET ......................................................... 6 HANGING TOILET/ HANGING LATRINE......................................................... 7 NO FACILITIES ON PREMISES/BUSH/FIELD ............................... 8 DON’T KNOW.......................................... 98 REFUSE.....................................................99 PROCESSING OF EQUIPMENTS FOR REUSE D19 Please tell me if the following items used for processing of equipment for reuse are available and functional in the facility today. IF AVAILABLE, ASK TO SEE IT AND INDICATE IF IT IS FUNCTIONING OR NOT A) AVAILABLE B) FUNCTI ONING OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON’T KNOW I8 D19_01 Electric autoclave (pressure & wet heat) 1 → B 2 → B 3 02 1 2 98 I8 D19_02 Non-electric autoclave 1 → B 2 → B 3 03 1 2 98 I8 D19_03 Electric dry heat sterilizer 1 → B 2 → B 3 04 1 2 98 D19_04 Electric boiler or steamer (no pressure) 1 → B 2 → B 3 05 1 2 98 H–78 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/D on’t Know D19_05 Non-electric pot with cover for boiling/steam 1 06 2 06 3 06 I8 D19_06 Heat source for non-electric equipment 1 → B 2 → B 3 D20 1 2 98 HEALTH CARE WASTE MANAGEMENT I9 D20 Now I would like to ask you a few questions about waste management practices for sharps waste, such as needles or blades. How does this facility finally dispose of sharps waste (e.g., filled sharps boxes)? PROBE TO ARRIVE AT CORRECT RESPONSE NOTE: IF ANY OF THE RESPONSES 2-9 TAKE PLACE OUTSIDE THE FACILITY, THEN THE CORRECT RESPONSE TO CIRCLE WILL BE IN THE CATEGORY OF "REMOVE OFFSITE" BURN INCINERATOR 2-CHAMBER INDUSTRIAL (800-1000+° C) ..................................................................... 2 1-CHAMBER DRUM/BRICK....................... 3 OPEN BURNING FLAT GROUND - NO PROTECTION ... 4 PIT OR PROTECTED GROUND ............. 5 DUMP WITHOUT BURNING FLAT GROUND - NO PROTECTION ... 6 COVERED PIT OR PIT LATRINE ............. 7 OPEN-PIT - NO PROTECTION............... 8 PROTECTED GROUND OR PIT ............. 9 REMOVE OFFSITE STORED IN COVERED CONTAINER.. 10 STORED IN OTHER PROTECTED ENVIRONMENT ......................................... 11 STORED UNPROTECTED....................... 12 OTHER _______________________ 96 (SPECIFY) NEVER HAS SHARP WASTE................... 95 DON’T KNOW.......................................... 98 REFUSE............................................................99 I10 D21 Now I would like to ask you a few questions about waste management practices for medical waste other than sharps, such as used bandages. How does this facility finally dispose of medical waste other than sharps boxes? PROBE TO ARRIVE AT CORRECT RESPONSE NOTE: IF ANY OF THE RESPONSES 2-9 TAKE PLACE OUTSIDE THE FACILITY, THEN THE CORRECT RESPONSE TO CIRCLE WILL BE IN THE CATEGORY OF "REMOVE OFFSITE" SAME AS FOR SHARPS ITEMS .................. 1 BURN INCINERATOR 2-CHAMBER INDUSTRIAL (800-1000+° C) ..................................................................... 2 1-CHAMBER DRUM/BRICK....................... 3 OPEN BURNING FLAT GROUND - NO PROTECTION ... 4 PIT OR PROTECTED GROUND ............. 5 DUMP WITHOUT BURNING FLAT GROUND - NO PROTECTION ... 6 COVERED PIT OR PIT LATRINE ............. 7 OPEN-PIT - NO PROTECTION............... 8 PROTECTED GROUND OR PIT ............. 9 REMOVE OFFSITE STORED IN COVERED CONTAINER.. 10 STORED IN OTHER PROTECTED ENVIRONMENT ......................................... 11 STORED UNPROTECTED....................... 12 ANNEX H. DATA COLLECTION TOOLS n H–79 Indicator code Number Question Result Skip/D on’t Know OTHER _______________________ 96 (SPECIFY) NEVER HAS SHARP WASTE................... 95 DON’T KNOW.......................................... 98 REFUSE............................................................99 D22 CHECK QD20 AND QD21: INCINERATOR USED (EITHER "2" OR "3" SELECTED) INCINERATOR NOT USED (NEITHER "2" NOR "3" SELECTED) ➔D25 I9 I10 D23 Is the incinerator functional today? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 2, 98, 99➔D 25 I9 I10 D24 Is fuel for the incinerator available today? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 SUPERVISION D25 When was the last time this facility received a supervision visit from the higher level (DHMT or other)? THIS MONTH ............................................... 1 IN THE LAST 3 MONTHS.......................... 2 MORE THAN 3 MONTHS AGO.............. 3 DON’T KNOW.......................................... 98 REFUSE............................................................99 3, 98, 99➔D 27 D26 During the supervision visit, did the supervisor assess the following? YES NO DK RE F D26_01 Pharmacy (e.g. drug availability, expiry, records, etc.) 1 2 98 99 D26_02 Staffing (e.g. staff available and training) 1 2 98 99 D26_03 Data (e.g. completeness, quality, and timely reporting) 1 2 98 99 D27 Does this Health Facility have a Facility Management Committee/Health Management Committee? 1 2 DK REF ➔D29 if 2, 98, 99 98 99 D27_01 How many FMC/HMC meetings were held in the last 3 months? More than 4 meetings………………1 Three meetings……………………..2 Two meetings………………………3 One meeting………………………..4 None………………………………..5 DON’T KNOW.......................................... 98 REFUSE.....................................................99 YES NO DK RE F D28 Do any of the catchment communities near this facility have a Village Development Committee/Ward Development Committee? 1 2 98 99 H–80 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/D on’t Know BASIC EQUIPMENT D29 Please tell me if the following basic equipment and supplies used in the provision of client services are available and functional in this facility today. ASK TO SEE THE ITEMS A) AVAILABLE B) FUNCTIONING OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK RE F E1 D29_01 Adult weighing scale 1 → B 2 → B 3 02 1 2 98 99 E2 E38 D29_02 Child weighing scale- 250 gram gradation 1 → B 2 → B 3 03 1 2 98 99 E38 D29_03 Infant weighing scale – 100 gram gradation 1 → B 2 → B 3 04 1 2 98 99 E18 D29_04 Measuring tape-height board/stadiometre 1 → B 2 → B 3 05 1 2 98 99 E3 D29_05 Thermometer 1 → B 2 → B 3 06 1 2 DK REF E4 D29_06 Stethoscope 1 → B 2 → B 3 07 1 2 98 99 E5 D29_07 Blood pressure apparatus (may be digital or manual sphygmomanometer with stethoscope) 1 → B 2 → B 3 08 1 2 98 99 E6 D29_08 Light source (flashlight acceptable) 1 → B 2 → B 3 09 1 2 98 99 M27 D29_09 Intravenous infusion kits 1 10 2 10 3 10 98 99 E45 D29_10 Oxygen concentrators 1 → B 2 → B 3 11 1 2 98 99 E45 D29_11 Oxygen cylinders 1 → B 2 → B 3 12 1 2 98 99 E45 D29_12 Central oxygen supply 1 → B 2 → B 3 13 1 2 98 99 E45 D29_13 Flowmeter for oxygen therapy (with humidification) 1 → B 2 → B 3 14 1 2 98 99 E45 D29_14 Oxygen delivery apparatus (key connecting tubes and mask/nasal prongs) 1 → B 2 → B 3 D30 1 2 98 99 E45 D30 At any time during the past 3 months has oxygen been unavailable for any reason? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 INFECTION CONTROL T1 D31 Does this facility have any guidelines on standard precautions for infection prevention? IF YES, ASK TO SEE THE DOCUMENT YES, OBSERVED ........................................... 1 YES, REPORTED NOT SEEN..................... 2 NO................................................................... 3 DON’T KNOW.......................................... 98 ANNEX H. DATA COLLECTION TOOLS n H–81 Indicator code Number Question Result Skip/D on’t Know REFUSE............................................................99 D32 Please tell me if the following resources/supplies used for infection control are available in the general outpatient area of this facility today. ASK TO SEE THE ITEMS OBSERVED REPORTED NOT SEEN NOT AVAILABLE DK RE F I15 D32_01 Clean running water (piped, bucket with tap, or pour pitcher) 1 2 3 98 9 9 I15 D32_02 Hand-washing soap/liquid soap 1 2 3 98 9 9 I15 D32_03 Alcohol based hand rub 1 2 3 98 9 9 I16 D32_04 Disposable latex gloves 1 2 3 98 9 9 I12 D32_05 Waste receptacle (pedal bin) with lid and plastic bin liner 1 2 3 98 9 9 I11 D32_06 Sharps container ("safety box") 1 2 3 98 9 9 I13 D32_07 Environmental disinfectant (e.g. chlorine, alcohol) 1 2 3 98 9 9 I14 D32_08 Disposable syringes with disposable needles 1 2 3 98 9 9 I14 D32_09 Auto-disable syringes 1 2 3 98 9 9 USER FEES D33 Does this facility have any routine user-fees or charges for client services? YES................................................................... 1 NO................................................................... 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 2, 98, 99➔ D39 D34 Does this facility charge a fixed fee that covers all services that a client receives, or are there separate fees for different components of the services provided by the facility? PROBE FIXED FEE COVERING ALL SERVICES... 1 NO, CHARGE FEE FOR SEPARATE ITEMS .............................................................. 2 DON’T KNOW.......................................... 98 REFUSE............................................................99 1➔D3 6 D35 Does this facility have a fee for the following items: READ OUT EACH RESPONSE CATEGORY AND CIRCLE APPROPRIATELY YES NO DK RE F D35_01 Consultation 1 2 98 9 9 D35_02 Client health card 1 2 98 9 9 D35_03 Medicines 1 2 98 9 9 D35_04 Vaccines 1 2 98 9 9 D35_05 Contraceptive commodities 1 2 98 9 9 D35_06 Normal child deliveries 1 2 98 9 9 H–82 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/D on’t Know D35_07 Caesarean Section 1 2 98 9 9 D35_08 HIV Diagnostic Test 1 2 98 9 9 D35_09 Malaria Rapid Diagnostic Test 1 2 98 9 9 D35_10 Malaria Microscopy 1 2 98 9 9 D35_11 Other lab tests 1 2 98 9 9 D35_12 ARV for Treatment 1 2 98 9 9 D35_13 ARV for PMTCT 1 2 98 9 9 D35_14 Minor surgical procedures 1 2 98 9 9 D35_15 Syringes and needles 1 2 98 9 9 D35_16 Registration 1 2 98 9 9 D36 Are the official fees posted or displayed so that the client can easily see them? YES……………………………………….1 NO POSTED FEES………………………2 DON’T KNOW.......................................... 98 REFUSE............................................................99 2, 98, 99➔D 38 D37 May I see the posted fees? OBSERVED, ALL FEES POSTED ………1 OBSERVED, SOME BUT NOT ALL FEES……………………………………..2 REFUSE..........................................................99 D38 What is the procedure if a client is unable to pay for any of the fees associated with health care provided in this facility? Fee exempted/discounted, no payment expected………………………………… 1 Fee exempted/discounted, payment expected later …………………………...2 Service not provided, asked to come Back when able to pay …………………..3 Accept payment in kind …………………4 Other (Specify)________________________ 5 DON’T KNOW.......................................... 98 REFUSE........................................................... 99 SOURCE OF REVENUE D39 I would like to ask about the sources of revenue or funding for this facility. Tell me if the facility received any revenue or funding from any of the listed resources during the _________ financial year. (Circle all that apply, prompt each) Ministry of Health ………………………1 Other public ministries …………………2 Medical schemes/insurance ……………..3 Social Security fund ……………………..4 Reimbursement by employer …………...5 Government contribution to private ……6 Faith-based ………………………………7 Community programs …………………..8 ANNEX H. DATA COLLECTION TOOLS n H–83 Indicator code Number Question Result Skip/D on’t Know USAID............................................... 9 JSI......................................................10 CDC.................................................11 MSCP................................................12 HC3..................................................13 Other Donor (Specify) ..............................14 Other (Specify)__________________15 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D40 Does this health facility have an operational budget for this fiscal year? Yes …………………………………… 1 No ……………………………………. 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D41 What does the budget cover? CHECK ALL THAT APPLY Salaries ………………………………… A Supplies …………………………………B Medicines ……………………………….C Equipment ………………………………D Utilities ………………………………… E Other (SPECIFY)______________________ F DON’T KNOW.......................................... 98 REFUSE............................................................99 D42 Does this health facility have sufficient funds to meet its operating budget? Yes ………………………………………1 No ………………………………………2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 CLIENT OPINION AND FEEDBACK D43 Does this facility have any system for determining clients’ opinions about the health facility or its services? Yes ………………………………………1 No ………………………………………2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 2,98,99 ➔D46 D43_01 Please tell me all the methods that this facility uses to elicit client opinion. Yes ………………………………………1 No ………………………………………2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_02 Suggestion box Yes ………………………………………1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_03 Client survey form Yes ………………………………………1 No ………………………………………2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_04 Client interview form Yes ………………………………………1 No ……………………………………… 2 H–84 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/D on’t Know DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_05 Official meeting with community leaders Yes ………………………………………1 No ………………………………………2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_06 Informal discussion with clients or the community Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_07 email Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_08 Facility’s website Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_09 Letters from the community Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D43_10 Other Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 D44 Is there a procedure for reviewing or reporting on clients’ opinion? If yes, ask to see a report or form on which data are compiled or discussion is reported Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 2,98,99 ➔D46 D45 May I see a report on the review of client opinions, or any document on such a review? Observed ………………………………1 Not observed ………………………… 2 REFUSE.....................................................99 Policy/Regulations Supporting Facility Operations D46 Are there any existing government regulations that help this facility operate the way it is operating today? Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 2,98,99 ➔E0 D47 What aspects of facility operations do they support? CHECK ALL THAT ARE MENTIONED Staffing……………………………………1 Supplies ………………………………… 2 Operational costs ……………………… 3 Service quality …………………………………. 4 Others. Specify ………………………… 5 ANNEX H. DATA COLLECTION TOOLS n H–85 Indicator code Number Question Result Skip/D on’t Know DON’T KNOW.......................................... 98 REFUSE.....................................................99 D48 Were these regulations introduced as a result of the Ebola epidemic? Yes ……………………………………… 1 No ……………………………………… 2 DON’T KNOW.......................................... 98 REFUSE.....................................................99 H–86 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Other SECTION E: BASIC SERVICES This section will focus on questions related to available services. FAMILY PLANNING SERVICES S7 E0 Does this facility offer family planning services? YES...................................................................... 1 NO....................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 2,98,99 èE8 E0_01 Does this facility offer family planning services at the community level? YES...................................................................... 1 No.......................................................................2 DON’T KNOW............................................ 98 REFUSE............................................................99 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE FAMILY PLANNING SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT FAMILY PLANNING SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E1 Does this facility provide or prescribe any of the following modern methods of family planning: YES NO DK REF S7_01 E1_01 Combined estrogen progesterone oral contraceptive pills 1 2 98 99 S7_02 E1_02 Progestin-only contraceptive pills 1 2 98 99 S7_03 E1_03 Combined estrogen progesterone injectable contraceptives 1 2 98 99 S7_04 E1_04 Progestin-only injectable contraceptives 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–87 Indicator code Number Question Result Skip/Other S7_05 E1_05 Male condoms 1 2 98 99 S7_06 E1_06 Female condoms 1 2 98 99 S7_07 E1_07 Intrauterine contraceptive device (IUCD) 1 2 98 99 S7_08 E1_08 Implants 1 2 98 99 S7_09 E1_09 Cycle beads for standard days method 1 2 98 99 S7_10 E1_10 Emergency contraceptive pills 1 2 98 99 S7_11 E1_11 Male sterilization 1 2 98 99 S7_12 E1_12 Female sterilization 1 2 98 99 E2 Does this facility provide or prescribe any of the following modern methods of family planning for unmarried adolescents: YES NO DK REF S12_02 S12_03 E2_01 Combined estrogen progesterone oral contraceptive pills 1 2 98 99 S12_02 S12_04 E2_02 Male condoms 1 2 98 99 S12_02 S12_06 E2_03 Emergency contraceptive pills 1 2 98 99 S12_02 S12_07 E2_04 Intrauterine contraceptive device (IUCD) 1 2 98 99 H–88 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Other E3 Please tell me if the following documents are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T2 E3_01 National family planning guidelines 1 2 3 98 99 T62 E3_02 Any family planning check-lists and/or job- aids 1 2 3 98 99 E4 Have you or any provider(s) of family planning services: YES NO DK REF T3 E4_01 Received any family planning training in the last two years? 1 2 98 99 T16 E4_02 Received any training in adolescent sexual and reproductive health in the last two years? 1 2 98 99 E5 Does this facility stock contraceptive commodities at this service site? YES...................................................................... 1 NO....................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 2, 98, 99 èE8 E6 Are any of the following reproductive health medicines and commodities available in this service site today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF ANNEX H. DATA COLLECTION TOOLS n H–89 Indicator code Number Question Result Skip/Other M15 E6_01 Combined estrogen progesterone oral contraceptive pills 1 2 3 4 5 98 99 M96 E6_02 Progestin-only contraceptive pills 1 2 3 4 5 98 99 M16 M97 E6_03 Combined estrogen progesterone injectable contraceptives 1 2 3 4 5 98 99 M16 M98 E6_04 Progestin-only injectable contraceptives 1 2 3 4 5 98 99 M17 E6_05 Male condoms 1 2 3 4 5 98 99 M99 E6_06 Female condoms 1 2 3 4 5 98 99 M108 E6_07 Implant (e.g. levonorgestrel, etonogestrel) 1 2 3 4 5 98 99 M109 E6_08 Emergency contraceptive pills (e.g. levonorgestrel tablet, ulipristal acetate tablet, mifepristone tablet 10- 25 mg) 1 2 3 4 5 98 99 M105 E6_09 Intrauterine contraceptive device (IUCD) 1 2 3 4 5 98 99 E7 For each of the following items, please check in the facility records if the drug has been unavailable any time in the past 3 months: UNAVAILABLE IN THE PAST 3 MONTHS ALWAYS AVAILABLE IN PAST 3 MONTHS NOT INDICATED PRODUCT NOT OFFERED FACILITY RECORD NOT AVAILABLE M99_A E7_01 Female condoms 1 2 3 4 5 H–90 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Other M108_A E7_02 Implant (e.g. levonorgestrel, etonogestrel) 1 2 3 4 5 M109_A E7_03 Emergency contraceptive pills (e.g. levonorgestrel tablet, ulipristal acetate tablet, mifepristone tablet 10- 25 mg 1 2 3 4 5 ANTENATAL CARE SERVICES S8 E8 Does this facility offer antenatal care (ANC) services? YES...................................................................... 1 NO....................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 2,98,99 èE12 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE ANTENATAL CARE SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT ANTENATAL CARE SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E9 Do ANC providers provide any of the following services to pregnant women as part of routine ANC services? YES NO DK REF S8_01 E9_01 Iron supplementation 1 2 98 99 S8_02 E9_02 Folic acid supplementation 1 2 98 99 S8_03 E9_03 Intermittent preventive treatment in pregnancy (IPTp) for malaria 1 2 98 99 S8_04 E9_04 Tetanus toxoid immunization 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–91 Indicator code Number Question Result Skip/Other S8_05 E9_05 Monitoring for hypertensive disorder of pregnancy 1 2 98 99 E10 Please tell me if the following documents are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T4 E10_01 National ANC guidelines 1 2 3 98 99 T63 E10_02 Any ANC check-lists and/or job-aids 1 2 3 98 99 T19 E10_03 IPTp guidelines, check￾lists and/or job-aids (including wall charts) ACCEPTABLE IF PART OF ANC GUIDELINES. 1 2 3 98 99 E11 Have you or any provider(s) of ANC services: YES NO DK REF T5 E11_01 Received any ANC training in the last two years? 1 2 98 99 T21 E11_02 Received any training in IPTp in the last two years? 1 2 98 99 PREVENTION OF MOTHER-TO-CHILD TRANSMISSION OF HIV S20 E12 Does this facility offer services for the prevention of mother￾to-child transmission of HIV (PMTCT)? YES...................................................................... 1 NO....................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 2,98,99èE17 H–92 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Other ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE PMTCT SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT PMTCT SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E13 As part of PMTCT services, please tell me if this facility provides the following services to clients: YES NO DK REF S20_01 E13_01 Provide HIV counselling and testing services to pregnant women for PMTCT 1 2 98 99 S20_02 E13_02 Provide HIV counselling and testing services to infants born to HIV positive pregnant women for PMTCT 1 2 98 99 S20_03 E13_03 Provide ARV prophylaxis to HIV positive pregnant women for PMTCT 1 2 98 99 S20_04 E13_04 Provide ARV prophylaxis to newborns of HIV positive pregnant women for PMTCT 1 2 98 99 S20_05 E13_05 Provide infant and young child feeding counselling for PMTCT 1 2 98 99 S20_06 E13_06 Provide nutritional counselling for HIV positive pregnant women and their infants for PMTCT 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–93 Indicator code Number Question Result Skip/Other S20_07 E13_07 Provide family planning counselling to HIV positive pregnant women for PMTCT 1 2 98 99 E14 Please tell me if the following guidelines are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T37 E14_01 National guidelines for PMTCT 1 2 3 98 99 T38 E14_02 Guidelines for infant and young child feeding counselling 1 2 3 98 99 E15 Have you or any provider(s) of PMTCT services: YES NO DK REF T39 E15_01 Received any training in PMTCT in the last two years? 1 2 98 99 T40 E15_02 Received any training in infant and young child feeding in the last two years? 1 2 98 99 I24 E16 Is the PMTCT service room or area a private room/area with auditory and visual privacy? AUDITORY PRIVACY ONLY..................... 1 VISUAL PRIVACY ONLY............................. 2 BOTH AUDITORY AND VISUAL PRIVACY……3 NO PRIVACY.................................................. 4 DON’T KNOW............................................ 98 REFUSE............................................................99 H–94 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES CHILD PREVENTIVE AND CURATIVE CARE SERVICES S11 E17 Does this facility offer preventative and curative care services for children under 5? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 2,98,99 èE22 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE CHILD PREVENTATIVE AND CURATIVE CARE SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT CHILD PREVENTATIVE AND CURATIVE CARE SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E18_ Please tell me if this facility provides the following services: YES NO DK REF S11_01 E18_01 Diagnose and/or treat child malnutrition 1 2 98 99 S11_02 E18_02 Provide vitamin A supplementation 1 2 98 99 S11_03 E18_03 Provide iron supplementation 1 2 98 99 S11_04 E18_04 Provide ORS to children with diarrhoea 1 2 98 99 S11_04 E18_05 Provide zinc supplementation to children with diarrhoea 1 2 98 99 S11_05 E18_06 Child growth monitoring 1 2 98 99 S11_06 E18_07 Treatment of pneumonia 1 2 98 99 S11_07 E18_08 Administration of amoxicillin for the treatment of pneumonia in children 1 2 98 99 S11_08 E18_09 Treatment of malaria in children 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–95 S11_09 E18_10 Deworming of children with Albendazole 1 2 98 99 E19 Please tell me if the following documents are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T10 E19_01 IMNCI guidelines for the diagnosis and management of childhood illnesses 1 2 3 98 99 T11 E19_02 National guidelines for growth monitoring 1 2 3 98 99 E19_03 Any check-lists and/or job-aids for IMNCI 1 2 3 98 99 E19_04 National protocol for treatment of malnutrition 1 2 3 98 99 E20 Have you or any provider(s): YES NO DK REF T12 E20_01 Of curative care services for sick children received any training in the Integrated Management of Newborn and Childhood Illnesses (IMNCI) in the last two years? 1 2 98 99 T13 E20_02 Of growth monitoring services for children received any training in growth monitoring in the last two years? 1 2 98 99 H–96 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES E21 Please tell me if the following basic equipment items are available and functional in this service area today. ASK TO SEE THE ITEMS A) AVAILABLE B) FUNCTIONNING OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF E16 E21_01 Length/height measuring equipment 1 → B 2 → B 3 02 1 2 98 99 E17 E21_02 Growth charts 1 03 2 03 3 03 98 99 E21_03 Mid Upper Arm Circumference (MUAC) tape 1 04 2 04 3 04 98 99 E21_04 Weighing scale 1 E22 2 E22 3 E22 98 99 IMMUNIZATION S10 E22 Does this facility offer immunization services? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE41 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE IMMUNIZATION SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT IMMUNIZATION SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E23 Is this facility providing immunization services today? YES............................................................... 1 NO............................................................... 2 DON’T KNOW..................................... 98 REFUSE.....................................................99 ANNEX H. DATA COLLECTION TOOLS n H–97 E24 Does this facility provide any of the following immunization services in the facility only, as outreach at fixed post only, or both? *VACCINES SCHEDULE SHOULD BE ADAPTED ACCORDING TO THE COUNTRY’S IMMUNIZATION SCHEDULE BOTH IN THE FACILITY AND AS OUTREACH IN THE FACILITY ONLY OUTREACH ONLY SERVICE NOT OFFERED DK REF S10_07 E24_01 Birth doses (e.g. hepB0, BCG, OPV0, …) 1 2 3 4 98 99 S10_08 E24_02 Infant vaccines (under 1 year) 1 2 3 4 98 99 S10_09 E24_03 Adolescent/adult vaccines (e.g. HPV, tetanus, flu) 1 2 3 4 98 99 S10_10A S10_10B S10_10C S10_10D S10_10E E25 How often does this facility offer routine full child immunization services at the facility? DAILY................................................................ 1 WEEKLY............................................................ 2 MONTHLY....................................................... 3 QUARTERLY................................................... 4 NEVER...............................................................5 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE............................................................99 H–98 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES S10_11A S10_11B S10_11C S10_11D S10_11E E26 How often does this facility offer routine full child immunization services as outreach? DAILY................................................................ 1 WEEKLY............................................................ 2 MONTHLY....................................................... 3 QUARTERLY................................................... 4 NEVER...............................................................5 OTHER_______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE............................................................99 T8 E27 Do you have the checklist/job aid on routine child immunization available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT *NATIONAL GUIDELINES SHOULD BE USED FOR EACH COUNTRY YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE............................................................99 E28 Have you or any provider(s) of immunization service delivery received any training in any of the following child immunization services in the last two years? IF YES: Pease specify if it was through formal training or supportive supervision YES, FORMAL TRAINING YES, SUPPORTIVE SUPERVISION NO TRAINING DK REF T9 E28_01 Immunization service delivery (Immunization in practice (IIP) or any similar) 1 2 3 98 99 ANNEX H. DATA COLLECTION TOOLS n H–99 T9 E28_02 Vaccine management/handling and cold chain 1 2 3 98 99 T9 E28_03 Data reporting and monitoring of service delivery (e.g. Data Quality Self-Assessment (DQS)) 1 2 3 98 99 T9 E28_04 Disease surveillance and reporting 1 2 3 98 99 T9 E28_05 Injection safety and waste management 1 2 3 98 99 T9 E28_06 RED (Reaching Every District) 1 2 3 98 99 T9 E28_07 Training on new vaccine* prior to introduction * NEW VACCINE LIST SHOULD BE ADAPTED ACCORDING TO COUNTRY’S IMMUNIZATION SCHEDULE 1 2 3 98 99 E29 I would like to know if the following items for immunization are available in this service area today. For each item, please tell me if it is available today. ASK TO SEE THE ITEMS OBSERVED REPORTED NOT SEEN NOT AVAILABLE DK REF I14 I22 E29_01 Auto-disable syringes 1 2 3 98 99 I21 E29_02 Sharps container/safety box 1 2 3 98 99 E14 E29_03 Vaccine carrier(s)/cold box 1 2 3 98 99 H–100 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES E14 E29_04 Set of ice packs for vaccine carriers (Note: 4-5 ice packs make one set) 1 2 3 98 99 E41 E29_05 Immunization cards (or child health booklet) 1 2 3 98 99 E42 E29_06 National immunization tally sheets or integrated tally sheet 1 2 3 98 99 E29_07 National immunization registers or equivalent 1 2 3 98 99 E15 E47 E30 Does this facility have a refrigerator available and functioning for the storage of vaccines? NOTE: FOR A REFRIGERATOR TO BE FUNCTIONAL IT MUST HAVE SUFFICIENT CAPACITY TO ACCOMMODATE ALL NEEDED VACCINES. AVAILABLE AND FUNCTIONAL............. 1 AVAILABLE NOT FUNCTIONAL............. 2 AVAILABLE DON’T KNOW IF FUNCTIONING............................................. 3 NOT AVAILABLE........................................... 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 4,98,99 èE38 E40 E40_A E40_B E40_C E40_D E40_E E40_F E31 What type of energy source is used for the vaccine refrigerator? ELECTRICITY (GRID OR GENERATOR) 1 SOLAR (WITH OR WITHOUT BATTERIES) 2 GAS .................................................................... 3 KEROSENE....................................................... 4 MIXED (ELECTRIC WITH GAS KEROSENE) 5 OTHER............................................................ 96 DON’T KNOW............................................ 98 REFUSE.....................................................99 E40 E32 Does this energy source supply power to the refrigerator for 24 hours a day and for 7 days in the week? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE............................................................99 ANNEX H. DATA COLLECTION TOOLS n H–101 E33 A) AVAILABLE Which of the following devices for monitoring refrigerator temperature are AVAILABLE in the refrigerator today: ASK TO SEE THE ITEMS B) FUNCTIONING Which of the following devices for monitoring refrigerator temperature are FUNCTIONING in the refrigerator today: OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF E39 E47 E33_01 Thermometer 1 → B 2 → B 3 02 1 2 98 99 E39 E47 E33_02 Continuous temperature recorder/logger (e.g. Fridge tag, Multilog, etc.) 1 → B 2 → B 3 E34 1 2 98 99 E49 E47 E34 Is the temperature of the refrigerator monitored twice daily? IF YES: PLEASE ASK TO SEE THE LOG USED TO RECORD THE TEMPERATURE YES, LOG OBSERVED................................... 1 YES, LOG REPORTED NOT SEEN........... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,3,98,99 èE37 E49 E47 E35 Has the temperature log been completed for the last 30 days? PLEASE REVIEW LOG AND CHECK FOR COMPLETENESS (TEMPERATURE RECORDED 2 TIMES / DAY DURING THE LAST 30 DAYS) YES...................................................................... 1 YES, PARTIALLY............................................. 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 3,98,99 èE37 H–102 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES E49 E47 E36 Has the temperature been out of the range 2 to 8 oC inclusive in the last 30 days? PLEASE CHECK THE TEMPERATURE RECORD AND VERIFY THE TEMPERATURE FOR THE LAST 30 WORKING DAYS IN ORDER TO ANSWER THE QUESTION OBSERVED IN RANGE ................................ 1 REPORTED IN RANGE BUT NOT SEEN2 OUT OF RANGE ........................................... 3 RECORD NOT AVAILABLE....................... 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 E37 Are there any other items stored in the refrigerator? (e.g. food, drinks, reagents etc.) YES, OBSERVED ............................................. 1 YES, NOT SEEN.............................................. 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 E38 CHECK QE23 AND QE30: FACILITY IS OFFERING IMMUNIZATION SERVICES TODAY (QE23 =“1” ) OR HAS A FUNCTIONNING REFRIGERATOR FOR THE STORAGE OF VACCINES (QE30 = “1”) FACILITY DOES NOT OFFER IMMUNIZATION SERVICES TODAY (QE23 = “2”,”98”,”99”) AND DOES NOT HAVE A FUNCTIONAL REFRIGERATOR FOR THE STORAGE OF VACCINES (QE30 = “2”, “3”,“4”,”98”,”99”) èE40 E39 OBSERVED AVAILABLE NOT OBSERVED ANNEX H. DATA COLLECTION TOOLS n H–103 Are any of the following vaccines available in this service site today? * THE LIST OF VACCINES SHOULD BE ADAPTED ACCORDING TO COUNTRY’S IMMUNIZATION SCHEDULE SELECT ONE OF EACH VACCINE AT RANDOM AND CHECK IF THE VACCINE IS VALID: 1. VIAL MONITOR (VVM) ON THE VACCINE VIAL HAS NOT TURNED AND 2. THE EXPIRY DATE HAS NOT PASSED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M28 E39_01 Measles vaccine and diluent 1 2 3 4 5 98 99 M29 E39_02 DPT-Hib+HepB (pentavalent) 1 2 3 4 5 98 99 M30 E39_03 Oral polio vaccine 1 2 3 4 5 98 99 M31 E39_04 BCG vaccine and diluent 1 2 3 4 5 98 99 M92 E39_05 Rotavirus vaccine 1 2 3 4 5 98 99 M93 E39_06 Pneumococcal vaccine 1 2 3 4 5 98 99 M142 M143 H–104 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES E40 In the past three months were you unable to give any of the vaccines listed below because of unavailable stock? FOR EACH OF THE FOLLOWING ITEMS, PLEASE CHECK IN THE FACILITY RECORDS IF THE DRUG HAS BEEN UNAVAILABLE ANY TIME IN THE PAST 3 MONTHS * THE LIST OF VACCINES SHOULD BE ADAPTED ACCORDING TO COUNTRY’S IMMUNIZATION SCHEDULE YES, UNAVAILABLE NO, ALWAYS AVAILABLE NOT INDICATED PRODUCT NOT OFFERED FACILITY RECORD NOT AVAILABLE M28_A E40_01 Measles vaccine and diluent 1 2 3 4 5 M29_A E40_02 DPT-Hib-HepB (pentavalent) vaccine 1 2 3 4 5 M30_A E40_03 Oral polio vaccine 1 2 3 4 5 M31_A E40_04 BCG vaccine and diluent 1 2 3 4 5 M92_A E40_05 Rotavirus vaccine 1 2 3 4 5 M93_A E40_06 Pneumococcal vaccine 1 2 3 4 5 M142_A M143_A ADOLESCENT HEALTH SERVICES S12 E41 Does this facility offer adolescent health services? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE44 ANNEX H. DATA COLLECTION TOOLS n H–105 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE ADOLESCENT YOUTH FRIENDLY HEALTH SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT ADOLESCENT HEALTH SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. T14 E42 Do you have the national guidelines for service provision to adolescents available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T15 E43 Have you or any providers of adolescent health services received any training on the provision of adolescent health services in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 SEXUALLY TRANSMITTED INFECTIONS S21 E44 Does this facility offer diagnosis or treatment of STIs other than HIV? YES...................................................................... 1 NO......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE49 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE STI SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT STI SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. S21_01 E45 Do providers in this facility diagnose STIs? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 S21_02 E46 Do providers in this facility prescribe treatment for STIs? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 H–106 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES T41 E47 Do you have the national guidelines for the diagnosis and treatment of STIs available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT Observed........................................................... 1 Reported Not Seen........................................ 2 Not available..................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T42 E48 Have you or any provider(s) of STI services received any training in STI diagnosis and treatment in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 HIV TESTING AND COUNSELLING S17 E49 Does this facility offer HIV counselling and testing services? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE57 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE HIV COUNSELLING AND TESTING SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT HIV COUNSELLING AND TESTING SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. T30 E50 Do you have the national HIV counselling and testing guidelines available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 E51 Have you or any provider(s) of HIV/AIDS counselling and testing services: YES NO DK REF ANNEX H. DATA COLLECTION TOOLS n H–107 T31 E51_01 Received any training in voluntary counselling and testing (VCT) in the last two years? 1 2 98 99 T17 E51_02 Received any training in HIV/AIDS prevention, care, and management for adolescents in the last two years? 1 2 98 99 S12_01 E52 Does this facility provide HIV counselling and testing services to adolescents? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 I23 E53 Is the HIV testing and counselling service room or area a private room/area with auditory and visual privacy? AUDITORY PRIVACY ONLY..................... 1 VISUAL PRIVACY ONLY............................. 2 BOTH AUDITORY AND VISUAL PRIVACY......3 NO PRIVACY.................................................. 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 D6 E54 Does this facility have HIV rapid test kits (with valid expiration date) in stock in this service site today? CHECK TO SEE IF VALID (NOT EXPIRED) YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 M17 M91 E55 Does this facility have condoms available in this service site today to give to clients receiving services? IF YES, ASK TO SEE CONDOMS YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 H–108 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES E56 Please tell me if the following resources/supplies used for infection control are available in this service area today. ASK TO SEE THE ITEMS OBSERVED REPORTED NOT SEEN NOT AVAILABLE DK REF I15 E56_01 Clean running water (piped, bucket with tap, or pour pitcher) 1 2 3 98 99 I15 E56_02 Hand-washing soap/liquid soap 1 2 3 98 99 I15 E56_03 Alcohol based hand rub 1 2 3 98 99 I16 E56_04 Disposable latex gloves 1 2 3 98 99 I12 E56_05 Waste receptacle (pedal bin) with lid and plastic bin liner 1 2 3 98 99 I11 E56_06 Sharps container ("safety box") 1 2 3 98 99 I13 E56_07 Environmental disinfectant (e.g., chlorine, alcohol) 1 2 3 98 99 I14 E56_08 Disposable syringes with disposable needles 1 2 3 98 99 I14 E56_09 Auto-disable syringes 1 2 3 98 99 HIV TREATMENT S19 E57 Does this facility offer HIV & AIDS antiretroviral prescription or antiretroviral treatment follow-up services? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE62 ANNEX H. DATA COLLECTION TOOLS n H–109 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE HIV TREATMENT SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT HIV TREATMENT SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. E58 Do providers in this facility: YES NO DK REF S19_01 E58_01 Prescribe ART 1 2 98 99 S12_09 E58_02 Prescribe ART to adolescents 1 2 98 99 S19_02 E59 Does this facility provide treatment follow-up services for persons on ART, including providing community-based services? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T35 E60 Do you have the national ART guidelines available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T36 E61 Have you or any provider(s) of ART received any training in ART prescription and management in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 MALARIA S15 E62 Does this facility offer diagnosis or treatment of malaria? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE75 H–110 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE MALARIA SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT MALARIA SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. S15_01 E63 Do providers in this facility diagnose malaria? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE68 E64 Which of the following methods are used at this facility for diagnosing malaria: YES NO DK REF S15_05 E64_01 Clinical symptoms 1 2 98 99 S15_02 S15_06 E64_02 Rapid diagnostic testing (RDT) 1 2 98 99 S15_02 S15_07 E64_03 Microscopy 1 2 98 99 CHECK QE64_02: IF FACILITY CONDUCTS MALARIA RDTS: IF FACILITY DOES NOT CONDUCT MALARIA RDTS OR DK OR REF: E68 D3 D34 D36 E65 Does this facility have malaria rapid diagnostic test kits (with valid expiration date) in stock in this service site today? CHECK TO SEE IF VALID (NOT EXPIRED) YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 D36_A E66 Have malaria RDT kits been unavailable any time in the past 4 weeks? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE68 ANNEX H. DATA COLLECTION TOOLS n H–111 D36_B E67 How many days was it unavailable? LESS THAN 7 DAYS...................................... 1 7 TO 14 DAYS................................................ 2 MORE THAN 14 DAYS................................ 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 S15_03 E68 Do providers in this facility prescribe treatment for malaria? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 E68A Do providers in this facility prescribe long lasting insecticide treated nets (LLIN) for malaria prevention? YES...................................................................... 1 NO................................................................................. 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T18 E69 Do you have the national guidelines for the diagnosis and treatment of malaria available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T20 D34 E70 Have you or any provider(s) of malaria services received any training in malaria diagnosis with RDTs in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T20 E71 Have you or any provider(s) of malaria services received any training in malaria treatment in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 H–112 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES S15_04 E72 Does this facility provide Intermittent preventive treatment for malaria? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 >E76 E73 Are Intermittent Preventive Treatment (IPT) drugs for malaria unavailable today? (Day of visit) YES...................................................................... 1 NO......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 E74 If YES, for how long they have been unavailable LESS THAN 1 MONTH........................................... 1 ONE MONTH............................................................ 2 TWO MONTHS........................................................ 3 THREE MONTHS..................................................... .4 MORE THAN 3 MONTHS..................................... 5 DON’T KNOW.......................................................98 REFUSE.....................................................99 NON-COMMUNICABLE DISEASES S22 S23 S24 S29 E75 Does this facility offer diagnosis or management of non- communicable diseases, such as diabetes, cardiovascular disease, chronic respiratory disease, or cervical cancer? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF0_1 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE NON-COMMUNICABLE DISEASE SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT NCD SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. S22 E76 Do providers in this facility diagnose and/or manage diabetes in patients? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE78 ANNEX H. DATA COLLECTION TOOLS n H–113 T43 T44 E77 Have you or any provider(s) of diabetes services received any training in the diagnosis and management of diabetes in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 S23 E78 Do providers in this facility diagnose and/or manage cardiovascular diseases such as hypertension in patients? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE80 T46 E79 Have you or any provider(s) of services for cardiovascular diseases received any training in the diagnosis and management of cardiovascular diseases such as hypertension in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 S24 E80 Do providers in this facility diagnose and/or manage chronic respiratory diseases in patients? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èE84 T47 E81 Do you have the national guidelines for the diagnosis and management of chronic respiratory disease available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED ............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 H–114 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES T48 E82 Have you or any provider(s) of chronic respiratory disease services received any training in the diagnosis and management of chronic respiratory diseases in the last two years? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 E83_ Please tell me if the following basic equipment items are available and functional in this service area today. ASK TO SEE THE ITEMS A) AVAILABLE Please tell me if the following basic equipment items are AVAILABLE in this service area today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following basic equipment items are FUNCTIONAL in this service area today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF E19 E83_01 Peak flow meters 1 → B 2 → B 3 02 1 2 98 99 E20 E83_02 Spacers for inhalers 1 → B 2 → B 3 E84 1 2 98 99 S29 E84 Do providers in this facility diagnose cervical cancer in patients? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF0_1 T61 E85 Have you or any provider(s) received any training in cervical cancer prevention and control? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 E86 A) AVAILABLE Please tell me if the following basic equipment/items are AVAILABLE in this service area today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following basic equipment/items are FUNCTIONING in this service area today. ANNEX H. DATA COLLECTION TOOLS n H–115 OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D37 E86_01 Acetic acid 1 02 2 02 3 02 98 99 E44 E86_02 Speculum 1 → B 2 → B 3 F0_1 1 2 98 99 Indicator code Number Question Result Skip/Don’t Know SECTION F. SPECIAL SERVICES 600a Now I am going to read a list of a different set of services. Please tell me which ones are offered as this facility. If they are, we will go into each one of those services in more detail. (Read the list and not all those that apply). F0 Does this facility offer: Yes No F0_01 Obstetric and Newborn Care ………….…………… 1 2 If YesèF0 F0_02 Caesarean Section………………………………….. 1 2 If YesèF9 F0_03 Surgery ……………………………………….……… 1 2 If Yes èF14 F0_04 HIV Care and Support ……………………………… 1 2 If Yes èF24 F0_05 Tuberculosis ………………………………………… 1 2 If Yes èF29 F0_06 Ebola Virus Disease Survivors, Care & Support …… 1 2 If Yes èF52 F0_07 Blood Transfusion …………………………………… 1 2 If Yes èF40 H–116 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F0_08 Diagnostics ………………………………………… 1 2 If Yes èF61 F0_09 Fistula Care ………………………………………... 1 2 F0_10 Gender Based Violence……………………………… 1 2 OBSTETRIC AND NEWBORN CARE SERVICES S9 F0_11 Does this facility offer delivery (including normal delivery, basic emergency obstetric care, and/or comprehensive emergency obstetric care) and/or new-born care services? YES....................................................................... 1 NO ...................................................................... 2 èF9 F0_12 In the past 30 days, how many deliveries occurred at this health facility? NUMBER OF DELIVERY DON’T KNOW............................................ 98 REFUSE............................................................99 F0_13 In the past 30 days, how many deliveries were conducted by a health care provided trained in normal delivery & life-saving obstetric care (i.e. doctors, nurses, physicians assistants or midwives) NUMBER OF DELIVERY DON’T KNOW............................................ 98 REFUSE............................................................99 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE OBSTETRIC AND NEWBORN CARE SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT OBSTETRIC AND NEWBORN CARE SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. F1 Please tell me if the following interventions are routinely carried out by providers of delivery services in this facility: YES NO DK REF ANNEX H. DATA COLLECTION TOOLS n H–117 Indicator code Number Question Result Skip/Don’t Know S9_13 F1_01 Administration of oxytocin injection immediately after birth to all women for the prevention of post-partum haemorrhage 1 2 98 99 S9_14 F1_02 Monitoring and management of labour using partograph 1 2 98 99 S9_15 F1_03 Immediate and exclusive breastfeeding 1 2 98 99 S9_16 F1_04 Hygienic cord care (cut with sterile item and apply disinfectant to tip and stump, and no application of other substances) 1 2 98 99 S9_17 F1_05 Thermal protection (drying baby immediately after birth and wrapping) 1 2 98 99 F2 Please tell me if any of the following interventions for the management of complications during and after pregnancy and childbirth have been carried out in the last 12 months by providers of delivery services as part of their work in this facility. YES NO DK REF S9_01 S9_18 S26_03 F2_01 Parenteral administration of antibiotics (IV or IM) for mothers 1 2 98 99 S9_02 S9_18 S26_03 F2_02 Parenteral administration of oxytocic for treatment of post-partum haemorrhage (IV or IM) 1 2 98 99 S9_03 S9_18 S26_03 F2_03 Parenteral administration of magnesium sulphate for management of preeclampsia and eclampsia (IV or IM) 1 2 98 99 H–118 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know S9_04 S9_18 S26_03 F2_04 Assisted vaginal delivery 1 2 98 99 S9_05 S9_18 S26_03 F2_05 Manual removal of placenta 1 2 98 99 S9_06 S9_18 S26_03 F2_06 Removal of retained products of conception 1 2 98 99 S9_07 S9_19 S26_03 F2_07 Neonatal resuscitation with bag and mask 1 2 98 99 S26_01 S26_03 F2_08 Caesarean section 1 2 98 99 S26_02 S26_03 F2_09 Blood transfusion 1 2 98 99 S9_09 S9_19 F2_10 Antibiotics for preterm or prolonged PROM (premature rupture of membranes) to prevent infection 1 2 98 99 S9_10 S9_19 F2_11 Corticosteroids in preterm labour 1 2 98 99 S9_11 S9_19 F2_12 KMC (Kangaroo mother care) for premature/very small babies 1 2 98 99 S9_12 S9_19 F2_13 Injectable antibiotics for neonatal sepsis 1 2 98 99 F3 Are the following documents available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T6 F3_01 Any national guidelines for essential childbirth care 1 2 3 98 99 ANNEX H. DATA COLLECTION TOOLS n H–119 Indicator code Number Question Result Skip/Don’t Know T64 F3_02 Any check-lists and/or job-aids for Essential childbirth care 1 2 3 98 99 T66 F3_03 Any national guidelines for essential newborn care 1 2 3 98 99 F4 Have you or any provider(s) of delivery services: YES NO DK REF T65 F4_01 Received training in newborn resuscitation using the newborn bag and mask in the last two years 1 2 98 99 T7 F4_02 Apart from newborn resuscitation, received training in essential childbirth care in the last two years 1 2 98 99 F5 A) AVAILABLE I would like to know if the following basic equipment items are available in this service area today. For each equipment or item, please tell me if it is AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING I would like to know if the following basic equipment items are FUNCTIONING in this service area today. For each equipment or item, please tell me if it is FUNCTIONING. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF E7 F5_01 Examination light (flashlight ok) 1 → B 2 → B 3 02 1 2 98 99 E8 F5_02 Delivery pack 1 → B 2 → B 3 03 1 2 98 99 E8 F5_03 Cord clamp 1 → B 2 → B 3 04 1 2 98 99 E8 F5_04 Episiotomy scissors 1 → B 2 → B 3 05 1 2 98 99 H–120 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know E8 F5_05 Scissors or blade to cut cord 1 → B 2 → B 3 06 1 2 98 99 E8 F5_06 Suture material with needle 1 07 2 07 3 07 98 99 E8 F5_07 Needle holder 1 → B 2 → B 3 08 1 2 98 99 E10 F5_08 Manual vacuum extractor 1 → B 2 → B 3 09 1 2 98 99 E11 F5_09 Vacuum aspirator or D&C kit 1 → B 2 → B 3 10 1 2 98 99 E30 F5_10 Incubator 1 → B 2 → B 3 11 1 2 98 99 I20 F5_11 Disposable latex gloves 1 12 2 12 3 12 98 99 E13 F5_12 Blank partograph 1 13 2 13 3 13 98 99 E37 F5_13 Delivery bed 1 → B 2 → B 3 14 1 2 98 99 E50 F5_14 Resuscitation table (with heat source) (for newborn resuscitation) 1 → B 2 → B 3 15 1 2 98 99 E12 E43 F5_15 Newborn bag and mask size 1 for term babies (for newborn resuscitation) 1 → B 2 → B 3 16 1 2 98 99 E12 E43 F5_16 Newborn bag and mask size 0 for pre-term babies (for newborn resuscitation) 1 → B 2 → B 3 17 1 2 98 99 E9 E43 F5_17 Electric suction pump (for suction apparatus) 1 → B 2 → B 3 18 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–121 Indicator code Number Question Result Skip/Don’t Know E9 E43 F5_18 Suction catheter (for suction apparatus) for suctioning newborn 1 → B 2 → B 3 19 1 2 98 99 E9 E43 F5_19 Suction bulb, single use 1 → B 2 → B 3 20 1 2 98 99 E9 E43 F5_20 Suction bulb, sterilizable multi-use 1 → B 2 → B 3 21 1 2 98 99 E44 F5_21 Speculum 1 → B 2 → B 3 22 1 2 98 99 E51 F5_22 Infant weighting scale 1 → B 2 → B 3 23 1 2 98 99 E52 F5_23 Blood pressure apparatus (may be digital or manual sphygmomanometer with stethoscope) 1 → B 2 → B 3 24 1 2 98 99 I25 F5_24 Hand-washing soap/liquid soap 1 25 2 25 3 25 98 99 I25 F5_25 Alcohol based hand rub 1 F6 2 F6 3 F6 98 99 F6 Does this facility stock any medicines for obstetric care in this service site? YES....................................................................... 1 NO ......................................................................2 èF09 F7 Are any of the following medicines and commodities available in this service site today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M21 F7_01 Antibiotic eye ointment for newborn 1 2 3 4 5 98 99 H–122 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know M72 M23 M110 M141 F7_02 Gentamicin injection 1 2 3 4 5 98 99 M71 M23 F7_03 Ampicillin powder for injection 1 2 3 4 5 98 99 M106 F7_04 Hydralazine injection 1 2 3 4 5 98 99 M73 F7_05 Metronidazole injection 1 2 3 4 5 98 99 M75 F7_06 Azithromycin cap/tab or oral liquid 1 2 3 4 5 98 99 M76 F7_07 Cefixime cap/tab 1 2 3 4 5 98 99 M77 F7_08 Benzathine benzylpenicillin powder for injection 1 2 3 4 5 98 99 M79 F7_09 Nifedipine cap/tab (10mg) 1 2 3 4 5 98 99 M107 F7_10 Methyldopa tablet 1 2 3 4 5 98 99 M70 F7_11 Calcium gluconate injection 1 2 3 4 5 98 99 M24 F7_12 Magnesium sulphate injectable 1 2 3 4 5 98 99 M26 F7_13 Skin disinfectant 1 2 3 4 5 98 99 M27 F7_14 Intravenous solution with infusion set 1 2 3 4 5 98 99 M69 F7_15 Sodium chloride injectable solution 1 2 3 4 5 98 99 M78 F7_16 Betamethasone injection 1 2 3 4 5 98 99 M78 M129 F7_17 Dexamethasone injection 1 2 3 4 5 98 99 M22 F7_18 Oxytocin injection 1 2 3 4 5 98 99 ANNEX H. DATA COLLECTION TOOLS n H–123 Indicator code Number Question Result Skip/Don’t Know IF OXYTOCIN IS OBSERVED AVAILABLE (QF7_18 is “1” OR “2”) IF OXYTOCIN IS NOT OBSERVED AVAILABLE (QF7_18 is “3”,”4”,” 5”,”98”OR”99”) QF9 F8 Is the oxytocin stored in cold storage? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 CESAREAN SECTION F9 Does this facility offer caesarean sections? YES....................................................................... 1 NO……………………………………….2 èF14 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE CAESAREAN SECTION SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT CAESAREAN SECTION SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. T51 F10 Do you have the national guidelines for Comprehensive Emergency Obstetric Care (CEmOC) available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED.............................................. 1 YES, REPORTED NOT SEEN ...................... 2 NO ......................................................................3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T52 F11 Have you or any provider(s) of delivery service received any training in Comprehensive Emergency Obstetric Care (CEmOC) in the last two years? YES....................................................................... 1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T53 F12 Does this facility have a health professional who can perform caesarean section present in the facility or on call 24 hours a day (including weekends and on public holidays)? YES....................................................................... 1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 H–124 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know T54 F13 Does this facility have an anaesthetist (or doctor with anaesthetics training) present in the facility or on call 24 hours a day (including weekends and on public holidays)? YES....................................................................... 1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 SURGICAL SERVICES S25 S28 F14 Does this facility offer any surgical services (including minor surgery such as suturing, circumcision, wound debridement, etc.), or caesarean section? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF24 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE SURGICAL SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT SURGICAL SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. F15 Please tell me if this facility provides the following services: YES NO DK REF S25_01 F15_01 Incision and drainage of abscesses 1 2 98 99 S25_02 F15_02 Wound debridement 1 2 98 99 S25_03 F15_03 Acute burn management 1 2 98 99 S25_04 F15_04 Suturing 1 2 98 99 S25_05 F15_05 Closed repair of fracture 1 2 98 99 S25_06 F15_06 Cricothyroidotomy 1 2 98 99 S25_07 F15_07 Male circumcision 1 2 98 99 S25_08 F15_08 Hydrocele reduction 1 2 98 99 S25_09 F15_09 Chest tube insertion 1 2 98 99 S25_10 F15_10 Closed repair of dislocated joint 1 2 98 99 S25_11 F15_11 Biopsy of lymph node or mass or other 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–125 Indicator code Number Question Result Skip/Don’t Know S25_12 F15_12 Removal of foreign body (throat, eye, ear or nose) 1 2 98 99 CHECK QA9: IF HOSPITAL: IF NOT HOSPITAL: QF16 S28_01 F15_13 Tracheostomy 1 2 98 99 S28_02 515_14 Tubal ligation 1 2 98 99 S28_03 F15_15 Vasectomy 1 2 98 99 S28_04 F15_16 Dilatation & Curettage 1 2 98 99 S28_05 F15_17 Obstetric fistula repair 1 2 98 99 S28_06 F15_18 Episiotomy, cervical and vaginal laceration 1 2 98 99 S28_07 F15_19 Appendectomy 1 2 98 99 S28_08 F15_20 Hernia repair (strangulated) 1 2 98 99 S28_22 F15_21 Hernia repair (elective) 1 2 98 99 S28_09 F15_22 Cystostomy 1 2 98 99 S28_10 F15_23 Urethral stricture dilatation 1 2 98 99 S28_11 F15_24 Laparotomy (uterine rupture, ectopic pregnancy, acute abdomen, intestinal obstruction, perforation, injuries) 1 2 98 99 S28_12 F15_25 Congenital hernia repair 1 2 98 99 S28_13 F15_26 Neonatal surgery (abdominal wall defect, colostomy imperforate anus, intussusceptions) 1 2 98 99 S28_14 F15_27 Cleft palate repair 1 2 98 99 S28_23 F15_28 Contracture release 1 2 98 99 H–126 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know S28_23 F15_29 Skin grafting 1 2 98 99 S28_17 F15_30 Open reduction and fixation for fracture 1 2 98 99 S28_18 F15_31 Amputation 1 2 98 99 S28_19 F15_32 Cataract surgery 1 2 98 99 S28_20 F15_33 Club foot repair (casting or open club foot release) 1 2 98 99 S28_21 F15_34 Drainage of osteomyelitis-septic arthritis 1 2 98 99 F16 Please tell me if the A) AVAILABLE Please tell me if the following surgical equipment and supplies are AVAILABLE in this facility today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following surgical equipment and supplies are FUNCTIONAL in this facility today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF E29 E27 F16_01 Resuscitator bag and mask- adult 1 → B 2 → B 302 1 2 98 99 E29 E27 F16_02 Resuscitator bag and mask￾paediatric 1 → B 2 → B 3 03 1 2 98 99 E21 F16_03 Needle holder 1 → B 2 → B 3 04 1 2 98 99 E22 F16_04 Scalpel handle with blades 1 → B 2 → B 3 05 1 2 98 99 E23 F16_05 Retractor 1 → B 2 → B 3 06 1 2 98 99 E24 F16_06 Surgical scissors 1 → B 2 → B 3 07 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–127 Indicator code Number Question Result Skip/Don’t Know E25 F16_07 Nasogastric tubes 1 → B 2 → B 3 08 1 2 98 99 E26 F16_08 Tourniquet 1 → B 2 → B 3 09 1 2 98 99 E28 F16_09 Suction pump (manual or electric) with catheter 1 → B 2 → B 3 10 1 2 98 99 F16_10 CHECK QA7 AND QF02_08: IF HOSPITAL OR HEALTH FACILITY OFFERS CESAREAN SECTION: IF NOT HOSPITAL AND CESAREAN SECTION NOT OFFERED: QF18 E29 F16_11 Oropharyngeal airway- adult 1 → B 2 → B 3 12 1 2 98 99 E29 F16_12 Oropharyngeal airway- paediatric 1 → B 2 → B 3 13 1 2 98 99 E29 F16_13 Magills forceps- adult 1 → B 2 → B 3 14 1 2 98 99 E29 F16_14 Magills forceps- paediatric 1 → B 2 → B 3 15 1 2 98 99 E29 F16_15 Endotracheal tube neonatal – uncuffed size below 3 1 → B 2 → B 316 1 2 98 99 E29 F16_16 Endotracheal tube paediatric￾uncuffed sizes 3.0 to 5.0 1 → B 2 → B 3 17 1 2 98 99 E29 F16_17 Endotracheal tube adult- cuffed sizes 5.5 to 9.0 1 → B 2 → B 3 18 1 2 98 99 E29 F16_18 Laryngoscope handle and blade￾adult 1 → B 2 → B 3 19 1 2 98 99 H–128 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know E29 F16_19 Laryngoscope handle and blade- paediatric 1 → B 2 → B 3 20 1 2 98 99 E29 F16_20 Laryngoscope handle and blade￾neonatal 1 → B 2 → B 321 1 2 98 99 E29 F16_21 Anaesthesia machine 1 → B 2 → B 3 22 1 2 98 99 E29 F16_22 Tubings and connectors (to connect endotracheal tube) 1 → B 2 → B 3 23 1 2 98 99 E29 F16_23 Stylet 1 → B 2 → B 3 24 1 2 98 99 E32 F16_24 Spinal needle 1 → B 2 → B 3 25 1 2 98 99 E29 F16_25 Newborn bag and mask size 1 for term babies (for newborn resuscitation) 1 → B 2 → B 3 26 1 2 98 99 E48 F16_26 Oxygen concentrators 1 → B 2 → B 3 27 1 2 98 99 E48 F16_27 Oxygen cylinders 1 → B 2 → B 3 28 1 2 98 99 E48 F16_28 Central oxygen supply 1 → B 2 → B 329 1 2 98 99 E48 F16_29 Flowmeter for oxygen therapy (with humidification) 1 → B 2 → B 330 1 2 98 99 E48 F16_30 Oxygen delivery apparatus (key connecting tubes and mask/nasal prongs) 1 → B 2 → B 3 F17 1 2 98 99 E48 F17 At any time during the past 3 months has oxygen been unavailable for any reason? YES....................................................................... 1 NO ...................................................................... 2 ANNEX H. DATA COLLECTION TOOLS n H–129 Indicator code Number Question Result Skip/Don’t Know F18 Please tell me if any of the following materials or medicines are available in this service site today. I would like to see those that are available. CHECK TO SEE IF AT LEAST ONE OF EACH MATERIAL/MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M63 F18_01 Suture material (any type) 1 2 3 4 5 98 99 M26 F18_02 Skin disinfectant 1 2 3 4 5 98 99 M64 F18_03 Ketamine (injection) 1 2 3 4 5 98 99 M65 F18_04 Lidocaine 1% or 2% (anaesthesia) 1 2 3 4 5 98 99 M148 F18_05 Splints for extremities 1 2 3 4 5 98 99 M149 F18_06 Material for cast 1 2 3 4 5 98 99 CHECK QA9 AND QF0: IF HOSPITAL OR HEALTH FACILITY OFFERS CESAREAN SECTION: IF NOT HOSPITAL AND CESAREAN SECTION NOT OFFERED: QF19 M84 F18_07 Thiopental (powder) 1 2 3 4 5 98 99 M85 F18_08 Suxamethonium bromide (powder) 1 2 3 4 5 98 99 M86 F18_09 Atropine (injection) 1 2 3 4 5 98 99 M25 F18_10 Diazepam (injection) 1 2 3 4 5 98 99 M87 F18_11 Halothane (inhalation) 1 2 3 4 5 98 99 M88 F18_12 Bupivacaine (injection) 1 2 3 4 5 98 99 M89 F18_13 Lidocaine 5% (heavy spinal solution) 1 2 3 4 5 98 99 M62 F18_14 Epinephrine (injection) 1 2 3 4 5 98 99 M90 F18_15 Ephedrine (injection) 1 2 3 4 5 98 99 H–130 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know T49 F19 Do you have materials on Integrated Management of Emergency and Essential Surgical care (IMEESC) (e.g. best practices, protocols, etc.) available in this facility today? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED.............................................. 1 YES, REPORTED NOT SEEN ...................... 2 NO ...................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 T50 F20 Have you or any provider(s) of basic surgical services received any training in IMEESC in the last two years? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T57 F21 Does this facility have a staff member trained in surgery, including caesarean section, (clinical officer, general physician, or surgeon) present in the facility or on call 24 hours a day (including weekends and on public holidays)? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 T58 F22 Does this facility have a staff member trained in anaesthesia (nurse, clinical officer, general physician, surgeon, or anaesthesiologist) present in the facility or on call 24 hours a day (including weekends and on public holidays)? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F23 I am interested in knowing if the following resources/supplies used for infection control are available in this service area today. ASK TO SEE THE ITEMS OBSERVED REPORTED NOT SEEN NOT AVAILABLE I15 F23_01 Clean running water (piped, bucket with tap, or pour pitcher) 1 2 3 ANNEX H. DATA COLLECTION TOOLS n H–131 Indicator code Number Question Result Skip/Don’t Know I15 F23_02 Hand-washing soap/liquid soap 1 2 3 I15 F23_03 Alcohol based hand rub 1 2 3 I16 F23_04 Disposable latex gloves 1 2 3 I12 F23_05 Waste receptacle (pedal bin) with lid and plastic bin liner 1 2 3 I11 F23_06 Sharps container ("safety box") 1 2 3 I13 F23_07 Environmental disinfectant (e.g., chlorine, alcohol) 1 2 3 I14 F23_08 Disposable syringes with disposable needles 1 2 3 I14 F23_09 Auto-disable syringes 1 2 3 HIV CARE AND SUPPORT S18 F24 Does this facility offer HIV & AIDS care and support services, including treatment of opportunistic infections and provisions of palliative care? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF29 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE HIV CARE AND SUPPORT SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT HIV CARE AND SUPPORT SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. F25 Please tell me if this facility provides the following services for HIV/AIDS clients: YES NO DK REF S18_01 F25_01 Prescribe treatment for any opportunistic infections or symptoms related to HIV/AIDS? This includes treating topical fungal infections. 1 2 98 99 H–132 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know S18_02 F25_02 Provide or prescribe palliative care for patients, such as symptom or pain management, or nursing care for the terminally ill, or severely debilitated clients? 1 2 98 99 S18_03 F25_03 Provide systemic intravenous treatment of specific fungal infections such as cryptococcal meningitis? 1 2 98 99 S18_04 F25_04 Provide treatment for Kaposi's sarcoma? 1 2 98 99 S18_05 F25_05 Provide nutritional rehabilitation services? e.g., client education and provision of nutritional supplements? 1 2 98 99 S18_06 F25_06 Prescribe or provide fortified protein supplementation (FPS)? 1 2 98 99 S18_07 F25_07 Care for paediatric HIV/AIDS patients? 1 2 98 99 S18_08 F25_08 Prescribe or provide preventive treatment for TB (INH + Pyridoxine)? 1 2 98 99 S18_09 F25_09 Primary preventive treatment for opportunistic infections, such as co- trimoxazole preventive treatment (CPT)? 1 2 98 99 S18_10 F25_10 Provide or prescribe micronutrient supplementation, such as vitamins or iron? 1 2 98 99 S18_11 F25_11 Family planning counselling for HIV/AIDS clients? 1 2 98 99 S18_12 F25_12 Provide condoms for preventing further transmission of HIV? 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–133 Indicator code Number Question Result Skip/Don’t Know D14 F26 Do providers in this facility screen or test HIV clients for TB or have a system for diagnosis of TB among HIV positive clients? IF YES, ASK TO SEE A REGISTER OR RECORD OF HIV-POSITIVE CLIENTS TESTED FOR TB YES, OBSERVED.............................................. 1 YES, REPORTED NOT SEEN ...................... 2 YES, REGISTER NOT MAINTAINED........ 3 NO ...................................................................... 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 F27 Please tell me if the following guidelines are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T32 F27_01 National guidelines for the clinical management of HIV/AIDS 1 2 3 98 99 T33 F27_02 Guidelines for palliative care 1 2 3 98 99 T34 F28 Have you or any provider(s) of HIV care and support services received any training in the clinical management of HIV/AIDS in the last two years? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 TUBERCULOSIS S16 F29 Does this facility offer diagnosis, treatment prescription, or treatment follow-up of tuberculosis? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF40 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE TUBERCULOSIS SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT TUBERCULOSIS SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. H–134 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know S16_01 F30 Do providers in this facility diagnose TB? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èF32 F31 Which of the following methods are used at this facility for diagnosing TB: YES NO DK REF S16_03 F31_01 Clinical symptoms 1 2 98 99 S16_02 S16_04 F31_02 Sputum smear microscopy examination 1 2 98 99 S16_02 S16_05 F31_03 Culture 1 2 98 99 S16_02 S16_06 F31_04 Rapid test (GeneXpert MTB/RIF) 1 2 98 99 S16_02 S16_07 F31_05 Chest X-ray 1 2 98 99 S16_08 F32 Does this facility prescribe drugs for TB patients? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 S16_09 F33 Does this facility provide drugs to TB patients? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 S16_10 F34 Does this facility manage and provide treatment follow-up for TB patients? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 ANNEX H. DATA COLLECTION TOOLS n H–135 Indicator code Number Question Result Skip/Don’t Know D13 F35 Do providers in this facility screen or test TB patients for HIV or have a system for diagnosis of HIV among TB patients? IF YES, ASK TO SEE A REGISTER OR RECORD OF TB CLIENTS TESTED FOR HIV YES, OBSERVED.............................................. 1 YES, REPORTED NOT SEEN ...................... 2 YES, REGISTER NOT MAINTAINED........ 3 NO ...................................................................... 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 F36 Please tell me if the following guidelines are available in the facility today: IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED YES, REPORTED NOT SEEN NO DK REF T22 F36_01 Diagnosis and treatment of TB 1 2 3 98 99 T23 F36_02 Management of HIV and TB co￾infection 1 2 3 98 99 T24 F36_03 MDR-TB 1 2 3 98 99 T25 F36_04 TB infection control 1 2 3 98 99 F37 Have any providers of TB services at this facility received training in the following topics in the last two years? YES NO DK REF T26 F37_01 Diagnosis and treatment of TB 1 2 98 99 T27 F37_02 Management of HIV and TB co￾infection 1 2 98 99 T28 F37_03 MDR-TB 1 2 98 99 T29 F37_04 TB infection control 1 2 98 99 H–136 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F38 Does this facility stock any medicines for TB treatment? YES, IN SERVICE SITE.................................... 1 YES, ELSEWHERE (E.G BULK STORE/PHARMACY) ......2 YES, IN BOTH LOCATIONS ...................... 3 NO, TB MEDS NOT STOCKED................ 4 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,4,98,99 èF4 0 F39 Are any of the following medicines available in this service site today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK M41 F39_01 Ethambutol 100mg 1 2 3 4 5 98 M41 F39_02 Isoniazid 100mg 1 2 3 4 5 98 M41 F39_03 Pyrazinamide 1 2 3 4 5 98 M41 F39_04 Isoniazid 75mg + Rifampicin 150mg (RH150/75) 1 2 3 4 5 98 M41 F39_05 Isoniazid + Rifampicin + Pyrazinamide (RHZ 60/30/150) 1 2 3 4 5 98 M41 F39_06 Isoniazid + Rifampicin + Ethambutol (RHE) (3FDC) 1 2 3 4 5 98 M41 F39_07 Isoniazid + Rifampicin + Pyrazinamide + Ethambutol (4FDC) 1 2 3 4 5 98 F39_08 Streptomycin Injectable 1 2 3 4 5 98 F39_09 Isoniazid 30mg + Rifampicin 60mg (RH60/30) 1 2 3 4 5 98 BLOOD TRANSFUSION ANNEX H. DATA COLLECTION TOOLS n H–137 Indicator code Number Question Result Skip/Don’t Know S27 F40 Does this facility offer blood transfusion services? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99èF52 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE BLOOD IS COLLECTED, PROCESSED, TESTED, STORED, OR HANDLED PRIOR TO TRANSFUSION. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT BLOOD TRANSFUSION SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. M66 F41 Have there been any interruptions in blood availability during the past 3 months? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 M67 F42 Does this facility obtain blood from a national or regional blood centre? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 M67 F43 Does this facility obtain ANY blood from sources other than the national or regional blood centre? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 M67 F44 Does any place in this facility do blood screening for infectious diseases prior to transfusion? YES....................................................................... 1 NO ...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99>F46 F45 Please tell me if the blood that is transfused in the facility is "always", "sometimes", ”rarely”, or "never" screened for any of the following infectious diseases. ALWAYS SOMETIMES RARELY NEVER DK REF H–138 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know M67 F45_01 HIV 1 2 3 4 98 99 M67 F45_02 Syphilis 1 2 3 4 98 99 M67 F45_03 Hepatitis B 1 2 3 4 98 99 M67 F45_04 Hepatitis C 1 2 3 4 98 99 E31 F46 Does this facility have a refrigerator available and functioning in this service area for the storage of blood? AVAILABLE AND FUNCTIONAL............. 1 AVAILABLE NOT FUNCTIONAL............. 2 AVAILABLE DON’T KNOW IF FUNCTIONING.............................................. 3 NOT AVAILABLE ........................................... 4 DON’T KNOW............................................98 REFUSE.....................................................99 4,98,99 èF50 E31 F47 Is the temperature of the refrigerator monitored at least once every 24 hours? IF YES: PLEASE ASK TO SEE THE LOG USED TO RECORD THE TEMPERATURE YES, LOG OBSERVED................................... 1 YES, LOG REPORTED NOT SEEN ........... 2 NO...................................................................... 3 DON’T KNOW............................................98 REFUSE.....................................................99 3,98,99 èF50 E31 F48 Has the temperature log been completed for the last 30 days? PLEASE REVIEW LOG AND CHECK FOR COMPLETENESS (TEMPERATURE RECORDED AT LEAST ONCE EVERY 24 HOURS DURING THE LAST 30 DAYS) YES ...................................................................... 1 YES, PARTIALLY ............................................. 2 YES, REPORTED NOT SEEN.........................3 NO...................................................................... 4 èF50 E31 F49 Has the temperature been out of the range 2 to 8 oC inclusive in the last 30 days? PLEASE CHECK THE TEMPERATURE RECORD AND VERIFY THE TEMPERATURE FOR THE LAST 30 WORKING DAYS IN ORDER TO ANSWER THE QUESTION OBSERVED IN RANGE................................. 1 REPORTED IN RANGE BUT NOT SEEN2 OUT OF RANGE............................................ 3 RECORD NOT AVAILABLE........................ 4 ANNEX H. DATA COLLECTION TOOLS n H–139 Indicator code Number Question Result Skip/Don’t Know T55 F50 Do you have any guidelines on the appropriate use of blood and safe transfusion practices? IF AVAILABLE, ASK TO SEE THE DOCUMENT YES, OBSERVED.............................................. 1 YES, REPORTED NOT SEEN...................... 2 NO...................................................................... 3 DON’T KNOW............................................98 REFUSE.....................................................99 T56 F51 Have any provider(s) of blood transfusion services received any training in the appropriate use of blood and safe transfusion practices in the last two years? YES ...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................98 REFUSE.....................................................99 EBOLA VIRUS DISEASE SURVIVORS CARE AND SUPPORT F52 Does this facility offer care and support services for survivors of Ebola virus disease? YES ...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................98 REFUSE.....................................................99 2,98,99 èF62 ASK TO BE SHOWN THE LOCATION IN THE FACILITY WHERE EBOLA VIRUS DISEASE SURVIVORS CARE AND SUPPORT SERVICES ARE PROVIDED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT EBOLA VIRUS DISEASE SURVIVORS CARE AND SUPPORT SERVICES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. F53 Please tell me if this facility provides the following services for survivors of Ebola virus disease clients: INSTRUCTION TO INTERVIEWERS: IF NO, PROBE ‘DO YOU REFER PATIENTS TO ANOTHER HEALTH FACILITY?’ YES NO REFERRAL DK REF H–140 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F53_01 Musculoskeletal evaluation and clinical management 1 2 3 98 99 F53_02 Ocular evaluation and clinical management 1 2 3 98 99 F53_03 Auditory evaluation and clinical management 1 2 3 98 99 F53_04 Abdominal evaluation and clinical management 1 2 3 98 99 F53_05 Neurological evaluation and clinical management 1 2 3 98 99 F53_06 Mental health evaluation and support 1 2 3 98 99 F53_07 Sexual health evaluation and clinical management 1 2 3 98 99 F53_08 Consultation with social worker to address issues including stigma, economic status and employment, dependents, social support, domestic abuse, substance misuse or dependency etc. 1 2 3 98 99 ANNEX H. DATA COLLECTION TOOLS n H–141 Indicator code Number Question Result Skip/Don’t Know F53_09 Routine complete blood count laboratory test 1 2 3 98 99 F53_10 Routine creatinine laboratory test 1 2 3 98 99 F53_11 Any of the optional tests including Ebola RT-PCR or IgG or IgM antibody, Hepatitis transaminases (ALT and AST) and amylases, thyroid function tests, Erythrocyte sedimentation rate or C reactive protein, pregnancy test, malaria rapid diagnostic test, stool examination for ova, cyst and parasites, urine dipstick for protein, syphilis test and HIV tests 1 2 3 98 99 F53_12 Does this facility provide IPC guidance for survivors of Ebola virus disease to use at home? 1 2 3 98 99 F54 Please tell me if any of the following materials or medicines are available in this service site today. I would like to see those that are available. CHECK TO SEE IF AT LEAST ONE OF EACH MATERIAL/MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABL E BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF H–142 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F54_01 Paracetamol 1 2 3 4 5 98 99 F54_02 Ibuprofen 1 2 3 4 5 98 99 F54_03 Prednisone (oral and IV) 1 2 3 4 5 98 99 F54_04 Methotrexate 1 2 3 4 5 98 99 F54_05 Ranitidine 1 2 3 4 5 98 99 F54_06 Omeprazole 1 2 3 4 5 98 99 F54_07 Amoxicillin 1 2 3 4 5 98 99 F54_08 Ivermectin 1 2 3 4 5 98 99 Country Specific to specify F54_09 Antimalarial drug as per current national policy 1 2 3 4 5 98 99 F54_10 Fluoxetine 1 2 3 4 5 98 99 F54_11 Diazepam 1 2 3 4 5 98 99 F54_12 Haloperidol 1 2 3 4 5 98 99 F54_13 Amitriptyline 1 2 3 4 5 98 99 F54_14 Prednisolone acetate drops 1% 1 2 3 4 5 98 99 F54_15 Atropine drops 1% 1 2 3 4 5 98 99 F54_16 Cyclopentolate 1% 1 2 3 4 5 98 99 F54_17 Timolol drops 0.5% 1 2 3 4 5 98 99 F54_18 Tetracycline ointment 1 2 3 4 5 98 99 F54_19 Artificial tears 1 2 3 4 5 98 99 F55 I am interested in knowing if the following resources/supplies used for infection control are available in this service area today. ASK TO SEE THE ITEMS OBSERVED REPORTED NOT SEEN NOT AVAILABLE DK REF ANNEX H. DATA COLLECTION TOOLS n H–143 Indicator code Number Question Result Skip/Don’t Know F55_01 Clean running water (piped, bucket with tap, or pour pitcher) 1 2 3 98 99 F55_02 Hand-washing soap/liquid soap 1 2 3 98 99 F55_03 Alcohol based hand rub 1 2 3 98 99 F55_04 Disposable latex gloves 1 2 3 98 99 F55_05 Waste receptacle (pedal bin) with lid and plastic bin liner 1 2 3 98 99 F55_06 Sharps container ("safety box") 1 2 3 98 99 F55_07 Environmental disinfectant (e.g., chlorine, alcohol) 1 2 3 98 99 F55_08 Disposable syringes with disposable needles 1 2 3 98 99 F55_09 Auto-disable syringes 1 2 3 98 99 F56 Please tell me if the following equipment items are available and functional in this service area today. ASK TO SEE THE ITEMS A) AVAILABLE B) FUNCTIONING OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF F56_01 Slit lamps 1 → B 2 → B 3 02 1 2 98 99 F56_02 Growth charts 1 B 2 03 3 03 1 2 98 99 F56_03 Mid Upper Arm Circumference (MUAC) tape 1 B 2 04 3 04 1 2 98 99 H–144 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F57 Does this facility have a separate room for suspected cases of Ebola virus disease? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F58 Does this facility have an EVD IPC precautions and PPE when handling potentially infectious specimens? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F59 Does this facility have a guidance on disposing waste potentially infected with Ebola virus? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F60 Does this facility provide IPC guidance upon discharged for survivors of Ebola virus disease to use at home? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F61 Does this facility have a mortuary on site? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 ANNEX H. DATA COLLECTION TOOLS n H–145 Indicator code Number Question Result Skip/Don’t Know DIAGNOSTICS F62 Does this facility conduct any diagnostic testing including any rapid diagnostic testing? YES................................................................... 1 NO................................................................... 2 DON’T KNOW........................................ 98 REFUSE.....................................................99 2,98,99èG0 ASK TO BE SHOWN THE MAIN LABORATORY OR LOCATION IN THE FACILITY WHERE MOST TESTING IS DONE TO START DATA COLLECTION. INTRODUCE YOURSELF AND EXPLAIN THE PURPOSE OF THE SURVEY, THEN ASK THE FOLLOWING QUESTIONS. I would like to know if the following diagnostic tests and associated equipment are available today in this facility. F63 Does this facility offer any of the following tests on-site? YES (ONSITE) NO DK REF F63_01 Malaria rapid testing 1 2 98 99 D9 F63_02 Rapid syphilis testing 1 2 98 99 D6 F63_03 HIV rapid testing 1 2 98 99 D11 F63_04 Urine rapid tests for pregnancy 1 2 98 99 D4 F63_05 Urine protein dipstick testing 1 2 98 99 D5 F63_06 Urine glucose dipstick testing 1 2 98 99 D20 F63_07 Urine ketone dipstick testing 1 2 98 99 D7 F63_08 Dry Blood Spot (DBS) collection for HIV viral load or EID 1 2 98 99 F64 I would like to know if the following items for rapid diagnostic testing are available or not available today. CHECK TO SEE IF AT LEAST ONE OF EACH RDT IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILAB LE NON VALID REPOR TED AVAIL ABLE BUT NOT SEEN NOT AVAIL ABLE TODAY NEVER AVAIL ABLE DK REF D3 D34 D36 F64_01 Malaria rapid diagnostic kit 1 2 3 4 5 98 99 D9 F64_02 Syphilis rapid test kit 1 2 3 4 5 98 99 D6 F64_03 HIV rapid test kit 1 2 3 4 5 98 99 D11 F64_04 Urine pregnancy test kit 1 2 3 4 5 98 99 D4 F64_05 Dipsticks for urine protein 1 2 3 4 5 98 99 D5 F64_06 Dipsticks for urine glucose 1 2 3 4 5 98 99 D20 F64_07 Dipsticks for urine ketone bodies 1 2 3 4 5 98 99 D7 F64_08 Filter paper for collecting DBS 1 2 3 4 5 98 99 H–146 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know CHECK QF64_01: IF FACILITY CONDUCTS MALARIA RDTS (QF64_01 = 1, 2, 3, OR 4): IF FACILITY DOES NOT CONDUCT MALARIA RDTS (QF64_01 = 5, 98, 99): QF67 D36_A F65 Have malaria RDT kits been unavailable any time in the past 4 weeks? YES................................................................... 1 NO................................................................... 2 DON’T KNOW...........................................98 REFUSE.....................................................99 2,98,99èF67 D36_B F66 How many days were they unavailable? LESS THAN 7 DAYS................................... 1 7 TO 14 DAYS............................................. 2 MORE THAN 14 DAYS............................. 3 DON’T KNOW...........................................98 REFUSE.....................................................99 F67 Does this facility conduct the following tests onsite or offsite? YES, ONSITE YES, OFFSITE DON’T CONDUCT THE TEST DK REF D2 F67_01 Blood glucose tests using a glucometer 1 2 3 98 99 D1 F67_02 Haemoglobin testing 1 2 3 98 99 D10 F67_03 General microscopy/wet￾mounts 1 2 3 98 99 D3 F67_04 Malaria smear tests 1 2 3 98 99 D6 D23 F67_05 HIV antibody testing by ELISA 1 2 3 98 99 F68 A) AVAILABLE I would like to know if the following general equipment items are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING I would like to know if the following general equipment items are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D3 D10 D35 D8 D31 D32 D33 F68_01 Light microscope 1 → B 2 → B 3 02 1 2 98 99 D3 D10 D35 D8 D31 D32 F68_02 Glass slides and cover slips 1 → B 2 → B 3 03 98 99 ANNEX H. DATA COLLECTION TOOLS n H–147 Indicator code Number Question Result Skip/Don’t Know F68_03 Refrigerator 1 → B 2 → B 3 04 1 2 98 99 D2 F68_04 Glucometer 1 → B 2 → B 3 05 1 2 98 99 D2 F68_05 Glucometer test strips (with valid expiration date) 1 → B 2 → B 3 06 98 99 D1 F68_06 Colorimeter or haemoglobinometer 1 → B 2 → B 3 07 1 2 98 99 D1 F68_07 HemoCue 1 → B 2 → B 3 08 1 2 98 99 D3 D35 F68_08 Wright-Giemsa stain or other acceptable malaria parasite stain (e.g. Field Stain A and B) 1 → B 2 → B 3 09 1 2 98 99 D6 D23 F68_09 ELISA washer 1 → B 2 → B 3 10 1 2 98 99 D6 D23 F68_10 ELISA reader 1 → B 2 → B 3 11 1 2 98 99 D6 D23 F68_11 Incubator 1 → B 2 → B 3 12 1 2 98 99 D6 D23 F68_12 Specific assay kit- HIV antibody testing by ELISA 1 → B 2 → B 3 F69 98 99 T59 D35 F69 Does this facility have an accredited/certified microscopist? YES................................................................... 1 NO................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 F70 CHECK QF29: TB SERVICES OFFERED TB SERVICES NOT OFFERED QF75 D8 F71 Does this facility do Ziehl￾Neelsen testing for TB (AFB) onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,3,98,99 èF73 F72 A) AVAILABLE I would like to know if the following equipment items for TB testing are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING I would like to know if the following equipment items for TB testing are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF D8 F72_01 Fluorescence microscope (FM) 1 → B 2 → B 3 02 1 2 98 99 D8 F72_02 Ziehl-Neelsen stain 1 → B 2 → B 3 03 1 2 98 99 H–148 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know D8 F72_03 Auramine Rhodamine stain for fluorescent microscopy 1 → B 2 → B 3 F73 1 2 98 99 F73 Does this facility conduct Xpert MTB/RIF diagnostic testing for TB onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW...........................................98 REFUSE.....................................................99 2,3,98,99 èF75 F74 A) AVAILABLE Please tell me if the following equipment items for Xpert MTB/RIF diagnostic testing for TB are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items for Xpert MTB/RIF diagnostic testing for TB are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF F74_01 GeneXpert 4 module unit with laptop 1 → B 2 → B 3 02 1 2 98 99 F74_02 TB rapid test cartridge 1 → B 2 → B 3 F75 1 2 98 99 F75 Does this facility conduct liver function /renal function tests and/or white blood counts onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW...........................................98 REFUSE.....................................................99 3,98,99 èF78 F76 Does this facility conduct the following liver and renal function tests onsite or offsite? YES, ONSITE YES, OFFSITE DON’T CONDUCT THE TEST DK REF D19 F76_01 ALT testing 1 2 3 98 99 D19 F76_02 Other liver function testing (such as bilirubin) 1 2 3 98 99 D18 F76_03 Serum creatinine testing 1 2 3 98 99 D18 F76_04 Other renal function testing (such as urea nitrogen) 1 2 3 98 99 F76_05 CHECK QF75 liver function/renal function: IF "YES, ONSITE" SELECTED FOR ANY TEST IF ONLY "YES, OFFSITE" OR "NO" OR “DK” OR “REFUSE”ARE SELECTED QF78 ANNEX H. DATA COLLECTION TOOLS n H–149 Indicator code Number Question Result Skip/Don’t Know F77 A) AVAILABLE Please tell me if the following equipment items and reagents for liver and kidney function testing are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items and reagents for liver and kidney function testing are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D18 D19 F77_01 Biochemistry analyser 1 → B 2 → B 3 02 1 2 98 99 D18 D19 F77_02 Centrifuge 1 → B 2 → B 3 03 1 2 98 99 D19 F77_03 Specific assay kit(s)- liver function test 1 → B 2 → B 3 04 1 2 98 99 D18 F77_04 Specific assay kit(s)- renal function test 1 → B 2 → B 3 F78 1 2 98 99 D15 D25 F78 Does this facility do full blood count and differential testing onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW........................................... 98 REFUSE.....................................................99 2,3,98,99 èF80 F79 A) AVAILABLE Please tell me if the following equipment items and reagents for full blood count testing are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items and reagents for full blood count testing are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D15 D25 F79_01 Haematology analyzer (for full blood count) 1 → B 2 → B 3 02 1 2 98 99 D15 D25 F79_02 Stains for full blood count and differential 1 → B 2 → B 3 F80 1 2 98 99 D16 F80 Does this facility do CD4 count (absolute and percentage) testing onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW........................................... 98 REFUSE.....................................................99 2,3,98,99èF82 H–150 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know F81 A) AVAILABLE Please tell me if the following equipment items for CD4 testing are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items for CD4 testing are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D16 F81_01 CD4 counter 1 → B 2 → B 3 02 1 2 98 99 D16 F81_02 Specific assay kit- CD4 test 1 → B 2 → B 3 F82 1 2 98 99 D21 D22 F82 Does this facility conduct blood group serology onsite or offsite? YES, ONSITE................................................. 1 YES, OFFSITE................................................ 2 NO................................................................... 3 DON’T KNOW........................................... 98 REFUSE.....................................................99 3,98,99 èF84 F83 Does this facility conduct the following blood group serology tests onsite or offsite? YES, ONSITE YES, OFFSITE DON’T CONDU CT THE TEST DK REF D21 F83_01 ABO blood grouping testing 1 2 3 98 99 D21 F83_02 Rhesus blood grouping testing 1 2 3 98 99 D22 F83_03 Cross-match testing by direct agglutination 1 2 3 98 99 D22 F83_04 Cross-match testing by indirect anti-globulin testing or other test with equivalent sensitivity 1 2 3 98 99 F84 CHECK QF83_01 to _QF83_04 Blood typing and cross match: IF "YES, ONSITE" SELECTED FOR ANY TEST IF ONLY "YES, OFFSITE" OR "NO" OR “DK” OR “REFUSE” ARE SELECTED QG0 F85 A) AVAILABLE Please tell me if the following equipment items and reagents for blood typing and cross match are AVAILABLE today. ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items and reagents for blood typing and cross match are FUNCTIONAL today. OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DK REF D21 D22 F85_01 Centrifuge 1 → B 2 → B 3 02 1 2 98 99 ANNEX H. DATA COLLECTION TOOLS n H–151 Indicator code Number Question Result Skip/Don’t Know D22 F85_02 37° C incubator 1 → B 2 → B 3 03 1 2 98 99 D22 F85_03 Grouping sera 1 → B 2 → B 3 F86 1 2 98 99 F86 CHECK QA9: IF HOSPITAL: IF NOT HOSPITAL: QG0 F87 Does this facility conduct the following tests onsite or offsite? YES, ONSITE YES, OFFSITE DON’T CONDUC T THE TEST DK REF D24 F87_01 Serum electrolyte testing 1 2 3 98 99 D32 F87_02 Urine microscopy testing 1 2 3 98 99 D29 F87_03 Syphilis serology testing 1 2 3 98 99 D31 F87_04 Gram stain testing 1 2 3 98 99 D33 F87_05 CSF/ body fluid counts 1 2 3 98 99 D30 F87_06 Cryptococcal antigen testing 1 2 3 98 99 D17 F87_07 Molecular biological technique for HIV viral load or HIV early￾infant diagnosis (PCR) 1 2 3 98 99 F87_08 Molecular -based assays for Ebola virus RNA (i.e. RT-PCR) 1 2 3 98 99 F88 A) AVAILABLE Please tell me if the following equipment items and reagents are AVAILABLE today: ASK TO SEE THE ITEMS B) FUNCTIONING Please tell me if the following equipment items and reagents are FUNCTIONAL today: OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF D24 F88_01 Specific assay kit- serum electrolyte test 1 → B 2 → B 3 02 1 2 98 99 D29 F88_02 Specific assay kit- syphilis serology 1 → B 2 → B 3 03 1 2 98 99 D31 F88_03 Gram stains 1 → B 2 → B 3 04 1 2 98 99 F88_04 White blood counting chamber 1 → B 2 → B 3 05 1 2 98 99 D30 F88_05 Specific assay kit- cryptococcal antigen test 1 → B 2 → B 3 06 1 2 98 99 D17 F88_06 Assay specific automated system for estimating HIV viral load 1 → B 2 → B 3 07 1 2 98 99 D17 D24 F88_07 Centrifuge 1 → B 2 → B 3 08 1 2 98 99 H–152 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/Don’t Know D17 F88_08 Vortex mixer 1 → B 2 → B 3 09 1 2 98 99 D17 F88_09 Pipettes 1 → B 2 → B 3 10 1 2 98 99 D24 F88_10 Biochemistry analyser 1 → B 2 → B 3 F89 1 2 98 99 F89 Does this facility perform diagnostic x-rays, ultrasound, or computerized tomography? YES...................................................................... 1 NO...................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99èG0 F90 Please tell me if the following imaging equipment items are available and functional today. ASK TO SEE THE ITEMS A) AVAILABLE B) FUNCTIONING OBSERVED REPORTED NOT SEEN NOT AVAILABLE YES NO DON'T KNOW REF E33 F90_01 X-ray machine 1 → B 2 → B 3 02 1 2 98 99 E35 F90_02 Ultrasound equipment 1 → B 2 → B 3 03 1 2 98 99 E36 F90_03 CT scan 1 → B 2 → B 3 04 1 2 98 99 E34 F90_04 ECG 1 → B 2 → B 3 G0 1 2 98 99 Indicator code Number Question Result Skip/ Other SECTION G: MEDICINES AND COMMODITIES. (The listed medicines and commodities will be checked against the list from the Ministry of Health and adapted accordingly) G0 Does this facility stock medicines, vaccines, or contraceptive commodities? YES.......................................................................1 NO ......................................................................2 èH0 ASK TO BE SHOWN THE MAIN LOCATION IN THE FACILITY WHERE MEDICINES AND OTHER SUPPLIES ARE STORED. FIND THE PERSON MOST KNOWLEDGEABLE ABOUT STORAGE AND MANAGEMENT OF MEDICINES AND SUPPLIES IN THE FACILITY. INTRODUCE YOURSELF, EXPLAIN THE PURPOSE OF THE SURVEY AND ASK THE FOLLOWING QUESTIONS. I would like to know if the following medicines are available today in this facility. I would also like to observe the medicines that are available. If any of the medicines I mention is stored in another location in the facility, please tell me where in the facility it is stored so I can go there to verify. G1 Are any of the following medicines for the treatment of infectious diseases available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF ANNEX H. DATA COLLECTION TOOLS n H–153 Indicator code Number Question Result Skip/ Other M43 G1_01 Co-trimoxazole cap/tab (Oral antibiotic) 1 2 3 4 5 98 99 M135 G1_02 Fluconazole cap/tab 1 2 3 4 5 98 99 M35 G1_03 Albendazole or Mebendazole cap/tab 1 2 3 4 5 98 99 M49 G1_04 Metronidazole cap/tab 1 2 3 4 5 98 99 M2 G1_05 Amoxicillin cap/tab 1 2 3 4 5 98 99 M5 M23 M110 G1_06 Ceftriaxone injection 1 2 3 4 5 98 99 M6 G1_07 Ciprofloxacin cap/tab 1 2 3 4 5 98 99 G2 Are any of the following medicines for the management of non-communicable diseases available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M50 G2_01 Metformin cap/tab 1 2 3 4 5 98 99 M51 G2_02 Insulin regular injection 1 2 3 4 5 98 99 M52 G2_03 Glucose 50% injection 1 2 3 4 5 98 99 M53 G2_04 ACE inhibitor (e.g. enalapril, lisinopril, ramipril, perindopril) 1 2 3 4 5 98 99 M54 G2_05 Thiazide (e.g. hydrochlorothiazide) 1 2 3 4 5 98 99 M55 G2_06 Beta blocker (e.g. bisoprolol, metoprolol, carvedilol, atenolol) 1 2 3 4 5 98 99 M56 G2_07 Calcium channel blocker (e.g. amlodipine) 1 2 3 4 5 98 99 M57 G2_08 Aspirin cap/tab 1 2 3 4 5 98 99 M59 G2_09 Beclomethasone inhaler 1 2 3 4 5 98 99 M60 G2_10 Prednisolone cap/tab 1 2 3 4 5 98 99 M61 G2_11 Hydrocortisone injection 1 2 3 4 5 98 99 M62 G2_12 Epinephrine injection 1 2 3 4 5 98 99 M114 G2_13 Furosemide cap/tab 1 2 3 4 5 98 99 M10 G2_14 Glibenclamide cap/tab 1 2 3 4 5 98 99 M115 G2_15 Gliclazide tablet or glipizide tablet 1 2 3 4 5 98 99 M116 G2_16 Glyceryl trinitrate sublingual tablet 1 2 3 4 5 98 99 M95 M44 G2_17 Ibuprofen tablet 1 2 3 4 5 98 99 H–154 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other M118 G2_18 Isosorbide dinitrate tablet 1 2 3 4 5 98 99 M11 G2_19 Omeprazole tablet or alternative such as pantoprazole, rabeprazole 1 2 3 4 5 98 99 M38 M44 G2_20 Paracetamol cap/tab (adult oral formulation) 1 2 3 4 5 98 99 M13 G2_21 Salbutamol inhaler 1 2 3 4 5 98 99 M14 G2_22 Simvastatin tablet or other statin e.g. atorvastatin, pravastatin, fluvastatin 1 2 3 4 5 98 99 M147 G2_23 Spironolactone tablets 1 2 3 4 5 98 99 G3 Are any of the following reproductive health medicines and commodities available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABL E DK REF M15 G3_01 Combined estrogen progesterone oral contraceptive pills 1 2 3 4 5 98 99 M96 G3_02 Progestin-only contraceptive pills 1 2 3 4 5 98 99 M16 M97 G3_03 Combined estrogen progesterone injectable contraceptives 1 2 3 4 5 98 99 M16 M98 G3_04 Progestin-only injectable contraceptives 1 2 3 4 5 98 99 M17 G3_05 Male condoms 1 2 3 4 5 98 99 M99 G3_06 Female condoms 1 2 3 4 5 98 99 M108 G3_07 Implant (e.g. levonorgestrel, etonogestrel) 1 2 3 4 5 98 99 M109 G3_08 Emergency contraceptive pill (e.g. levonorgestrel tablet, ulipristal acetate tablet, mifepristone tablet 10-25 mg) 1 2 3 4 5 98 99 M105 G3_09 Intrauterine contraceptive device (IUCD) 1 2 3 4 5 98 99 G4 For each of the following items, please check in the facility records if they have been unavailable anytime in the past 3 months: UNAVAIL ABLE IN THE PAST 3 MONTHS ALWAYS AVAILABLE IN PAST 3 MONTHS NOT INDICATED PRODUCT NOT OFFERED FACILITY RECORD NOT AVAILAB LE M99_A G4_01 Female condoms 1 2 3 4 5 ANNEX H. DATA COLLECTION TOOLS n H–155 Indicator code Number Question Result Skip/ Other M108_A G4_02 Implant (e.g. levonorgestrel, etonogestrel) 1 2 3 4 5 M109_A G4_03 Emergency contraceptive pill (e.g. levonorgestrel tablet, ulipristal acetate tablet, mifepristone tablet 10-25 mg) 1 2 3 4 5 G5 Are any of the following maternal health medicines available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M18 G5_01 Iron tablets 1 2 3 4 5 98 99 M19 G5_02 Folic acid tablets 1 2 3 4 5 98 99 M18 M19 G5_03 Iron and folic acid combined tablets 1 2 3 4 5 98 99 M20 G5_04 Tetanus toxoid vaccine 1 2 3 4 5 98 99 M69 G5_05 Sodium chloride injectable solution 1 2 3 4 5 98 99 M70 G5_06 Calcium gluconate injection 1 2 3 4 5 98 99 M24 G5_07 Magnesium sulphate injectable 1 2 3 4 5 98 99 M71 M23 G5_08 Ampicillin powder for injection 1 2 3 4 5 98 99 M72 M23 M110 M141 G5_09 Gentamicin injection 1 2 3 4 5 98 99 M106 G5_10 Hydralazine injection 1 2 3 4 5 98 99 M73 G5_11 Metronidazole injection 1 2 3 4 5 98 99 M74 G5_12 Misoprostol 200µg tablets 1 2 3 4 5 98 99 M75 G5_13 Azithromycin cap/tab or oral liquid 1 2 3 4 5 98 99 M76 G5_14 Cefixime cap/tab 1 2 3 4 5 98 99 M77 G5_15 Benzathine benzylpenicillin powder for injection 1 2 3 4 5 98 99 M78 G5_16 Betamethasone injection 1 2 3 4 5 98 99 M78 M129 G5_17 Dexamethasone injection 1 2 3 4 5 98 99 M79 G5_18 Nifedipine cap/tab (10mg) 1 2 3 4 5 98 99 M107 G5_19 Methyldopa tablet 1 2 3 4 5 98 99 M22 G5_20 Oxytocin injection 1 2 3 4 5 98 99 H–156 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other IF OXYTOCIN IS OBSERVED AVAILABLE (QG5_20 is “1” OR “2”) IF OXYTOCIN IS NOT OBSERVED AVAILABLE (QG5_20 is “3”,”4”,“5”,”98”OR”99”) èG7 G6 Is the oxytocin stored in cold storage? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 G7 For each of the following items, please check in the facility records if they have been unavailable any time in the past 3 months: UNAVAIL ABLE IN THE PAST 3 MONTHS ALWAYS AVAILABLET IN PAST 3 MONTHS NOT INDICATED PRODUCT NOT OFFERED FACILITY RECORD NOT AVAILAB LE M22_A G7_01 Oxytocin injection 1 2 3 4 5 M74_A G7_02 Misoprostol 200µg tablets 1 2 3 4 5 M24_A G7_03 Magnesium sulphate injection 1 2 3 4 5 M72_A G7_04 Gentamicin injection 1 2 3 4 5 M80_A G7_05 Procaine benzylpenicillin injection 1 2 3 4 5 M5_A G7_06 Ceftriaxone injection 1 2 3 4 5 M78_A G7_07 Betamethasone injection 1 2 3 4 5 M78_B G7_08 Dexamethasone injection 1 2 3 4 5 G8 Are any of the following child health medicines available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABL E D K REF M80 M110 G8_01 Procaine benzylpenicillin injection 1 2 3 4 5 98 99 M32 G8_02 Oral Rehydration Salts (ORS) sachets 1 2 3 4 5 98 99 M36 G8_03 Zinc sulphate tablets 1 2 3 4 5 98 99 M36 G8_04 Zinc sulphate syrup or dispersible tablets 1 2 3 4 5 98 99 M34 G8_05 Vitamin A (retinol) capsules 1 2 3 4 5 98 99 M21 G8_06 Antibiotic eye ointment for newborn 1 2 3 4 5 98 99 M7 G8_07 Co-trimoxazole syrup/suspension 1 2 3 4 5 98 99 M12 G8_08 Paracetamol syrup/suspension 1 2 3 4 5 98 99 ANNEX H. DATA COLLECTION TOOLS n H–157 Indicator code Number Question Result Skip/ Other M33 G8_09 Amoxicillin 250 mg or 500 mg dispersible tablet or syrup/suspension 1 2 3 4 5 98 99 G8_10 Paracetamol suppositories 1 2 3 4 5 98 99 IF AMOXICILLIN DISPERSIBLE TABLETS ARE OBSERVED AVAILABLE (QG8_09 is “1”) AMOXICILLIN DISPERSIBLE TABLETS NOT OBSERVED èG11 G9 Is the product stored so that identification labels and expiry dates and manufacturing dates are visible? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 G10 Check the expiry dates of the stored product. Are they stored in first-to-expire, first￾out (FEFO) order (i.e. the stock that will expire first is the closest to the front)? CHECK THE EXPIRY DATES OF THE STORED PRODUCT AT THE FRONT AND AT THE BACK OF THE SHELF. IF THE PRODUCT AT THE FRONT EXPIRES FIRST, ANSWER “YES”. IF THE PRODUCT AT THE BACK EXPIRES FIRST, ANSWER “NO”. YES.......................................................................1 NO ......................................................................2 G11 For each of the following items, please check in the facility records if they have been unavailable any time in the past 3 months: UNAVAIL ABLE IN THE PAST 3 MONTHS ALWAYS AVAILABLE IN PAST 3 MONTHS NOT INDICATED PRODUCT NOT OFFERED FACILITY RECORD NOT AVAILAB LE M33_A G11_01 Amoxicillin 250mg or 500mg dispersible tablet or syrup/suspension 1 2 3 4 5 M32_A G11_02 Oral rehydration salts (ORS) 1 2 3 4 5 M36_A G11_03 Zinc sulphate tablets 1 2 3 4 5 M36_B G11_04 Zinc sulphate syrup or dispersible tablets 1 2 3 4 5 G12 Does this facility stock any medicines for malaria treatment? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èG16 OBSERVED AVAILABLE NOT OBSERVED H–158 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other G13 Are any of the following malaria medicines and commodities available today in this facility? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE D K REF M81 M37 G13_01 ACT 1 2 3 4 5 98 99 M136 G13_02 Artemisinin monotherapy (oral) 1 2 3 4 5 98 99 M82 G13_03 Artesunate rectal or injection dosage forms 1 2 3 4 5 98 99 M39 G13_04 SP (Sulfadoxine + Pyrimethamine) 1 2 3 4 5 98 99 M40 G13_05 Insecticide treated bed nets for patients and their families and households 1 2 3 4 5 98 99 G13_06 G13_07 M139 G13_08 Quinine (oral) 1 2 3 4 5 98 99 CHECK QG13_01: IF FACILITY STOCKS ACT (QG13_01 = 1, 2, 3, OR 4): IF FACILITY DOES NOT STOCK ACT (QG13_01 = 5, 98, 99): èG16 M37_A G14 Has ACT been unavailable any time in the past 4 weeks? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èG16 M37_B G15 How many days of was it not available? LESS THAN 7 DAYS.......................................1 7 TO 14 DAYS.................................................2 MORE THAN 14 DAYS.................................3 DON’T KNOW............................................ 98 REFUSE.....................................................99 G16 Does this facility stock any medicines for tuberculosis treatment? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èG18 G17 Are any of the following TB medicines available today in this facility? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABL E DK REF ANNEX H. DATA COLLECTION TOOLS n H–159 Indicator code Number Question Result Skip/ Other M41 G17_01 Ethambutol 1 2 3 4 5 98 99 M41 G17_02 Isoniazid 1 2 3 4 5 98 99 M41 G17_03 Pyrazinamide 1 2 3 4 5 98 99 M41 G17_04 Rifampicin 1 2 3 4 5 98 99 M41 G17_05 Isoniazid + Rifampicin (2FDC) 1 2 3 4 5 98 99 M41 G17_06 Isoniazid + Ethambutol (EH) (2FDC) 1 2 3 4 5 98 99 M41 G17_07 Isoniazid + Rifampicin + Pyrazinamide (RHZ) (3FDC) 1 2 3 4 5 98 99 M41 G17_08 Isoniazid + Rifampicin + Ethambutol (RHE) (3FDC) 1 2 3 4 5 98 99 M41 G17_09 Isoniazid + Rifampicin + Pyrazinamide + Ethambutol (4FDC) 1 2 3 4 5 98 99 G17_10 Streptomycin injectable 1 2 3 4 5 98 99 G18 Does this facility stock any antiretroviral medicines? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èG20 G19 Are any of the following ARVs available today in this facility? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M45 M48 G19_01 Zidovudine (ZDV, AZT) 1 2 3 4 5 98 99 M46 G19_02 Zidovudine (ZDV, AZT) syrup 1 2 3 4 5 98 99 M45 M48 G19_03 Abacavir (ABC) 1 2 3 4 5 98 99 M45 M48 G19_04 Lamivudine (3TC) 1 2 3 4 5 98 99 M45 M48 G19_05 Tenofovir Disoproxil Fumarate (TDF) 1 2 3 4 5 98 99 M45 M48 G19_06 Nevirapine (NVP) 1 2 3 4 5 98 99 M47 G19_07 Nevirapine (NVP) syrup 1 2 3 4 5 98 99 M45 M48 G19_08 Efavirenz (EFV) 1 2 3 4 5 98 99 M45 M48 G19_09 1 2 3 4 5 98 99 M45 M48 G19_10 Lamivudine + Abacavir (3TC + ABC) 1 2 3 4 5 98 99 H–160 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other M45 M48 G19_11 Zidovudine + Lamivudine (AZT + 3TC) 1 2 3 4 5 98 99 M45 M48 G19_12 Zidovudine + Lamivudine + Abacavir (AZT + 3TC + ABC) 1 2 3 4 5 98 99 M45 M48 G19_13 Zidovudine + Lamivudine + Nevirapine (AZT + 3TC + NVP) 1 2 3 4 5 98 99 M45 M48 G19_14 Tenofovir + Emtricitabine (TDF + FTC) 1 2 3 4 5 98 99 M45 M48 G19_15 Tenofovir + Lamivudine (TDF + 3TC) 1 2 3 4 5 98 99 M45 M48 G19_16 Tenofovir + Lamivudine + Efavirenz (TDF + 3TC + EFV) 1 2 3 4 5 98 99 M45 M48 G19_17 Tenofovir + Emtricitabine + Efavirenz (TDF + FTC + EFV) 1 2 3 4 5 98 99 M45 G19_18 Didanosine (DDI) 1 2 3 4 5 98 99 G19_19 Lamivudine (3TC) syrup 1 2 3 4 5 98 99 M45 G19_20 Stavudine 30 or 40 (D4T) 1 2 3 4 5 98 99 G19_21 Stavudine syrup 1 2 3 4 5 98 99 G19_22 Efavirenz (EFV) syrup 1 2 3 4 5 98 99 M45 G19_23 Delavirdine (DLV) 1 2 3 4 5 98 99 M45 G19_24 Enfuvirtide (T-20) 1 2 3 4 5 98 99 M45 G19_25 Stavudine + Lamivudine (D4T + 3TC) 1 2 3 4 5 98 99 G20 Does this facility stock any protease inhibitors for the treatment of HIV/AIDS? YES.......................................................................1 NO ......................................................................2 DON’T KNOW............................................ 98 REFUSE.....................................................99 2,98,99 èG22 G21_ Are any of the following protease inhibitors available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE DK REF M48 G21_01 Lopinavir (LPV) 1 2 3 4 5 98 99 G21_02 1 2 3 4 5 98 99 G21_03 1 2 3 4 5 98 99 G21_04 1 2 3 4 5 98 99 G21_05 5 1 2 3 4 5 98 99 G21_06 Atazanavir (ATV) 1 2 3 4 5 98 99 G21_07 1 2 3 4 5 98 99 ANNEX H. DATA COLLECTION TOOLS n H–161 Indicator code Number Question Result Skip/ Other G21_08 1 2 3 4 5 98 99 G21_09 1 2 3 4 5 98 99 G22_ Are any of the following other medicines and commodities available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE D K REF M27 G22_01 Normal saline IV solution 1 2 3 4 5 98 99 M27 G22_02 Ringers lactate IV solution 1 2 3 4 5 98 99 M27 G22_03 5% dextrose IV solution 1 2 3 4 5 98 99 M42 G22_04 IV treatment for fungal infections 1 2 3 4 5 98 99 M26 G22_05 Skin disinfectant 1 2 3 4 5 98 99 G22_06 Gowns 1 2 3 4 5 98 99 G22_07 Eye protection (goggles, face shields) 1 2 3 4 5 98 99 G22_08 Medical (surgical or procedural) masks 1 2 3 4 5 98 99 M63 G22_09 Absorbable suture material 1 2 3 4 5 98 99 M63 G22_10 Non-absorbable suture material 1 2 3 4 5 98 99 M64 G22_11 Ketamine (injection) 1 2 3 4 5 98 99 M65 G22_12 Lidocaine 1% or 2% (anaesthesia) 1 2 3 4 5 98 99 M25 G22_13 Diazepam (injection) 1 2 3 4 5 98 99 G22_14 IV cannula 1 2 3 4 5 98 99 G22_15 Water for Injection 1 2 3 4 5 98 99 G22_16 Syringes 1 2 3 4 5 98 99 G22_17 Needles 1 2 3 4 5 98 99 G22_18 Cotton wool 1 2 3 4 5 98 99 CHECK QA9 AND QF2_08: IF HOSPITAL OR HEALTH FACILITY OFFERS CESAREAN SECTION: IF NOT HOSPITAL AND CESAREAN SECTION NOT OFFERED: QG23 M84 G22_19 Thiopental (powder) 1 2 3 4 5 98 99 M85 G22_20 Suxamethonium bromide (powder) 1 2 3 4 5 98 99 M86 G22_21 Atropine (injection) 1 2 3 4 5 98 99 M87 G22_22 Halothane (inhalation) 1 2 3 4 5 98 99 H–162 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other M88 G22_23 Bupivacaine (injection) 1 2 3 4 5 98 99 M89 G22_24 Lidocaine 5% (heavy spinal solution) 1 2 3 4 5 98 99 M62 G22_25 Epinephrine (injection) 1 2 3 4 5 98 99 M90 G22_26 Ephedrine (injection) 1 2 3 4 5 98 99 G23 Are any of the following mental health and neurological medicines available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE D K REF M1 G23_01 Amitriptyline tablet 1 2 3 4 5 98 99 M119 G23_02 Carbamazepine tablet 1 2 3 4 5 98 99 M120 G23_03 Chlorpromazine injection 1 2 3 4 5 98 99 M121 G23_04 Diazepam tablet 1 2 3 4 5 98 99 M122 G23_05 Diazepam injection or diazepam rectal tubes 1 2 3 4 5 98 99 M94 G23_06 Fluoxetine tablet 1 2 3 4 5 98 99 M123 G23_07 Fluphenazine injection 1 2 3 4 5 98 99 M124 G23_08 Haloperidol tablet 1 2 3 4 5 98 99 M125 G23_09 Lithium tablet 1 2 3 4 5 98 99 M126 G23_10 Phenobarbital tablet 1 2 3 4 5 98 99 M127 G23_11 Phenytoin tablet 1 2 3 4 5 98 99 M128 G23_12 Valproate sodium tablet 1 2 3 4 5 98 99 M144 G23_13 Lorazepam injection 1 2 3 4 5 98 99 M145 G23_14 Levodopa + carbidopa tablet 1 2 3 4 5 98 99 G24 Are any of the following palliative care medicines available in the facility today? CHECK TO SEE IF AT LEAST ONE OF EACH MEDICINE/COMMODITY IS VALID (NOT EXPIRED) OBSERVED AVAILABLE NOT OBSERVED AT LEAST ONE VALID AVAILABLE NON VALID REPORTED AVAILABLE BUT NOT SEEN NOT AVAILABLE TODAY NEVER AVAILABLE D K REF M129 G24_01 Dexamethasone injection 1 2 3 4 5 98 99 M130 G24_02 Haloperidol injection 1 2 3 4 5 98 99 M131 G24_03 Hyoscine butylbromide injection 1 2 3 4 5 98 99 M132 G24_04 Lorazepam tablet 1 2 3 4 5 98 99 M133 G24_05 Metoclopramide injection 1 2 3 4 5 98 99 M83 M44 G24_06 Morphine granules, tablet 1 2 3 4 5 98 99 ANNEX H. DATA COLLECTION TOOLS n H–163 Indicator code Number Question Result Skip/ Other M83 M44 G24_07 Morphine injection 1 2 3 4 5 98 99 M134 G24_08 Senna preparation (laxative) 1 2 3 4 5 98 99 M146 G24_09 Loperamide tab/cap 1 2 3 4 5 98 99 G24_10 Pethidine injection 1 2 3 4 5 98 99 G24_11 Tramadol Tablets 1 2 3 4 5 98 99 SUPPLY CHAIN G25 Who is the principal person responsible for managing the ordering of medical supplies at this facility? NURSE................................................................1 CLINICAL OFFICER.......................................2 PHARMACY TECHNICIAN.........................3 PHARMACY ASSISTANT.............................4 PHARMACIST..................................................5 MEDICAL ASSISTANT...................................6 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE.....................................................99 G26 Which of the following mechanisms is used to determine this facility’s resupply quantities? ASK FOR EACH OF THE BELOW YES NO DON’T KNOW REF G26_01 The facility itself (pull distribution system) 1 2 98 99 G26_02 A higher level facility (push distribution system) 1 2 98 99 G26_03 Other _______________________ (SPECIFY) G27 How are the facility’s resupply quantities determined? FORMULA (ANY CALCULATION)..........1 OTHER MEANS (SPECIFY)............................3 DON’T KNOW............................................ 98 REFUSE.....................................................99 G28 What is the main source of your routine pharmaceutical commodity supplies? By this I mean who is the direct supplier to your facility? NATIONAL MEDICAL STORES.................1 JOINT MEDICAL STORES............................2 NGO/DONORS..............................................3 PRIVATE SOURCES........................................4 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE.....................................................99 If 2, show G28_01 and G28_02. If not 2, go to G29 H–164 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other G28_01 Central Medical Stores YES.......................................................................1 NO ..................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 G28_02 District Medical Stores Country specific YES.......................................................................1 NO ..................................................................... 2 DON’T KNOW............................................ 98 REFUSE.....................................................99 G29 How are your pharmaceutical commodity supplies from the main supplier of your routine pharmaceuticals delivered to this facility? SUPPLIER DELIVERS TO FACILITY...........1 FACILITY MUST ARRANGE DELIVERY TO FACILITY......2 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE.....................................................99 G30 Who is responsible for transporting products from central medical stores to your facility? YES NO DK REF G30_01 Local supplier delivers 1 2 98 99 G30_02 Higher level delivers 1 2 98 99 G30_03 This facility collects 1 2 98 99 G30_04 Other _______________________ (SPECIFY) G31 For the most recent order, how long did it take between ordering and receiving products? LESS THAN 2 WEEKS ...................................1 2 WEEKS TO 1 MONTH..............................2 BETWEEN 1 AND 2 MONTHS..................3 MORE THAN 2 MONTHS...........................4 DON’T KNOW............................................ 98 REFUSE.....................................................99 G32 YES NO DK REF G33 Does the facility have adequate storage for medical supplies? 1 2 98 99 G34 Does this health facility suffer from delay in delivery of medicines and commodities? 1 2 98 99 G35 Does this health facility submit the logistic management information system (LMIS) reports? YES.......................................................................1 NO ......................................................................2 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE.....................................................99 1èG36 2,96,98,99 èH1 ANNEX H. DATA COLLECTION TOOLS n H–165 Indicator code Number Question Result Skip/ Other G36 How often does this health facility submit the LMIS? WEEKLY............................................................1 BI-WEKLY .........................................................2 MONTHLY........................................................3 OTHER _______________________ 96 (SPECIFY) DON’T KNOW............................................ 98 REFUSE.....................................................99 SECTION H: RECORD KEEPING H1 May I see the form used to collect service statistics? Form exist............................................................... …1 Form does not exit................................................... 2 Tablets are used………………………………...3 REFUSE.....................................................99 2,99èH 4 H2 How often is this form filled out/data entered? Weekly ......................................................................... 1 Bi-weekly...................................................................... 2 Monthly......................................................................... 3 DON’T KNOW...................................... 98 REFUSE.....................................................99 H3 What is done with the form once it is filled out/data are entered? (Check all that apply) Sent to district office for collation ........................ 1 Used to generate facility bulletin .......................... 2 Used in decision making........................................... 3 DON’T KNOW...................................... 98 REFUSE.....................................................99 1èH3A 2,3,98,9 9 èH4 H3A In which form is the data sent to the district office? PAPER.................................................................1 ELECTRONIC/EMAIL.....................................2 OTHER _______________________ 96 (SPECIFY) DON’T KNOW...................................... 98 REFUSE.....................................................99 H4 May I see the form used to record disease surveillance data? Form exists........................................................1 Form does not exist........................................2 DON’T KNOW...................................... 98 REFUSE.....................................................99 2,98,99 èH6 H5 How often is this form filled out/data entered? Weekly ...............................................................1 Bi-weekly............................................................2 Monthly...............................................................3 DON’T KNOW...................................... 98 REFUSE.....................................................99 H6 What is done with the form once it is filled out/data entered? (Check all that apply) Sent to district office for collation .............1 Used to generate facility bulletin ................2 Used in decision making ...............................3 DON’T KNOW...................................... 98 REFUSE.....................................................99 1èH7 2,3,98,9 9 èH8 H–166 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Indicator code Number Question Result Skip/ Other H7 How often is this form sent to the district? WEEKLY............................................................1 BI-WEKLY .........................................................2 MONTHLY........................................................3 OTHER _______________________ 96 (SPECIFY) DON’T KNOW...................................... 98 REFUSE.....................................................99 H7A What was the total number of outpatient client visits during the following fiscal/calendar years? Country specific (use of fiscal or calendar year depends on the country) Calendar year 2017/ Fiscal year 2016- 2017:___________________, DK: Calendar year 2016/ Fiscal year 2015-2016:_-------------------, DK: Calendar year 2015/ Fiscal year 2014- 2015:________________, DK: Calendar year 2014/ Fiscal year 2013-2014:_______________, DK Calendar year 2013/ Fiscal year 2012-2013:____________, DK DON’T KNOW............................................ 98 REFUSE.....................................................99 GENERAL CONDITIONS OF THE HEALTH FACILITY H8 ASSESS THE GENERAL CONDITION OF THE HEALTH FACILITY YES NO H8_01 FLOOR: SWEPT. NO OBVIOUS DIRT OR WASTE 1 2 H8_02 COUNTER/TABLES/CHAIRS: WIPED CLEAN- NO OBVIOUS DUST OR WASTE 1 2 H8_03 NEEDLES, SHARPS OUTSIDE SHARPS BOX 1 2 H8_04 SHARPS BOX OVERFLOWING OR TORN/PERCED 1 2 H8_05 BANDAGES/INFECTIOUS WASTE LYING UNCOVERED 1 2 H8_06 WALLS: CONSIDERABLE DAMAGE 1 2 H8_07 DOORS: CONSIDERABLE DAMAGE 1 2 H8_08 CEILING: WATER STAINS OR DAMAGE 1 2 H9 HOW MANY BUILDINGS COMPRISE THIS HEALTH FACILITY? 1 BUILDING............................................................... 1 2 BUILDING............................................................... 2 3+ BUILDINGS.......................................................... 3 ANNEX H. DATA COLLECTION TOOLS n H–167 Indicator code Number Question Result Skip/ Other H9A Has this health facility had any renovations done or any additional support provided over the last 3 years? YES.......................................................................1 NO ......................................................................2 DON’T KNOW...................................... 98 REFUSE.....................................................99 2,98,99 èH11 H10 When were the last renovations of this building completed? # MONTHS;_________ DON’T KNOW 98 REFUSE 99 H10A Who provided the additional support or funds for the renovation? [PROBE FOR DONOR, IMPLEMENTING PARTNERS OR PROJECT NAME] Ministry of Health ..................................................... 1 Other public ministries ........................................... 2 Health Facility Fund .................................................. 3 Non-Profit Organization.......................................... 4 Government contribution to private.................... 5 Donor agencies ......................................................... 6 Faith-based organization ......................................... 7 Community programs ............................................. 8 Other____________________________9 DON’T KNOW...................................... 98 REFUSE.....................................................99 H11 Was this health facility closed during the Ebola crisis (in 2014- 2015) YES.......................................................................1 NO ......................................................................2 DON’T KNOW...................................... 98 REFUSE.....................................................99 2,98,99 èI0 H12 When did it reopen? MONTH: _______ YEAR:___ 98 DK 99 REFUSE We have now finished this interview. Many thanks for your assistance. We have now completed all of the questions in this module of the survey. Thank you for your participation. Number Question Result Skip SECTION I: INTERVIEWER'S OBSERVATIONS I0 INTERVIEW END TIME (use the 24 hour-clock system) : H–168 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Number Question Result Skip I1 RESULT CODES (LAST VISIT): I1: RESULT CODES (FINAL VISIST): 1= FACLITY COMPLETED [LAST QUESTION ASNWERED] 2= FACILITY RESPONDENTS NOT AVAILABLE [CODE 2 IN INTERIM OUTCOME FOR ALL 3 VISITS] 3= FACILITY REFUSED [CODE 4 IN INTERIM OUTCOME] 4= PARTIALLY COMPLETED [INTERVIEW BREAKOFF] 5=INELIGIBLE (code 3 in Interim Outcome) COMMENTS ABOUT THE RESPONDENT: ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ COMMENTS ON SPECIFIC QUESTIONS: ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ _____________________________________________________________________________________ ______________________________________________________________________________________ ANY OTHER COMMENTS: ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ SUPERVISOR'S OBSERVATIONS: ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ______________________________________________________________________________________ ANNEX H. DATA COLLECTION TOOLS n H–169 Number Question Result Skip NAME OF SUPERVISOR: ____________________________________ DATE: - ______________________ H–170 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SURVEY— HOUSEHOLDS ANNEX H. DATA COLLECTION TOOLS n H–171 H–172 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–173 H–174 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–175 H–176 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–177 H–178 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–179 H–180 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–181 H–182 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–183 H–184 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–185 H–186 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX H. DATA COLLECTION TOOLS n H–187 H–188 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX I. INTERVIEWS AND DISCUSSIONS n I–1 ANNEX I. LIST OF PERSONS INTERVIEWED, INTERVIEW AND FGD SITES Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document 1. Key Informant Interviews No Name Designation Organization Location of Interview Washington, DC 1 Andrea Long-Wagar Fellow, Emerging Pandemic Threats Advisor USAID/AFR Washington, DC 2 Ross Herbert Democracy Fellow USAID/AFR/SD Washington, DC 3 Jeff Muschell GH Ebola Team Lead USAID/GH/GHET Washington, DC 4 Dennis Carroll (Kendra Chittenden) Director, Emerging Threats USAID/GH Washington, DC 5 Cheryl Klein FAFR backup, currently front office USAID/F Washington, DC 6 Sonali Korde Director, Legislative Affairs USAID/LPA Washington, DC 7 Gamma Roberts ICT for Development Advisor USAID/Global Development Lab Washington, DC 8 Denise Rollins Senior Coordinator USAID Africa Ebola Unit Washington, DC 9 Alexandra Robinson M&E Specialist (TBC) USAID/Global Development Lab Washington, DC 10 Nadine Ritcheson Program Lead USAID Africa Ebola Unit Washington, DC 11 Mette Karlsen Senior Regional Advisor USAID/Food for Peace Washington, DC 12 Amanda Boachie Liberia Desk Officer, COR for Ebola Pillar II MEL USAID/AFR Washington, DC 13 Sarah Glass Director of Special Projects USAID/Global Development Lab Washington, DC 14 Monica Bautista Program Assistant USAID/GH Washington, DC 15 Dirk Dijkerman Former Executive Coordinator USAID Ebola Secretariat Washington, DC 16 Katherine Alexander Congressional Liaison Officer USAID Washington, DC 17 Philip Rihm Program Analyst, Private￾Public Partnerships (PPPs) USAID/Global Development Lab Washington, DC 18 Jen Fluder (Avery White) Innovation Team Lead and Senior Partnership Advisor (Program Analyst) USAID Global Health Center for Innovation and Impact Washington, DC 19 Robbin Boyer Senior Program Analyst USAID/GH Washington, DC Guinea 1 General Abdoulaye Keleba Keita Secretary General Prefecture Boufero, Beyla 2 Dr Koikoi Guilavogui General Supervisor Prefectoral Hospital Boufero, Beyla I–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 3 Jean Mathieu Konomou Director of Programmes and Focal Point for Search for Common Ground Rural Radio of Beyla Keme Bourema, Beyla 4 Alphonse Lamah Director of Social Action Prefectoral Department of Social Action Keme Bourema, Beyla 5 Daouda Kaba General Secretary Urban Commune of Beyla Keme Bourema, Beyla 6 Elhadj Adjouba Camara Secretary General of Islamic League Town of Beyla Keme Bourema, Beyla 7 General Balde (Issa Camara) Secretary General (Director for Children Support) Prefecture of Boffa Boffa 8 Dr. Balla Traoré DPS Prefectural Health Directorate of Boffa Almamya, Boffa 9 Alseny Camara Prefectoral Director of Agriculture of Boffa Ministry of Agriculture Boffa 10 Amadou Soumah President of the special delegation of Boffa Special Delegation of Boffa Boffa 11 Ramatoulaye Bah President Boffa Women's Association Boffa 12 Momo Blaise Bangoura General Secretary in charge of Administrative Affairs AA Boffa Foulamadina, Boffa 13 Dr. Sekou Ahmed Tidiane Hospital Director Regional Hospital, Boke Boké 14 Morlaye KEITA Prefectural Director of Agriculture of Boke Ministry of Agriculture Boké 15 Dr. El Hadj Mamadou Diouhé BARRY DRS of Boke Boké 16 Elhadj sekou Souare (Kade Dabas Keita) President of the Regional Council of Boké Civil Society Organizations (CROSC) (Coordinator of the activities of the CEC of Boké) Boke Civil Society (Center for Civic and Electoral Education Boké) Dibia, Boké 17 Nouhan Kaba General Secretary of the Urban Commune of Boké Municipality of Boké Koulifanyah, Boké 18 General Jean Beavogui Boke Prefecture Secretary General Urban Commune, Boké 19 Ibrahima Sory KABA Head of Division National Directorate of Electoral Administration Almamya, Conakry 20 Kourouma Danssa President of the Civil Society Camayenne, Conakry 21 Edouard Nizeyimana (Mr. Fakoli) Country Representative (Head of Programs) WFP Conakry 22 Jennifer Mbabazi Acting Head of Health USAID/Guinea Conakry 23 Jean Damascene Butora (Dr. Sangare, Koly Koivogue, Lancinet Conde) Head of Projects ABT Associates Conakry 24 Meba Kagone (Simbe Sokiba, Sabou Sow) Chief of Projects (Country Director) JSI Conakry 25 Marc Rubin (Moise Tounkar) Country Representative (Emergency Specialist) UNICEF Conakry 26 Kathy Middleton Deputy Country Director CDC Conakry ANNEX I. INTERVIEWS AND DISCUSSIONS n I–3 1. Key Informant Interviews No Name Designation Organization Location of Interview 27 Ismael Diallo (Ousmane Kourouma, Mohamed Lamine Barry, Saliou Mamadou Diallo) Director of Programs (Chief Accountant, Technical Assistance, IT) OICI Conakry 28 Julien Bolomou Program Coordinator SFCG Conakry 29 Dr. Rafi (Dr. Souma) Head of Health Promotion (Community Development) MOH Conakry 30 Claude Bahati Country Director Chemonics Conakry 31 Dr. Yerebhoye Camara Chief, Office of Strategy & Development MOH Conakry 32 Dr. Sakoba Keita (Dr. Facinet Yattara) Director General (Head of Department) National Agency of Health Security/Safety MOH, Former Anti-Ebola Coordination Office (Surveillance Department) Conakry 33 Dr. Conte Aboubacar Deputy National Director Ministry of Health (DNEHS) Conakry 34 Keita Fode Nyagassola IT specialist, assembly manager RTG Koloma Conakry 35 Dr. Mamadou Oury Diallo President of the National Network of Associations of Survivors of Ebola of Guinea National Network of Associations of Survivors of Ebola of Guinea Conakry 36 Dr. Ferida Mara Formal Youth and Youth Health Section Head Youth and Youth Health Section Conakry 37 Hadja Kany Diakite Head of Cabinet Ministry of Social Action Conakry 38 Dr. Konate Almamy Doctor Pediatrician CMC Matam Conakry 39 Lansana Camara Senior M&E Specialist MSH Conakry 40 Dr Barry Ibrahima Sory (Marie Chesnay) Head of Training and Research (Technical Advisor) GIZ Conakry 41 Hadja Fatou Sikhe Camara Director of Hospital Donka National Hospital Conakry 42 Keita Mamady Chairman of commission National Assembly Conakry 43 Karolina Lagiewka European Union Conakry 44 Dr Mariama Cire Camara Deputy Director Department of Health Conakry City Conakry 45 Dr Yolande Hyjazi (Dr. Mamadou Diallo) Country Director Jhpiego Conakry 46 Bocar Cissoko Secretary General CENI Conakry 47 Moundjirou Cherif Governorate Chief of Cabinet Kaloum Couranthie, Conakry 48 Dr. Robert Camara Director DNPSC Kaloum, Conakry 49 Prof Kouloubaly Moussa Deputy General Director University Hospital Ignace Deen Kaloum, Conakry 50 El Hadj Mamadou Aliou Bah Head of the Division of Prevention and Operations National Service for Humanitarian Actions (SENAH) Kaloum-Almamya, Conakry 51 Dr. Bafode Boua Soumah (Alseny Camara) DCN (Adjoining DCN) Ministry of Commerce Kaloum-Almamyah, Conakry I–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 52 Jean Traore (Cheick Magassouba) Head of Media Unit (Head of Social Mobilization) ANSS Kaloum-Coronthie, Conakry 53 Jean Luc Faber Division Manager of Development Strategies Ministry of Agriculture Kaloum-Coronthie, Conakry 54 CONDE Fode Studies Director DNETFP Kaloum-Tombo, Conakry 55 Dr Doussou Toure (Fatoumata Conde) Director CMC Coleah Matam-Coléah, Conakry 56 Camara Aboubacar General Manager URTELGUI Matoto, Conakry 57 Hadja Saran Kandé Secretary UFDG Minière, Conakry 58 Abdoulaye Toure President of Association Association of People Cured and Affected by Ebola Matoto, Conakry 59 Dr Conde Sekou Permanent Secretary RPG Arc en Ciel Matoto, Conakry 60 Mohamed Rahim Kebe Measure Evaluation Ratoma, Conakry 61 Dr Achille Guemou President Ebola Survivors Association Ratoma, Conakry 62 Bolamou Julien Monitoring and Evaluation Coordinator SFCG Taouyah, Conakry 63 Honorable Dr. Cisse Member of the Health Commission National Assembly Tombo, Conakry 64 Toure Kerfala Head of Service National Assembly Tombo, Conakry 65 Bandjougou OULARE Head of Planning Faranah Municipal Directorate Abattoire, Faranah 66 Aly Facinet CAMARA Programs Coordinator Local Radio of Faranah Abattoire, Faranah 67 Ibrahima Sylla (Sory Doumboya) Head of Administrative and Finance Department (Head of Public Health) Regional Health Directorate of Faranah Faranah 68 Keita Ayouba District Communication Officer SLEGG (Free Teachers and Researchers Union of Guinea/Faranah) Faranah 69 Kouyate Amara In charge of Programs and Planning Association of young people at the service of humanity Faranah 70 Lansana Tito Camara Secretary General of Communications Civil Society Organization (NTC) Market 1, Faranah 71 Kaman Camara General Secretary of the Municipality Urban Municipality Fatako I, Forécariah 72 Aminata CAMARA Chairwoman Alafama Association Fatako II, Forécariah 73 El Hadj Hafiou Fofana Imam Islamic League Fatako1, Forécariah 74 Dr. Lansana Kerouane Camara (Dr. Souleymane Sylla DPS (Disease Control Doctor) Prefectural Health Directorate, Forécariah Forécariah 75 Karamoko Oulare Director of Micro Achievement (DMR) Prefecture of Forécariah Forécariah 76 Alseny Ben Soumah Director Prefectoral Directorate of Agriculture Forécariah 77 Lamah Foromo Sapin Chairman of the Board of Directors MC2 Koutoumania, Forécariah 78 Kadiatou YOULA Housewife/survivor Madina, Forécariah ANNEX I. INTERVIEWS AND DISCUSSIONS n I–5 1. Key Informant Interviews No Name Designation Organization Location of Interview 79 Diané Mohamed Union President Union of Planters of Forecariah Maférinya, Forécariah 80 Dopavogui Tannou Head of the social Action Section Prefectural Direction of Social Action Tatagui, Forécariah 81 Jules Aly KONDONO Doctor Chief of the CPE Prefectural Hospital Tatagui, Forécariah 82 Lamah Kovana Director Plan Prefectoral Office in Forecariah Tatagui, Forécariah 83 Pivi Barre (Dr. Toure Wilalam Boua, Moussa Toure) Leprosy and TB Specialist (Nutritionist, In charge of Community Health) DPS Guéckédou 84 Mm Goubou Benemou Supplanting primary curative consultation Mangala Health Center Bambo, Guéckédou 85 Abel Leno Administrative Secretary Farako Town Farako 1, Guéckédou 86 Saa Sabas Temessadouno President of Association Association of People Infected and Cured of Ebola (APAGE) Farako, Guéckédou 87 Mamadou Bailo Diallo (Albert Boye Lelano) Area Head (Assistant Area Head) Local government Gbangbangyessa, Guéckédou 88 Daniel Mano Millimouno (Tabma Edouart Kamano) Pastor, VP of District Assembly (Pastor, President of Protestant Church) EPEG Guéckédou 89 Isaac Lelano Interim Head, Childhood Section District Directorate of Social Action, advancement of women and children Guéckédou 90 Saa Télliano (Lamine Diallo) Houndonin Sector Head (Sage of Houndonin) Houndonin, Guéckédou 91 Dr. Hawa Mady Camara Hospital General Supervisor Prefectural Hospital Macenta Koura, Guéckédou 92 Christine YOMBOUNO Representative Group of Women Macenta Koura, Guéckédou 93 Faya Bawa Kamano Prefecture director of micro￾implementation DMR Macenta Koura, Guéckédou 94 Saa bernabe TOLNO (Saa Michel Millimouno) Agriculture Engineer (Head of financial resources) ANASA (DPA, ANPROCA) Mangalan Batico, Guéckédou 95 Felix Faya Kamano Correspondence Guinean Press Agency Macenta Kola, Guéckédou 96 Gnouma CAMARA General Manager Radio Rurale Sandia, Guéckédou 97 Faya Moussa II KAMANO Bilingual Regional Correspondent Evasion group in Guinea Sandia, Guéckédou 98 Gnouma Bongo no Head of Administration Service Town Hall Sandia, Guékédou 99 Sekou Keita National Agency for Agricultural Statistics (NAAS) Almamya, Kaloum 100 Dioubaté Souleymane Program Director Kankan Rural Radio Station Kankan 101 Kouyaté Ayoub Spokesperson DjélyTômba (association of artists and storytellers) Kankan 102 Sylé CAMARA Head of Administrative and Finance Services Regional Office for Health in Kankan Dalako, Kankan 103 Amiata KABA Vice President, Special Delegation Municipality of Kankan Energy, Kankan I–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 104 Mohamed Mariame KEITA Head of Children Affairs section District Directorate for Social, Feminine Promotion and Children Affairs Farako, Kankan 105 Aziz Diop (General Yaya Conde) Prefect (Secretary General) Prefecture Gare, Kankan 106 Dr Souleymane Camara Data Manager Regional Health Directorate Lako, Kankan 107 Bouneyamine Traore Spokesperson Regional Islamic League/ President of the Renacode Meteo, Kankan 108 Lounceny Chérif President Civil Society Kankan (CNOCKS) Sogbe, Kankan 109 Dr. Mory Togba DPS Prefectural Health Directorate, Kindia Kindia 110 Dr. Bakary Conde Hospital Director Regional Hospital Kindia Kindia 111 Kabinet Diawara Secretary General of Decentralized Communities Kindia Prefecture Government Kindia 112 Dr MoryTogba Prefectural Director of Health Prefectural Directorate of Health Kindia 113 Sekouba NANSOCO (Alkaly Traore) Director, Micro-realization (Director of Planning) Prefectoral development department Kindia 114 Dr Faroukou Dansoko Regional Director of Kindia Ministry of Health Kindia 115 Dr Togba Mory Prefectural Director of Health Prefectoral Directorate of Health Abatoir, Kindia 116 Aboubacar Thomas Camara Co-Responsible of Sponsorship Child Fund Condetta, Kindia 117 Karamoko Damba Manager SAREF Friguiagbé, Kindia 118 Aboubacar Sherif Coordinator of the union Group of Satakhui planters Gnougouya, Kindia 119 Dr Alpha Ousmane Barry (Ismatou Diallo) Head of Health Center (Focal Point of Prevention & Mother and Child Transmission) Manquepas Health Center Manquepas, Kindia 120 Mamadi Meme Camara (Sita Mamoudou Conde) President of Regional Council on CSOs (Administrative Secretary of Regional Council of CSOs) Organization of Kindia Civil Society Manquepas, Kindia 121 Fode Moussa Sayon CAMARA In charge of children issues Prefectural department of Social action and women promotion Manquepas, Kindia 122 Hadja Aissatou Savané General Secretary in charge of Administration Prefecture of Kindia Manquepas, Kindia 123 Marie Josephine DORE General Secretary City Council Manquepas, Kindia 124 Moussa Camara President Federation of Planters of Fruit in Lower Guinea Rural Friguiagbé, Kindia 125 Dr. Diallo Mamadou Hady DPS Prefecture Health Directorate of Labé Kouroula, Labé 126 Kadiatou Bailo SOUMANO Prefectoral Director of Social Action Department of Social Action Labé 127 Dr Abdoulaye Tounkoura Balde Head of Prevention Services Regional Directorate of Health Kouroula, Labé 128 Sadou Keita Governor of Region Labé Administration Kouroula, Labé A N N E X F. D O C U M E N T S C O N S U LT E D n I–7 1. Key Informant Interviews No Name Designation Organization Location of Interview 129 Lanciné Sangare Secretary General in charge of Decentralized Communities Prefectoral Government of Labé Kouroula, Labé 130 Dr Abdoulaye Ibrahima DIALLO Head of Prevention and Disease Control Section Prefectural Health Directorate of Labé Kouroula, Labé 131 Ismael DIENG Vice-President AT TADRIS WADDA WATI LALLAHI Kouroula, Labé 132 BARRY Boubacar UGAL Chairman Farmers Union Kouroula, Labé 133 Dr. Kaba Keita Hospital Director Prefectoral Hospital of Lola Cote Koly, Lola 134 Louopou Traore (Foromo Kolamou) Prefectoral Director of Social Action (In charge of children issues) Department of Social Action Beyla-Tigbemo 2, Lola 135 Bakary Komara General Secretary in charge of Administration Prefecture of Lola Cote Koly 2, Lola 136 Togba Gbamou President Association of Young Graduates and Friends of Lola Cote Koly 2, Lola 137 Dr. Madjou Diallo Deputy Director/Hospital DPS Woroyapo, Lola 138 Jean President of Association Association of Victims and Survivors of Ebola Macenta 139 Abel Woloba Guilavogui Mediator Post Ebogui Macenta 140 Karifa Camara (Bakary Camara) President of Macenta Koura District (Member, Brother) Community Office (Office of the District) Macenta 141 Rene Cece Sagno Director Radio Station Macenta 142 Dr Mamadou Saidou Camara Head of Health Center Hermakono Health Center Bouse, Macenta 143 Dr. Bangaly Doumbouya Deputy Director Hospital Prefecture Hospital Kamandou Kamandou, Macenta 144 Gomo Maleyogui (Dr. Sekou Keita, Bakary Camara, Emmanuel Koropogui, Oye Beavogui) Responsible for Cold Chain (Responsible for Enlarged Vaccination Program, Responsible for Leprosy and TB Program, Surveillance Focal Point, Bureau of Leprosy and TB Program) DPS Bowa, Macenta 145 Marcel Koiffi Onivogui Chairman of the Association Youths Association Kamandou Cité, Macenta 146 Patricia Akoi Mara (Dobo Beavogui) Director (Assistant) Prefectoral Direction of Social Action Zezebokone, Macenta 147 Abdoulaye Mory Conde General Secretary of Decentralization Civil Servant Zezebokone, Macenta 148 Dr. Diallo Mamadou Regional Hospital Director Faranah DPS Marché, Faranah 149 Dr. Adama Baba (Dr. Mathieu Loua) Health Regional Director (In Charge of Disease Control) Health Regional Directorate (DRS) Goniau, N'Zérékoré 150 Elhadj Sékou Kourouma Imam Mosque Bomani, N'Zérékoré 151 Jean Marie Guemou Priest, Coordinator of regional catholic schools Catholic Church Dorota, N'Zérékoré 152 Pierre Lancei SIDIBE Chief of Quartier Quartier Nya Sokoura II, N'Zérékoré I–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 153 Ibrahima Soumahoro Chief of Cultural Section Radio Zaly FM Liberte N'Zérékoré 154 Dr Zoba Guilavogui Deputy Director Regional Hospital N'Zérékoré 155 Michel Camara Head of Health Center Commercial Health Center N'Zérékoré 156 Pé Mamadi Bamy (Saa Dimio Sandouno) Mayor (District Secretary General) District Government N'Zérékoré 157 Christophe Millimono Director Regional Rural Radio of Forest Guinea N'Zérékoré 158 Dr Mathieu Loua Acting Regional Director of Health Ministry of Health N'Zérékoré 159 Soromou Gononan Prefectural Director of Agriculture Ministry of Agriculture N'Zérékoré 160 Elhadj Sékou Kaba Coordinating President Coordination the Malinke people of Nzerekore N'Zérékoré 161 Moussa Fofana District Director of Social Action District Directorate for Social Action for Women and Children's Advancement N'Zérékoré 162 Florentin Lamah 2nd Vice President of the Federation of Coffee-Cocoa Planters Federation of Coffee-Cocoa Planters Ossud, N'Zérékoré 163 Dr. Zobe Guilavogui Deputy Director Regional Hospital Quartier Commercial, N'Zérékoré 164 Benoit Thea Farmer Association of farmers Samoé, N'Zérékoré 165 Charles Kpogomou Technical Coordinator PRIDE Guinea Samoé, N'Zérékoré 166 Molou Foromo Zogbélén President Coordination of Forest Guinea, N’Zérékoré Bellevue, N’Zérékoré Liberia 1 Kowah Flomo District Health Officer Ministry of Health Sinje, Bomi 2 Joseph Andrews Community Group Leader Tubmanburg, Bomi 3 Z Edmond Greaves Ministry of Agriculture Tubmanburg, Bomi 4 Kieh W. Wisseh Principal C.H Dewy Central High School Tubmanburg, Bomi 5 Paul K Tyes Farmer Tubmanburg, Bomi 6 Boakai Karney District Health Officer Ministry of Health Tubmanburg, Bomi 7 Mr. Seo Davis County Education Officer Ministry of Education Tubmanburg, Bomi 8 Madam Jagani Nile BRAC Tubmanburg, Bomi 9 Mr. Samuel Y. Koenig County Education Officer Ministry of Education Bong 10 Kortoe D Woloquelli Agriculturist Gbarnga, Bong 11 Obe G Smith Radio Gbarnga Gbarnga, Bong 12 Stephen Matthew Ministry of Agriculture Gbarnga, Bong 13 Livingstone Mulubah Proprietor Ma Mary and Sons business center Gbarnga, Bong 14 Sianeh Y. Diabol LNRCS Gbarnga, Bong 15 Leon T. Harris DEN-L Gbarnga, Bong ANNEX I. INTERVIEWS AND DISCUSSIONS n I–9 1. Key Informant Interviews No Name Designation Organization Location of Interview 16 Mr. Dormea A. Jue District Education Officer Ministry of Education Gbarnga, Bong 17 Henry Sackey District Health Officer Ministry of Health Suakoko, Bong 18 Joseph G. Garteh District Health Officer Ministry of Health Zota, Bong 19 Joyce Korveh County Agriculture Coordinator Ministry of Agriculture Buchanan, Grand Bassa 20 Nancy Bryant Chairlady Rise and Shine Multi-Purpose Cooperative Society Buchanan, Grand Bassa 21 James Z Zekie Community Leader Buchanan, Grand Bassa 22 Koboi Sirmenee District Health Officer Ministry of Health Buchanan, Grand Bassa 23 Yassah V. S. Sumo District Health Officer Ministry of Health Buchanan, Grand Bassa 24 Morris B Zorbah Bassa Community College Buchanan, Grand Bassa 25 Joseph S Venbo Farmer Buchanan, Grand Bassa 26 Varlee S Kamara District Health Officer Ministry of Health Damaballa, Grand Cape Mount 27 Dwight Harvey County Education Officer, Grand Cape Mount county Ministry of Education Grand Cape Mount 28 Oretha Koffa Businesswoman/EVD Survivor Kru Town, Robertsport, Grand Cape Mount 29 Saah Hudson Reporter Peace FM Robertsport, Grand Cape Mount 30 Charles W. Singbe Educator Robertsport high school Robertsport, Grand Cape Mount 31 Maima Kamara Farmer Robertsport, Grand Cape Mount 32 Ben Sabinah Executive Director CHAP Robertsport, Grand Cape Mount 33 Austin V. Markel Chairman Rural Community Empowerment Robertsport, Grand Cape Mount 34 Flomo K Konie District Health Officer Ministry of Health Sinje, Grand Cape Mount 35 Elibe B. Koroma District Education Officer Ministry of Education Tewor District, Grand Cape Mount 36 Watta Borbor District Health Officer Ministry of Health Kakata, Margibi 37 Sis Musu Tuahyonn Ministry of Agriculture Kakata, Margibi 38 Danileatte D. Asiton Office of the county Supt. Ministry of Gender Kakata, Margibi 39 Mama J. K. Philip Reproductive Health Supervisor Reproductive Health Group Kakata, Margibi 40 Eric Sumogode Administrative Officer BRAC Kakata, Margibi 41 Gorma Minnie County Education Officer Ministry of Education Kakata, Margibi 42 Amanda W Zota District Education Officer Ministry of Education Kakata, Margibi 43 Joseph Saah District Health Officer Ministry of Health Unification, Margibi 44 Josephine Goodlin Community Health Worker City Clinic Bundu Kakata, Margibi 45 Joshua Sehkehporh Principal School for the Deaf Central Virginia, Montserrado 46 Garisin Turry Principal Gbanjor Public School Gbanjor, Montserrado I–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 47 Cecelia Reeves County Education Officer Ministry of Education Greater Monrovia, Montserrado 48 David Bikaba (Edwin Rogers) WASH Officer Monrovia, Montserrado 49 Paul Wolstenholme Chief of Party Tetra Tech Monrovia, Montserrado 50 Rose Macauley (Yvonne Kodl) Chief of Party (DCOP) JSI Monrovia, Montserrado 51 Chea Sanford Wesseh Assistant Minister for Vital and Health Statistics MOH, GOL Monrovia, Montserrado 52 Prince Allison Reporter Red Power Radio Monrovia, Montserrado 53 Kenneth Martin Community Safety Initiative Monrovia, Montserrado 54 Lincoln H Nagbe District Health Officer, St. Paul Ministry of Health Monrovia, Montserrado 55 Gbemie Teta Horace-Kollie Deputy Minister of Operations Ministry of Internal Affairs Monrovia, Montserrado 56 Anne Fiedler (2 unnamed additional respondents) COP (IPC Trainers) Jhpiego Monrovia, Montserrado 57 Augustine M. Kpahe District Health Officer Ministry of Health Monrovia, Montserrado 58 Ma Amie Subah EVD Survivor Monrovia, Montserrado 59 Pewee S. Flomoku Chief of Party Carter Center Monrovia, Montserrado 60 Bud Randall Acting Chief of Party Management Sciences for Health Monrovia, Montserrado 61 Michael Musili (Aron) Deputy Director of WFP (M&E Specialist) WFP Monrovia, Montserrado 62 Vicki Cooper Chief of Party Digital Liberia Montserrado 63 Madama Kamara Community Health Worker Montserrado 64 Amos G E Zeon Ministry of Agriculture Montserrado 65 Cleopatra Gibson Planning Officer Ministry of Agriculture, Dept of Planning and Development Montserrado 66 Del Francis Wreh Executive Director LIMPAC, Ministry of Finance and Development Planning Montserrado 67 Hon. Felicia Sumah Assistant Minister for Instruction Ministry of Education Montserrado 68 Aji James Kalou Deputy Director for Operation eHealth Africa Montserrado 69 Pastor Robert Oberly Coordinator EVD Survivors Network Paynesville, Montserrado 70 Mr. Kolubah H. Flomo District Education Officer Ministry of Education Paynesville, Montserrado 71 Ma Watta Jabiteh EVD Survivor Pregina Paynesville, Montserrado 72 Samuka Konneh Capacity Building Development Specialist IREX Sinkor, Montserrado 73 Harold Aldoo Executive Director Institute for research and democratic development Sinkor, Montserrado 74 Philip S Bestman West Point Saving Group Network West Point, Montserrado 75 Amos M. Constance Governance Officer Office of the Commissioner West Point, Montserrado ANNEX I. INTERVIEWS AND DISCUSSIONS n I–11 1. Key Informant Interviews No Name Designation Organization Location of Interview 76 Ruth Zansi Administrator Hope foster family the nation Bain-gaa, Nimba 77 N Samuel Keheay Ministry of Agriculture Ganta, Nimba 78 Aaron Glay District Health Officer Ministry of Health Gbehley-Geh, Nimba 79 Darius Dan Wehyee ESSPD Nimba 80 Mr. Moses S. Dologbay County Education Officer Ministry of Education Nimba 81 Cooper Karnue District Health Officer Ministry of Health Saclepea, Nimba 82 Yaah Bellah Suah Rural Women & Girls Pronestor Sanniquellie, Nimba 83 Sarah Layweh Community Health Worker County Health Team Sanniquellie, Nimba 84 Mr. Johnson G. Korto District Education Officer Ministry of Education Sanniquellie, Nimba 85 Moses F. Momolu District Health Officer Ministry of Health Foya, Lofa 86 Mr. Lee W. Dahn District Education Officer Ministry of Education Kolahun, Lofa 87 David K. Sumo District Health Officer Ministry of Health Voinjama, Lofa 88 Nyan Twayen Sr. County Education Officer Ministry of Education Voinjama, Lofa 89 Comfort K. Sumo Community Health Worker Lofa Health Team Voinjama, Lofa 90 Tokpa Tarnue Radio ELBC Voinjama, Lofa 91 Henry Burkdleh Ministry of Agriculture Voinjama, Lofa Sierra Leone 1 Mohammed Kamara Survivor, Cash Recipient N/A Bombali 2 Moses Kargbo Asst. Program Director World Hope International Makeni, Bombali 3 Alpha J. Koroma District Agriculture Officer MAFFS Makeni, Bombali 4 Shaku Kamara Trader Union Makeni, Bombali 5 Mohamed Sankoh Program Officer Action Aid Makeni, Bombali 6 Alice James Regional Manager Action Aid Makeni, Bombali 7 Amadu Bundu Conteh District Disease Surveillance Officer Ministry of Health & Sanitation Makeni, Bombali 8 Emil Jengo Radio Maria FM 101.1 Makeni, Bombali 9 Alie A Fofanah District M&E Officer Ministry of Agriculture Forestry and Food Security Makeni, Bombali 10 Ibrahim Alpha Turay Project Officer MADAM Makeni, Bombali 11 John Shanghai Kamara District Council Chairman Bombali District Council Makeni, Bombali 12 Sheku Samuel Kamara District Council Chair GOSL Kabala, Koinadugu 13 Korefa Koroma Survivor N/A Kabala, Koinadugu 14 Dr. Adrian Hanciles (Sannie Samura) Medical Officer/ Superintendent MOHS Kabala, Koinadugu 15 Mohamed Sesay (Mohamed Amara Kamara) District Extension Officer MAFFS Kabala, Koinadugu 16 Chief Abubakar Sherriff Traditional Leader N/A Koinadugu 17 Alhaji Babar Kamara Bintumarin Radio Station Koinadugu 18 Yusufu Jarloh Town Chief Kumala, Koinadugu 19 Samuel Alvan Browne Station Manager Bintumani FM Radio 97.3 Yogomaia, Koinadugu I–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 20 Mohamed Koroma EVD Survivor Yogomaia, Koinadugu 21 Yusifu Kamara Deputy Director of Education Ministry of Education, Science and Technology Magburaka, Tonkolili 22 Mr. Ibrahim Koroma Student Magbekor, Tonkolili 23 Mohamed I. Turay Student Magburaka, Tonkolili 24 Alhaji Sankoh Farmer Survivor Male Magburaka, Tonkolili 25 Dr. Augustine Sahr Jimmisa District Medical Officer Ministry of Health and Sanitation Magburaka, Tonkolili 26 Augustine Bockarie Program Manager RODA Makeni, Tonkolili 27 Abdul Richard Kamara District Chairman Sierra Leone Association for Ebola Survivors in Tonkolili Tonkolili 28 Gbawuru Sillah Chief Administrator Tonkolili District Council Tonkolili 29 Isatu Kamara N/A N/A Yele, Tonkolili 30 Lahai Koroma Sustainable Empowerment for the People of Koinadugu Yogomaia, Tonkolili 31 Rev. Gibrilla Kamara District Superintendent/ Chairman Council WCSL, Kenema Council of Churches Kenema 32 Zainab Musa Recipient N/A Kenema 33 Dr Prince Masuba Physician & Medical Superintendent Kenema District Government Hospital Kenema 34 Sheik Mohamed Zaki Waganneh Religious Leader Muslim Kenema 35 Bockarie Buanie Chief Administrator Kenema District Council Kenema 36 Mohamed SK Sesay Chairperson Sierra Leone Ebola Survivors Association in Kenema Kenema 37 Almamy Kargbo District Agricultural Officer MAFFS Kenema 38 Gerald Amara Sama District Agricultural Officer Ministry of Agriculture Kailahun 39 Kolleh Michael Fartoma Assistant District Social Welfare Officer Ministry of Social Welfare Kailahun 40 Tenneh Koroma Recipient N/A Kailahun 41 Hawanatu Koroma (Jabbie M. Luseine) Sector Head (Social Worker) Ministry of Social Welfare, Children and Gender Affairs Kailahun 42 Adu G Lamin Principal Methodist Junior Secondary School Kailahun 43 Jane Turay Matron Kailahun Government Hospital Kailahun 44 Foday Konteh Deputy Director of Education Ministry of Education, Science and Technology Kailahun 45 Momoh Ensah Tailor Self employed Kailahun 46 Fudia Sesay Recipient N/A Kailahun 47 Momoh Musa Mambu Recipient N/A Kailahun 48 Christian Joel Benjamin News Editor Sierra Leone Broadcasting cooperation (SLBC) Kailahun 49 Alimany Fannah Sankoh Chairman Chief Ruling house Port Loko 50 Mohamed Conteh Farmer/Fisherman N/A Maikanu, Port Loko ANNEX I. INTERVIEWS AND DISCUSSIONS n I–13 1. Key Informant Interviews No Name Designation Organization Location of Interview 51 Ahmid Munirr Chairman Port Loko District Council Port Loko 52 Chief Ya-Alimamy Bangura Deputy Chairlady Sierra Leone Women Farmers Forum Port Loko 53 Dr Tom Sesay District Medical Officer Port Loko District Council Port Loko 54 Alhaji Hassan Sesay (Jinnah Bockarie) M&E Officer (District Crop Officer) Ministry of Agriculture Forestry and Food Security Port Loko 55 Mr. Issa Kabba Manager Sierra Leone Community Bank Port Loko 56 Alhaji Alusine Kamara Chief Imam Allen Town Mosque Allen Town, Western Area Urban 57 Dr. Clifford Kamara Government of Sierra Leone Fouray Bay College, Western Area Urban 58 Kinday Ndella Samba Deputy Country Director WFP Freetown, Western Area Urban 59 Dauda Wurie Coordinator, Water Sector Presidential Delivery Team on Transition and Recovery Freetown, Western Area Urban 60 Janna de Jong Grant and Reporting Officer Action Against Hunger Freetown, Western Area Urban 61 Dr. Aurelien Pekezou Sr. Public Health Coordinator IOM Freetown, Western Area Urban 62 Kshitji Joshi Chief Communication for Development UNICEF Freetown, Western Area Urban 63 Veronica Avila Livelihood Advisor CARE International Freetown, Western Area Urban 64 Anna Fraenzel Country Director GOAL Sierra Leone Country Office Freetown, Western Area Urban 65 Francis Kyobe Program Manager Catholic Relief Services Freetown, Western Area Urban 66 Sheku J. Kamara Presidential Adviser, Strategy & Research Office of the President, Strategy and Policy Unit, GoSL Freetown, Western Area Urban 67 Edward Foday Research and Publication Lead Directorate for Policy Planning MoHS Freetown, Western Area Urban 68 Grace Nancy Dumbuya Gender and Women's Health Officer Health for All Coalition Freetown, Western Area Urban 69 Laurentiu Stan Chief of Party JSI Freetown, Western Area Urban 70 Marc Cunningham Monitoring and Evaluation JSI Freetown, Western Area Urban 71 Samuel A.S Kargbo Director, Planning, Policy and Information Ministry of Health and Sanitation Freetown, Western Area Urban 72 Juan Diez Bertran Senior Health Advisor Save the Children Freetown, Western Area Urban 73 Musa Ansumana Soko Director WASH-Net Freetown, Western Area Urban 74 Idriss Toure Director, Social Protection National Commission for Social Action Freetown, Western Area Urban I–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 1. Key Informant Interviews No Name Designation Organization Location of Interview 75 Tamba Bandagagba Head Rural Water Services Ministry of Water Resources Freetown, Western Area Urban 76 Dr. SAS Kargbo Ministry of Health and Sanitation Freetown, Western Area Urban 77 Suliamon Conteh Program Manager for Reproductive Health and Family Planning MoHS Freetown, Western Area Urban 78 Dr Asunmana Director, Environmental Health Dept MOHS Freetown, Western Area Urban 79 Dalton Communications Officer Health Alert Freetown, Western Area Urban 80 Alison Jenkins Deputy Representative UNICEF Freetown, Western Area Urban 81 Hassan Kamara Chairperson SLAES Jui, Western Area Urban 82 Binti Bangura Senior Matron MOHS Rokupa, Western Area Urban 83 Amadu A. Juana Medical Officer Ministry of Health & Sanitation Rokupa, Western Area Urban 84 Ibrahim Sessay Program Manager Campaign for Good Governance Tengbe Town, Western Area Urban 85 Dr Alieu Wurrie Ministry of Health and Sanitation Western Area Urban 86 Harry Kpangai Medicos Western Area Urban 87 Joe Ben-Davis SALWACO Western Area Urban 88 Dr Joseph Kandeh PHC (MHS) Western Area Urban 89 Aiah Tholley MAFFS Western Area Urban 90 Nanah Bockarie MEST Western Area Urban 91 John Lasher Executive Director, PIH SL Partners In Health Western Area Urban 92 Salamatu Koroma MEST Western Area Urban 93 Davidson Jonah Child Fund Waterloo, Western Area Urban 94 Alhassan Yillah Development & Planning Officer Western Area Rural District Council Waterloo, Western Area Rural 95 Market Joe (Unnamed Market Worker) Market worker (Market Worker and EVD survivor) Western Area Rural 96 Kabbah Kamara Recipient Okada Rider Western Area Rural 97 Dr David Joefrey Koroma Physician/Deputy Medical Officer Adventist Health System Waterloo Hospital Western Area Rural ANNEX I. INTERVIEWS AND DISCUSSIONS n I–15 2. Focus Group Discussions No Type of Group Respondent Demographics Location Guinea 1 Beneficiaries – Recipients Young Women Beyla 2 Beneficiaries – Recipients Adult Male Beneficiaries Keme Bourema, Beyla 3 Beneficiaries – Recipients Adult Female Beneficiaries Beyla 4 Beneficiaries – Recipients Young Male Beneficiaries Keme Bourema, Beyla 5 Beneficiaries – Service Providers Hospital Boffa 6 Beneficiaries – Recipients Community Beneficiaries, Adult Men Boké 7 Beneficiaries – Recipients Adult Women Tamaranssy, Boké 8 Beneficiaries – Recipients Young Male Beneficiaries Boké 9 Beneficiaries – Recipients EVD Survivors/Affected, Young Women Boké 10 Beneficiaries - Service Providers Hospital Boké 11 Beneficiaries – Recipients Farmers Associations Boffa, Boké 12 Beneficiaries – Recipients Adult Male Farmers Boffa, Boké 13 Beneficiaries – Recipients Women Farmers Boffa, Boké 14 Beneficiaries – Recipients Young Female Farmer Beneficiaries Boffa, Boké 15 USG Stakeholders USAID Conakry 16 Beneficiaries – Service Providers APERGUE Conakry 17 Beneficiaries - Recipients EVD Survivors Conakry 18 Beneficiaries – Service Providers Communal Health Center of Ratoma Conakry 19 Beneficiaries – Service Providers Health Care Workers & 2 CSO members Matam, Conakry 20 Beneficiaries - Recipients Community Beneficiaries Matam, Conakry 21 Beneficiaries - Recipients Community Beneficiaries Matam, Conakry 22 Beneficiaries - Recipients EVD Survivors Matam, Conakry 23 Beneficiaries – Recipients Community Beneficiaries Forécariah, Kindia 24 Beneficiaries – Recipients Community Beneficiaries Forécariah, Kindia 25 Beneficiaries – Service Providers Hospital Forécariah, Kindia 26 Beneficiaries – Recipients Young Men Under 30 Forécariah, Kindia 27 Beneficiaries – Recipients EVD Survivors Forécariah, Kindia 28 Beneficiaries – Recipients Male Survivors Forécariah, Kindia 29 Beneficiaries – Recipients Community Beneficiaries Forécariah, Kindia 30 Beneficiaries – Recipients Community Beneficiaries Forécariah, Kindia 31 Beneficiaries – Recipients EVD Survivors Forécariah, Kindia 32 Beneficiaries – Service Providers Hospital Forécariah, Kindia 33 Beneficiaries – Recipients Small-Scale Traders Friguiagbé, Kindia 34 Beneficiaries – Recipients Adult Women Beneficiaries, Farmers/ Traders Friguiagbé, Kindia 35 Beneficiaries – Recipients Young Men under 30 Friguiagbé, Kindia 36 Beneficiaries – Recipients Adult Men Friguiagbé, Kindia 37 Beneficiaries – Recipients Women under 30 Friguiagbé, Kindia I–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 2. Focus Group Discussions No Type of Group Respondent Demographics Location 38 CSOs/CBOs Radio, CBO, Community Health Kindia 39 Beneficiaries – Recipients EVD Survivors/Farmers Kindia 40 Beneficiaries – Service Providers Hospital Kindia 41 Beneficiaries – Service Providers Hospital Kamandou 42 Beneficiaries – Recipients Adult Women Faranah 43 Beneficiaries – Recipients Young Women Faranah 44 Beneficiaries – Recipients Young Men Faranah 45 Beneficiaries – Service Providers Hospital Faranah 46 Beneficiaries – Recipients Adult Male Beneficiaries Market 1, Faranah 47 Beneficiaries – Service Providers Hospital Beyla, N’Zérékoré 48 Beneficiaries – Recipients Young Men Guéckédou, N’Zérékoré 49 Beneficiaries – Service Providers Adult Men Beneficiaries Guéckédou, N’Zérékoré 50 Beneficiaries – Recipients Adult Men EVD Survivors Guéckédou, N'Zérékoré 51 Beneficiaries – Service Providers Health Providers Lola, N’Zérékoré 52 Beneficiaries – Recipients Adult Male Beneficiaries Macenta, N’Zérékoré 53 Beneficiaries – Service Providers NGO Service Providers Macenta, N’Zérékoré 54 Beneficiaries – Recipients Adult Men N’Zérékoré 55 Beneficiaries – Recipients EVD Survivors N’Zérékoré 56 Beneficiaries – Recipients Adult Women Survivors and Affected N’Zérékoré 57 Beneficiaries – Recipients Affected Adult Women N’Zérékoré 58 Beneficiaries – Recipients EVD Survivors/Affected, Adult Men N’Zérékoré 59 Beneficiaries – Recipients Young Male Survivors N'Zérékoré 60 Beneficiaries – Recipients Adult Women N’Zérékoré, Guéckédou 61 Beneficiaries – Service Providers Hospital N’Zérékoré, Guéckédou 62 Beneficiaries – Recipients Community Beneficiaries Kouroula, Labé 63 Beneficiaries – Recipients Community Beneficiaries Kouroula, Labé 64 Beneficiaries – Recipients Community Beneficiaries Kouroula, Labé 65 Beneficiaries – Recipients Men under 30 Kouroula, Labé 66 Beneficiaries – Recipients Adult Men Beneficiaries Labé 67 Beneficiaries – Recipients Young Male Beneficiaries Kankan 68 Beneficiaries – Recipients Young Female Beneficiaries Kankan 69 Beneficiaries – Recipients Adult Men Kankan 70 Beneficiaries – Recipients Adult Women Beneficiaries Kankan 71 Beneficiaries – Recipients Adult Female Beneficiaries Kankan 72 Beneficiaries – Recipients Young Women's Group Macenta 73 Beneficiaries – Recipients Young Adult Women's Group Macenta 74 Beneficiaries – Recipients Association of EVD Survivors and Affected, Male Group Macenta 75 Beneficiaries – Recipients Association of EVD Survivors and Affected, Female Group Macenta ANNEX I. INTERVIEWS AND DISCUSSIONS n I–17 2. Focus Group Discussions No Type of Group Respondent Demographics Location 76 Beneficiaries – Service Providers Local Health Providers Macenta 77 Beneficiaries – Recipients EVD Survivors/Beneficiaries, Young Men Macenta 78 Beneficiaries – Recipients Association of Survivors/Affected, Adult Women Koura, Macenta, Guéckédou 79 Beneficiaries – Recipients Young Female Beneficiaries Guéckédou 80 CSOs/CBOs CSOs, rural radio, health service providers Guéckédou 81 Beneficiaries – Recipients Male Survivors and Affected Lola 82 Beneficiaries – Recipients Adult Women Community Group Lola 83 Beneficiaries – Recipients Young Female Beneficiaries Lola 84 Beneficiaries – Recipients Community Beneficiaries Lola 85 Beneficiaries – Service Providers Hospital Lola, N’Zérékoré 86 Beneficiaries – Recipients Adult Male Beneficiaries Cote Koly 2, Lola, N’Zérékoré 87 Beneficiaries – Recipients Young Male Beneficiaries Cote Koly 2, Lola 88 Beneficiaries – Recipients Female EVD Survivors Lola, N’Zérékoré Liberia 1 Beneficiaries – Recipients Young Male Survivors Buchanan, Grand Bassa 2 Beneficiaries – Recipients Farmers Buchanan, Grand Bassa 3 Beneficiaries – Recipients Young Female Survivors Buchanan, Grand Bassa 4 Beneficiaries – Recipients Students Buchanan, Grand Bassa 5 Beneficiaries – Recipients Adult Men Beneficiaries Buchanan, Grand Bassa 6 Beneficiaries – Recipients Petty Traders Buchanan, Grand Bassa 7 Beneficiaries – Recipients Male Social Club Members Buchanan, Grand Bassa 8 Beneficiaries – Recipients Adult Female Beneficiaries Buchanan, Grand Bassa 9 Beneficiaries – Recipients Civil Society Group Members Grand Bassa 10 Beneficiaries – Service Providers Community Groups/Farmers Ganta, Nimba 11 Beneficiaries – Service Providers Petty Trade Groups Ganta, Nimba 12 CSOs/CBOs Zankpa Street Community Ganta, Nimba 13 Beneficiaries – Recipients EVD Survivors/Farmers Ganta, Nimba 14 Beneficiaries – Recipients Young Male Survivors Ganta, Nimba 15 Beneficiaries – Recipients Students Ganta, Nimba 16 Beneficiaries – Recipients Adult Men Beneficiaries Ganta, Nimba 17 Beneficiaries – Recipients Adult Women Beneficiaries Ganta, Nimba 18 CSOs/CBOs Rural Women and Men Groups Garnulin, Nimba 19 Beneficiaries – Recipients Petty Trade Organization Gbarnga, Bong 20 Beneficiaries – Recipients Susu Group Gbarnga, Bong 21 Beneficiaries – Recipients Young Female Survivors Gbarnga, Bong 22 Beneficiaries – Recipients Students Gbarnga, Bong 23 Beneficiaries – Recipients EVD Survivors/Farmers Gbarnga, Bong 24 Beneficiaries – Recipients Young Male EVD Survivors Gbarnga, Bong I–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 2. Focus Group Discussions No Type of Group Respondent Demographics Location 25 Beneficiaries – Recipients Adult Male Beneficiaries Totota, Bong 26 Beneficiaries – Recipients Adult Male Beneficiaries Totota, Bong 27 Beneficiaries – Recipients Adult Female Beneficiaries Totota, Bong 28 Beneficiaries – Recipients Women's Group/VSLA Jeneweni, Grand Cape Mount 29 Beneficiaries – Recipients Community Members Jeneweni, Grand Cape Mount 30 Beneficiaries – Recipients Young Male Survivors Jeneweni, Grand Cape Mount 31 CSOs/CBOs Female traders/village savings group Kakata, Margibi 32 Beneficiaries – Recipients Petty Trade Organization Kakata, Margibi 33 Beneficiaries – Recipients Young Men's Group Kakata, Margibi 34 Beneficiaries – Recipients EVD Survivors/Farmers Kakata, Margibi 35 Beneficiaries – Recipients Young Female Survivors Kakata, Margibi 36 Beneficiaries – Recipients Young Male EVD Survivors Kakata, Margibi 37 Beneficiaries – Recipients Students Kakata, Margibi 38 Beneficiaries – Recipients Adult Male Beneficiaries Kakata, Margibi 39 Beneficiaries – Recipients Adult Female Beneficiaries Kakata, Margibi 40 Beneficiaries – Service Providers Community Based Group Brewerville, Montserrado 41 Beneficiaries – Recipients Adult Male Beneficiaries Lower, Montserrado 42 Beneficiaries – Recipients Adult Female Beneficiaries Lower, Montserrado 43 Beneficiaries – Recipients Young Female Beneficiaries Lower, Montserrado 44 Beneficiaries – Recipients Petty Traders Monrovia, Montserrado 45 Beneficiaries – Recipients Community Members Monrovia, Montserrado 46 Beneficiaries – Recipients Students' Health Club Monrovia, Montserrado 47 Beneficiaries – Recipients EVD Survivors/Farmers Monrovia, Montserrado 48 Beneficiaries – Recipients EVD Survivors/Farmers Monrovia, Montserrado 49 Beneficiaries – Recipients Social Club Members Monrovia, Montserrado 50 Beneficiaries – Recipients Social club members/Susu Group Monrovia, Montserrado 51 Beneficiaries – Recipients Petty Traders Monrovia, Montserrado 52 Beneficiaries – Recipients Young Male Survivors Monrovia, Montserrado 53 Beneficiaries – Recipients Students, Members of 4H Montserrado 54 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors Paynesville, Montserrado 55 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors Paynesville, Montserrado 56 Beneficiaries – Recipients Adult Men Beneficiaries Porkaph, Grand Cape Mount 57 Beneficiaries – Recipients Adult Female Beneficiaries Porkaph, Grand Cape Mount 58 Beneficiaries – Recipients Farmers Robertsport, Grand Cape Mount 59 Beneficiaries – Recipients Students Robertsport, Grand Cape Mount 60 Beneficiaries – Recipients Community Beneficiaries Tubmanburg, Bomi 61 Beneficiaries – Recipients Young Male Survivors Tubmanburg, Bomi 62 Beneficiaries – Recipients Petty Traders Tubmanburg, Bomi 63 Beneficiaries – Recipients Farmers Tubmanburg, Bomi ANNEX I. INTERVIEWS AND DISCUSSIONS n I–19 2. Focus Group Discussions No Type of Group Respondent Demographics Location 64 Beneficiaries – Recipients Students age 17-22 Tubmanburg, Bomi 65 Beneficiaries – Recipients Adult Female Beneficiaries Tubmanburg, Bomi 66 Beneficiaries – Recipients Young Female Survivors Voinjama, Lofa 67 Beneficiaries – Service Providers Community Based Group Voinjama, Lofa 68 Beneficiaries – Recipients Young Male EVD Survivors Voinjama, Lofa 69 Beneficiaries – Recipients Students Voinjama, Lofa 70 Beneficiaries – Recipients Social Club Members Voinjama, Lofa 71 Beneficiaries – Recipients Farmers Voinjama, Lofa 72 Beneficiaries – Recipients Petty Traders Voinjama, Lofa 73 Beneficiaries – Recipients Adult Female Beneficiaries Voinjama, Lofa 74 Beneficiaries – Recipients Adult Men EVD Survivors Voinjama, Lofa Sierra Leone 1 Beneficiaries – Service Providers Health Service Providers Makeni, Bombali 2 Beneficiaries – Recipients Community Beneficiaries Bombali 3 Beneficiaries – Recipients EVD Survivors, Cash Recipients - Male Makeni, Bombali 4 Beneficiaries – Recipients Young Female Survivor Cash Recipients Makeni, Bombali 5 Beneficiaries – Service Providers Health and Agriculture Service Providers Makeni, Bombali 6 Beneficiaries – Recipients Farmers/Traders from Sierra Leone Ebola Survivors Association (SLAES) Patebana Marank, Bombali 7 Beneficiaries – Recipients Farmers/Traders from Sierra Leone Ebola Survivors Association (SLAES) Patebana Marank, Bombali 8 Beneficiaries – Recipients Ebola Survivors/Farmers Patebana Marank, Bombali 9 Beneficiaries – Recipients Ebola Survivors/Farmers Patebana Marank, Bombali 10 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Kailahun 11 Beneficiaries – Service Providers Health Workers Daru, Kailahun 12 Beneficiaries – Recipients Young Ebola Survivors Kailahun 13 Beneficiaries – Recipients Young Ebola Survivors Kailahun 14 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Farmers Kailahun 15 Beneficiaries – Recipients Male Farmers Kailahun 16 Beneficiaries – Recipients Cash Recipients, Farmers and Petty Traders, Beneficiaries of RESSNER Program Luawa Chiefdom, Kailahun 17 Beneficiaries – Service Providers Health Facility/MNCH Koinadugu 18 Beneficiaries – Recipients Young Male Survivor Cash Recipients Kailan, Kumala, Koinadugu 19 Beneficiaries – Recipients Community Beneficiaries Koinadugu 20 Beneficiaries – Service Providers Kabala Government Hospital Koinadugu 21 Beneficiaries – Recipients Ebola Survivors/Farmers Kumala, Koinadugu 22 Beneficiaries – Recipients EVD Survivors Eastern Region, Kenema I–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 2. Focus Group Discussions No Type of Group Respondent Demographics Location 23 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Farmers Kailahun, Kenema 24 Beneficiaries – Service Providers Kenema Govt Hospital Kenema 25 Beneficiaries – Service Providers Ministry of Agriculture Kenema 26 Beneficiaries – Service Providers Kenema Govt Hospital Kenema 27 Beneficiaries – Recipients Members from Sierra Leone Association of Ebola Survivors Nongowa, Kenema 28 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Port Loko 29 Beneficiaries – Recipients Ebola Survivors/Farmers Port Loko 30 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Port Loko 31 Beneficiaries – Service Providers WASH Services Providers Port Loko 32 Beneficiaries – Service Providers Health Workers Port Loko 33 Beneficiaries – Recipients Young Female Survivor Cash Recipients Port Loko 34 Beneficiaries – Recipients Ebola Survivors/Farmers Port Loko 35 Beneficiaries – Recipients EVD Survivor Female Cash Recipients Port Loko town, Tonkolili 36 Beneficiaries – Recipients EVD Survivor Young Female Cash Recipients Magburaka, Tonkolili 37 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Magburaka, Tonkolili 38 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Magburaka, Tonkolili 39 Beneficiaries – Service Providers Agriculture Service Providers from Ministry of Agriculture, Concern Worldwide, Red Cross Magburaka, Tonkolili 40 Beneficiaries – Service Providers District Health Medical Team Magburaka, Tonkolili 41 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Farmers Magburaka, Tonkolili 42 Beneficiaries – Recipients Community Beneficiaries: EVD Survivors, Cash Recipients Yele, Tonkolili 43 Beneficiaries – Service Providers Cash Transfer Service Providers from CARITAS Brookfield, Western Area Rural 44 Beneficiaries – Recipients EVD Survivor Female Farmers Kissi Town, Western Area Rural 45 Beneficiaries – Recipients Community Beneficiaries: Cash Recipients Lumpa, Western Area Rural 46 Beneficiaries – Recipients Young Male Cash Recipients LUMPA-55, Western Area Rural 47 Beneficiaries – Recipients Farmers Newton, Western Area Rural 48 Beneficiaries – Recipients Community Beneficiaries: Cash Recipients Waterloo, Western Area Rural 49 Beneficiaries – Service Providers Health Workers Waterloo, Western Area Rural 50 Beneficiaries – Service Providers Agriculture Service Providers and Farmers Waterloo, Western Area Rural 51 Beneficiaries – Service Providers Agriculture Service Providers of Develop Initiative Program (DIP), FHM, and CARITAS Jui, Freetown, Western Area Urban ANNEX I. INTERVIEWS AND DISCUSSIONS n I–21 2. Focus Group Discussions No Type of Group Respondent Demographics Location 52 Beneficiaries – Service Providers Cash Transfer Service Providers from ACF, WHO, German Emergency Drug, WHH, SAZE, GOAL, EFSL Jui, Freetown, Western Area Urban 53 Beneficiaries – Recipients EVD Survivor Female Farmers Jui, Western Area Urban 54 Beneficiaries – Recipients Young Female Ebola Survivors Cash Recipients Jui, Western Area Urban 55 Beneficiaries – Service Providers Health Workers Wellington, Western Area Urban 56 Beneficiaries – Recipients EVD Survivors, Cash Recipients - Female Western Area Urban 57 Beneficiaries – Recipients Young Male Ebola Survivors Western Area Urban 58 IPs/Beneficiaries – Service Providers Staff from PIH, Caritas, IMC, RS, Goal, HCS, IOM, and KIH Western Area Urban I–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX J. ACTIVITY MAPPING TOOL n J–1 ANNEX J. ACTIVITY MAPPING TOOL Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document Legend Missing info Activity has ended FLWS = Frontline Worker Support ICT = Information, Communications, Technology, IE = Institutional Enhancements MCP = Management, Coordination, Partnerships SBCC = Social and Behavioral Change Communication SP = Social Protection Guinea Pillar II Activity Mapping by Location and Duration - Guinea Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Grand Challenge: The Stop Ebola Collective music and media campaign to prevent transmission 3D Family Production Innovation and Communication Technology ICT PPP SBCC $ 268,455 GH Jun-15 Aug-15 2 Health Financing and Governance Abt Associates Governance and Economic Crisis Mitigation Governance MCP $ 1,000,000 Bilateral 6/1/2016 6/1/2018 24 Comprehensive Health Systems Strengthening Activity: supporting governance, financing, and HRH Abt Associates Health Systems Recovery Health MCP $ 5,000,000 GH Jun-16 Jun-18 24 Expert advisors embedded in Guinea MOH HMIS unit for HIS technical assistance Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Innovation and Communication Technology Health ICT $ 500,000 LAB 10/1/2015 12/31/ 2015 2 Increasing the availability and quality of health service delivery data Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Health Systems Recovery Health ICT $ 2,000,000 GH Oct-15 Dec-17 26 Food Vouchers in Fores Region, nutrition messaging, and livelihoods fairs Catholic Relief Services (CRS) Food Security Agriculture & Food Security SP $ 3,253,136 FFP 2/24/2015 12/30/ 2016 22 J–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Pillar II Activity Mapping by Location and Duration - Guinea Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter- vention Type(s) September Obligation FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) Procurement and Supply Chain Management: Inventory Assessment of expired commodities, controlled disposal, coordination and certification Chemonics Health Systems Recovery Health IE $ 1,250,000 GH Jun-17 Dec-17 6 Strengthened Governance for Ebola Recovery in Guinea FHI 360 Governance and Economic Crisis Mitigation Governance MCP $ 10,000,000 Bilateral 6/21/2017 6/20/2020 35 PPPs to support economic recovery, disease surveillance and health systems strengthening within the agriculture and health sectors GOG Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 3,000,000 LAB 10/1/2016 9/30/2018 23 Health Systems Recovery Activity: Health Mobile Clinic component GOG Health Systems Recovery Health $ 5,500,000 Bilateral 3/15/2018 3/14/2019 12 Support participative, credible, and peaceful elections GOG Governance and Economic Crisis Mitigation Governance MCP $ 500,000 Bilateral 3/1/2015 3/31/2017 24 Support credible, legitimate, participatory, transparent, and peaceful elections GOG Governance and Economic Crisis Mitigation Governance MCP $ 1,500,000 Bilateral 2/26/2015 2/28/2018 36 Maternal and Child Health Program (MCSP) - Health Systems Strengthening: ensure essential integrated care package for MNCH is provided with consistency and quality Jhpiego Health Systems Recovery Health IE, MCP $ 7,733,289 GH Jun-16 Sep-17 15 Heath Service Delivery (HSD) - Strengthen health systems with a focus on quality management and supervision at health facilities and at the district level Jhpiego Health Systems Recovery Health IE, FLWS $ 2,750,000 GH Mar-16 Dec-17 21 MCSP Restoration of Health Services (RHS) - restoration of routine health service delivery and strengthening IPC Jhpiego Emergency Non-Ebola Health Services Health MCP, Frontline Worker Support $ 4,000,000 GH Jul-15 Dec-16 17 HC3 Building Resilient Health Systems through SBCC and Initiative for improving the Johns Hopkins CCP Emergency Non-Ebola Health Services Health FLWS, IE, SBCC $ 5,500,000 GH Jul-15 Aug-17 25 ANNEX J. ACTIVITY MAPPING TOOL n J–3 Pillar II Activity Mapping by Location and Duration - Guinea Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) quality and increasing the demand for RMNCH System for Improved Access to Pharmaceuticals and Services (SIAPS) - Restoration and expanding the capability of the public health supply chain MSH Health Systems Recovery Health IE, MCP $ 2,500,000 GH Apr-16 Sep-17 17 Improved Livelihoods and Agriculture Development Program - Partnership with Rio Tinto Simfer SA to support the community of Beyla by providing education and livelihoods development, youth and women Opportunities Industrialization Centers International (OICI) Governance and Economic Crisis Mitigation Economic Crisis Mitigation Social Protection $ 1,500,000 LAB 10/1/2016 9/30/2018 24 Health messaging and community outreach to reduce violence against health workers Plan Guinea, Helen Keller International, RTI, Jhpiego Health Systems Recovery Health SBCC $ 3,482,000 Bilateral 11/7/2014 9/6/2015 8 Program Support Program Support Health Systems Recovery Health MCP $ 288,219 GH Apr-16 Sep-18 29 Rebuilding Together: Support to Local Election Reconciliation and National Unity and Consortium for Elections and Political Process Strengthening (CEPPS) for credible, legitmate, participatory, transparent, and peaceful elections Search For Common Ground ($500k) / CEPPS ($1.5m) Governance and Economic Crisis Mitigation Governance MCP, SBCC and SP $ 2,000,000 Bilateral 8/15/2015 2/28/2018 30 Recovery school feeding in EVD-affected areas World Food Programme Food Security Agriculture & Food Security Social Protection $ 7,182,907 FFP 3/25/2015 12/31/ 2016 21 Non-Ebola Health Monitoring and Evaluation: collect data, examine country strategies, provide support to more effectively define causal pathways and conduct end-term evaluations World Vision, Inc. Emergency Non-Ebola Health Services Health MCP $ 600,820 GH Dec-16 Sep-18 21 Total number of activities: 24 Total funding: $71,314,235 J–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Liberia Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter- vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) Ebola Recovery and Resilience Program (ERRP): targeted cash transfers and agricultural input vouchers for emergency food assistance and market recovery ACDI/VOCA Food Security Agriculture & Food Security SP $ 8,975,418 FFP 3/17/2015 9/9/2016 17 Project Link: Strengthening and developing communications infrastructure to address critical weaknesses C-Squared Liberia LLC (formerly a subsidiary of Google) Innovation and Communication Technology Health ICT $ 2,000,000 LAB ICT 10/1/2016 9/30/2021 59 Project Link: Strengthening and developing communications infrastructure to address critical weaknesses C-Squared Liberia LLC (formerly a subsidiary of Google) Governance and Economic Crisis Mitigation Economic Crisis Mitigation ICT $ 1,000,000 LAB GECM 10/1/2016 9/30/2021 59 Expert advisors embedded in Liberia MOH HMER unit for HIS technical assistance Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Innovation and Communication Technology Health ICT $ 500,000 LAB 5/15/2015 12/31/2015 7 Procurement and Supply Chain Management: Restoration of functionality of health sector supply chain and LMIS strengthening Chemonics Health Systems Recovery Health IE $ 4,500,000 GH Oct-16 Sep-18 23 eGovernance programming to address specific weaknesses to effectively prevent, detect, and respond to future outbreaks. Chemonics/IBI Innovation and Communication Technology Governance MCP $ 4,722,831 LAB ICT 10/1/2016 9/30/2021 59 HIS Landscape Assessment: Identify how information collected via mHero can support health workers beyond central level Ministries Dalberg Consulting Innovation and Communication Technology Health FLWS, ICT $ 38,739 LAB 6/1/2016 9/30/2016 3 Development of a digital payment system - mSTAR FHI 360 Innovation and Communication Technology Health ICT $ 562,113 LAB 6/1/2016 3/30/2018 21 Support strategic planning and rollout of mobile money sensitization, registration, and FHI 360 Health Systems Recovery Health ICT $ 1,500,000 GH Jun-16 Dec-17 18 ANNEX J. ACTIVITY MAPPING TOOL n J–5 Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) validation for health workers - mSTAR Social Protection Cash Transfer Give Directly Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 3,888,722 Bilateral 2/15/2018 1/31/2021 35 Project Last Mile: Pharmaceutical Supply Chain Management Global Environment and Technology Foundation Governance and Economic Crisis Mitigation Health MCP $ 750,000 LAB 10/1/2016 6/30/2018 20 PPPs to support economic recovery, disease surveillance and health systems strengthening in Liberia GOL MOFDP Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 1,000,000 LAB 10/1/2016 9/30/2018 23 PPPs to support economic recovery, disease surveillance and health systems strengthening in Liberia, including the President's Young Professionals Program (PYPP). GOL MOFDP Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 3,000,000 LAB 10/1/2016 9/30/2018 23 Expansion and scaling of mHero GOL MOFDP Innovation and Communication Technology Health ICT $ 300,000 LAB 10/1/2017 9/30/2018 11 eGovernance programming to address specific weaknesses to effectively prevent, detect, and respond to future outbreaks. GOL MOFDP Innovation and Communication Technology Governance ICT $ 1,000,000 LAB 11/1/2016 9/30/2021 58 Policy support planning with GOL for connectivity and eGovernance programs GOL MOFDP Innovation and Communication Technology ICT PPP MCP $ 121,231 LAB 10/1/2017 9/30/2018 11 Payment of Health Worker Salaries GOL MOHSW Emergency Non-Ebola Health Services Health FLWS $ 3,400,000 Bilateral 9/1/2014 2/28/2015 5 e-GOVERNANCE - Sub￾obligation to Google Inc. with USAID/LAB ($2M) and Chemonics/IBI with Lab ($2M) GOL MOPEA Governance and Economic Crisis Mitigation Governance MCP, IE, ICT $ 4,000,000 Bilateral 9/29/2016 9/27/2019 35 Technical Assistance to Liberia Economic Impact Fund (LEIF) International Executive Services Corps. (IESC) Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 1,000,000 LAB 10/1/2016 9/30/2021 59 Partnership for Advancing Community-Based Services (PACS) - train and strengthen International Rescue Committee & Partners Health Systems Recovery Health FLWS, SBCC $ 1,000,000 GH Health Oct-15 Dec-17 26 J–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter- vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) curriculum for community health workers and other health professionals with MOH Systems Recovery Partnership for Advancing Community-Based Services (PACS) - restore routine health services, increase utilization, and expand health worker capacity International Rescue Committee & Partners Emergency Non-Ebola Health Services Health FLWS, SBCC $ 6,000,000 GH Non- Ebola Health Services Oct-15 Dec-17 26 Ebola Recovery Connectivity Initiative: expand affordable, reliable, and sustainable communications networks for health facilities and Ebola recovery organizations in rural and remote areas. Inveneo Governance and Economic Crisis Mitigation Economic Crisis Mitigation ICT $ 250,000 LAB GECM 10/1/2016 10/31/2018 24 Ebola Recovery Connectivity Initiative Inveneo Innovation and Communication Technology Health ICT $ 497,695 LAB ICT 10/1/2016 10/31/2018 24 Civil Society and Media Leadership (CSML) IREX Governance and Economic Crisis Mitigation Governance MCP $ 2,200,000 Bilateral 6/30/2015 11/30/2015 5 Human Resources for Health (HRH): Maternal and Child Survival Program – Pre-Service Training (MSCP/PST) for midwives and lab personnel Jhpiego Health Systems Recovery Health FLWS $ 10,000,000 GH Apr-16 Jun-18 26 Restoration of Health Services in Liberia and strengthening IPC Jhpiego Emergency Non-Ebola Health Services Health FLWS, MCP, IE $ 10,500,000 GH Aug-15 Jun-18 34 IPC renovation activities to FARA facilities Jhpiego Health Systems Recovery Health IE $ 4,757,000 GH Sep-16 Jun-18 21 DELIVER (TO4, TO7) - Strengthening health service delivery and providing technical assistance to build the capacity of the public health supply chain. John Snow, Inc. (JSI) Health Systems Recovery Health IE $ 1,714,611 GH Oct-15 Jun-16 8 HC3 Building Resilient Health Systems through SBCC, Returning to RMNCH Services Johns Hopkins CCP Emergency Non-Ebola Health Services Health SBCC $ 2,600,000 GH Jun-15 Feb-17 20 ANNEX J. ACTIVITY MAPPING TOOL n J–7 Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Continuing support to MOH for utilization of mHero for HIS K4Health Innovation and Communication Technology Health ICT $ 293,090 LAB 10/1/2016 10/31/2017 12 Deployment of HIS experts to support MOH Liberia Interoperability between HIS K4Health Innovation and Communication Technology Health ICT $ 60,001 LAB 3/24/2016 5/31/2016 2 Direct technical support for Liberia PPPs KaizenCo Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 1,000,000 LAB 4/1/2015 3/15/2019 47 Establish a Center of Excellence in Infectious Disease Control (CEIDC) with JFK Hospital in Monrovia and Chevron Management Sciences for Health (MSH) Governance and Economic Crisis Mitigation Economic Crisis Mitigation FLWS, MCP $ 1,000,000 LAB 10/1/2016 9/30/2018 23 Collaborative Support for Health (CSH) - Support MOH to scale up IPC, accelerate development and integration of HIS, and expand county-level capacity building. Management Sciences for Health (MSH) Emergency Non-Ebola Health Services Health MCP $ 5,000,000 GH Feb-17 Feb-19 24 Economic Recovery from Ebola for Liberia (EREL): targeted cash transfers and agricultural input vouchers Mercy Corps Food Security Agriculture & Food Security SP $ 13,493,176 FFP 1/7/2015 12/31/2016 23 Engineering and policy technical experts providing direct support to GOL and USAID to implement ICT infrastructure and policy improvements NetHope Innovation and Communication Technology Governance IE, ICT, MCP $ 357,304 LAB 5/1/2015 9/30/2018 40 Program to enable GOL ICT capacity to better respond to future outbreaks NetHope Innovation and Communication Technology ICT PPP MCP, IE, ICT $ 80,377 LAB 6/30/2015 10/9/2015 3 Support for Liberian telecommunications regulator to deploy equipment to address connectivity breakdowns (with FCC) NetHope Innovation and Communication Technology ICT PPP IE $ 1,002,305 LAB 10/1/2016 5/1/2018 19 Purchase and deliver essential health commodities to Ebola￾affected health facilities PFSCM Health Systems Recovery Health IE $ 2,350,000 GH Jul-16 Sept-16 2 Program Support Program Support Health Systems Recovery Health MCP $ 685,056 GH Apr-16 Sep-18 23 J–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter- vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) Targeted cash transders, cash- for-work, and agricultural input vouchers to EVD-affected individuals for emergency food assistance and market recovery Project Concern International Food Security Agriculture & Food Security SP $ 8,030,564 FFP 2/12/2015 9/11/2016 18 Emergency Food Assistance for Ebola Affected Families: targeted cash transfers and agricultural input vouchers Save the Children Food Security Agriculture & Food Security SP $ 4,574,526 FFP 8/31/2015 2/28/2017 17 Connectivity Demand Analysis for planning Monrovia communications infrastructure SIA Innovation and Communication Technology Health ICT, IE $ 19,488 LAB 12/10/2015 3/17/2016 3 Liberia Municipal Water Program (LMWP) - rehabilitation of water infrastructure, sub-ob to United Infrastructure Projects Tetra Tech/GOL MOPEA Governance and Economic Crisis Mitigation Economic Crisis Mitigation IE $ 10,900,000 Bilateral 2/1/2015 4/1/2019 50 Technical assistance to GOL to learn from success of digital payments to health care workers in SL UNCDF Innovation and Communication Technology Governance ICT $ 342,000 LAB 10/1/2016 9/30/2017 11 Strengthening Routine Immunization in Liberia UNICEF Emergency Non-Ebola Health Services Health MCP, FLWS $ 2,000,000 GH Jul-15 Jul-17 24 Education Crisis Response in Liberia - WASH in schools UNICEF, GOL MOPEA Governance and Economic Crisis Mitigation Education IE $ 8,600,000 Bilateral, AFR 2/1/2016 12/17/2017 22 Workshop on ICT in Liberia with PPP to develop policy recommendations for responding to future outbreaks W3C Innovation and Communication Technology ICT PPP MCP $ 20,000 LAB 4/1/2015 5/31/2015 2 Technical assistance to GOL MoPT to modernize national ICT and Telecom Policy to strengthen communications systems and networks W3C Innovation and Communication Technology Governance ICT, IE $ 165,000 LAB 10/1/2016 9/30/2017 11 Database for receiving government assistance World Bank Governance Governance MCP, IE, ICT $ 6,411,278 Bilateral Recovery school feeding in EVD-affected areas World Food Programme Food Security Agriculture & Food Security SP $ 7,370,323 FFP 4/22/2015 11/30/2015 7 ANNEX J. ACTIVITY MAPPING TOOL n J–9 Pillar II Activity Mapping by Location and Duration - Liberia Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septemb er Obligatio n FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Relief and Recovery Operation for EVD-affected Ivoirian refugees World Food Programme Food Security Agriculture & Food Security SP $ 8,921,600 FFP 3/7/2014 7/31/2016 28 Emergency non-Ebola Health Services M&E: collect data, examine country strategies, provide support to more effectively determine causal pathways and conduct end-term evaluations World Vision, Inc. Emergency Non-Ebola Health Services Health MCP $ 600,820 GH Dec-16 Sep-18 21 Total number of activities: 53 Total funding: $165,055,268 J–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Sierra Leone Pillar II Activity Mapping by Location and Duration – Sierra Leone Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septembe r Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Sustainable Nutrition and Agriculture Promotion (SNAP) Program Plus - Title II and locally procured CSB to children at risk of MAM, agricultural input vouchers, seed loans to agricultural business centers, targeted cash transfers, grants to VSLAs and women’s small enterprise groups. ACDI/VOCA Food Security Agriculture & Food Security SP $ 13,740,816 FFP 4/14/2015 12/31/2017 32 MCSP: Community health systems strengthening with PPP Aspen Innovation Partnership for Health/MCSP Governance and Economic Crisis Mitigation Health ICT, MCP $ 500,000 LAB 10/1/2016 9/30/2018 23 Rapid Ebola Social Safety Network and Economic Recovery (EFSP RESSNER): targeted cash transfers CARE Food Security Agriculture & Food Security SP $ 4,550,110 FFP 8/15/2015 12/31/2017 28 Expert advisors embedded in Sierra Leone MOHS for HIS technical assistance Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Innovation and Communication Technology Health MCP, ICT $ 500,000 LAB 10/1/2015 4/30/2016 6 Targeted cash transfers to reduce acute food insecurity Catholic Relief Services (CRS) Food Security Agriculture & Food Security SP $ 4,868,711 FFP 8/26/2015 12/31/2017 28 Project Last Mile: Pharmaceutical Supply Chain Management Global Environment and Technology Foundation Governance and Economic Crisis Mitigation Health IE $ 750,000 LAB 10/1/2016 6/22/2019 32 PPPs to support economic recovery, disease surveillance and health systems strengthening in Sierra Leone GOSL Governance and Economic Crisis Mitigation Economic Crisis Mitigation MCP $ 3,000,000 LAB 10/1/2016 9/30/2018 23 Grand Challenge: Citizen Engagement Platform to Develop Effective Behavior Change Policies: Epidemic Preparedness in Communities (EPiC) IBM Research Innovation and Communication Technology ICT PPP SBCC $ 526,355 GH Jul-15 Mar-17 20 ANNEX J. ACTIVITY MAPPING TOOL n J–11 Pillar II Activity Mapping by Location and Duration – Sierra Leone Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septembe r Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) IPC Phase II Training: pre￾training and renovation and equipping of academic health institutions International Organization on Migration (IOM) Health Systems Recovery Health FLWS $ 3,000,000 GH Jan-17 Sep-18 20 Implementation of mHero in SL IntraHealth International Innovation and Communication Technology Health MCP, ICT $ 250,000 LAB 10/1/2015 9/30/2016 11 Restoration of Health Services: Support defining and documenting CHW and digital health efforts Jhpiego Emergency Non￾Ebola Health Services Health MCP $ 15,000 GH Oct-16 Oct-16 1 Advancing Partners and Communities (APC) - Restoration of routine health service delivery, and strengthening IPC practices John Snow, Inc. (JSI) Research and Training Institute Emergency Non￾Ebola Health Services Health MCP, FLWS $ 17,000,000 GH Aug-15 Sep-17 25 HC3 Using SBCC Rebuild Trust and Address RMNCH, Building Resilient Health Systems Johns Hopkins CCP Emergency Non￾Ebola Health Services Health SBCC $ 5,485,000 GH Jul-15 Apr-17 21 Training to integrate mobile and digital technology to support CHWs and improve health systems strengthening mPowering Frontline Health Workers Innovation and Communication Technology Health ICT, FLWS $ 15,000 LAB 5/15/2016 11/30/2016 6 System for Improved Access to Pharmaceuticals and Services (SIAPS) - Restoration and expanding the capability of the public health supply chain. MSH Health Systems Recovery Health IE $ 4,500,000 GH Oct-15 Mar-18 29 Program Support Program Support Health Systems Recovery Health MCP $ 408,921 GH Jul-16 Sep-18 26 ReGrow West Africa: financial and technical support to local SMEs in Ebola affected communities RESOLVE, Chevron Governance and Economic Crisis Mitigation Economic Crisis Mitigation SP $ 800,000 LAB 10/1/2016 4/30/2018 18 Targeted cash transfers and cash grants to traders for emergency food assistance and market recovery Save the Children Food Security Agriculture & Food Security SP $ 9,947,904 FFP 3/1/2015 12/31/2017 33 Digital Financial Inclusion in Fragile and Conflict Affected States (DFI-FCAS) - ePayments: UNCDF Innovation and Communication Technology ICT PPP FLWS $ 480,000 LAB 11/1/2016 10/31/2020 48 J–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Pillar II Activity Mapping by Location and Duration – Sierra Leone Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Inter￾vention Type(s) Septembe r Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Supporting the Central Bank and the Ministry of Finance in the development of a digital payment systems to pay salaried healthcare workers through mobile payments Procurement of essential medications and commodities for GoSL UNICEF Emergency Non￾Ebola Health Services Health IE $ 6,751,231 GH Jun-15 Jun-18 36 Improve oversight of drug distribution, boost data reporting, create demand for MCH services, and grow capacity of media to report on health activities UNICEF Health Systems Recovery Health * $ 4,000,000 Bilateral 6/1/2017 5/31/2019 23 Senior Health Advisor from the World Health Organization (WHO) embedded into the Sierra Leone MOHS World Health Organization Innovation and Communication Technology Health ICT $ 399,986 LAB 10/1/2015 9/30/2016 11 Emergency non-Ebola Health M&E: collect data, examine country strategies, provide support to more effectively define causal pathways and conduct end-term evaluations World Vision, Inc. Emergency Non￾Ebola Health Services Health MCP $ 600,820 GH Dec-16 Sep-18 21 Targeted cash transfers and agriculture input vouchers for emergency food assistance and market recovery World Vision Food Security Agriculture & Food Security Social Protectio n $ 6,584,958 FFP 7/28/2015 12/31/2017 29 * Information on this activity was not received by the January 21, 2018 cut-off date for new data used in this analysis, and therefore it was not categorized into an intervention type. Total number of activities: 24 Total funding: $88,674,812 ANNEX J. ACTIVITY MAPPING TOOL n J–13 Regional Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Support to Cash Learning Partnership (CaLP) to document lessons learned from the Ebola response in Liberia and Sierra Leone and build local capacity in cash programming Action Contre le Faim (ACF)/Action Against Hunger Food Security Agriculture & Food Security SP, MCP $ 694,028 FFP 4/14/2016 11/30/2017 19 MCSP: Community health systems strengthening with PPP Aspen Innovation Partnership for Health/MCSP Governance and Economic Crisis Mitigation Health ICT $ 1,000,000 LAB 10/1/2016 9/30/2018 23 Grand Challenge: Repurposed shipping containers as rapidly deployable ETUs Baylor College of Medicine Innovation and Communication Technology Health ICT $ 613,927 GH Jun-15 Jun-17 24 Grand Challenge: Technical assistance for introduction and scale-up of innovations in the Fighting Ebola pipeline Boston Consulting Group Innovation and Communication Technology Health ICT $ 500,000 GH Nov-16 May-17 6 Accra Data Harmonization Summit Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Innovation and Communication Technology Health ICT $ 142,381 LAB 2/1/2015 12/30/2015 10 Invitational travel for a thought leader embedded in MOH for technology solution development under the HIS Interoperability BAA Carolina Institute for Developmental Disabilities/Carolina Population Center/MEASURE Innovation and Communication Technology Health ICT $ 5,409 LAB 11/7/2015 11/14/ 2015 0 J–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) Inter-agency agreement for Ebola response activities Centers for Disease Control (CDC) CDC Inter- agency Agreement - Ebola Response Health MCP $ 80,000,000 GH May-16 Dec-18 32 FEWSNET: early warning, analysis, and reporting on acute food insecurity Chemonics Food Security Agriculture & Food Security SP, MCP, ICT, IE $ 2,865,965 FFP 9/9/2015 6/30/2017 21 Survivor support: Develop EVD survivor medicines and supplies list, quantification and procurement Chemonics Ebola Transmission Prevention (Survivors) Health IE $ 905,000 GH Jan-17 Aug-18 19 Transportation of RUTF CMA CGM Food Security Agriculture & Food Security SP, IE $ 64,010 FFP 5/20/15 (Liberia), 7/1/15 (Sierra Leone) 6/30/15 (Liberia), 9/11/15 (Sierra Leone) 1 (Liberia), 2 (Sierra Leone) Grand Challenge: Colored bleach mist for visualization of sprayed surfaces Columbia University Innovation and Communication Technology Health ICT $ 649,342 GH Jun-15 Jul-16 13 Strengthening Health Information Systems toward Interoperability in the West Africa Region (Broad Agency Announcement) DAI Washington Innovation and Communication Technology Health ICT $ 178,613 LAB 9/15/2015 11/30/2016 14 Grand Challenge: human-centered design assistance for Fighting Ebola innovators Dalberg Consulting Innovation and Communication Technology Health ICT $ 952,369 GH Sep-16 Sep-16 1 Grand Challenge: Open source mobile platform for health data collection, decision making, Dimagi, Inc Innovation and Communication Technology Health ICT $ 298,996 GH May-15 May-16 12 ANNEX J. ACTIVITY MAPPING TOOL n J–15 Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) client tracking, SMS communication, and map-based visuals Procurement of RUTF for programming in Ebola-affected countries Edesia Food Security Agriculture & Food Security SP, IE $ 2,238,000 FFP 4/17/2015 4/15/2016 11 Independent M&E for WAHIT and additional selected HIS programs Evidence 4 Development (E4D) Innovation and Communication Technology Health ICT $ 107,388 LAB 8/15/2016 12/30/2018 28 HIS Learning Agenda FHI 360 Innovation and Communication Technology Health ICT $ 142,612 LAB 5/25/2015 1/10/2017 19 Training and mentoring to software engineers in HIS for MOHs Health Policy Plus (HP+)/CAPS Innovation and Communication Technology Health ICT $ 666,750 LAB 9/15/2016 3/31/2018 18 Monitoring, Evaluation, and Learning for USAID Ebola Recovery Efforts IBTCI Monitoring and Evaluation Activities MCP $ 8,500,000 AFR 10/1/2016 9/30/2019 35 Grand Challenge: A new clothing system for improved heat stress relief, full body liquid integrity, and ease of doffing International Personnel Protection Innovation and Communication Technology Health ICT $ 323,845 GH May-15 Aug-16 15 Grand Challenge: mHero to support frontline health workers IntraHealth International Innovation and Communication Technology Health ICT $ 700,000 GH Jun-15 Sep-16 15 Grand Challenge: Safer and faster doffing PPE for Frontline Workers and new PPE for Community/Family Care Johns Hopkins University Innovation and Communication Technology Health ICT $ 793,635 GH Jun-15 Jul-16 13 J–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) APC Ebola Transmission Prevention and Survivor Services JSI Research and Training Institute Ebola Transmission Prevention (Survivors) Health FLWS, SP, MCP, SBCC $ 21,925,000 GH Jun-16 Jun-18 24 Training and technical support to MOHs in West African Nations (with WAHO) Leadership Capacity Strengthening Project (CAPS) Innovation and Communication Technology Health ICT $ 833,250 LAB 3/1/2016 1/30/2019 34 Transportation of RUTF Maersk, Inc. Food Security Agriculture & Food Security Social Protection, IE $ 146,250 FFP 5/19/2015 7/13/2015 1 Grand Challenge: Redesigned ETU with ergonomic features to promote heat and air exchange Makerere University (MAK) Innovation and Communication Technology Health ICT $ 482,231 GH Jun-15 Apr-16 10 Grand Challenge: Modular and rapidly deployable treatment units Modula S Inc. Innovation and Communication Technology Health ICT $ 500,000 GH May-15 Dec-15 7 Technical support and training for key West African government and private sector stakeholders on Interoperability Standards mPowering Frontline Health Workers Innovation and Communication Technology Health ICT $ 72,000 LAB 5/15/2015 3/15/2016 10 Build the capacity of tertiary-level specialty care to treat and monitor survivors & improve referral network National Academy of Sciences Ebola Transmission Prevention (Survivors) Health MCP $ 1,000,000 GH Feb-17 Jun-21 53 Train local clinicians to support disease surveillance and provide long-term care to survivors National Academy of Sciences (PEER) Ebola Transmission Prevention (Survivors) Health Frontline Worker Support $ 5,000,000 LAB Feb-17 Jun-21 53 ANNEX J. ACTIVITY MAPPING TOOL n J–17 Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) ICT policy guidance for successful implementation of HIS advancements NetHope Innovation and Communication Technology Health ICT $ 13,769 LAB 9/30/2015 10/1/2015 0 ePayments Learning and evaluation survey with FFP IPs in Sierra Leone and Liberia NetHope Innovation and Communication Technology Agriculture & Food Security ICT, IE $ 16,147 LAB 3/15/2016 4/30/2016 1 Grand Challenge: Communications support for the Fighting Ebola Grand Challenge Oneworld Communications, Inc. Innovation and Communication Technology Health ICT $ 200,000 GH Nov-16 Dec-17 13 The DIGITAL Health Initiative under HIS Interoperability BAA PATH Innovation and Communication Technology Health ICT $ 854,876 LAB 10/1/2016 9/30/2021 59 Development of open source tool to support use of Big Data to fight against Ebola resurgence Real Impact Analytics Governance and Economic Crisis Mitigation Economic Crisis Mitigation ICT $ 300,000 LAB 10/1/2016 9/30/2018 23 Grand Challenge: Wearable health technologies including a Bluetooth sensor for remote monitoring of Ebola patients’ vital signs Scripps health Innovation and Communication Technology Health ICT $ 632,058 GH Jun-15 May-16 11 Grand Challenge: Low-cost, battery￾powered infusion monitors Shift Labs, Inc. Innovation and Communication Technology Health ICT $ 318,682 GH May-15 Aug-16 15 ePayments agent network strengthening in Sierra Leone and Liberia SIA Innovation and Communication Technology Health Frontline Worker Support $ 1,599,740 LAB 10/1/2016 3/31/2018 17 Grand Challenge: Long-lasting, spray SPR Advanced Technologies, Inc. Innovation and Communication Technology Health ICT $ 655,788 GH May-15 Aug-16 15 J–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform- ance Start Date Perform- ance End Date Duration of Ebola Funded Activities (months) on barrier for PPE materials Grand Challenge: Easy to assemble decontamination chambers for health workers TOMI Environmental Solutions, Inc. Innovation and Communication Technology Health ICT $ 559,003 GH May-15 Feb-16 9 HIS Landscape Assessment: Analysis of ICT issues and challenges that resulted in failures during the Ebola outbreak UN Foundation Innovation and Communication Technology Health ICT $ 961,261 LAB 3/15/2015 10/23/2015 7 Health systems recovery in Ebola- affected countries through private sector engagement with Paul Allen Foundation UN OCHA Governance and Economic Crisis Mitigation Health ICT $ 150,000 LAB 10/1/2016 3/31/2017 5 Support to restart screening, referral, treatment services for children with SAM, including RUTF UNICEF Food Security Agriculture & Food Security Social Protection, IE $ 3,374,348 FFP 6/22/2015 6/21/2016 11 Program Support USAID Monitoring and Evaluation Activities MCP $ 1,500,000 AFR nd nd nd Improving health data and information sharing with mHealth tools and services under HIS Interoperability BAA VOTO Mobile, Inc. Innovation and Communication Technology Health ICT $ 1,178,300 LAB 8/1/2016 7/31/2020 47 Food assistance for EVD-affected Ivoirian refugees repatriating to Cote d'Ivoire from Liberia refugee camps World Food Programme Food Security Agriculture & Food Security Social Protection $8,300,000 FFP 4/1/2015 12/31/2017 32 ANNEX J. ACTIVITY MAPPING TOOL n J–19 Activity IP/Agency USAID Category/ Sector Thematic Area based on Theory of Change Intervention Type(s) September Obligation FY17 Funding Source Perform￾ance Start Date Perform￾ance End Date Duration of Ebola Funded Activities (months) Evaluation of survivor programs and further Ebola transmission prevention interventions World Vision, Inc. Ebola Transmission Prevention (Survivors) Health MCP $670,000 GH Dec-16 Sep-18 21 Total number of activities: 47 Total funding: $153,584,973 J–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–1 ANNEX K. HOUSEHOLD AND HEALTH FACILITY DATA TABLES Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document HOUSEHOLD SURVEY RESULTS PE1 (HHS, 2017) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–4 GUINEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–4 Table K–G1. Percent distribution of households (HHs) by selected background characteristics, Guinea . . . . . . . . . . . . . . . . . K–4 Table K–G2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–5 Table K–G3. Among surveyed households (HHs) that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . K–6 Table K–G4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . K–7 Table K–G5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . K–7 Table K–G6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–8 Table K–G7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by prefecture and EVD survivorship, Guinea . . . . . . . . . . . . . . . K–8 Table K–G8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–9 Table K–G9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–10 Table K–G10. Percent distribution of household (HH) members age 15 or older by their employment status, according to prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–10 Table K–G11. Primary school net attendance ratio (NAR) by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–11 Table K–G12. Percentage of heads of household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–11 Table K–G13. Among heads of the household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–12 Table K–G14. Percentage of heads of the household (HH)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–13 Table K–G15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–14 Table K–G16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–15 Table K–G17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–15 Table K–L1. Percent distribution of households (HHs) by selected background characteristics, Liberia . . . . . . . . . . . . . . . . K–18 LIBERIA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–18 Table K–L2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–19 Table K–L3. Among surveyed households (HHs) that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . K–20 K–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . K–20 Table K–L5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . K–21 Table K–L6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–21 Table K–L7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by county and EVD survivorship, Liberia . . . . . . . . . . . . . . . . . K–22 Table K–L8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–22 Table K–L9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–23 Table K–L10. Percent distribution of household (HH) members age 15 or older by their employment status, according to county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–23 Table K–L11. Primary school net attendance ratio (NAR) by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–23 Table K–L12. Percentage of heads of household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–24 Table K–L13. Among heads of the household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–24 Table K–L14. Percentage of heads of the household (HH)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–25 Table K–L15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–26 Table K–L16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–27 Table K–L17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–27 Table K–SL1. Percent distribution of households by selected background characteristics, Sierra Leone . . . . . . . . . . . . . . . . K–29 SIERRA LEONE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–29 Table K–SL2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–30 Table K–SL3. Among surveyed households that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by district, Sierra Leone . . . . . . . . . . . . . . . . K–31 Table K–SL4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by district, Sierra Leone . . . . . . . . . . . . . . . . . K–31 Table K–SL5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by district, Sierra Leone . . . . . . . . . . . . . . . . . . K–32 Table K–SL6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–32 Table K–SL7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by district and EVD survivorship, Sierra Leone . . . . . . . . . . . . K–33 Table K–SL8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–33 Table K–SL9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–34 Table K–SL10. Percent distribution of household (HH) members age 15 or older by their employment status, according to district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–34 Table K–SL11. Primary school net attendance ratio (NAR) by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–34 Table K–SL12. Percentage of heads of households (HHs)/household respondents who took specific actions to avoid being infected with Ebola virus, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–35 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–3 Table K–SL13. Among heads of the households (HHs)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Sierra Leone . . . . . . . . . . . . . . . . . . . . K–35 Table K–SL14. Percentage of heads of the households (HHs)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . K–36 Table K–SL15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–37 Table K–SL16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–38 Table K–SL17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–38 HEALTH FACILITY SURVEY RESULTS PE1 (HFS, 2017) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–40 GUINEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–40 Table K–GF1. Number, types, location and managing authority of surveyed health facilities (HFs) by prefecture, Guinea . . K–40 Table K–GF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–40 Table K–GF3. Infrastructure and record keeping of surveyed health facilities (HFs) by prefecture, Guinea . . . . . . . . . . . . . K–42 Table K–GF4. Facility management of surveyed health facilities (HFs) by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . K–42 Table K–GF5. Status of health facilities (HFs) during the EVD epidemic by prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . K–43 Table K–GF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and prefecture, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–43 Table K–GF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Guinea K–44 LIBERIA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–45 Table K–LF1. Number, types, location and managing authority of surveyed health facilities (HFs) by county, Liberia . . . . . . K–45 Table K–LF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by county, Liberia . K–45 Table K–LF3. Infrastructure and record keeping of surveyed health facilities (HFs) by county, Liberia . . . . . . . . . . . . . . . . . K–46 Table K–LF4. Facility management of surveyed health facilities (HFs) by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–47 Table K–LF5. Status of health facilities (HFs) during the EVD epidemic by county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . K–48 Table K–LF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and county, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–48 Table K–LF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–49 SIERRA LEONE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–50 Table K–SLF1. Number, types, location and managing authority of surveyed health facilities (HFs) by district, Sierra Leone K–50 Table K–SLF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–50 Table K–SLF3. Infrastructure and record keeping of surveyed health facilities (HFs) by district, Sierra Leone . . . . . . . . . . . K–51 Table K–SLF4. Facility management of surveyed health facilities (HFs) by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . K–52 Table K–SLF5. Status of health facilities (HFs) during the EVD epidemic by district, Sierra Leone . . . . . . . . . . . . . . . . . . . . K–53 Table K–SLF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and district, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–53 Table K–SLF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . K–53 K–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES HOUSEHOLD SURVEY RESULTS PE1 (HHS, 2017) Guinea Table K–G1. Percent distribution of households (HHs) by selected background characteristics, Guinea Characteristic Percent (%) Number of HHs Household head Female 21.4 1109 Male 78.6 4080 Total 100.0 5189 Number of usual household members 1 0.2 13 2 3.2 165 3 10.7 555 4 15.6 809 5 19.5 1014 6 13.4 694 7 11.1 576 8 8.2 423 9+ 18.1 940 Total 100.0 5189 Residence Urban 48.2 2502 Rural 51.8 2687 Total 100.0 5189 Prefecture Beyla 4.9 256 Boffa 3.6 187 Boke 7.9 409 Conakry 30.5 1581 Faranah 4.4 230 Forécariah 4.2 218 Gueckedou 5.9 305 Kankan 6.0 313 Kindia 8.1 420 Labe 7.3 379 Lola 3.7 189 Macenta 5.7 294 N’Zérékoré 7.8 406 Total 100.0 5189 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–5 Table K–G1. Percent distribution of households (HHs) by selected background characteristics, Guinea Characteristic Percent (%) Number of HHs Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 13.9 395 Primary 32.2 838 Secondary 20.9 507 Higher 33.0 666 Total 100.0 2406 Employment of the head of household Employed in the last 7 days 72.5 3757 Employed in the last 12 months but not in the last 7 days 7.2 373 Employed in the last 12 months 79.7 4134 Ebola virus diseease (EVD) Household with no member diagnosed with EVD 88.0 4567 Household with member diagnosed with Ebola who survived 1.2 64 Household with member diagnosed with Ebola who died 1.9 100 Don't know/Not sure/Refused 8.8 458 Total 100.0 5189 Socio-economic quintile Lowest 21.0 1464 Second 19.5 1331 Middle 19.8 1174 Fourth 20.4 788 Highest 19.4 432 Total 100.0 5187 Table K–G2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by prefecture, Guinea Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs Prefecture Beyla 94.5 9.7 77.3 21.2 79.0 401 Boffa 61.2 54.5 43.1 11.0 90.0 387 Boke 78.9 66.6 55.2 38.3 94.7 392 Conakry 97.7 65.9 62.3 96.4 93.2 472 Faranah 93.1 34.0 58.5 33.4 85.0 373 Forécariah 76.4 64.0 41.8 30.9 82.8 415 Gueckedou 93.0 30.1 45.1 11.7 74.2 378 Kankan 93.9 39.9 68.3 36.8 76.0 426 Kindia 77.1 49.1 53.4 46.4 90.4 383 K–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by prefecture, Guinea Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs Labe 96.6 60.4 76.4 40.0 92.8 394 Lola 80.9 20.3 29.1 5.4 84.9 404 Macenta 92.6 13.0 35.8 9.4 82.4 386 N’Zérékoré 89.5 49.2 52.4 24.4 85.8 378 Total 89.8 49.9 56.9 48.8 87.8 5189 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. Table K–G3. Among surveyed households (HHs) that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by prefecture, Guinea Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs Prefecture Beyla 100.0 19.8 79.9 0.0 90.0 10 Boffa 88.1 46.4 54.0 14.0 96.0 28 Boke 94.1 67.7 58.7 44.0 100.0 27 Conakry 100.0 64.7 58.8 97.1 97.1 34 Faranah 100.0 39.3 76.2 60.7 100.0 21 Forécariah 80.6 41.8 80.6 41.8 80.6 5 Gueckedou 100.0 17.1 46.4 20.9 70.7 17 Kankan 100.0 50.0 100.0 0.0 100.0 2 Kindia 94.3 76.4 76.4 65.3 100.0 17 Labe 100.0 62.2 79.5 62.2 100.0 19 Lola 73.0 26.8 23.0 11.4 73.1 26 Macenta 93.5 30.2 37.4 17.2 81.3 16 N’Zérékoré 75.8 51.5 50.5 27.8 82.8 43 Total 93.3 55.2 58.4 59.0 92.4 265 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–7 Table K–G4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by prefecture, Guinea Location % HHs whose main water source was con￾structed, built, or renovated in the last 3 years Number of HHs Among households whose main water source was constructed/built/renovated in the last 3 years, percentage (%) by source of construction/renovation Number of HHs Govt. Local/Intl. NGO Donor Orgn. Community Group Other Prefecture Beyla 51.7 401 18.0 13.8 1.0 62.5 4.7 207 Boffa 30.9 387 43.9 17.7 6.5 20.4 11.5 120 Boke 41.9 392 21.7 7.5 9.9 26.0 34.9 164 Conakry 37.6 472 19.9 4.8 13.0 18.5 43.8 177 Faranah 47.2 373 39.7 7.9 0.0 22.2 30.2 176 Forécariah 44.4 415 23.3 10.5 0.6 23.3 42.3 184 Gueckedou 38.3 378 16.3 34.1 4.9 31.9 12.7 145 Kankan 49.3 426 31.2 13.0 5.9 35.6 14.3 210 Kindia 53.1 383 29.1 16.9 1.6 22.4 30.0 203 Labe 46.0 394 10.4 3.0 3.1 39.0 44.5 181 Lola 35.0 404 27.0 29.5 1.6 23.9 18.2 141 Macenta 33.1 386 26.2 17.2 3.4 34.9 18.3 128 N’Zérékoré 25.6 378 7.5 19.2 2.9 29.0 41.3 97 Total 40.5 5189 22.5 12.1 6.0 28.2 31.3 2099 Table K–G5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by prefecture, Guinea Location % HHs with an observed handwashing station Number of HHs % HHs with an observed handwashing station AND water and soap or other cleansing agent(s)* Number of HHs Prefecture Beyla 76.0 401 32.1 305 Boffa 57.5 387 31.4 223 Boke 72.2 392 34.7 283 Conakry 49.4 472 51.1 233 Faranah 21.2 373 55.2 79 Forécariah 32.8 415 36.2 136 Gueckedou 28.9 378 40.2 109 Kankan 64.2 426 49.4 274 Kindia 38.5 383 42.2 147 Labe 37.5 394 39.4 148 Lola 16.1 404 15.3 65 Macenta 10.3 386 9.7 40 N’Zérékoré 17.9 378 63.3 68 Total 42.9 5189 43.3 2225 * Excluding mud, ash, or sand. K–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by prefecture, Guinea Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any as￾sistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs Prefecture Beyla 84.0 9.5 6.5 0.0 399 Boffa 57.6 37.6 4.3 0.4 374 Boke 55.9 40.0 3.0 1.1 385 Conakry 61.3 31.5 6.5 0.7 460 Faranah 57.6 33.6 6.6 2.1 367 Forécariah 85.0 9.0 5.7 0.2 407 Gueckedou 67.5 26.6 5.7 0.2 373 Kankan 72.7 23.0 4.3 0.0 412 Kindia 72.9 13.8 12.5 0.8 372 Labe 61.0 36.4 2.6 0.0 386 Lola 68.9 20.2 10.7 0.3 385 Macenta 65.7 22.7 10.8 0.8 371 N’Zérékoré 64.2 22.4 10.7 2.6 357 Total 65.4 27.0 6.9 0.7 5048 * Excludes households that state received assistance from “other” sources. Table K–G7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by prefecture and EVD survivorship, Guinea Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any assistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs Prefecture Beyla 0.0 0.0 0.0 0.0 0 Boffa 0.0 0.0 100.0 0.0 1 Boke 0.0 0.0 100.0 0.0 1 Conakry 33.3 16.7 50.0 0.0 6 Faranah 100.0 0.0 0.0 0.0 1 Forécariah 81.6 9.2 6.1 3.1 32 Gueckedou 70.8 29.2 0.0 0.0 10 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–9 Table K–G7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by prefecture and EVD survivorship, Guinea Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any assistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs Kankan 76.5 0.0 23.5 0.0 4 Kindia 100.0 0.0 0.0 0.0 1 Labe 65.5 34.5 0.0 0.0 3 Lola 100.0 0.0 0.0 0.0 1 Macenta 72.9 16.9 10.2 0.0 30 N’Zérékoré 61.9 0.0 38.1 0.0 5 Total 63.7 14.8 21.0 0.6 95 * Excludes households that state received assistance from “other” sources. Table K–G8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by prefecture, Guinea Location Percentage of households that received the following types of assistance in the last 3 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs Prefecture Beyla 4.7 5.0 4.3 2.7 1.5 1.0 0.8 66 Boffa 14.5 19.5 10.2 10.1 4.7 1.6 3.4 171 Boke 18.7 23.1 8.7 8.4 3.1 0.2 2.2 176 Conakry 15.7 19.3 5.7 7.6 1.1 0.6 3.6 190 Faranah 12.6 18.1 9.7 6.2 1.8 1.1 1.9 161 Forécariah 5.3 4.3 3.3 2.9 0.5 0.0 0.2 69 Gueckedou 12.7 11.9 6.5 9.2 1.1 0.5 1.2 126 Kankan 11.5 9.8 8.1 3.2 2.0 0.7 0.5 127 Kindia 7.9 8.9 7.8 10.2 0.3 0.0 2.9 112 Labe 20.9 21.3 10.5 10.1 2.3 0.5 3.3 159 Lola 15.1 12.3 7.9 6.9 2.5 0.5 4.7 139 Macenta 11.8 9.9 10.0 12.1 2.3 0.3 2.3 142 N’Zérékoré 17.5 14.9 13.7 11.9 2.5 1.1 1.1 149 Total 14.0 15.4 7.8 8.0 1.7 0.6 0.6 1885 K–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by prefecture, Guinea Location Percentage of households that received the following types of assistance in the last 12 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs Prefecture Beyla 5.5 5.2 5.0 4.0 2.5 1.5 2.5 401 Boffa 23.4 24.4 17.1 15.5 7.4 2.1 6.7 387 Boke 26.5 28.5 15.4 13.1 5.8 0.4 6.2 392 Conakry 19.5 22.9 8.7 12.9 1.9 1.3 7.2 472 Faranah 19.8 23.4 15.4 10.4 5.6 2.2 5.7 373 Forécariah 8.4 5.5 4.1 6.6 1.5 0.0 1.2 415 Gueckedou 18.6 15.3 8.7 12.4 2.7 1.2 4.5 378 Kankan 14.5 11.6 12.2 5.1 2.9 1.6 0.5 426 Kindia 14.0 11.6 12.0 15.9 1.6 0.5 4.5 383 Labe 25.5 25.1 13.4 12.4 3.3 0.5 4.8 394 Lola 17.1 13.8 9.1 9.6 3.0 0.7 6.4 404 Macenta 16.3 14.3 13.9 17.6 3.9 0.8 4.1 386 N’Zérékoré 20.9 18.1 15.9 14.9 4.2 1.6 11.6 378 Total 18.5 18.7 11.1 12.2 3.1 1.1 5.7 5189 Table K–G10. Percent distribution of household (HH) members age 15 or older by their employment status, according to prefecture, Guinea Location Percentage of HH members age 15 or older that were employed during the: Number of HH members age 15 or older Last 7 days Last 12 months but NOT in last 7 days Last 12 months Prefecture Beyla 85.1 3.5 88.5 401 Boffa 71.4 11.5 83.1 387 Boke 73.3 5.1 78.4 392 Conakry 69.0 5.9 74.8 472 Faranah 59.0 11.1 70.1 373 Forécariah 68.5 17.5 86.0 415 Gueckedou 74.4 8.3 82.7 378 Kankan 77.4 4.7 82.2 426 Kindia 70.9 8.8 79.6 383 Labe 64.5 6.4 70.9 394 Lola 20.3 6.4 85.9 404 Macenta 19.6 8.6 89.0 386 N’Zérékoré 17.1 6.4 89.2 378 Total 72.5 7.2 79.7 5189 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–11 Table K–G11. Primary school net attendance ratio (NAR) by prefecture, Guinea Location Primary school NAR* Number of children aged 6–11 years Prefecture Beyla 66.6 213 Boffa 56.7 388 Boke 60.1 313 Conakry 84.7 425 Faranah 55.0 257 Forécariah 66.3 480 Gueckedou 66.0 326 Kankan 47.2 363 Kindia 70.5 351 Labe 70.3 411 Lola 71.6 503 Macenta 70.5 482 N’Zérékoré 71.1 407 Total 71.1 4919 * The NAR for primary school is the percentage of the primary-school age (6–11 years) population that is attending primary school. Table K–G12. Percentage of heads of household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, Guinea Action Taken Percent Wash hands with soap/water more often 87.9 Wash hands with disinfectant more often 75.6 Avoid crowded places 9.9 Drink bitter-cola 0.7 Drink a lot of water/juice 0.9 Take traditional herbs 0.5 Take antibiotics 2.0 Wear gloves/protective barriers 1.2 Avoid touching people suspected to have Ebola 12.6 Avoid touching everyone 20.4 Avoid visiting homes of people with Ebola 7.1 Avoid touching dead bodies during or preparing for burial ceremonies 1.9 Wash with salt/hot water 0.6 Pray/pray more often 1.0 Report suspected cases of Ebola 0.6 Number of heads of households/household respondents 4380 K–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G13. Among heads of the household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Guinea Action Prompts Percent Spoke to health worker/community health education 59.3 Listened to radio program(s) 58.8 Watched TV program(s) 30.5 Read billboard message(s)/education materials 3.8 Received advice from family/friends 14.5 Received instruction(s) from workplace/school 4.8 Lost someone important to me 0.9 Number of heads of households/household respondents who took specific actions 4380 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–13 Table K–G14. Percentage of heads of the household (HH)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Guinea Location % HH respondents who would NOT go to HF if they had a fever Among heads of the households/household respondents who would NOT go to a health facility if they had fever, percentage by reason: No money for treat￾ment No HF nearby No money for trans￾port HF is con￾taminated with EVD HF is dirty/un￾sanitary I do not trust the health care staff Health care is of poor quality Health staff are not re￾spectful People may think I have EVD/ stigma I prefer to go to a pharmacy I prefer to go to a traditional healer I prefer to stay at home to recover Prefecture Beyla 4.0 43.6 6.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 12.5 31.3 12.5 Boffa 4.6 59.7 37.0 18.5 0.0 12.3 12.3 18.5 6.2 0.0 0.0 12.3 0.0 Boke 0.7 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 66.7 0.0 0.0 Conakry 6.4 50.0 6.7 3.3 0.0 3.3 3.3 0.0 0.0 0.0 20.0 3.3 3.3 Faranah 20.7 61.3 54.0 5.5 0.0 0.0 1.4 5.3 0.0 0.0 1.4 1.4 1.4 Forécariah 12.6 42.6 13.3 15.2 0.0 0.0 1.9 3.8 1.9 0.0 44.2 30.3 0.0 Gueckedou 7.1 76.8 69.7 15.5 3.9 0.0 0.0 0.0 0.0 0.0 11.6 0.0 0.0 Kankan 7.1 74.5 20.4 17.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 11.1 3.4 Kindia 9.8 60.5 13.2 5.3 0.0 0.0 2.6 0.0 0.0 0.0 10.5 13.2 2.6 Labe 9.3 55.8 0.0 7.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 19.7 38.9 Lola 8.9 50.0 26.5 8.9 0.0 0.0 0.0 3.0 0.0 0.0 11.6 11.8 11.7 Macenta 7.4 43.0 57.4 28.7 0.0 0.0 3.6 0.0 3.6 0.0 3.6 3.2 3.6 N’Zérékoré 11.1 51.9 5.4 15.1 2.2 0.0 0.0 0.0 0.0 0.0 23.2 37.9 8.1 Total 7.7 54.7 21.0 9.6 0.5 1.1 1.9 1.4 0.5 0.0 14.0 12.8 6.8 , by prefecture, Guinea K–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Guinea Location % women age 15–49 who would NOT go to HF if they had a fever Among female respondents aged 15–49 who would NOT go to a health facility if they had fever, percentage by reason: No mon- ey for treatment No HF nearby No money for trans- port HF is con- taminated with EVD HF is dirty/ unsanitary I do not trust the health care staff Health care is of poor quality Health staff are not re- spectful People may think I have EVD/ stigma I prefer to go to a traditional healer I prefer to stay at home to recover Prefecture Beyla 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Boffa 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Boke 1.0 50.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 50.0 0.0 Conakry 5.2 80.0 20.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 20.0 0.0 Faranah 20.9 84.8 34.8 0.0 0.0 5.1 0.0 0.0 0.0 0.0 0.0 0.0 Forécariah 5.2 64.6 0.0 11.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 11.8 Gueckedou 3.9 75.0 100.0 50.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Kankan 2.8 72.7 27.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Kindia 8.9 70.0 20.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Labe 3.6 79.2 0.0 18.7 0.0 0.0 0.0 0.0 0.0 0.0 20.8 0.0 Lola 6.3 74.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Macenta 8.5 83.3 33.3 50.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 16.7 N’Zérékoré 7.5 89.8 0.0 23.1 10.3 0.0 0.0 0.0 0.0 0.0 23.1 12.8 Total 5.2 78.5 21.4 10.4 1.3 0.9 0.0 0.0 0.0 0.0 9.9 3.4 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–15 Table K–G16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by prefecture, Guinea Location Mother received ANC during pregnancy Mother received ANC by a skilled health provider* Mother received 4 or more ANC visits Delivery oc￾curred at a health facility Delivery was as￾sisted by a skilled health provider* Number of HHs Prefecture Beyla 9.1 9.1 0.0 82.1 72.9 11 Boffa 94.1 90.7 72.2 62.7 77.8 123 Boke 93.3 90.1 72.0 75.6 77.7 130 Conakry 90.9 90.2 78.8 87.9 95.5 132 Faranah 75.2 73.2 46.7 51.8 40.2 102 Forécariah 93.6 93.6 69.2 64.0 69.1 141 Gueckedou 85.6 83.7 56.6 60.1 72.4 109 Kankan 57.7 54.3 13.4 64.2 91.2 144 Kindia 91.1 88.7 67.6 56.7 64.0 124 Labe 92.3 92.3 76.7 72.8 83.5 140 Lola 95.0 91.2 56.2 68.6 72.4 80 Macenta 91.9 88.3 55.6 65.9 69.5 86 N’Zérékoré 94.3 91.8 66.3 72.8 79.5 118 Total 88.2 86.4 65.8 72.6 80.5 1440 * Skilled provider includes doctor, nurse, midwife, health/physicians’ assistant or auxiliary health worker. Table K–G17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Guinea Characteristic HDDS Number of HHs HHS Number of HHs Household head Female 6.5 726 0.8 943 Male 6.5 2819 0.9 4189 Number of usual household members 1 6.7 10 0.6 13 2 5.8 103 0.7 167 3 6.4 370 0.7 540 4 6.5 578 0.7 810 5 6.3 697 0.8 1021 6 6.3 490 0.9 703 7 7.1 351 0.8 524 8 6.7 307 0.8 417 9+ 6.8 639 1.2 937 Residence Urban 7.0 1165 0.6 1644 Rural 6.0 2380 1.1 3488 K–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–G17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Guinea Characteristic HDDS Number of HHs HHS Number of HHs Prefecture Beyla 6.2 317 0.7 384 Boffa 4.9 246 0.7 399 Boke 7.0 317 0.6 390 Conakry 7.1 378 0.5 471 Faranah 5.5 281 1.0 361 Forécariah 6.2 294 1.5 406 Gueckedou 6.0 241 1.3 376 Kankan 6.5 205 1.1 423 Kindia 6.5 298 1.0 378 Labe 7.3 333 0.6 394 Lola 4.8 219 1.5 400 Macenta 5.5 187 1.4 377 N’Zérékoré 5.5 229 1.2 373 Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 6.3 493 1.0 711 Primary 6.6 1141 0.9 1653 Secondary 6.8 742 0.7 1075 Higher 7.3 1169 0.5 1693 Employment of the head of household Employed in the last 7 days 6.6 2546 0.8 3761 Employed in the last 12 months but not in the last 7 days 6.4 286 1.0 409 Employed in the last 12 months 6.6 2835 0.8 4174 Received assistance in the last 12 months None 6.2 2337 1.0 3486 Family/friends 7.1 916 0.6 1266 Government/NGO/CBO/community group 7.0 256 1.0 344 Family/friends AND Government/NGO/ CBO/community group 7.1 36 0.7 36 Experienced shock in past 12 months Yes 6.6 3354 0.9 4855 No 5.9 191 0.6 277 Ebola Virus Disease (EVD) Household with no member diagnosed with EVD 6.6 3098 0.8 4420 Household with member diagnosed with EVD who survived 6.2 34 1.4 63 Household with member diagnosed with EVD who died 6.4 46 1.2 100 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–17 Table K–G17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Guinea Characteristic HDDS Number of HHs HHS Number of HHs Socio-economic quintile Lowest 5.5 974 1.3 1448 Second 6.1 896 1.1 1310 Middle 7.0 825 0.8 1165 Fourth 6.9 522 0.6 781 Highest 6.9 328 0.5 428 K–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L1. Percent distribution of households (HHs) by selected background characteristics, Liberia Characteristic Percent (%) Number of HHs Household head Female 39.5 1408 Male 60.5 1889 Total 100.0 3297 Number of usual household members 1 3.1 90 2 9.1 251 3 16.6 491 4 21.2 636 5 15.8 535 6 10.4 370 7 8.6 322 8 4.2 179 9+ 11.2 423 Total 100.0 3297 Residence Urban 53.7 1152 Rural 46.3 2145 Total 100.0 3297 County Bomi 3.0 392 Bong 11.6 382 Grand Bassa 7.6 422 Grand Cape Mount 4.4 425 Lofa 9.6 408 Margibi 7.3 429 Montserrado 38.8 403 Nimba 17.7 436 Total 100.0 3297 Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 5.9 196 Primary 20.3 555 Secondary 33.1 803 Higher 40.7 788 Total 100.0 2342 Liberia ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–19 Table K–L1. Percent distribution of households (HHs) by selected background characteristics, Liberia Characteristic Percent (%) Number of HHs Employment of the head of household Employed in the last 7 days 60.1 1832 Employed in the last 12 months but not in the last 7 days 13.6 476 Employed in the last 12 months 73.7 2315 Ebola virus diseease (EVD) Household with no member diagnosed with EVD 93.8 3091 Household with member diagnosed with Ebola who survived 1.6 54 Household with member diagnosed with Ebola who died 2.4 80 Don't know/Not sure/Refused 2.2 72 Total 100.0 3297 Socio-economic quintile Lowest 19.3 977 Second 18.3 768 Middle 18.1 675 Fourth 22.7 506 Highest 21.6 371 Total 100.0 3297 Table K–L2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by county, Liberia Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs County Bomi 84.5 53.5 19.3 7.8 61.6 392 Bong 91.8 46.2 46.3 10.5 74.4 382 Grand Bassa 59.4 53.7 20.5 4.2 53.3 422 Grand Cape Mount 94.5 60.3 21.2 12.8 66.1 425 Lofa 96.3 47.7 47.3 8.5 73.8 408 Margibi 65.0 55.7 38.7 7.9 72.3 429 Montserrado 51.8 49.9 50.3 56.1 76.7 403 Nimba 87.4 69.0 43.5 6.5 58.4 436 Total 71.4 53.9 43.0 26.6 69.9 3297 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. K–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L3. Among surveyed households (HHs) that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by county, Liberia Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs County Bomi 74.2 56.4 10.9 6.9 65.4 55 Bong 84.3 44.8 56.0 17.9 81.6 40 Grand Bassa 74.5 65.5 44.8 11.4 73.1 34 Grand Cape Mount 97.4 62.9 23.1 25.6 79.4 42 Lofa 92.6 59.4 51.6 14.7 72.6 41 Margibi 78.0 78.0 43.9 12.0 76.0 50 Montserrado 44.6 56.2 64.2 73.0 97.1 35 Nimba 94.9 67.4 29.6 5.1 63.3 28 Total 70.4 59.5 49.7 35.6 82.2 325 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. Table K–L4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by county, Liberia Location % HHs whose main water source was con￾structed, built, or renovated in the last 3 years Number of HHs Among households whose main water source was constructed/built/renovated in the last 3 years, percentage (%) by source of construction/renovation Number of HHs Govt. Local/Intl. NGO Donor Orgn. Community Group Other County Bomi 84.4 392 2.1 58.7 5.1 18.8 15.2 331 Bong 78.8 382 10.2 23.9 2.9 37.1 25.9 301 Grand Bassa 74.1 422 4.5 18.1 7.4 51.7 18.3 313 Grand Cape Mount 83.3 425 8.4 67.7 4.9 12.0 7.1 354 Lofa 76.3 408 14.9 45.8 2.0 25.7 11.7 311 Margibi 85.3 429 4.9 19.3 4.2 34.1 37.5 366 Montserrado 78.1 403 17.2 31.0 4.5 33.2 14.0 315 Nimba 84.3 436 12.5 35.7 2.0 33.2 16.6 367 Total 80.0 3297 11.9 33.0 3.7 33.3 18.2 2638 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–21 Table K–L5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by county, Liberia Location % HHs with an observed handwashing station Number of HHs % HHs with an observed handwashing station AND water and soap or other cleansing agent(s)* Number of HHs County Bomi 12.0 392 30.7 47 Bong 6.8 382 68.0 26 Grand Bassa 50.1 422 20.5 211 Grand Cape Mount 21.3 425 43.5 91 Lofa 42.1 408 54.0 172 Margibi 34.7 429 24.1 149 Montserrado 37.4 403 56.9 151 Nimba 24.9 436 70.3 109 Total 31.4 3297 50.9 1035 * Excluding mud, ash, or sand. Table K–L6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by county, Liberia Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any as￾sistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs County Bomi 40.4 53.0 5.3 1.3 386 Bong 48.3 47.9 3.0 0.8 374 Grand Bassa 69.9 23.4 6.0 0.7 414 Grand Cape Mount 45.1 48.9 5.0 1.0 416 Lofa 48.2 45.7 5.3 0.8 391 Margibi 45.5 50.4 2.2 2.0 409 Montserrado 48.5 45.7 5.3 0.5 398 Nimba 48.0 46.4 4.4 1.2 426 Total 49.2 45.2 4.6 1.0 3214 * Excludes households that state received assistance from “other” sources. K–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by county and EVD survivorship, Liberia Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any assistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs County Bomi 48.6 51.4 0.0 0.0 21 Bong 50.0 50.0 0.0 0.0 2 Grand Bassa 100.0 0.0 0.0 0.0 3 Grand Cape Mount 26.3 63.1 10.6 0.0 11 Lofa 46.8 46.8 6.4 0.0 13 Margibi 40.1 49.9 10.1 0.0 20 Montserrado 75.0 25.0 0.0 0.0 4 Nimba 0.0 82.4 17.6 0.0 5 Total 48.4 46.1 5.6 0.0 79 * Excludes households that state received assistance from “other” sources. Table K–L8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by county, Liberia Location Percentage of households that received the following types of assistance in the last 3 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs County Bomi 38.0 27.7 13.2 11.9 7.8 5.6 10.9 199 Bong 23.4 26.2 13.4 14.8 3.1 3.1 6.7 183 Grand Bassa 17.7 14.0 5.4 6.7 3.1 2.3 5.1 84 Grand Cape Mount 28.4 27.6 13.2 18.5 5.0 4.3 8.2 294 Lofa 29.1 30.3 14.4 19.5 6.7 4.1 8.2 192 Margibi 32.6 30.3 10.7 16.6 6.1 6.1 9.6 238 Montserrado 23.6 33.5 11.4 19.3 3.2 4.5 7.1 202 Nimba 26.4 20.2 10.8 13.7 6.0 4.1 3.4 337 Total 25.4 27.8 11.4 16.4 4.5 4.2 6.7 1763 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–23 Table K–L9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by county, Liberia Location Percentage of households that received the following types of assistance in the last 12 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs County Bomi 49.0 34.8 18.8 17.9 13.1 8.4 14.2 392 Bong 34.3 34.1 17.3 18.7 6.2 5.8 10.3 382 Grand Bassa 23.0 18.3 9.0 8.6 5.2 4.0 8.1 422 Grand Cape Mount 37.3 34.0 17.7 24.2 6.7 28.4 9.8 425 Lofa 35.2 35.7 18.5 24.6 10.0 5.6 10.0 408 Margibi 41.0 39.4 16.1 20.5 7.7 6.5 11.7 429 Montserrado 32.0 40.2 16.3 22.8 4.5 6.2 8.9 403 Nimba 35.1 28.1 15.4 18.0 9.1 5.9 6.6 436 Total 33.8 34.8 16.0 20.3 6.7 5.9 9.1 3297 Table K–L10. Percent distribution of household (HH) members age 15 or older by their employment status, according to county, Liberia Location Percentage of HH members age 15 or older that were employed during the: Number of HH members age 15 or older Last 7 days Last 12 months but NOT in last 7 days Last 12 months County Bomi 60.8 14.3 75.1 392 Bong 52.9 16.0 68.9 382 Grand Bassa 41.1 12.7 53.7 422 Grand Cape Mount 47.9 16.9 65.3 425 Lofa 63.7 16.3 80.0 408 Margibi 58.2 15.4 73.4 429 Montserrado 69.3 12.7 81.8 403 Nimba 55.0 11.2 66.3 436 Total 60.1 13.6 73.7 3297 Table K–L11. Primary school net attendance ratio (NAR) by county, Liberia Location Primary school NAR* Number of children aged 6–11 years County Bomi 72.4 399 Bong 75.4 355 Grand Bassa 73.6 269 Grand Cape Mount 76.6 403 Lofa 89.2 378 K–24 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L11. Primary school net attendance ratio (NAR) by county, Liberia Location Primary school NAR* Number of children aged 6–11 years Margibi 72.4 421 Montserrado 81.9 219 Nimba 72.1 453 Total 77.5 2897 * The NAR for primary school is the percentage of the primary-school age (6–11 years) population that is attending primary school. Table K–L12. Percentage of heads of household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, Liberia Action Taken Percent Wash hands with soap/water more often 85.9 Wash hands with disinfectant more often 36.0 Avoid crowded places 26.5 Drink bitter-cola 2.8 Drink a lot of water/juice 1.8 Take traditional herbs 1.3 Take antibiotics 2.3 Wear gloves/protective barriers 8.2 Avoid touching people suspected to have Ebola 19.4 Avoid touching everyone 28.5 Avoid visiting homes of people with Ebola 15.1 Avoid touching dead bodies during or preparing for burial ceremonies 10.2 Wash with salt/hot water 1.2 Pray/pray more often 0.5 Report suspected cases of Ebola 0.8 Number of heads of households/household respondents 2686 Table K–L13. Among heads of the household (HH)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Liberia Action Prompts Percent Spoke to health worker/community health education 55.8 Listened to radio program(s) 43.3 Watched TV program(s) 13.1 Read billboard message(s)/education materials 9.3 Received advice from family/friends 15.4 Received instruction(s) from workplace/school 3.4 Lost someone important to me 3.3 Number of heads of households/household respondents who took specific actions 2686 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–25 Table K–L14. Percentage of heads of the household (HH)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by county, Liberia Location % HH respondents who would NOT go to HF if they had a fever Among heads of the households/household respondents who would NOT go to a health facility if they had fever, percentage by reason: No money for treat￾ment No HF nearby No money for trans￾port HF is con￾taminated with EVD HF is dirty/un￾sanitary I do not trust the health care staff Health care is of poor quality Health staff are not re￾spectful People may think I have EVD/ stigma I prefer to go to a pharmacy I prefer to go to a traditional healer I prefer to stay at home to recover County Bomi 3.6 29.2 14.6 7.3 0.0 0.0 6.2 34.3 0.0 0.0 13.5 7.3 0.0 Bong 0.5 45.4 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 54.6 0.0 Grand Bassa 2.7 75.0 16.7 41.7 0.0 0.0 0.0 33.3 0.0 0.0 50.0 8.3 16.7 Grand Cape Mount 5.1 48.8 30.7 23.1 0.0 0.0 2.5 10.3 2.5 2.5 35.9 0.0 0.0 Lofa 1.7 20.9 0.0 0.0 0.0 0.0 20.9 20.9 0.0 0.0 58.2 0.0 0.0 Margibi 5.6 47.7 8.7 8.7 8.7 13.1 4.4 30.6 4.4 0.0 39.2 0.0 4.4 Montserrado 5.2 57.9 0.0 10.5 5.3 0.0 10.5 0.0 5.3 21.1 10.5 0.0 15.8 Nimba 0.4 50.0 0.0 0.0 50.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 50.0 Total 3.3 54.2 4.9 12.3 5.5 1.7 8.3 8.9 4.0 13.2 20.2 1.9 12.6 K–26 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by prefecture, Liberia Location % women age 15–49 who would NOT go to HF if they had a fever Among female respondents aged 15–49 who would NOT go to a health facility if they had fever, percentage by reason: No money for treat- ment No HF nearby No money for trans- port HF is con- taminated with EVD HF is dirty/ unsanitary I do not trust the health care staff Health care is of poor quality Health staff are not respectful People may think I have EVD/stigma I prefer to go to a traditional healer I prefer to stay at home to recover Prefecture Bomi 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Bong 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Grand Bassa 1.0 0.0 0.0 0.0 5.0 0.0 0.0 0.0 0.0 0.0 100.0 0.0 Grand Cape Mount 0.4 100.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 0.0 0.0 0.0 Lofa 0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Margibi 4.1 33.3 0.0 0.0 16.5 0.0 0.0 0.0 0.0 16.8 0.0 50.2 Montserrado 1.9 50.0 0.0 0.0 0.0 0.0 50.0 0.0 0.0 0.0 0.0 0.0 Nimba 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Total 1.3 43.0 0.0 0.0 5.0 0.0 31.0 0.0 1.7 5.3 5.4 15.7 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–27 Table K–L16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by county, Liberia Location Mother received ANC during pregnancy Mother received ANC by a skilled health provider* Mother received 4 or more ANC visits Delivery oc￾curred at a health facility Delivery was as￾sisted by a skilled health provider* Number of HHs County Bomi 98.2 98.2 84.2 67.1 92.0 106 Bong 96.6 96.6 75.2 86.3 96.8 93 Grand Bassa 94.8 94.8 53.8 58.9 96.8 95 Grand Cape Mount 97.8 97.8 71.3 76.1 92.9 139 Lofa 91.6 86.8 73.1 80.7 93.9 84 Margibi 98.6 97.3 87.8 79.6 96.6 147 Montserrado 95.6 94.9 87.3 89.8 90.4 110 Nimba 87.6 86.0 69.6 75.2 89.0 113 Total 94.5 93.5 78.8 81.8 92.3 887 * Skilled provider includes doctor, nurse, midwife, health/physicians’ assistant or auxiliary health worker. Table K–L17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Liberia Characteristic HDDS Number of HHs HHS Number of HHs Household head Female 5.5 1115 1.5 1396 Male 6.2 1453 1.3 1872 Number of usual household members 1 5.6 82 1.0 90 2 5.5 201 1.2 248 3 5.7 370 1.2 488 4 6.0 494 1.3 632 5 5.8 428 1.5 533 6 5.8 289 1.4 368 7 6.1 249 1.4 316 8 6.3 136 1.2 175 9+ 6.3 319 1.5 418 Residence Urban 5.9 911 1.3 1143 Rural 5.8 1657 1.4 2125 County Bomi 5.6 338 1.6 387 Bong 5.8 249 1.5 372 Grand Bassa 5.5 373 1.8 420 Grand Cape Mount 5.8 333 1.4 421 K–28 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–L17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Liberia Characteristic HDDS Number of HHs HHS Number of HHs Lofa 5.3 303 1.1 405 Margibi 5.7 383 1.6 426 Montserrado 6.0 312 1.2 402 Nimba 6.4 277 1.3 435 Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 5.8 140 1.5 193 Primary 5.6 447 1.5 552 Secondary 6.0 585 1.3 801 Higher 6.4 630 1.0 784 Employment of the head of household Employed in the last 7 days 6.0 1434 1.2 1814 Employed in the last 12 months but not in the last 7 days 5.7 377 1.4 474 Employed in the last 12 months 6.0 1818 1.3 2295 Received assistance in the last 12 months None 5.4 1251 1.5 1577 Family/friends 6.3 1121 1.2 1430 Government/NGO/CBO/community group 6.7 106 1.1 145 Family/friends AND Government/NGO/ CBO/community group 7.0 26 1.2 33 Experienced shock in past 12 months Yes 5.9 2402 1.4 3063 No 5.3 166 0.9 205 Ebola Virus Disease (EVD) Household with no member diagnosed with EVD 5.9 2385 1.3 3044 Household with member diagnosed with EVD who survived 5.5 43 2.3 53 Household with member diagnosed with EVD who died 5.8 56 1.8 80 Socio-economic quintile Lowest 5.5 774 1.8 967 Second 5.9 586 1.2 764 Middle 5.6 526 1.6 666 Fourth 5.7 388 1.4 503 Highest 6.6 294 0.8 368 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–29 Table K–SL1. Percent distribution of households by selected background characteristics, Sierra Leone Characteristic Percent (%) Number of HHs Household head Female 26.6 852 Male 73.4 2383 Total 100.0 3235 Number of usual household members 1 1.2 31 2 3.7 110 3 8.9 289 4 14.6 478 5 23.2 746 6 15.7 507 7 10.5 351 8 7.0 240 9+ 15.2 483 Total 100.0 3235 Residence Urban 49.9 1515 Rural 50.1 1720 Total 100.0 3235 District Bombali 12.6 420 Kailahun 11.0 390 Kenema 12.7 379 Koinadugu 8.5 385 Port Loko 12.8 397 Tonkolili 11.1 399 Western Area Rural 9.3 468 Western Area Urban 22.0 397 Total 100.0 3235 Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 15.2 273 Primary 42.6 718 Secondary 25.3 403 Higher 17.0 257 Total 100.0 1651 Employment of the head of household Employed in the last 7 days 66.1 2144 Sierra Leone K–30 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SL1. Percent distribution of households by selected background characteristics, Sierra Leone Characteristic Percent (%) Number of HHs Employed in the last 12 months but not in the last 7 days 12.5 410 Employed in the last 12 months 78.7 2560 Ebola virus diseease (EVD) Household with no member diagnosed with EVD 91.8 2967 Household with member diagnosed with Ebola who survived 1.27 41 Household with member diagnosed with Ebola who died 2.29 74 Don't know/Not sure/Refused 4.7 153 Total 100.0 3235 Socio-economic quintile Lowest 20.8 702 Second 19.4 676 Middle 19.3 674 Fourth 20.0 655 Highest 20.5 528 Total 100.0 3235 Table K–SL2. Percentage of surveyed households (HHs) with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by district, Sierra Leone Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs District Bombali 86.3 51.3 56.0 21.7 71.6 420 Kailahun 67.7 33.0 22.4 2.9 53.6 390 Kenema 79.6 46.1 28.3 12.5 56.2 379 Koinadugu 82.0 67.1 46.7 4.1 59.7 385 Port Loko 58.1 29.2 40.2 11.5 71.2 397 Tonkolili 70.9 39.4 49.7 12.7 69.8 399 Western Area Rural 90.0 38.5 57.9 38.1 87.5 468 Western Area Urban 77.1 34.3 35.5 78.1 91.9 397 Total 76.0 40.9 40.9 28.6 72.3 3235 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–31 Table K–SL3. Among surveyed households (HHs) that received WASH input in the last 12 months, percentage with an improved source of drinking water, using an appropriate treatment method for their drinking water, using an improved non-shared latrine/toilet facility, and having selected household possessions, by district, Sierra Leone Location % HHs with im￾proved source of drinking water* % HHs using ap￾propriate treat￾ment method for drinking water** % HHs with improved non￾shared latrine/ toilet facililty*** % HHs with electricity % HHs that possess a mobile phone Number of HHs District Bombali 87.6 53.6 47.5 26.2 81.2 29 Kailahun 87.2 48.0 33.7 0.0 48.0 20 Kenema 94.8 56.3 40.0 23.5 72.2 37 Koinadugu 81.3 81.3 56.1 0.0 74.1 11 Port Loko 58.3 33.3 41.7 8.3 91.7 12 Tonkolili 68.8 42.2 47.1 21.7 84.4 16 Western Area Rural 96.9 46.9 71.9 43.8 100.0 18 Western Area Urban 83.7 41.9 44.2 83.7 90.7 72 Total 85.0 48.4 46.0 41.4 81.8 215 * Improved drinking water sources include a piped source within the dwelling, yard, or plot; a public tap, tube well, or borehole; a hand pump/protected well or protected spring; and rainwater or bottled water (WHO and UNICEF, 2012a) ** Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting *** A household is classified as having an improved toilet if the toilet is used only by members of one household (i.e., it is not shared) and if the facility used by the household separates the waste from human contact (WHO and UNICEF, 2012b). The types of facilities considered improved are toilets that flush or pour flush into a piped sewer system, septic tank, or pit latrine; ventilated improved pit (VIP) latrines; and pit latrines with a slab. Table K–SL4. Percentage of households (HHs) whose main water source was constructed, built, or renovated in the last 3 years, and among those, percentage by source of construction/renovation, by district, Sierra Leone Location % HHs whose main water source was con￾structed, built, or renovated in the last 3 years Number of HHs Among households whose main water source was constructed/built/renovated in the last 3 years, percentage (%) by source of construction/renovation Number of HHs Govt. Local/Intl. NGO Donor Orgn. Community Group Other District Bombali 90.0 420 16.0 41.4 3.2 14.9 24.5 378 Kailahun 51.2 390 19.0 48.8 0.9 13.6 17.6 199 Kenema 69.8 379 14.7 43.5 4.6 18.0 19.1 264 Koinadugu 49.2 385 32.1 23.7 4.3 17.6 22.4 190 Port Loko 48.5 397 14.5 37.0 6.9 24.1 17.6 192 Tonkolili 64.5 399 18.2 25.3 0.8 34.0 21.6 257 Western Area Rural 50.6 468 14.8 19.7 1.0 19.3 45.3 237 Western Area Urban 70.8 397 21.8 5.9 5.5 15.0 51.8 281 Total 63.4 3235 18.4 29.4 3.7 19.1 29.4 2051 K–32 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SL5. Percentage of surveyed households (HHs) with a handwashing station (observed), and among those, percentage that have water and soap, or detergent/other cleansing agents, by district, Sierra Leone Location % HHs with an observed handwashing station Number of HHs % HHs with an observed handwashing station AND water and soap or other cleansing agent(s)* Number of HHs District Bombali 25.7 420 37.4 108 Kailahun 25.2 390 28.2 98 Kenema 23.5 379 37.8 89 Koinadugu 43.0 385 19.5 166 Port Loko 20.9 397 20.3 83 Tonkolili 61.9 399 46.9 247 Western Area Rural 41.1 468 37.0 192 Western Area Urban 70.0 397 87.4 278 Total 41.4 3235 54.3 1339 * Excluding mud, ash, or sand. Table K–SL6. Percentage of surveyed households (HHs) by whether they received any type of assistance in the last 12 months and main sources of that assistance,* by district, Sierra Leone Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any as￾sistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs District Bombali 61.5 17.5 18.5 2.5 408 Kailahun 64.6 13.6 21.8 0.0 382 Kenema 64.9 21.9 12.4 0.8 375 Koinadugu 78.8 6.1 15.1 0.0 363 Port Loko 81.6 7.6 10.2 0.5 393 Tonkolili 81.9 8.5 9.3 0.3 376 Western Area Rural 65.0 32.4 2.2 0.4 443 Western Area Urban 54.4 38.5 5.7 1.4 351 Total 69.1 18.3 11.9 0.7 3091 * Excludes households that state received assistance from “other” sources. ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–33 Table K–SL7. Percentage of surveyed households (HHs) with at least one EVD death that received any type of assistance in the last 12 months and the main sources of assistance,* by district and EVD survivorship, Sierra Leone Location Percentage of HHs that did not receive any assistance in the last 12 months Percentage of HHs that received any assistance in the last 12 months from family or friends Percentage of HHs that received any assistance in the last in the last 12 months from the government, local/ intl. NGOs, CBOs, or community groups (e.g., farmers coops or associations) Percentage of HHs that received any assistance in the last 12 months from both family/friends and government, local/ intl. NGOs, CBOs, or community groups Number of HHs District Bombali 80.0 0.0 20.0 0.0 10 Kailahun 61.2 4.6 34.2 0.0 18 Kenema 50.0 34.2 15.9 0.0 6 Koinadugu 0.0 0.0 0.0 0.0 0 Port Loko 77.7 5.6 16.7 0.0 18 Tonkolili 100.0 0.0 0.0 0.0 6 Western Area Rural 100.0 0.0 0.0 0.0 7 Western Area Urban 62.5 25.0 12.5 0.0 8 Total 66.4 8.7 12.4 0.0 73 * Excludes households that state received assistance from “other” sources. Table K–SL8. Percentage of surveyed households (HHs) that received various types of assistance in the last 3 months, by district, Sierra Leone Location Percentage of households that received the following types of assistance in the last 3 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs District Bombali 9.7 10.5 12.1 5.7 3.4 1.2 4.9 167 Kailahun 13.4 9.2 3.7 6.2 2.7 2.3 3.9 176 Kenema 9.4 12.0 7.4 9.0 3.9 1.7 3.6 114 Koinadugu 5.4 3.7 3.2 7.8 4.5 0.8 2.6 0 Port Loko 5.6 5.1 2.5 4.8 2.0 0.8 2.0 73 Tonkolili 7.0 6.6 5.0 6.5 1.5 0.8 2.5 44 Western Area Rural 17.1 16.4 8.3 9.6 2.7 3.0 4.1 187 Western Area Urban 26.5 21.4 10.1 11.6 1.8 0.5 7.8 176 Total 13.3 11.8 6.9 8.0 2.7 1.3 4.4 1053 K–34 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SL9. Percentage of surveyed households (HHs) that received various types of assistance in the last 12 months, by district, Sierra Leone Location Percentage of households that received the following types of assistance in the last 12 months: Number of HHs Food Cash HH items Education Crop Inputs Livestock Inputs WASH Inputs District Bombali 15.8 18.4 11.7 14.3 8.8 3.8 7.7 420 Kailahun 12.1 16.1 13.2 9.3 4.7 1.7 6.1 390 Kenema 19.1 12.9 5.3 9.2 4.5 4.4 9.8 379 Koinadugu 8.7 5.4 4.7 12.4 5.3 0.8 4.1 385 Port Loko 9.3 7.6 3.8 6.6 2.8 0.8 3.0 397 Tonkolili 11.0 9.1 6.1 9.8 3.8 2.0 4.8 399 Western Area Rural 23.7 22.9 12.0 12.9 3.9 4.0 6.6 468 Western Area Urban 36.5 33.3 14.6 16.4 2.8 1.8 10.8 397 Total 19.1 17.6 9.5 11.8 4.4 4.4 7.1 3235 Table K–SL10. Percent distribution of household (HH) members age 15 or older by their employment status, according to district, Sierra Leone Location Percentage of HH members age 15 or older that were employed during the: Number of HH members age 15 or older Last 7 days Last 12 months but NOT in last 7 days Last 12 months District Bombali 54.3 17.0 71.3 420 Kailahun 79.4 3.8 83.2 390 Kenema 56.2 7.4 63.6 379 Koinadugu 57.5 25.6 83.3 385 Port Loko 66.7 16.7 83.4 397 Tonkolili 84.9 6.8 91.7 399 Western Area Rural 68.5 10.5 79.0 468 Western Area Urban 64.5 13.6 78.1 397 Total 66.1 12.5 78.7 3235 Table K–SL11. Primary school net attendance ratio (NAR) by district, Sierra Leone Location Primary school NAR* Number of children aged 6–11 years District Bombali 81.1 451 Kailahun 76.4 322 Kenema 73.4 317 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–35 Table K–SL11. Primary school net attendance ratio (NAR) by district, Sierra Leone Location Primary school NAR* Number of children aged 6–11 years Koinadugu 77.8 330 Port Loko 79.6 500 Tonkolili 90.0 298 Western Area Rural 91.7 433 Western Area Urban 95.5 311 Total 83.7 2962 * The NAR for primary school is the percentage of the primary-school age (6–11 years) population that is attending primary school. Table K–SL12. Percentage of heads of households (HHs)/household respondents who took specific actions to avoid being infected with Ebola virus, Sierra Leone Action Taken Percent Wash hands with soap/water more often 72.3 Wash hands with disinfectant more often 25.6 Avoid crowded places 32.1 Drink bitter-cola 0.4 Drink a lot of water/juice 0.6 Take traditional herbs 0.5 Take antibiotics 3.3 Wear gloves/protective barriers 8.2 Avoid touching people suspected to have Ebola 24.6 Avoid touching everyone 40.0 Avoid visiting homes of people with Ebola 21.0 Avoid touching dead bodies during or preparing for burial ceremonies 10.6 Wash with salt/hot water 1.2 Pray/pray more often 1.9 Report suspected cases of Ebola 1.6 Number of heads of households/household respondents 3235 Table K–SL13. Among heads of the households (HHs)/household respondents who took specific actions to avoid being infected with Ebola virus, reasons that prompted them to take those actions, Sierra Leone Action Prompts Percent Spoke to health worker/community health education 54.2 Listened to radio program(s) 48.5 Watched TV program(s) 11.7 Read billboard message(s)/education materials 3.2 Received advice from family/friends 15.4 Received instruction(s) from workplace/school 4.2 Lost someone important to me 2.7 Number of heads of households/household respondents who took specific actions 3235 K–36 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SL14. Percentage of heads of the households (HHs)/household respondents who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by district, Sierra Leone Location % HH respondents who would NOT go to HF if they had a fever Among heads of the households/household respondents who would NOT go to a health facility if they had fever, percentage by reason: No money for treat- ment No HF nearby No money for trans- port HF is con- taminated with EVD HF is dirty/un- sanitary I do not trust the health care staff Health care is of poor quality Health staff are not re- spectful People may think I have EVD/ stigma I prefer to go to a pharmacy I prefer to go to a traditional healer I prefer to stay at home to recover District Bombali 2.1 62.2 12.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 25.5 Kailahun 0.9 23.3 23.3 0.0 0.0 0.0 0.0 0.0 0.0 23.3 0.0 0.0 30.0 Kenema 4.7 53.5 6.2 23.5 0.0 6.2 0.0 0.0 5.8 17.3 12.0 17.3 5.8 Koinadugu 3.3 32.1 44.7 8.0 0.0 0.0 0.0 8.0 0.0 0.0 0.0 0.0 31.2 Port Loko 8.0 78.2 15.7 12.6 0.0 0.0 0.0 3.1 0.0 6.2 3.2 3.2 6.2 Tonkolili 2.8 45.5 9.0 0.0 0.0 0.0 0.0 0.0 0.0 45.0 9.0 9.5 0.0 Western Area Rural 10.2 40.2 2.1 0.0 0.0 0.0 2.1 11.3 0.0 20.7 2.1 10.3 25.8 Western Area Urban 16.1 50.0 5.0 10.0 0.0 0.0 0.0 1.7 0.0 31.7 3.3 10.0 31.7 Total 7.1 52.3 8.7 9.1 0.0 0.5 0.3 3.2 0.5 23.1 3.8 8.7 23.2 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–37 Table K–SL15. Percentage of women age 15–49 who would NOT go to a health facility (HF) if they had fever and the reasons for doing so, by district, Sierra Leone Location % women age 15–49 who would NOT go to HF if they had a fever Among female respondents aged 15–49 who would NOT go to a health facility if they had fever, percentage by reason: No money for treat￾ment No HF nearby No money for trans- port HF is con- taminated with EVD HF is dirty/ unsanitary I do not trust the health care staff Health care is of poor quality Health staff are not respectful People may think I have EVD/stigma I prefer to go to a traditional healer I prefer to stay at home to recover District Bombali 2.9 100.0 33.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Kailahun 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Kenema 5.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Koinadugu 4.2 27.3 72.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Port Loko 1.1 100.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Tonkolili 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Western Area Rural 4.2 14.3 0.0 0.0 0.0 0.0 0.0 28.6 0.0 0.0 0.0 28.6 Western Area Urban 8.8 70.0 10.0 20.0 0.0 0.0 0.0 0.0 10.0 0.0 10.0 0.0 Total 4.1 59.4 15.7 12.0 0.0 0.0 0.0 4.1 6.0 0.0 6.0 4.1 K–38 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SL16. Percentage of most recent live births since October 2014 by antenatal care received (ANC) and delivery care characteristics, by district, Sierra Leone Location Mother received ANC during pregnancy Mother received ANC by a skilled health provider* Mother received 4 or more ANC visits Delivery oc￾curred at a health facility Delivery was as￾sisted by a skilled health provider* Number of HHs District Bombali 100.0 100.0 90.1 90.2 96.2 101 Kailahun 99.1 98.2 61.5 96.5 97.6 89 Kenema 95.2 95.2 67.6 90.8 72.4 22 Koinadugu 95.8 93.5 60.6 92.6 91.5 91 Port Loko 97.0 95.9 76.4 73.2 73.2 97 Tonkolili 92.0 92.0 76.0 77.5 79.1 62 Western Area Rural 97.1 97.1 68.6 82.4 85.9 167 Western Area Urban 99.1 99.1 70.4 85.2 90.4 115 Total 97.7 97.3 72.1 85.4 87.7 744 * Skilled provider includes doctor, nurse, midwife, health/physicians’ assistant or auxiliary health worker. Table K–SL17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Sierra Leone Characteristic HDDS Number of HHs HHS Number of HHs Household head Female 5.3 634 1.5 832 Male 5.3 1754 1.5 2346 Number of usual household members 1 5.2 24 1.8 31 2 5.0 80 1.2 106 3 5.4 202 1.5 281 4 5.1 360 1.4 468 5 5.3 553 1.6 735 6 5.3 382 1.6 496 7 5.4 253 1.4 349 8 5.2 175 1.4 238 9+ 5.4 359 1.4 474 Residence Urban 5.7 1100 1.2 1485 Rural 4.9 1288 1.7 1693 District Bombali 5.4 282 1.8 411 Kailahun 4.0 316 1.5 384 Kenema 5.4 262 2.0 374 ANNEX K. HOUSEHOLD SURVEY DATA TABLES n K–39 Table K–SL17. Household Dietary Diversity Score (HDDS) and Household Hunger Score (HHS), by household characteristics, Sierra Leone Characteristic HDDS Number of HHs HHS Number of HHs Koinadugu 4.4 250 1.8 372 Port Loko 4.8 303 1.5 397 Tonkolili 5.6 328 1.3 396 Western Area Rural 5.7 355 1.1 460 Western Area Urban 6.2 292 1.1 384 Highest level of schooling completed by the head of household Pre-primary/Primary incomplete 5.2 208 1.6 266 Primary 5.5 542 1.5 705 Secondary 6.2 299 1.1 397 Higher 6.3 175 0.8 255 Employment of the head of household Employed in the last 7 days 5.5 1595 1.4 2110 Employed in the last 12 months but not in the last 7 days 4.9 293 1.6 398 Employed in the last 12 months 5.4 1892 1.4 2513 Received assistance in the last 12 months None 4.9 1601 1.6 2098 Family/friends 6.4 393 1.1 565 Government/NGO/CBO/community group 5.5 263 1.5 354 Family/friends AND Government/NGO/ CBO/community group 7.2 18 1.0 22 Experienced shock in past 12 months Yes 5.3 2278 1.5 3040 No 4.8 110 1.0 138 Ebola Virus Disease (EVD) Household with no member diagnosed with EVD 5.4 2208 1.5 2921 Household with member diagnosed with EVD who survived 4.5 26 2.0 39 Household with member diagnosed with EVD who died 4.3 48 2.0 71 Socio-economic quintile Lowest 4.3 528 2.0 689 Second 5.0 516 1.6 667 Middle 5.2 488 1.6 662 Fourth 5.6 485 1.4 642 Highest 6.4 371 0.9 518 K–40 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES HEALTH FACILITY SURVEY RESULTS PE1 (HFS, 2017) Guinea Table K–GF1. Number, types, location and managing authority of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Number of surveyed HFs 248 16 19 19 17 19 20 20 20 24 19 19 19 17 Type of health facility Clinic 32 0 1 7 11 0 5 0 0 4 2 0 0 2 Health Center 12 1 1 1 2 1 1 1 1 1 1 1 0 0 Hospital 66 5 3 4 2 6 4 4 6 8 8 4 5 7 Health Post 138 10 14 7 2 12 10 15 13 11 8 14 14 8 Location Urban 68 0 4 11 17 2 7 2 4 9 6 1 0 5 Rural 180 16 15 8 0 17 13 18 16 15 13 18 19 12 Managing authority Public 211 16 17 11 4 19 15 20 20 19 17 19 19 15 Private not for profit 4 0 0 0 0 0 4 0 0 0 0 0 0 0 Private for profit 32 0 2 8 12 0 1 0 0 5 2 0 0 2 Mission 1 0 0 0 1 0 0 0 0 0 0 0 0 0 Table K–GF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Number of surveyed HFs 248 16 19 19 17 19 20 20 20 24 19 19 19 17 Staff IPC training and supervision HF reported sufficient HR to provide services offered at the HF 145 10 18 19 13 3 10 10 11 15 11 7 9 9 HW received IPC protocol training within past 2 years 186 13 12 11 13 15 16 14 14 21 18 11 12 13 HF has standard guidelines for IPC, observed by interviewer 147 8 10 13 13 13 13 13 8 13 14 8 10 11 ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–41 Table K–GF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré HF received supervision from a higher level at least once in last 3 months 197 11 16 14 9 16 16 18 17 21 17 13 16 13 Availability of basic services HF provides FP services 153 10 11 15 4 16 14 11 14 17 14 10 7 10 HF provides FP services at community level 135 10 5 5 4 16 14 11 13 16 14 10 7 10 HF provides ANC services 162 8 16 9 9 16 16 11 12 20 16 8 9 12 HF provides child preventive & curative services for children <5 years 171 8 17 9 8 17 14 13 14 19 17 11 11 13 HF provides immunization services 114 6 7 9 5 13 11 6 8 11 13 8 7 10 HF provides adolescent health services 190 8 19 15 13 14 17 13 14 21 17 13 12 14 HF provides diagnosis and treatment for malaria services 246 16 19 19 17 19 20 20 20 24 19 18 19 16 Availability of specialized services HF offers care and support for Ebola survivors 5 0 0 0 1 1 1 1 0 0 1 0 0 0 Medicines and commodities HF stocks medicines, vaccines and contraceptive commodities 87 3 9 13 3 6 5 3 8 12 11 4 4 6 HF suffers delay in delivery of medicines and commodities 34 1 4 4 1 2 3 1 3 6 6 0 1 2 HF submits the LMIS reports 59 3 3 6 1 2 4 3 6 10 9 3 4 5 IPC Infrastructure HF has improved water source 190 12 10 13 15 17 17 16 17 17 16 13 15 12 HF has safe methods of sharp waste disposal 197 14 18 14 13 17 16 14 15 19 16 15 14 12 HF has clean running water and soap, observed by the interviewer 129 4 6 11 11 7 13 7 13 12 15 8 11 11 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage K–42 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–GF3. Infrastructure and record keeping of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Number of surveyed HFs 248 16 19 19 17 19 20 20 20 24 19 19 19 17 HF infrastructure Had any renovations done in last 3 years 63 3 5 9 6 8 5 3 4 8 1 3 4 4 —Source of provider for renovation Other Other MOH+ other Other Other Other Other MOH+ other NGO Other Other Other Other MOH+ other Has functioning telephone 76 6 4 6 8 5 5 4 3 10 9 10 3 3 Has access to internet 52 4 3 5 8 2 3 1 1 8 4 5 3 5 Has a source of electricity 128 7 6 15 16 5 11 5 8 15 16 5 9 10 —Most common source of electricity Solar Solar Solar Solar Gener￾ator Solar Gener￾ator Solar Solar Gener￾ator Solar Solar Solar Solar —Most common backup source of electricity None None None None None None Solar None None None None None None None Record keeping HF collects service statistics at least monthly 165 10 8 8 9 12 13 14 16 16 16 16 13 14 HF sends the service statistics to district office 154 11 5 5 7 12 12 14 16 15 16 16 11 14 —Sends data in paper form 89 6 1 0 3 10 6 7 13 10 8 8 8 9 —Sends data in electronic form 7 1 1 0 0 0 3 0 0 1 1 0 0 0 HF records disease surveillance data 155 11 3 8 8 13 10 13 16 14 17 16 12 14 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage Table K–GF4. Facility management of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Number of surveyed HFs 248 16 19 19 17 19 20 20 20 24 19 19 19 17 Management HF has a Facility Management Committee 125 10 7 7 10 11 9 8 11 13 13 7 10 9 HF has Village development Committee in the catchment communities 57 6 1 0 3 5 5 4 7 4 9 2 6 5 HF has an operational budget for this fiscal year 62 6 2 2 9 2 6 7 3 9 7 3 2 4 ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–43 Table K–GF4. Facility management of surveyed health facilities (HFs) by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré HF has a system for determining client's opinion about the HF services 95 4 1 3 12 9 10 6 10 12 11 7 3 7 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 141 10 9 11 14 11 8 14 12 9 13 11 7 12 —if yes, these regulations were introduced as a result of Ebola epidemic 76 5 5 8 6 3 5 8 8 3 12 5 3 5 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50–79% coverage; Red: less than 50% coverage Table K–GF5. Status of health facilities (HFs) during the EVD epidemic by prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Number of surveyed HFs 248 16 19 19 17 19 20 20 20 24 19 19 19 17 HF was closed during EVD crisis (2014–2015) If closed, the year it opened again: 34 0 1 6 0 2 2 2 3 3 1 6 4 4 2014 4 0 1 0 0 0 0 0 0 1 0 0 0 2 2015 10 0 0 2 0 0 0 1 0 2 1 0 3 1 2016 10 0 0 4 0 0 2 0 2 0 0 0 2 1 2017 10 0 0 0 0 2 0 1 1 0 0 6 0 0 Table K–GF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and prefecture, Guinea GUINEA Total Beyla Boffa Boké Conakry Faranah Forécariah Guéckédou Kankan Kinda Labé Lola Macenta N’Zérékoré N n n n n n n n n n n n n n Mean number of outpatient clients per calendar year 2013 1035 569 223 1858 2237 700 1024 1254 662 1106 1001 211 1231 1474 2014 837 440 229 1943 1683 612 627 622 825 1052 1160 284 601 792 2015 1101 509 265 2745 1898 662 752 627 1545 1199 1712 332 622 1467 2016 1304 623 564 2730 2156 891 1003 1536 1571 1543 1853 418 687 1282 2017 1804 931 583 3297 2886 1671 1382 1809 1938 2848 2529 521 938 1830 K–44 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–GF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Guinea GUINEA Total GHET-supported non-GHET￾supported p-value (chi2 ) N % n % n % Number of HFs 248 100% 39 16% 209 84% General indicators HF has sufficient HR to provide services that are offered at the HF 145 58% 25 64% 120 57% 0.44 HF has standard guidelines for IPC, observed 147 59% 36 92% 111 53% 0.00 HF offers care and support for Ebola survivors 5 2% 2 5% 3 1% 0.01 HF has clean running water 129 52% 25 64% 104 50% 0.10 HF received supervision from a higher level at least once in last 3 months 197 79% 37 95% 160 77% 0.03 HF has village development Committee in the catchment communities 58 23% 16 41% 42 20% 0.59 HF reported sufficient funds to meet its operating budget 18 25% 5 19% 13 28% 0.40 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 141 57% 30 77% 111 53% 0.01 HF has forms to collect service statistics 166 67% 35 90% 131 63% 0.00 HF has recording forms to record disease surveillance data 155 63% 36 92% 119 57% 0.00 Service Statistics: Mean number of outpatient clients of surveyed health facilities, by calendar year Total GHET-supported non-GHET￾supported p-value (t-test) 2017 1804 2999 1580 0.02 2016 1304 2835 1018 0.00 2015 1101 2366 865 0.00 2014 837 2132 596 0.00 2013 1035 2744 717 0.00 * Note: 39 HFs (6 in Beyla, 3 in Boffa, 3 in Boke, 1 in Conakry, 4 in Forécariah, 5 in Guéckédou, 5 in Kindia, 5 in Lola, 5 in Macenta and 2 in N’Zérékoré prefectures) are Global Health Ebola Team (GHET)-supported (e.g. MCPS). ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–45 Liberia Table K–LF1. Number, types, location and managing authority of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 Type of health facility Clinic 133 19 18 16 17 17 14 14 18 Health Center 12 0 0 1 2 1 4 3 1 Hospital 8 1 1 2 0 1 1 2 0 Health Post 153 20 19 19 19 19 19 19 19 Location Urban 48 2 2 9 0 0 0 17 9 Rural 105 18 17 10 19 19 10 2 10 Managing authority Public 108 18 14 14 19 18 9 2 14 Private not for profit 13 1 2 2 0 0 3 3 2 Private for profit 32 1 3 3 0 1 7 14 3 Table K–LF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 Staff IPC training and supervision HF reported sufficient HR to provide services offered at the HF 98 17 10 13 8 8 13 18 11 HW received IPC protocol training within past 2 years 127 16 16 19 17 14 17 13 15 HF has standard guidelines for IPC, observed by interviewer 112 11 11 16 16 13 17 13 15 HF received supervision from a higher level at least once in last 3 months 142 17 16 19 19 18 18 16 18 Availability of basic services HF provides FP services 129 13 14 15 19 19 19 12 18 HF provides FP services at community level 111 11 14 15 14 16 18 6 17 HF provides ANC services 123 10 15 15 18 18 19 11 17 K–46 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–LF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba HF provides child preventive & curative services for children <5 years 92 6 15 9 18 11 12 7 14 HF provides immunization services 126 14 13 15 19 18 16 12 19 HF provides adolescent health services 90 8 13 9 15 17 8 10 10 HF provides diagnosis and treatment for malaria services 142 14 19 16 19 19 19 18 18 Availability of specialized services HF offers care and support for Ebola survivors 6 0 1 0 2 2 0 1 0 Medicines and commodities HF stocks medicines, vaccines and contraceptive commodities 90 11 11 12 12 14 7 11 12 HF suffers delay in delivery of medicines and commodities 68 8 10 7 10 12 5 6 10 HF submits the LMIS reports 86 11 11 11 11 13 7 10 12 IPC Infrastructure HF has improved water source 134 18 17 15 18 15 18 15 18 HF has safe methods of sharp waste disposal 141 17 18 18 18 18 16 18 18 HF has clean running water and soap, observed by the interviewer 135 20 16 13 17 15 18 18 18 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage Table K–LF3. Infrastructure and record keeping of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 HF infrastructure Had any renovations done in last 3 years 60 11 6 12 10 6 5 5 5 —Source of provider for renovation Other Other MOH NGO Other NGO MOH Other MOH and other Has functioning telephone 38 6 3 4 4 4 8 4 5 Has access to internet 19 2 2 5 0 0 3 6 1 ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–47 Table K–LF3. Infrastructure and record keeping of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba Has a source of electricity 141 18 17 19 18 17 17 18 17 —Most common source of electricity Solar Solar Solar Solar Solar Solar Solar Central supply Solar —Most common backup source of electricity None Generator None None None None Generator Generator None Record keeping HF collects service statistics at least monthly 138 16 19 17 17 17 18 16 18 HF sends the service statistics to district office 126 15 13 15 17 16 17 16 17 —Sends data in paper form 93 14 10 12 16 14 7 8 12 —Sends data in electronic form 3 0 1 0 1 0 1 0 0 HF records disease surveillance data 140 17 18 17 17 17 17 18 19 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage Table K–LF4. Facility management of surveyed health facilities (HFs) by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 Management HF has a Facility Management Committee 97 13 13 9 14 12 8 13 15 HF has Village development Committee in the catchment communities 59 6 10 6 9 10 4 2 12 HF has an operational budget for this fiscal year 26 4 2 4 2 1 4 8 1 HF has a system for determining client's opinion about the HF services 68 9 11 10 10 5 3 9 11 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 127 15 16 17 19 14 15 14 17 —if yes, these regulations were introduced as a result of Ebola epidemic 42 2 7 4 7 9 3 6 4 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50–79% coverage; Red: less than 50% coverage K–48 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–LF5. Status of health facilities (HFs) during the EVD epidemic by county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 HF was closed during EVD crisis (2014–2015) If closed, the year it opened again: 23 4 3 3 0 1 7 4 1 2014 8 1 1 1 0 1 3 0 1 2015 10 1 1 2 0 0 4 2 0 2016 4 2 1 0 0 0 0 1 0 2017 1 0 0 0 0 0 0 1 0 Table K–LF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and county, Liberia LIBERIA Total Bomi Bong Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba N n n n n n n n n Number of surveyed HFs 153 20 19 19 19 19 19 19 19 Mean number of outpatient clients per calendar year 2013 4166 1656 4010 4262 2184 1100 1516 6290 12440 2014 2740 2410 4553 2494 2060 984 4100 394 4943 2015 3486 2057 4729 6064 2181 1674 2466 1642 7148 2016 5110 2896 7391 6585 3653 2366 3781 3627 10697 2017 8652 2620 7521 6544 3477 2146 3973 3892 39357 ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–49 Table K–LF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Liberia LIBERIA Total GHET-supported non-GHET￾supported p-value (chi2 ) N % n % n % Number of HFs 153 100% 35 23% 118 77% General indicators HF has sufficient HR to provide services that are offered at the HF 98 64% 19 54% 79 67% 0.17 HF has standard guidelines for IPC, observed 112 73% 17 49% 85 72% 0.55 HF offers care and support for Ebola survivors 6 4% 1 3% 5 4% 0.58 HF has clean running water 135 88% 28 80% 107 91% 0.08 HF received supervision from a higher level at least once in last 3 months 7 5% 2 6% 5 4% 0.74 HF has village development Committee in the catchment communities 59 39% 14 40% 45 38% 0.34 HF reported sufficient funds to meet its operating budget 8 5% 2 6% 6 5% 0.39 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 127 83% 32 91% 95 81% 0.131 HF has forms to collect service statistics 138 90% 32 91% 106 90% 0.78 HF has recording forms to record disease surveillance data 140 92% 32 91% 108 92% 0.98 Service Statistics: Mean number of outpatient clients of surveyed health facilities, by calendar year Total GHET-supported non-GHET￾supported p-value (t-test) 2017 8652 20580 5114 0.07 2016 5110 7752 4327 0.07 2015 3485 5292 2949 0.10 2014 2740 3270 2582 0.58 2013 4166 8047 3014 0.13 * Note: 35 HFs (19 in Grand Bassa, 7 in Lofa and 9 in Nimba counties) are Global Health Ebola Team (GHET)-supported (e.g. MCPS) K–50 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Sierra Leone Table K–SLF1. Number, types, location and managing authority of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 Type of health facility Clinic 10 1 0 1 1 1 1 3 2 Health Center 28 3 4 3 3 4 2 3 6 Hospital 11 1 1 1 1 1 1 2 3 Health Post 79 9 12 15 9 14 11 2 7 Location Urban 56 5 4 8 4 6 2 9 18 Rural 72 9 13 12 10 14 13 1 0 Managing authority Public 112 12 17 18 13 19 14 6 13 Private not for profit 8 1 0 2 0 1 1 2 1 Private for profit 6 1 0 0 1 0 0 1 3 Mission 2 0 0 0 0 0 0 1 1 Table K–SLF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 Staff IPC training and supervision HF reported sufficient HR to provide services offered at the HF 50 5 6 4 3 5 8 7 12 HW received IPC protocol training within past 2 years 120 14 17 18 12 17 15 9 18 HF has standard guidelines for IPC, observed by interviewer 91 12 17 10 11 16 13 1 11 HF received supervision from a higher level at least once in last 3 months 116 13 17 19 13 19 13 7 15 Availability of basic services HF provides FP services 118 13 17 18 14 19 14 8 15 HF provides FP services at community level 108 11 16 15 13 18 14 7 14 ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–51 Table K–SLF2. Staffing, availability of services, and IPC infrastructure of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban HF provides ANC services 120 14 17 19 13 20 14 9 14 HF provides child preventive & curative services for children <5 years 100 11 16 17 11 15 12 7 11 HF provides immunization services 115 13 17 18 14 19 13 8 13 HF provides adolescent health services 97 13 16 15 10 12 14 7 10 HF provides diagnosis and treatment for malaria services 127 14 17 19 14 20 15 10 18 Availability of specialized services HF offers care and support for Ebola survivors 24 8 2 1 4 3 3 2 1 Medicines and commodities HF stocks medicines, vaccines and contraceptive commodities 83 13 14 16 4 14 4 8 10 HF suffers delay in delivery of medicines and commodities 56 10 10 14 4 8 2 4 4 HF submits the LMIS reports 77 13 14 15 3 14 4 6 8 IPC Infrastructure HF has improved water source 106 11 14 16 12 19 9 8 17 HF has safe methods of sharp waste disposal 117 14 11 19 13 20 14 10 16 HF has clean running water and soap, observed by the interviewer 119 14 15 19 13 20 14 8 16 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage Table K–SLF3. Infrastructure and record keeping of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 HF infrastructure Had any renovations done in last 3 years 77 11 6 9 5 18 10 6 12 —Source of provider for renovation MOH other MOH MOH MOH other other MOH MOH Has functioning telephone 30 2 0 2 1 5 5 7 8 K–52 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SLF3. Infrastructure and record keeping of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban Has access to internet 12 0 0 0 1 2 2 2 5 Has a source of electricity 95 11 15 8 9 18 9 9 16 —Most common source of electricity Solar Solar Solar Solar Solar Solar Solar Solar and Gentr. Central supply —Most common backup source of electricity None None None None None None None None Generator Record keeping HF collects service statistics at least monthly 112 14 17 16 12 17 13 9 14 HF sends the service statistics to district office 107 14 17 15 12 17 14 7 11 —Sends data in paper form 81 12 17 9 12 9 12 2 8 —Sends data in electronic form 5 1 0 0 0 2 0 1 1 HF records disease surveillance data 114 14 14 16 12 18 15 10 15 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50-79% coverage; Red: less than 50% coverage Table K–SLF4. Facility management of surveyed health facilities (HFs) by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 Management HF has a Facility Management Committee 120 14 17 18 13 18 15 10 15 HF has Village development Committee in the catchment communities 99 12 16 17 11 15 14 6 8 HF has an operational budget for this fiscal year 22 3 1 0 4 3 5 3 3 HF has a system for determining client's opinion about the HF services 71 12 7 10 10 10 10 8 4 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 112 14 15 17 9 20 13 10 14 —if yes, these regulations were introduced as a result of Ebola epidemic 26 2 2 3 5 2 7 0 5 Legend: Green: Greater or equal to 80% coverage; Yellow: Between 50–79% coverage; Red: less than 50% coverage ANNEX K. HEALTH FACILITY SURVEY DATA TABLES n K–53 Table K–SLF5. Status of health facilities (HFs) during the EVD epidemic by district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 HF was closed during EVD crisis (2014–2015) If closed, the year it opened again: 11 0 1 0 1 3 1 2 3 2014 1 0 1 0 0 0 0 0 0 2015 5 0 0 0 1 1 1 1 1 2016 5 0 0 0 0 2 0 1 2 2017 0 0 0 0 0 0 0 0 0 Table K–SLF6. Service statistics for mean number of outpatient clients of surveyed health facilities (HFs) by calendar year and district, Sierra Leone SIERRA LEONE Total Bombali Kailahun Kenema Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban N n n n n n n n n Number of surveyed HFs 128 14 17 20 14 20 15 10 18 Mean number of outpatient clients per calendar year 2013 1261 1092 1492 3864 91 114 2179 552 98 2014 1459 1152 1648 4322 91 113 2941 612 109 2015 1380 640 386 5412 91 127 2840 428 121 2016 1414 1459 2017 3112 311 118 3030 1467 148 2017 1702 1857 1830 2318 1477 269 4371 1346 590 Table K–SLF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Sierra Leone SIERRA LEONE Total GHET-supported non-GHET￾supported p-value (chi2 ) N % n % n % Number of HFs 128 100% 15 12% 113 88% General indicators HF has sufficient HR to provide services that are offered at the HF 50 39% 4 27% 46 41% 0.40 HF has standard guidelines for IPC, observed 91 71% 12 80% 79 70% 0.55 HF offers care and support for Ebola survivors 24 19% 5 33% 19 17% 0.26 HF has clean running water 119 93% 15 100% 104 92% 0.60 HF received supervision from a higher level at least once in last 3 months 8 6% 1 7% 7 6% 1.00 K–54 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table K–SLF7. Comparison of selected indicators between GHET and non-GHET supported health facilities (HFs),* Sierra Leone SIERRA LEONE Total GHET-supported non-GHET￾supported p-value (chi2 ) N % n % n % HF has village development Committee in the catchment communities 99 77% 13 87% 86 76% 1.00 HF reported sufficient funds to meet its operating budget 7 5% 0 0% 7 6% 0.55 HF reported existence of any existing government regulations that help the facility operate the way it is operating today 112 88% 14 93% 98 87% 0.69 HF has forms to collect service statistics 112 88% 13 87% 99 88% 0.75 HF has recording forms to record disease surveillance data 114 89% 14 93% 100 88% 1.00 Service Statistics: Mean number of outpatient clients of surveyed health facilities, by calendar year TOTAL GHET-supported non-GHET￾supported p-value (t-test) 2017 1702 1465 1732 0.72 2016 1414 427 1545 0.26 2015 1380 713 1469 0.73 2014 1459 532 1582 0.56 2013 1261 479 1365 0.57 * Note: 15 HFs (4 in Porto Loko, 4 in Tinkolili and 3 in Western Area Urban ) are Global Health Ebola Team (GHET)-supported (e.g. MCPS) A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–1 ANNEX L. TREND TABLES, PRE- TO POST-EVD Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document GUINEA TREND TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–2 Table L–G1. Trends over time in antenatal and delivery care, Guinea, 2012–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–2 Table L–G2. Trends over time in water and sanitation, Guinea, 2012–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–4 Table L–G3. Trends over time in primary school net attendance ratio (NAR), Guinea, 2012–2017 . . . . . . . . . . . . . . . . . . . . . . L–6 Table L–G4. Trends over time in agriculture and food security, Guinea, 2012–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–7 Table L–G5. Trends over time in household electricity and communications technology, Guinea, 2012–2017 . . . . . . . . . . . . . . L–8 Table L–G6. Trends over time in economic growth, Guinea, 2013–2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–9 LIBERIA TREND TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–10 Table L–L1. Trends over time in antenatal and delivery care, Liberia, 2013–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–10 Table L–L2. Trends over time in water and sanitation, Liberia, 2013–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–11 Table L–L3. Trends over time in primary school net attendance ratio (NAR), Liberia, 2013–2017 . . . . . . . . . . . . . . . . . . . . . . L–13 Table L–L4. Trends over time in agriculture and food security, Liberia, 2015–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–14 Table L–L5. Trends over time in household electricity and communications technology, Liberia, 2013–2017 . . . . . . . . . . . . . . L–14 Table L–L6. Trends over time in economic growth, Liberia, 2013–2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–15 SIERRA LEONE TREND TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–16 Table L–SL1. Trends over time in antenatal and delivery care, Sierra Leone, 2013–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–16 Table L–SL2. Trends over time in water and sanitation, Sierra Leone, 2013–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–17 Table L–SL3. Trends over time in primary school net attendance ratio (NAR), Sierra Leone, 2013–2017 . . . . . . . . . . . . . . . . L–19 Table L–SL4. Trends over time in agriculture and food security, Sierra Leone, 2015–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–20 Table L–SL5. Trends over time in household electricity and communications technology, Sierra Leone, 2013–2017 . . . . . . . . L–20 Table L–SL6. Trends over time in health facility characteristics, Sierra Leone, 2012–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–21 Table L–SL7. Trends over time in economic growth, Sierra Leone, 2013–2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . L–22 L–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES GUINEA TREND TABLES In many instances in the following tables, baseline data in Guinea were available only for the full Region rather than by individual prefecture. Therefore, an (R) appearing next to certain baseline data indicates that the data were for the Region of that name, which may include more than one Prefecture. Data not available have been intentionally left blank. Table L–G1. Trends over time in antenatal and delivery care, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 20121 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of women age 15–49 who attended at least 4 ANC visits for most recent pregnancy National 56.6 National, rural 48.5 National, urban 77.1 Priority areas Total 65.8 Boké Boké 72.0 Boffa 72.2 Conakry Conakry 78.8 Faranah Faranah 46.7 Kankan Kankan 13.4 Kindia Kindia 67.6 Forécariah 69.2 Labé Labé 76.7 N’Zérékoré Beyla 0.0 Guéckédou 56.6 Lola 56.2 Macenta 55.6 N’Zérékoré 66.3 Percentage of live births attended by a skilled provider2 National 45.3 National, rural 31.6 National, urban 83.9 Priority areas Total 80.5 Boké Boké 39.9 (R) 77.7 Boffa 77.8 Conakry Conakry 91.1 95.5 Faranah Faranah 29.1 (R) 40.2 Kankan Kankan 43.5 (R) 91.2 Kindia Kindia 41.1 (R) 64.0 Forécariah 69.1 Labé Labé 31.9 (R) 83.5 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–3 Table L–G1. Trends over time in antenatal and delivery care, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 20121 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 N’Zérékoré Beyla 42.5 (R) 72.9 Guéckédou 72.4 Lola 72.4 Macenta 69.5 N’Zérékoré 79.5 Percentage of births delivered in a health facility3 National 40.3 National, rural 29.3 National, urban 71.2 Priority areas Total 72.6 Boké Boké 37.5 (R) 75.6 Boffa 62.7 Conakry Conakry 80.8 87.9 Faranah Faranah 29.0 (R) 51.8 Kankan Kankan 40.4 (R) 64.2 Kindia Kindia 36.0 (R) 56.7 Forécariah 64.0 Labé Labé 26.9 (R) 72.8 N’Zérékoré Beyla 34.2 (R) 82.1 Guéckédou 60.1 Lola 68.6 Macenta 65.9 N’Zérékoré 72.8 1. Institut National de la Statistique/Guinée and ICF International. 2013. Guinée Enquête Démographique et de Santé et à Indicateurs Multiples (EDS-MICS) 2012. Rockville, Maryland, USA: Institut National de la Statistique/Guinée et ICF International. Retrieved from: https://dhsprogram.com/publications/publication-FR280-DHS-Final-Reports.cfm 2. Skilled providers include doctors, nurses, midwives, and auxiliary nurses. 3. Health facilities include both private and public facilities. L–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–G2. Trends over time in water and sanitation, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households practicing appropriate treatment of drinking water1 National 25.4 National, rural 17.6 National, urban 41.5 Priority areas Total 49.9 Boké Boké 32.0 (R) 66.6 Boffa 54.5 Conakry Conakry 39.1 65.9 Faranah Faranah 13.6 (R) 34.0 Kankan Kankan 13.8 (R) 39.9 Kindia Kindia 40.1 (R) 49.1 Forécariah 64.0 Labé Labé 34.7 (R) 60.4 N’Zérékoré Beyla 5.5 (R) 9.7 Guéckédou 30.1 Lola 20.3 Macenta 13.0 N’Zérékoré 49.2 Percentage of households with access to improved water source2 National 74.9 National, rural 64.9 National, urban 95.5 Priority areas Total 89.8 Boké Boké 70.2 (R) 78.9 Boffa 61.2 Conakry Conakry 99.0 97.7 Faranah Faranah 78.6 (R) 93.1 Kankan Kankan 80.9 (R) 93.9 Kindia Kindia 56.0 (R) 77.1 Forécariah 76.4 Labé Labé 60.1 (R) 96.6 N’Zérékoré Beyla 81.3 (R) 94.5 Guéckédou 93.0 Lola 80.9 Macenta 92.6 N’Zérékoré 89.5 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–5 Table L–G2. Trends over time in water and sanitation, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households with access to improved sanitation facility3 National 19.0 National, rural 11.2 National, urban 34.8 Priority areas Total 56.9 Boké Boké 17.1 (R) 55.2 Boffa 43.1 Conakry Conakry 34.1 62.3 Faranah Faranah 9.0 (R) 58.5 Kankan Kankan 11.3 (R) 68.3 Kindia Kindia 17.0 (R) 53.4 Forécariah 41.8 Labé Labé 20.6 (R) 76.4 N’Zérékoré Beyla 15.0 (R) 77.3 Guéckédou 45.1 Lola 29.1 Macenta 35.8 N’Zérékoré 52.4 Percentage of households with handwashing station4 National 34.5 National, rural National, urban Priority areas Total 42.9 Boké Boké 72.2 Boffa 57.5 Conakry Conakry 49.4 Faranah Faranah 21.2 Kankan Kankan 64.2 Kindia Kindia 38.5 Forécariah 32.8 Labé Labé 37.5 N’Zérékoré Beyla 76.0 Guéckédou 28.9 Lola 16.1 Macenta 10.3 N’Zérékoré 17.9 L–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–G2. Trends over time in water and sanitation, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households with handwashing station where water and soap or other cleansing agents were present5 National 34.3 National, rural National, urban Priority areas Total 43.3 Boké Boké 34.7 Boffa 31.4 Conakry Conakry 51.1 Faranah Faranah 55.2 Kankan Kankan 49.4 Kindia Kindia 42.2 Forécariah 36.2 Labé Labé 39.4 N’Zérékoré Beyla 32.1 Guéckédou 40.2 Lola 15.3 Macenta 9.7 N’Zérékoré 63.3 1. Appropriate water treatment methods include boiling, bleaching, filtering, and solar disinfecting. 2. Improved sources include piped source within the dwelling or plot, public tap, tube well or borehole, protected well or spring, bottled water, and rainwater. 3. Improved facilities include unshared facilities: flush/pour to piped sewer system, septic tank, or pit latrine, ventilated improved pit latrine, pit latrine with slab or composting toilet. 4. Percentage of households where a handwashing station was observed. The most common reason for not observing handwashing stations was that there was no specific place for handwashing. 5. Out of all households where a handwashing station was observed, households where water and soap or other cleanser was present. This number does not include households who had only water or only a cleansing agent. Table L–G3. Trends over time in primary school net attendance ratio (NAR), Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Primary school net attendance ratio National 52.4 National, rural 38.3 National, urban 75.1 Priority areas Total 71.1 Boké Boké 54.7 (R) 60.1 Boffa 56.7 Conakry Conakry 79.6 84.7 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–7 Table L–G3. Trends over time in primary school net attendance ratio (NAR), Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Faranah Faranah 36.2 (R) 55.0 Kankan Kankan 31.2 (R) 47.2 Kindia Kindia 45.9 (R) 70.5 Forécariah 66.3 Labé Labé 43.1 (R) 70.3 N’Zérékoré Beyla 50.7 (R) 66.6 Guéckédou 66.0 Lola 71.6 Macenta 70.5 N’Zérékoré 71.1 Table L–G4. Trends over time in agriculture and food security,1 Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) During EVD (2014–2015) CRS Beneficiaries Baseline2 Post-EVD (2016–2019) IBTCI HHS 2017 Average Household Dietary Diversity Score (HDDS) National National, rural 6.0 National, urban 7.0 Priority areas Total Boké Boké 7.0 Boffa 4.9 Conakry Conakry 7.1 Faranah Faranah 5.5 Kankan Kankan 6.5 Kindia Kindia 6.5 Forécariah 6.2 Labé Labé 7.3 N’Zérékoré Beyla 6.2 Guéckédou 6.0 Lola 4.8 Macenta 6.8 (R) 5.5 N’Zérékoré 5.5 1. These indicators do not appear in the most readily available population-based surveys but are available in implementing partner baseline data for some prefectures shown here. 2. Catholic Relief Services (2015). Guinea EFSP: Baseline Report April/May 2015, survey of potential beneficiaries in N’Zérékoré and Macenta. Numbers here are a combined average from both areas. L–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–G5. Trends over time in household electricity and communications technology, Guinea, 2012–2017 Region Prefecture Pre-EVD (2008–2013) DHS 2012 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percent households having electricity National 26.2 National, rural 2.9 National, urban 74.2 Priority areas Total 48.8 Boké Boké 38.3 Boffa 11.0 Conakry Conakry 96.4 Faranah Faranah 33.4 Kankan Kankan 36.8 Kindia Kindia 46.4 Forécariah 30.9 Labé Labé 40.0 N’Zérékoré Beyla 21.2 Guéckédou 11.7 Lola 5.4 Macenta 9.4 N’Zérékoré 24.4 Percent households possessing mobile telephone National 65.3 National, rural 51.1 National, urban 94.6 Priority areas Total 87.8 Boké Boké 94.7 Boffa 90.0 Conakry Conakry 93.2 Faranah Faranah 85.0 Kankan Kankan 76.0 Kindia Kindia 90.4 Forécariah 82.8 Labé Labé 92.8 N’Zérékoré Beyla 79.0 Guéckédou 74.2 Lola 84.9 Macenta 82.4 N’Zérékoré 85.8 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–9 Table L–G6. Trends over time in economic growth, Guinea, 2013–2018 Region Prefecture Pre-EVD (2008–2013) World Bank 20131 During EVD (2014–2015) World Bank 20151 Post-EVD (2016–2019) African Econom￾ic Outlook 2018 (forecast)2 Percentage real GDP growth, annual National 2.3 3.5 6.2 National, rural National, urban Priority areas Total Boké Boké Boffa Conakry Conakry Faranah Faranah Kankan Kankan Kindia Kindia Forécariah Labé Labé N’Zérékoré Beyla Guéckédou Lola Macenta N’Zérékoré 1. World Bank (2018). Data based on World Bank national accounts data, and OECD National Accounts data files. GDP growth calculated using 2010 dollars. Accessed March 5, 2018 from https://data.worldbank.org/indicator/ NY.GDP.MKTP.KD.ZG?locations=GN 2. African Development Bank (2018). African Economic Outlook 2018. Accessed March 5, 2018 from https:// www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/African_Economic_Outlook_2018_-_EN.pdf L–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–L1. Trends over time in antenatal and delivery care, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 20131 During EVD (2014–2015) Post-EVD (2016–2019) MIS 20162 Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of women age 15–49 who attended at least 4 ANC visits for most recent pregnancy National 78.1 79.2 National, rural 72.0 73.7 National, urban 83.4 83.4 Priority areas Total 78.8 Bomi 84.2 Bong 75.2 Grand Bassa 53.8 Grand Cape Mount 71.3 Lofa 73.1 Margibi 87.8 Montserrado 87.3 Nimba 69.6 Percentage of live births attended by a skilled provider3 National 61.1 National, rural 49.6 National, urban 72.7 Priority areas Total 92.3 Bomi 69.0 92.0 Bong 44.5 96.8 Grand Bassa 41.9 96.8 Grand Cape Mount 44.2 92.9 Lofa 71.8 93.9 Margibi 56.9 96.6 Montserrado 81.0 90.4 Nimba 49.8 89.0 Percentage of births delivered in a health facility4 National 55.8 76.2 National, rural 45.5 71.2 National, urban 66.2 80.1 Priority areas Total 81.8 Bomi 64.1 67.1 Bong 34.6 86.3 Grand Bassa 40.2 58.9 LIBERIA TREND TABLES A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–11 Table L–L1. Trends over time in antenatal and delivery care, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 20131 During EVD (2014–2015) Post-EVD (2016–2019) MIS 20162 Post-EVD (2016–2019) IBTCI HHS 2017 Grand Cape Mount 39.0 76.1 Lofa 75.6 80.7 Margibi 51.3 79.6 Montserrado 73.3 89.8 Nimba 48.0 75.2 1. Liberia Institute of Statistics and Geo-Information Services (LISGIS), Ministry of Health and Social Welfare/Liberia, National AIDS Control Program/Liberia, & ICF International. 2014. Liberia Demographic and Health Survey 2013. Monrovia, Liberia: LISGIS and ICF International. Retrieved from: https://dhsprogram.com/publications/publication-FR291-DHS-Final-Reports.cfm 2. National Malaria Control Program (NMCP/Liberia), Ministry of Health/MOH, Liberia Institute of Statistics and Geo-Information Services (LISGIS), & ICF. 2017. Liberia Malaria Indicator Survey 2016. Monrovia, Liberia: MOH, LISGIS, and ICF. Retrieved from: https://dhsprogram.com/publications/publication-MIS27-MIS-Final-Reports.cfm 3. Skilled providers include doctors, nurses, midwifes, and auxiliary nurses and midwives. 4. Health facilities include both private and public facilities. Table L–L2. Trends over time in water and sanitation, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) MIS 2016 Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households practicing appropriate treatment of drinking water1 National 14.1 National, rural 9.0 National, urban 18.0 Priority areas Total 53.9 Bomi 53.5 Bong 46.2 Grand Bassa 53.7 Grand Cape Mount 60.3 Lofa 47.7 Margibi 55.7 Montserrado 49.9 Nimba 69.0 Percentage of households with access to improved water source2 National 72.6 84.5 National, rural 55.5 70.6 National, urban 85.8 95.3 Priority areas Total 71.4 Bomi 84.5 Bong 91.8 L–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–L2. Trends over time in water and sanitation, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) MIS 2016 Post-EVD (2016–2019) IBTCI HHS 2017 Grand Bassa 59.4 Grand Cape Mount 94.5 Lofa 96.3 Margibi 65.0 Montserrado 51.8 Nimba 87.4 Percentage of households with access to improved sanitation facility3 National 14.2 16.5 National, rural 4.1 4.4 National, urban 21.9 25.9 Priority areas Total 43.0 Bomi 19.3 Bong 46.3 Grand Bassa 20.5 Grand Cape Mount 21.2 Lofa 47.3 Margibi 38.7 Montserrado 50.3 Nimba 43.5 Percentage of households with handwashing station4 National 2 National, rural National, urban Priority areas Total 31.4 Bomi 12.0 Bong 6.8 Grand Bassa 50.1 Grand Cape Mount 21.3 Lofa 42.1 Margibi 34.7 Montserrado 37.4 Nimba 24.9 Percentage of households with handwashing station where water and soap or other cleansing agents were present5 National 49.0 National, rural National, urban A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–13 Table L–L2. Trends over time in water and sanitation, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) MIS 2016 Post-EVD (2016–2019) IBTCI HHS 2017 Priority areas Total 50.9 Bomi 30.7 Bong 68.0 Grand Bassa 20.5 Grand Cape Mount 43.5 Lofa 54.0 Margibi 24.1 Montserrado 56.9 Nimba 70.3 1. Appropriate water treatment methods include boiling, bleaching, PUR,™ WaterGuard,™ filtering, and solar disinfecting. 2. Improved sources include piped source within the dwelling or plot, public tap, tube well or borehole, protected well or spring, bottled water, and rainwater. DHS 2013 includes bottled water as an improved source. 3. Improved facilities include unshared facilities: flush/pour to piped sewer system, septic tank, or pit latrine, ventilated improved pit latrine, pit latrine with slab or composting toilet. 4. Percentage of households where a handwashing station was observed. The most common reason for not observing handwashing stations was that there was no specific place for handwashing. 5. Out of all households where a handwashing station was observed, households where water and soap or other cleanser was present. This number does not include households who had only water or only a cleansing agent. Table L–L3. Trends over time in primary school net attendance ratio (NAR), Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Primary school net attendance ratio National 38.3 National, rural 26.4 National, urban 47.8 Priority areas Total 77.5 Bomi 31.4 72.4 Bong 19.3 75.4 Grand Bassa 23.8 73.6 Grand Cape Mount 31.3 76.6 Lofa 36.2 89.2 Margibi 34.6 72.4 Montserrado 53.5 81.9 Nimba 34.4 72.1 L–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–L4. Trends over time in agriculture and food security,1 Liberia, 2015–2017 Region County Pre-EVD (2008–2013) During EVD (2014–2015) PCI Baseline 20152 Post-EVD (2016–2019) IBTCI HHS 2017 Average Household Dietary Diversity Score (HDDS) National National, rural 5.8 National, urban 5.9 Priority areas Total Bomi 3.8 5.6 Bong 5.8 Grand Bassa 5.5 Grand Cape Mount 4.0 5.8 Lofa 5.3 Margibi 5.7 Montserrado 6.0 Nimba 6.4 1. These indicators do not appear in the most readily available population-based surveys but are available in implementing partner baseline data for some counties shown here. 2. Project Concern International (2015). Protect and Empower for Ebola Resilience (PEER): Bomi and Grand Cape Mount Counties. Baseline Evaluation Report. This data is for a small sample of potential beneficiaries in these two counties only. Table L–L5. Trends over time in household electricity and communications technology, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) MIS 2016 Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households having electricity National 9.8 19.8 National, rural 1.2 1.3 National, urban 16.4 34.0 Priority areas Total 26.6 Bomi 7.8 Bong 10.5 Grand Bassa 4.2 Grand Cape Mount 12.8 Lofa 8.5 Margibi 7.9 Montserrado 56.1 Nimba 6.5 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–15 Table L–L5. Trends over time in household electricity and communications technology, Liberia, 2013–2017 Region County Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) MIS 2016 Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households possessing mobile telephone National 43.2 54.1 National, rural 20.7 30.3 National, urban 69.0 78.4 Priority areas Total 69.9 Bomi 61.6 Bong 74.4 Grand Bassa 53.3 Grand Cape Mount 66.1 Lofa 73.8 Margibi 72.3 Montserrado 76.7 Nimba 58.4 Table L–L6. Trends over time in economic growth, Liberia, 2013–2018 Region County Pre-EVD (2008–2013) World Bank 20131 During EVD (2014–2015) World Bank 20151 Post-EVD (2016–2019) African Econom￾ic Outlook 2018 (forecast)2 Percentage real GDP growth, annual National 8.7 0.0 3.9 National, rural National, urban Priority areas Total Bomi Bong Gbarpolu Grand Bassa Grand Cape Mount Lofa Margibi Montserrado Nimba 1. World Bank (2018). Data based on World Bank national accounts data, and OECD National Accounts data files. GDP growth calculated using 2010 dollars. Accessed March 5, 2018 from https://data.worldbank.org/indicator/ NY.GDP.MKTP.KD.ZG?locations=GN 2. African Development Bank (2018). African Economic Outlook 2018. Accessed March 5, 2018 from https:// www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/African_Economic_Outlook_2018_-_EN.pdf L–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–SL1. Trends over time in antenatal and delivery care, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 20131 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of women age 15–49 who attended at least 4 ANC visits for most recent pregnancy National 76.0 National, rural 74.6 National, urban 79.9 Priority areas Total 72.1 Kailahun 61.5 Kenema 67.6 Bombali 90.1 Koinadugu 60.6 Port Loko 76.4 Tonkolili 76.0 Western Area Rural 68.6 Western Area Urban 70.4 Percentage of live births attended by a skilled provider3 National 59.7 National, rural 53.2 National, urban 78.9 Priority areas Total 87.7 Kailahun 86.3 97.6 Kenema 82.5 72.4 Bombali 45.4 96.2 Koinadugu 33.0 91.5 Port Loko 46.0 73.2 Tonkolili 37.8 79.1 Western Area Rural 63.7 85.9 Western Area Urban 76.6 90.4 Percentage of births delivered in a health facility4 National 54.4 National, rural 49.7 National, urban 68.1 Priority areas Total na 85.4 Kailahun 84.3 96.5 Kenema 77.3 90.8 Bombali 41.4 90.2 SIERRA LEONE TREND TABLES A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–17 Table L–SL1. Trends over time in antenatal and delivery care, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 20131 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Koinadugu 32.8 92.6 Port Loko 39.2 73.2 Tonkolili 35.2 77.5 Western Area Rural 56.7 82.4 Western Area Urban 61.6 85.2 1. Statistics Sierra Leone (SSL) & ICF International. 2014. Sierra Leone Demographic and Health Survey 2013. Freetown, Sierra Leone: SSL and ICF International. Retrieved from: https://dhsprogram.com/publications/ publication-FR297-DHS-Final-Reports.cfm 2. Skilled providers include doctors, nurses, midwives, and MCH aides. 3. Health facilities include both private and public facilities. Table L–SL2. Trends over time in water and sanitation, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households practicing appropriate treatment of drinking water1 National 1.9 National, rural 1.3 National, urban 4.9 Priority areas Total 40.9 Kailahun 0.2 33.0 Kenema 1.6 46.1 Bombali 0.0 51.3 Koinadugu 1.0 67.1 Port Loko 0.7 29.2 Tonkolili 0.8 39.4 Western Area Rural 6.2 38.5 Western Area Urban 10.6 34.3 Percentage of households with access to improved water source2 National 59.5 National, rural 47.0 National, urban 89.0 Priority areas Total 76.0 Kailahun 67.7 Kenema 79.6 Bombali 86.3 Koinadugu 82.0 Port Loko 58.1 L–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–SL2. Trends over time in water and sanitation, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Tonkolili 70.9 Western Area Rural 90.0 Western Area Urban 77.1 Percentage of households with access to improved sanitation facility3 National 9.6 National, rural 5.0 National, urban 19.5 Priority areas Total 40.9 Kailahun 22.4 Kenema 28.3 Bombali 56.0 Koinadugu 46.7 Port Loko 40.2 Tonkolili 49.7 Western Area Rural 57.9 Western Area Urban 35.5 Percentage of households with handwashing station4 National 21.7 National, rural National, urban Priority areas Total 41.4 Kailahun 25.2 Kenema 23.5 Bombali 25.7 Koinadugu 43.0 Port Loko 20.9 Tonkolili 61.9 Western Area Rural 41.1 Western Area Urban 70.0 Percentage of households with handwashing station where water and soap or other cleansing agents were present5 National 34.3 National, rural National, urban Priority areas Total 54.3 Kailahun 28.2 Kenema 37.8 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–19 Table L–SL2. Trends over time in water and sanitation, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Bombali 37.4 Koinadugu 19.5 Port Loko 20.3 Tonkolili 46.9 Western Area Rural 37.0 Western Area Urban 87.4 1. Appropriate water treatment methods include boiling, bleaching, filtering, or solar disinfecting. 2. Improved sources include piped source within the dwelling or plot, public tap, tube well or borehole, protected well or spring, bottled water, and rainwater. 3. Improved facilities include unshared facilities: flush/pour to piped sewer system, septic tank, or pit latrine, ventilated improved pit latrine, pit latrine with slab or composting toilet. 4. Percentage of households where a handwashing station was observed. The most common reason for not observing handwashing stations was that there was no specific place for handwashing. 5. Out of all households where a handwashing station was observed, households where water and soap or other cleanser was present. This number does not include households who had only water or only a cleansing agent. Table L–SL3. Trends over time in primary school net attendance ratio (NAR), Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Primary school net attendance ratio1 National 71.2 National, rural 66.5 National, urban 83.1 Priority areas Total 83.7 Kailahun 73.5 76.4 Kenema 65.7 73.4 Bombali 77.5 81.1 Koinadugu 61.6 77.8 Port Loko 68.6 79.6 Tonkolili 71.2 90.0 Western Area Rural 79.6 91.7 Western Area Urban 82.9 95.5 1. The net attendance ratio (NAR) refers to the percentage of primary-school age (6-11) population who are attending primary school. L–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–SL4. Trends over time in agriculture and food security,1 Sierra Leone, 2015–2017 Region District During EVD (2014–2015) Save the Children Baseline 20152 Post-EVD (2016–2019) CRS Baseline 20163 Post-EVD (2016–2019) World Vision Baseline 20164 Post-EVD (2016–2019) IBTCI HHS 2017 Average Household Dietary Diversity Score (HDDS) National National, rural 4.9 National, urban 5.7 Priority areas Total Kailahun 4.5 4.0 Kenema 2.29 5.4 Bombali 5.4 Koinadugu 4.4 Port Loko 4.87 4.8 Tonkolili 5.6 Western Area Rural 5.7 Western Area Urban 6.2 1. These indicators do not appear in the most readily available population-based surveys but are available in implementing partner baseline data for some districts shown here. 2. Save the Children (2015). Kailahun Food for Emergency for Ebola Virus Disease Support (FEEDS) Baseline Report. Survey of potential beneficiaries. 3. Catholic Relief Services (2016). Baseline Survey Report: USAID/FFP Emergency Food Security Program in Kenema District, Sierra Leone. Survey of potential beneficiaries. 4. World Vision (2016). Emergency Food Assistance to Port Loko District Project Baseline Survey Report. Survey of potential beneficiaries. Table L–SL5. Trends over time in household electricity and communications technology, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households having electricity National 13.5 National, rural 0.7 National, urban 41.4 Priority counties Total 28.6 Kailahun 2.9 Kenema 12.5 Bombali 21.7 Koinadugu 4.1 Port Loko 11.5 Tonkolili 12.7 Western Area Rural 38.1 Western Area Urban 78.1 A N N E X L . T R E N D TA B L E S , P R E - TO P O S T- E B O L A n L–21 Table L–SL5. Trends over time in household electricity and communications technology, Sierra Leone, 2013–2017 Region District Pre-EVD (2008–2013) DHS 2013 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HHS 2017 Percentage of households possessing mobile telephone National 54.9 National, rural 41.1 National, urban 84.7 Priority areas Total Kailahun 53.6 Kenema 56.2 Bombali 71.6 Koinadugu 59.7 Port Loko 71.2 Tonkolili 69.8 Western Area Rural 87.5 Western Area Urban 91.9 Table L–SL6. Trends over time in health facility characteristics, Sierra Leone, 2012–2017 Region District Pre-EVD (2008–2013) SARA 20121 During EVD (2014–2015) Post-EVD (2016–2019) IBTCI HFS 2017 Percentage of health facilities offering family planning services National 96.0 National, rural National, urban Priority areas Total 92.2 Kailahun 100.0 Kenema 90.0 Bombali 92.9 Koinadugu 100.0 Port Loko 95.0 Tonkolili 93.3 Western Area Rural 80.0 Western Area Urban 83.3 1. Government of Sierra Leone Ministry of Health and Sanitation. 2012. Sierra Leone Service Availability and Readiness Assessment 2012 Report. L–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table L–SL7. Trends over time in economic growth, Sierra Leone, 2013–2018 Region District Pre-EVD (2008–2013) World Bank 20131 During EVD (2014–2015) World Bank 20151 Post-EVD (2016–2019) African Econom￾ic Outlook 2018 (forecast)2 Percentage real GDP growth, annual National 20.7 -20.5 6.1 National, rural National, urban Priority areas Total Kailahun Kenema Bombali Koinadugu Port Loko Tonkolili Western Area Rural Western Area Urban 1. World Bank (2018). Data based on World Bank national accounts data, and OECD National Accounts data files. GDP growth calculated using 2010 dollars. Accessed March 5, 2018 from https://data.worldbank.org/indicator/ NY.GDP.MKTP.KD.ZG?locations=GN 2. African Development Bank (2018). African Economic Outlook 2018. Accessed March 5, 2018 from https:// www.afdb.org/fileadmin/uploads/afdb/Documents/Publications/African_Economic_Outlook_2018_-_EN.pdf ANNEX M. DESK REVIEWS n M–1 ANNEX M. DESK REVIEWS Performance Evaluation of USAID Ebola Pillar II Activities / Final Report Annex Document Note: This annex was originally submitted in March 2018 as three separate reports. It was updated in December 2018 and again in March 2019, and is presented here as one annex in three parts, annexes appearing per country. AID-OAA-TO-16-00040/ AID-OAA-I-15-00022 Revised March, 2019 GUINEA LIBERIA SIERRA LEONE M–2 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ANNEX M GUINEA ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–6 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–8 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–10 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–13 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–15 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–25 SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED – GUINEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–31 SUB-ANNEX B. PILLAR II ACTIVITIES BY SECTOR AND USAID DEVELOPMENT OBJECTIVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–33 SUB-ANNEX C. DOCUMENTS CONSULTED FOR PILLAR II ACTIVITIES – GUINEA . . . . . . . . . . . . . . . . . . . . . . M–35 LIBERIA ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–40 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–42 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–45 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–48 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–52 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–64 RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–70 SUB- ANNEX A. REFERENCES AND SOURCES CONSULTED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–71 SUB-ANNEX B. LIBERIA PILLAR II ACTIVITIES WITH INFORMATION IN THE SBU FILE . . . . . . . . . . . . . . . . . . . M–78 SUB-ANNEX C. POPULATION SIZE AND HOUSEHOLD DENSITY OF PRIORITY AREAS, LIBERIA . . . . . . . . . . M–88 SIERRA LEONE ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–92 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–94 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–96 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–100 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–102 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–117 SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–123 ANNEX M. DESK REVIEWS – GUINEA n M–3 Photo by USAID GUINEA This report was made possible by the support of the American people through the United States Agency for International Development (USAID). It was prepared by Dr. Michael Toole of International Business and Technical Consultants, Inc. (IBTCI). M–4 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES GUINEA TABLE OF CONTENTS ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–6 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–8 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–8 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–8 Insights on the Four Evaluation Question Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–9 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–9 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–10 Overview of the Epidemic and its Impact on Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–10 Major Social and Economic Impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–11 Recovery Response 2015–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–12 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–13 The USAID Ebola Pillar II MEL Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–13 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–13 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–15 Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–15 Agriculture and Food Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–20 Governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–21 Innovation, Technology, and Partnerships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–23 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–25 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–25 Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–26 Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–26 Opportunities and Gaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–29 Management Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–30 SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED – GUINEA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–31 SUB-ANNEX B. PILLAR II ACTIVITIES BY SECTOR AND USAID DEVELOPMENT OBJECTIVE . . . . . . . . . . . . . . . . . . M–33 SUB-ANNEX C. DOCUMENTS CONSULTED FOR PILLAR II ACTIVITIES – GUINEA . . . . . . . . . . . . . . . . . . . . . . . . . M–35 ANNEX M. DESK REVIEWS n M–5 Contents TABLES Table M–G1. Selected human development indicators, Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–10 Table M–G2. Guinea Pillar II activities included in this desk review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–14 Table M–G3. Pillar II activities in the health sector, according to activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–16 Table M–G4. Differences in attitudes about health services and intention to access services, by sex, Guinea (N=3,000) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–19 Table M–G5. Fighting Ebola “Grand Challenge” innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–24 Table M–AB Activities by sector and USAID development objective – Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–33 Table M–AC. Documents consulted for Pillar II activities – Guinea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–35 FIGURES Figure M–G1. Distribution of EVD cases and deaths, 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–11 Figure M–G2. Gold Star award for service quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–19 Figure M–G3. Analytical framework to measure sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–28 M–6 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES AEU Africa Ebola Unit AFD Agence Française de Développement (French Agency for Development) AFR Africa Bureau AFS Agriculture and Food Security ANC Antenatal care ANSS Agence Nationale de Sécurité Sanitaire (National Health Security Agency) APC Advancing Partners and Communities ART Anti-Retroviral Therapy BEmONC Basic Emergency Obstetric and Neonatal Care CDC U.S. Centers for Disease Control and Prevention CDCS Country Development Cooperation Strategy (USAID) CEmONC Comprehensive Emergency Obstetric and Neonatal care CEPPS Consortium for Elections and Political Processes Strengthening CI Confidence Intervals CNLE Coordination Nationale de la Lutte contre Ebola (National Coordination of the Control of Ebola) COR Contracting Officer Representative CRS Catholic Relief Services CSO Civil Society Organization DEC Development Experience Clearinghouse DHIS2 District Health Information System 2 DO Development Objective DPA Department of Agriculture EFSP Emergency Access to Food for EVD-Affected Guineans EICP Essential and Integrated Care Package EmONC Emergency Obstetric and Neonatal care EOC Emergency Operations Center ET Evaluation Team ETC Ebola Treatment Center ETP&SS Ebola Transmission Prevention and Survivor Services EU European Union EVD Ebola Virus Disease FAO Food and Agriculture Organization FFP Food for Peace FP Family Planning GDP Gross Domestic Product GHSC/PSM Global Health Supply Chain Program/ Procurement and Supply Management GoG Government of Guinea HC3 Health Communication Capacity Collaborative HCW Healthcare Worker HDDS Household Dietary Diversity Score HFG Health Finance and Governance HFIAS Household Food Insecurity Access Scale HH Household HIS Health Information System HMIS Health Management Information System HRH Human Resources for Health HSD Health Service Delivery HSS Health Systems Strengthening IBTCI International Business & Technical Consultants, Inc. IFRC International Federation of Red Cross and Red Crescent Societies iHRS Open Source Human Resources Information Solutions ILADP Improved Livelihoods and Agriculture Development Program IMNCI Integrated Management of Newborn and Child Illness IOM International Organization on Migration IP Implementing Partner IPC Infection Prevention and Control IR Immediate Results ITP Innovation, Technology and Partnerships JHU Johns Hopkins University JSI John Snow Institute, Inc. KII Key Informant Interview M&E Monitoring and Evaluation MCP Management, Coordination and Partnership MCSP Maternal and Child Survival Program MEL Monitoring, Evaluation, & Learning MNCH Maternal, Neonatal and Child Health MOH Ministry of Health MSH Management Sciences for Health NDI National Democratic Institute OF Obstetric Fistula OFDA Office of U.S Foreign Disaster Assistance OU Operating Unit PAPP Post-Ebola Priority Action Plan PNDS Plan National de Développement Sanitaire (National Health Development Plan) RHS Restoration of Health Services ACRONYMS ANNEX M. DESK REVIEWS – GUINEA n M–7 RMNCH Reproductive, Maternal, Neonatal and Child Health SBCC Social and Behavioral Change Communication SFCG Search for Common Ground SIAPS System for Improved Access to Pharmaceuticals and Services SO Strategic Objective TFR Total Fertility Rate UMPP Unusable Medical and Pharmaceutical Products UN United Nations UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government WFP World Food Programme WHO World Health Organization M–8 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES EXECUTIVE SUMMARY Ebola Virus Disease (EVD) in West Africa was first reported during early March 2014 in Guinea’s three southeastern prefectures (Guéckédou, Macenta, and Kissidougou), which border Liberia and Sierra Leone. However, retrospective investigations indicate Ebola virus transmission might have occurred in Guinea almost three months earlier. The World Health Organization (WHO) published the official notification of Ebola on its website on March 23 (WHO, 2014). In the meantime, the virus quickly arrived in Conakry where the first cases – which multiplied fast – were confirmed there on March 27 (WHO, 2014). Thus began the first urban epidemic of EVD anywhere in the world. Eventually, a total of 3,804 cases and 2,536 deaths were reported in 32 of the country’s 34 prefectures (WHO, 2016). This report is based on a desk review of key efforts funded, either partially or entirely, by the United States Agency for International Development (USAID) between March 2015 and March 2017 to help the people of Guinea recover from the EVD crisis and “build back better.” Methodology The desk review is part of a broader performance evaluation of USAID’s Ebola Pillar II response. Based on activity documents provided by USAID and additional information found in the public domain, information is presented according to four thematic areas: (1) Health (Non-Ebola Essential Health Services and Health Systems Recovery and Survivor Programs); (2) Agriculture and Food Security (AFS); (3) Governance; and (4) Innovation, Technology and Partnerships (ITP). Based on activity mapping conducted in 2017, the evaluation team (ET) identified 24 activities in Guinea that were classified by USAID as Pillar II￾supported efforts. However, one key limitation of the desk review was the ET’s limited access to Pillar II activity documentation. This report is based on a review of documents made available for 13 Pillar II activities in Guinea, including one Pillar II “regional” activity whose purview included but was not limited to Guinea. To the extent that relevant information was available, the desk review provides insights on four sets of evaluation questions: (1) Performance (How are the Pillar II Ebola Recovery activities in Guinea contributing to the achievement of the mission and Ebola Pillar II Strategic Framework’s objectives?); (2) Sustainability (What Pillar II accomplishments are still observable after the activities have closed, and why?); (3) Gaps and opportunities (Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and missed?); and (4) Management (What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency?) Findings Health: The main Pillar II health activities addressed efforts to enhance care and support for EVD survivors, addressing training needs of service providers and working with relevant national bodies; frontline worker support (primarily management and technical training); management, coordination and partnerships (MCP); (health information system (HIS) strengthening), (ICT); social and behavior change communication (SBCC) to re-ignite and enhance demand for formal-sector health services, and institutional enhancements (e.g., minor health facility upgrades and renovations). In general, the Pillar II health activities were able to produce the majority of planned outputs (e.g., assessments completed, plans developed, health workers trained, facilities strengthened) and have documented some noteworthy outcomes related to health service recovery and improved performance in areas such as service utilization. AFS: Social protection was the main intervention type of Pillar II-supported AFS activities, with two activities implementing activities such as food vouchers and agricultural inputs. Catholic Relief Services (CRS)’s baseline and endline data documented major improvements in food security outcomes such as household dietary diversity and household hunger scores, while OIC International showed improvements in prices that rice producers received at market. Governance: Pillar II-supported activities in this thematic area focused on MCP interventions, as well as some SBCC interventions, to strengthen governance systems within the Government of Guinea (GoG), and to engage Civil Society Organizations (CSOs), including women’s and media organizations, to increase civic participation in local and national elections. The activities addressed human capacity, institutional capacity, and the enabling environment for good governance, transparency and accountability. ITP: Pillar II-supported efforts in this thematic area have addressed both hardware and ‘software’ (e.g., training, quality assurance, improved interoperability of different information systems/ platforms). In health, ICT activities receiving Pillar II funds include mHero, implemented by IntraHealth International. mHero involves a platform to allow two-way communication using mobiles to both gather data from health workers and support decision making. ANNEX M. DESK REVIEWS – GUINEA n M–9 Insights on the Four Evaluation Question Domains Performance: Pillar II activities were of short duration (36 months or shorter), and the actual implementation period was often shorter than the official award period due to the time needed to work with GoG and communities on activity introduction and start-up activities. Outcome data are limited, but the activities for which output data have been made available have achieved all or most of their intended outputs. Sustainability: Pillar II activities have supported national policies and/or worked with GoG to develop them where they either did not previously exist or needed revision. Pillar II activities were more likely to produce sustainable outcomes for those longer￾term activities that were “topped up” with Pillar II funds, such as Health Service Delivery (12/15-12/20), the Global Health Supply Chain Program (2012-2020), and System for Improved Access to Pharmaceuticals and Services (SIAPS - 10/11–9/17). Pillar II activities strengthened institutional support for particular activities by supporting GoG efforts with state-of-the-art information, tools, and approaches, as well as capacity building. Although Pillar II activities centered on approaches with proven effectiveness, specific technical choices have had implications in terms of program effectiveness and, ultimately, sustainability. Technical choices regarding how human capacity needs (e.g., embedment of technical experts with GoG structures, intensive training and capacity building of existing GoG staff) were addressed also have a bearing on sustainability. Enhanced accountability, which can also be a determinant of sustainability, was observed for activities in both the health and governance sectors. When considering how the ‘human dimension’ influences sustainability, community engagement is a common feature of Pillar II efforts – not just consultation on priorities and needs when implementation plans were being developed, but also the fostering of local ownership in implementation of specific strategies (e.g., in the rehabilitation of health facilities). Other contextual considerations such as sociopolitical instability may also affect sustainability, and some activities worked to build resilience in light of those risks/threats that affect the enabling environment. Opportunities and Gaps: While the Ebola epidemic was extremely disruptive, especially in the health sector, it has created opportunities to build stronger systems than existed prior to the epidemic. With the establishment of an effective coordination mechanism through the National Health Security Agency (In French: Agence Nationale de Sécurité Sanitaire (ANSS)), the capacity to respond to health security threats in a more effective and timely manner has been strengthened. Pillar II activities were adaptive, addressing gaps such as beneficiary biases and misinterpretations around governance mechanisms as they were discovered. However, there were missed opportunities, especially for activities which were not built on existing mechanisms and were therefore short-lived. The one livelihoods activity, with Catholic Relief Services (CRS), ended in December 2016. The final evaluation report found that beneficiaries were already starting to run out of stock of seeds by February 2017 (EFSP, 2017). Management: It is not possible to comment on the basis of activitie documents alone. In Key Informant Interviews (KIIs) with Implementing Partners (IPs) and GoG, discussed in further detail in the IBTCI Pillar II Final Performance Evaluation 1 Report, most stakeholders felt that USAID was very consultative and supportive with partners and government stakeholders, and praised USAID for its coordination with government and other partners. The supportive role of the mission’s Monitoring and Evaluation (M&E) section was highlighted. A number of stakeholders felt that the Pillar II activities were too short in duration and, therefore, unable to build sustainability. Some respondents, especially outside Conakry, felt that there had been inadequate consultation with government stakeholders in the planning of activities. Others felt that investments in health were at the expense of other sectors. Lessons Learned Key lessons learned are as follows: 1. No one intervention can stand alone; the various types of interventions across the different thematic areas (e.g., AFS and health) mutually supported each other. 2. The array of activities funded with Pillar II resources were on target to address the main challenges of EVD recovery, but they were constrained by the short timeframe of the Pillar II funding rules. 3. Building trust, both in the health system and the government more broadly, must be a priority in any public health crisis. 4. Community investment, inclusion, and feedback are vital to the success of recovery efforts. Social mobilization is paramount. 5. Good data availability, management, and sharing allow for more successful systems improvements and consistent improvement over time. 6. Building on existing activities is key to getting recovery activities going quickly in a crisis. 7. Crises, as difficult as they are, may provide opportunities for improving and strengthening systems that prove more difficult outside of a crisis. M–10 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES INTRODUCTION Ebola Virus Disease (EVD) in West Africa was first reported during early March 2014 in Guinea’s three southeastern prefectures (Guéckédou, Macenta, and Kissidougou), which border Liberia and Sierra Leone. However, retrospective investigations indicate Ebola virus transmission might have occurred in Guinea almost three months earlier. WHO published the official notification of Ebola on its website on March 23 (WHO, 2014). In the meantime, the virus quickly arrived in Conakry where the first cases – which multiplied fast – were confirmed there on March 27 (WHO, 2014). Thus began the first urban epidemic of EVD anywhere in the world. Eventually, a total of 3,804 cases and 2,536 deaths were reported in 32 of the country’s 34 prefectures (WHO, 2016). This report summarizes key efforts by the United States Agency for International Development (USAID) between March 2015 and March 2017 to support Guinea’s recovery from the EVD crisis and to “build back better” (USAID Africa Bureau, 2015). It also describes the context in which USAID efforts were implemented, and it presents preliminary conclusions and recommendations. Overview of the Epidemic and its Impact on Health Prior to the EVD outbreak the health outcomes in Guinea were already quite poor. In 2012, the key indicators were (Demographic Health Survey, 2012): ■ neonatal mortality rate was 33 per 1000 live births; ■ under five mortality rate was 123 per 1,000 live births; ■ delivery at health facilities with a skilled birth attendant was as low as 40%; ■ maternal mortality ratio was estimated at 724/100,000; ■ modern contraceptive prevalence was only 6%; and ■ total fertility rate (TFR) was 5.7, remaining almost unchanged over the previous ten years. Social resistance to the Ebola response was widespread in Guinea. In the Forest region, where social resistance was the most violent, anthropologists have described how efforts to isolate those infected and conduct safe burials offended traditional beliefs about the importance of observing proper funeral practices, which are linked to the family’s future prosperity (Cenciarelli et al, 2015). Communication messages early in the epidemic emphasized that there was neither a vaccine nor a cure; this made people fearful of seeking treatment. As a result, social resistance resulted in violent attacks on responders. Nationwide, an average of ten attacks per month were reported against Red Cross volunteers in Guinea in the last six months of 2014, ranging from verbal to physical assaults (International Federation of Red Cross and Red Crescent Societies [IFRC], February 2015). Prior to the Ebola epidemic in Guinea, the country suffered from a weak governance system, including poor healthcare infrastructure and a lack of health communications. When the epidemic occurred in 2014, those weaknesses contributed to misconceptions and Table M–G1. Selected human development indicators, Guinea Indicator Estimate Population Size 12.6 million Median Age 18.5 % of Population Living in Urban Areas 37.2% Human Development Index Ranking 183 Gross National Income Per Capita (2015) $1058 Life Expectancy at Birth 59.2 years Adult Literacy Rate 30.4%* Poverty Headcount (% of population @ $1.90/day 2011 PPP) 35.3% Maternal Mortality Ratio 679 * Rate obtained using DHS module, involving a reading test, and tends to be lower than simple reports of literacy. Source: United Nations Development Programs (UNDP) Human Development Report 2016: Country Data for Guinea ANNEX M. DESK REVIEWS – GUINEA n M–11 mistrust on the part of the Guinean people. The health impact was compounded by the fear around EVD, which led people to reject proper health practices: a 20% decline in use of health facilities for assisted births; a 25% decline in prenatal consultations; and the closing of 94 health centers and district hospitals by December 2014. Vaccination coverage also decreased by 30% (World Bank, June 2015). In addition, Guinea’s health system was impacted by the disproportionate EVD infection rates among health workers. By May 2015, 0.02% of Guinea’s population had died from Ebola, compared with 1.45% of the country’s doctors, nurses, and midwives (Evans et al, 2015). One study analyzed the impact of the loss of 78 doctors to EVD on other health indicators. The study postulated that the maternal mortality ratio in Guinea increased between 2013 and 2015 by 38% (95% CI 26–50%) and the infant mortality rate increased by 5-20% (Evans et al, 2015). According to Parpia et al (2016), more than 20,000 people died in Guinea during the EVD epidemic from malaria, HIV/AIDS and tuberculosis due to a dramatic decrease in attendance at health facilities (Parpia et al, 2016). They estimated that a 50% reduction in treatment coverage during the Ebola outbreak would have led to the deaths of 12,825 (95% CI 4,845–21,945) children <5 years of age from malaria. Given a 50% reduction in Anti-Retroviral Therapy (ART) coverage during the Ebola outbreak, they estimated that 5,151 (95% CI 3,099–7,333) adults aged 15-49 would have died. Using a 50% reduction in treatment coverage for both drug susceptible and multidrug-resistant TB, they estimated that 3,463 (95% CI 2,808–4,349) persons would have died from TB (Parpia et al, 2016). Major Social and Economic Impacts The Ebola pandemic has had a profound impact on Guinea’s economy and society, damaging the social fabric and retarding the country’s growth prospects. Overall, the negative impacts of EVD have severely affected all strata and sectors of the Guinean economy. Livelihoods were undermined, food security threatened, incomes of Guinean farmers compromised, employment opportunities and means of urban and peri-urban livelihoods greatly reduced, and foreign investment seriously slowed down (Government of Guinea, 2015). The World Bank estimates that the economic impact of the EVD epidemic in Guinea was USD $600 million (World Bank, 2016). Gross Domestic Product (GDP) per capita declined by USD $125. Real GDP growth in 2015 was 0.1%, compared to a pre-Ebola forecast of 4.0%. Services had very low growth, while mining contracted, and only agriculture displayed some resilience. The decline in the international price of bauxite (Guinea’s main export) exacerbated the Ebola impact (World Bank, 2016). Poverty affected about 55% of Guinea’s population in 2012, and this percentage is likely to have increased as a result of the Ebola crisis in 2014 and 2015 and the economic regression it contributed to. This is particularly true for the parts of the country most affected by Ebola that already had poverty rates above the national average (World Bank, 2016). A mobile phone survey conducted in September 2015, based on interviews with close to 2,500 households across Guinea, confirmed that Ebola had a strong impact on Guinean households. Welfare levels based on asset ownership had deteriorated – a result consistent with a pronounced decline of more than 30% in income for rural households and for women in areas severely affected by Ebola (World Bank, 2016). A decline in food consumption was also noted in these same households. In parallel, urban unemployment had doubled from 8% in 2012 to 16% in 2015, and close to 10% of households had withdrawn their children from school, with most citing Ebola as the main factor for doing so. Surprisingly, agricultural production remained resilient and food prices were stable (World Bank, 2016). In conclusion, the EVD epidemic revealed three major weaknesses in Guinea. First, the fragility of the health system in the country and the limitations imposed by its inability to prevent, detect and respond to epidemics and the need to institutionalize preparedness for response; second, the fragility of the economy, taking into account the vulnerability of the population and the weakness of the socioeconomic infrastructure and institutions; and third, weak governance and poorly developed institutions, combined with extensive political mistrust, especially in the Forest region and Forecariah prefecture. Figure M–G1. Distribution of EVD cases and deaths, 2014 M–12 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Recovery Response 2015–2017 GOG RESPONSE The Guinea Ebola Recovery Plan represents the Government of Guinea’s ambitious and wide-ranging attempt to end Ebola in the country and then relaunch the economy in the aftermath of Ebola (Government of Guinea, 2015). Essentially, the plan represents a multi-pronged approach to dealing with the immediate aftermath of Ebola and then focusing on the medium-term recovery by jumpstarting the key sectors. It tries to address a range of sectors and areas that have all been directly and indirectly impacted by Ebola. The plan is based on four fundamental pillars – social sector support, economic recovery, infrastructure development, and governance support. First, in the social sectors, the authorities envisage the strengthening of the health system (systems, human resources, and medicines) to meet the immediate needs of population post-Ebola. There are also plans to improve access to water and sanitation, accelerate literacy, promote gender equality, and ensure child protection. Second, in relation to economic recovery, the government focuses on rehabilitation of the systems of production in all their components, as well as recovery and acceleration of the diversification of economic activities. A third element of the plan is to improve the availability and quality of infrastructure necessary for economic growth and help promote the recovery of investment in economic infrastructure (roads, airports, ports etc.), which have been delayed by Ebola. Finally, the plan focuses on building infrastructure and strengthening governance through improved public administration and better service delivery (Government of Guinea, 2015). USAID’S SUPPORT FOR EBOLA RECOVERY In March 2015, USAID established the Africa Ebola Unit (AEU) in Washington to oversee coordination of Ebola-related activities within USAID, across agencies, and with the wider international community. In collaboration with other USAID units, the AEU provides support to USAID missions in affected countries to implement a robust set of development programs to address secondary impacts of the outbreak and ensure that Guinea, Sierra Leone, and Liberia are prepared to prevent, detect, and respond to future outbreaks. On September 30, 2016, AEU’s responsibilities were transferred to the Africa Bureau’s (AFR) Office of West African Affairs, which now oversees the United States Government’s (USG) “all of government” Ebola recovery. That effort has four pillars: (I) Controlling the epidemic at its source; (II) Mitigating second-order impacts including blunting the economic, social and political consequences in the region (i.e., health services, health systems, food security, governance and economic crisis mitigation; and innovation, technology and public-private partnerships); (III) Engaging and coordinating with a broader global audience; and (IV) Fortifying health security infrastructure in the region and beyond. Pillar II, which is the focus of this desk review, has three Strategic Objectives (SOs): (1) Prevent the loss of development gains; (2) Recover and strengthen existing institutions and infrastructures whose weaknesses enabled the rapid spread of Ebola or slowed the response; and (3) Build sustained systems through public-private partnerships, innovation, and capacity building. Some Pillar II inputs were designed and funded at the mission level, and others by USAID Operating Units (OUs) in Washington. In Guinea, USAID Pillar II investments mentioned in this review mainly address health system strengthening, with three governance activities and two food security activities. Pillar II aligns with the Government of Guinea’s (GoG) Post-Ebola Socio-Economic Recovery and Resilience Strategy 2015-2017. The development objectives of USAID Guinea’s recovery strategy are (USAID Guinea, 2015): DO-1: Utilization of Quality Health Services Increased DO-2: Democratic Governance and Economic Processes Strengthened See Annex B for an index of activities reviewed and which development objectives they address. OTHER SUPPORT In the health sector, the Global Fund is a key player in Guinea. A grant of USD $127 million was allocated to Guinea for 2015- 2017 for HIV, TB and malaria prevention and treatment, as well as for health system strengthening (Global Fund, 2017). UNICEF has a new three-year post-crisis strategy, 2018-2020, which has three elements: (1) Health system strengthening, including funding NGOs, governance, training and equipment with a strong focus on nutrition. (2) Community engagement through training and support to community health workers. (3) Disaster preparedness with ANSS, the National Emergency Forum, and other key ministries (personal communication, UNICEF, Conakry September 29, 2017). The European Union (EU) is providing approximately USD $20 million in health assistance to Guinea (European Commission, 2015). The French Agency for Development (AFD) has also provided $10 million dollars to the EU to implement basic health services with a focus on child survival, maternal health, family planning, and nutrition in the region of N’Zérékoré. AFD’s funding also focuses on strengthening the health information and pharmaceutical systems. Funding in food security has been minimal. AFD has worked for many years in both the Forest region and in coastal Guinea on integrated rice and fish production. The Food and Agriculture Organization (FAO) also invests in community fishery work. ANNEX M. DESK REVIEWS – GUINEA n M–13 The World Bank works on land management with the Ministry of Agriculture in 17 prefectures. The International Fund for Agricultural Development has a nationwide program working with Guinea’s extension service. Post-Ebola, World Food Programme (WFP) has focused almost exclusively on school feeding (personal communication, WFP, September 30, 2017). USAID’s Feed the Future initiative (not funded by post-Ebola recovery funds) has seven activities worth around USD $34 million (2016-2021) (personal communication, USAID, Conakry, October 2017). 1. IBTCI uses the term “project” for USAID agreements that obligate funds to implement specified activities to achieve specified outcomes or results. Given the multiplicity of actors supporting recovery, and overlapping themes across efforts (e.g., health system recovery), complementarity of effort is paramount. When examining USAID’s contributions, it will be important to consider concurrent efforts on the part of other development actors in Guinea, particularly in the same geographic areas, and with the same target groups. Also, the aforementioned efforts did not occur independent of USAID. The World Bank and United Nations (UN) agencies such as UNICEF, WFP and WHO received Pillar II funds to support some of their recovery efforts (USAID Office of Inspector General, 2015). APPROACH TO DESK REVIEW The USAID Ebola Pillar II MEL Project In October 2016, International Business & Technical Consultants, Inc. (IBTCI) was awarded a three-year project – Ebola Pillar II Monitoring, Evaluation, & Learning (MEL) – to conduct performance and impact evaluations and monitoring activities to examine USAID-funded Ebola recovery interventions in Guinea, Liberia, and Sierra Leone from March 2014 through December 2019. There are four key evaluation questions: (1) Performance: How are the Pillar II Ebola Recovery activities in Guinea contributing to the achievement of the mission and Ebola Pillar II Strategic Framework’s objectives? (2) Sustainability: What Pillar II accomplishments are still observable after the activities have closed, and why? (3) Gaps and Opportunities: Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and missed? (4) Management: What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency? Methodology PURPOSE The desk review is intended to collate available information on USAID’s Pillar II investments in Guinea, serving as one of several methods of data collection and analysis. It summarizes findings from documents from projects1 in Guinea that were funded partially or entirely with USAID Pillar II funds. KEY COMPONENTS The desk review began with a call for documents (with assistance from USAID/Washington and USAID/Guinea). Based on the set of activity documents provided by USAID, and supplemented by additional information gleaned from Implementing Partner (IP) documents and other sources that the Evaluation Team (ET) found in the public domain, the ET extracted information according to the following six thematic areas: (1) Health (Non-Ebola Essential Health Services and Health Systems Recovery and Survivor Programs); (2) Basic Education; (3) Agriculture and Food Security (AFS); (4) Governance; (5) Economic Crisis Mitigation (ECM); and (6) Innovation, Technology and Partnerships (ITP). Based on that evidence base, the ET: (a) assessed activity contributions to USAID Ebola Pillar II strategy and the GoG Post-Ebola Socio￾Economic Recovery and Resilience Strategy 2015-2017, using Ebola Pillar II Theory of Change; (b) analyzed contributions of the interventions to dimensions of resilience and sustainability; and (c) formulated recommendations for next phase of recovery from the EVD crisis. In exploring resilience, the team used USAID‘s resilience definition: “The ability of people, households, communities, countries and systems to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.” (USAID, n.d.). Focusing on health systems, Emrey (2016) suggested that resilience consists of: “Capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons during the crisis, reorganise if conditions require it.” USAID defines sustainability as: “The ability of a local system to produce desired outcomes over time, including ability to be both resilient and adaptive in the face of changing circumstances.” This definition emphasizes the persistence of project effects after interventions have ended. The ET developed an analytical framework for examining sustainability determinants such as policy, finances, accountability, institutional support, the human dimension, and external threats M–14 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–G2. Guinea Pillar II activities included in this desk review IP and Activity Thematic Areas of Focus Predominant Intervention Types Health Edcn AFS Govnc ECM ITP Soc Protn Ftln Wkr MCP ICT SBCC IE Abt Associates: Comprehensive Health Systems Strengthening (CHSS)/Health Finance and Governance* ✔ ✔ ✔ Carolina Population Center/ MEASURE Evaluation: Strengthening Data Availability and Use in Guinea ✔ ✔ Catholic Relief Services (CRS): Emergency Access to Food for EVD-Affected Guineans (EFSP) ✔ ✔ Chemonics: Global Health Supply Chain Program – Procurement and Supply Management ✔ ✔ FHI 360: Citizen Involvement in Health Governance ✔ ✔ JHPIEGO: Health Service Delivery (HSD) ✔ ✔ ✔ JHPIEGO: Maternal and Child Survival Program (MCSP)/ Health Systems Strengthening ✔ ✔ ✔ JSI/Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services (APC￾ETP&SS) ✔ ✔ ✔ ✔ ✔ Johns Hopkins Univ. Center for Communication Programs (JHU CCP): Health Communication Capacity Collaborative (HC3) ✔ ✔ ✔ ✔ Management Sciences for Health: System for Improved Access to Pharmaceuticals and Services (SIAPS) ✔ ✔ ✔ National Democratic Institute (NDI): Consortium for Elections and Political Processes Strengthening (CEPPS) ✔ ✔ OIC International (OICI): Improved Livelihoods and Agriculture Development Program (ILADP) ✔ ✔ Search for Common Ground (SFCG): Rebuilding Together: Support to Local Election Reconciliation and National Unity in Guinea ✔ ✔ ✔ ✔ ANNEX M. DESK REVIEWS – GUINEA n M–15 and the enabling environment. To the extent that relevant information exists, determinants have been explored under the auspices of this desk review. Actual measurement of sustainability and resilience, and assessment of outcomes, will be addressed in greater depth in the Pillar II MEL Project Performance Evaluation 2 at the end of 2018. SAMPLING Through activity mapping in 2017, the ET identified a total of 24 activities in Guinea that were classified by USAID as Pillar II efforts, and were expected to have M&E plans, regular results reports, and baseline, midline and/or end of activity evaluation reports. However, the ET was unable to obtain documentation on a large number of Pillar II activities. In the spring of 2017, IBTCI reached out to the mission and the IPs through the activity’s Contracting Officer Representative (COR) to request activity documentation. Second and third data calls were submitted in October and December 2017, respectively. Some of the documents are publicly available, but many were classified Sensitive but Unclassified (SBU) and have been reviewed only by staff who have signed Non-Disclosure Agreements with USAID/W and USAID/ Guinea. This desk review report is based on available documents from USAID/AFR and other Washington OUs, USAID/Guinea and/ or by IPs at the mission’s request, the USAID Development Experience Clearinghouse (DEC), and other documentation/data found in the public domain (e.g., organizational websites, peer￾review journals) for 13 Pillar II activities. Annex B contains basic information on the 13 activities. Table G2 classifies each activity according to the six thematic areas, as well as the following six intervention type(s): (1) Social Protection for EVD Survivors and Other Vulnerable Groups; (2) Frontline Worker Support; (3) Management, Coordination and Partnerships (MCP); (4) Information and Communication Technology (ICT); (5) Social and Behavior Change Communication (SBCC); and (6) Institutional Enhancements (IE). LIMITATIONS The desk review is not the sole source of evidence in answering the main evaluation questions; it represents IBTCI’s first contribution to compiling information and distilling insights on USAID’s contribution at a macro level. The following are specific limitations: (1) Measuring outcomes is challenged by the short duration of the interventions, the lack of baseline data and/or the poor quality of monitoring data. (2) The ET had very limited access to summative/endline reports, or other documents that presented cumulative results since activity inception. (3) Because the team did not have access to routine monitoring reports, this limited the ability to collate results and examine time-series data. (4) Most activity outputs are reported as ‘numerator’ only (i.e., IP reports note the number of people or health facilities impacted, and no denominators to assess how much of the need was addressed). (5) Beyond indicating prefecture(s) of operation, few reports offer detailed geographic information on their service delivery area. (6) There are a number of activities for which no data or documents were provided by USAID. PROGRESS WITHIN THEMATIC AREAS Health PROBLEM STATEMENT The 2014-2015 EVD epidemics in West Africa debilitated the healthcare systems of affected countries, hampering diagnosis and treatment for endemic diseases such as malaria, HIV/ AIDS, and tuberculosis. Prior to the Ebola epidemic in Guinea, the country suffered from a weak health governance system, including poor healthcare infrastructure and a lack of health communications. When the epidemic occurred in 2014, those weaknesses contributed to misconceptions and distrust on the part of the Guinean people. The health impact was compounded by the fear around EVD, which led people to reject positive health practices: A 20% decline in use of health facilities for assisted births; a 25% decline in prenatal consultations; and, the closing of 94 health centers and district hospitals by December 2014. Vaccination coverage also decreased by 30% (World Bank, June 2015). In addition, Guinea’s health system was impacted by the disproportionate EVD infection rates among health workers. By May 2015, 0.02% of Guinea’s population had died from Ebola, compared with 1.45% of the country’s doctors, nurses, and midwives (Evans et al, 2015). SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II Pillar II health activities entailed frontline worker support; management, coordination and partnerships; SBCC; and institutional enhancements. Table G3 lists the activities in the health sector that have been fully or partially funded by USAID post-Ebola recovery funds in Guinea and the corresponding intervention type. Information on activities by sector and development objective (DO) and expected Immediate Results (IR) in the USAID Country Development Cooperation Strategy (2015- 2019) can be found in Annex C. M–16 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Addressing Needs of EVD Survivors—A Complex Issue Requiring Multiple Types of Interventions: Available documentation on Pillar II activities suggests that addressing the needs of EVD survivors requires a mix of both supply-side and demand-side activities. Pillar II supported frontline worker support, SBCC and MCP activities to address EVD survivor needs. The Ebola Transmission Prevention and Survivor Services (ETP&SS) program works with the ANSS, in collaboration with international organizations and other stakeholders, to implement the Ministry of Health’s national plan for survivors to mitigate the risk of resurgence of the Ebola virus, support clinical care, and reduce stigma and other barriers for survivors to access needed healthcare (JSI, 2018). The ETP&SS program strengthens critical health services for survivors through direct assistance to 14 public health facilities in N’Zérékoré, Kindia, and Conakry that serve a high volume of survivors. The facilities receive training for health workers as well as clinical equipment, while some facilities will also be renovated to improve working conditions and strengthen the quality of care. A critical component of the program is community-based surveillance, which monitors survivors and their close contacts to track their health status, to ensure access to services, and to make sure that early signs of EVD are detected and action taken to prevent or mitigate a new outbreak. The community-based surveillance is implemented using sentinel sites in the 21 prefectures and five urban settings with the largest number of survivors. Achievements according to a Key Informant Interview (KII): ■ Developed a clinical guide for the treatment of post-Ebola complications; ■ Transformed the sentinel surveillance system; ■ Provided laboratory equipment for semen testing; and ■ Have built the capacity of survivor associations and provided opportunities to visit Liberia and Sierra Leone and organized a three-country meeting in Conakry (personal communication, John Snow Institute (JSI), Conakry, September 28, 2017). Frontline Worker Support Pillar II resources supported three activities which provided pre￾and in-service training of community health workers, nurses, midwives, and laboratory personnel. Topics were based on area priorities and documented gaps. Jhpiego’s Maternal and Child Survival Program (MCSP) Restoration of Health Services (RHS) & Health System Strengthening (HSS) was developed in April 2016 using USAID Pillar II funds, to address additional aspects of post-Ebola recovery with a focus on health systems including support to national level policy and coordination across several domains, as well as management capacity at the regional and prefectural level. This activity was implemented between June 2015 and December 2017 in Conakry, Boké, N’Zérékoré, Kissidougou, and Kindia regions. In June 2015, MCSP Guinea was asked to extend IPC training to five rural prefectures, with funds provided Table M–G3. Pillar II activities in the health sector, according to activity Interventions, Guinea Pillar II Activity Abt CHSS MSH SIAPS JHU HC3 Jhpiego MCSP Jhpiego HSD MEASURE Data APC ETC&SS Chemonics PSM Social Protection for EVD survivors and other Vulnerable Groups Addressing needs of EVD survivors ✔ Frontline Worker Support Health worker training ✔ ✔ ✔ MCP Leadership/management support ✔ ✔ ✔ Supply chain management ✔ ✔ SBCC Community engagement ✔ ✔ Other SBCC activities ✔ ICT Health information systems ✔ Institutional Enhancements Facility infrastructure upgrades ✔ ✔ ✔ ✔ ANNEX M. DESK REVIEWS – GUINEA n M–17 by Office of U.S Foreign Disaster Assistance (OFDA). This activity continued through May 2016 to extend IPC strengthening to Boké, Dabola, Dinguraye, Faranah and Mandiana through training and supportive supervision of 1,430 healthcare providers, and 271 support staff at 249 healthcare facilities (MCSP, July 2017). In all 13 priority prefectures, the activity helped develop capacity for the use of health service delivery data in decision making (MCSP, July 2017). As part of Jhpiego’s Health Service Delivery (HSD) activity, four private midwifery schools received models and materials to set up skills simulation labs following the training of teachers on simulation skills. HSD also supported to expansion of ENSK’s simulation lab with the donation of materials to set up additional skills stations (HSD, December 2017). As part of JSI’s Advancing Partners and Communities (APC) ETP&SS activity, health workers received training and a clinical guide was developed for the treatment of post-Ebola complications (personal communication, JSI Conakry, September 28, 2017). MCP Working together with GoG and other partners on national policy and local coordination helps strengthen the effect of Pillar II activities in Guinea. Nationally, the MCSP program facilitated the integration of the Comprehensive Approach into annual planning directives, developed and adapted management training modules, facilitated the re-establishment and continuation of the IPC Cluster working group, and introduced the use of tablets for IPC data collection. In nine of 13 priority prefectures, localities have successfully integrated findings from the Comprehensive Analysis approach promoted by MCSP HSS into their annual work plans (MCSP, July 2017). Jhpiego’s other Pillar II activity, HSD, is the mission’s flagship bilateral implementing mechanism supporting the collaborative development of Guinea’s health system. It is funded partly by Pillar II funds and partly by other bilateral health funds. 38% of the budget is Pillar 2 HSS funding ($4.6 M). This activity increased the availability of Maternal Neonatal Child Health (MNCH) and Family Planning (FP) care in health facilities affected by Ebola (see Institutional Enhancements) and facilitated the development of a National Strategic Plan for Obstetric Fistula Management. In terms of the supply chain, Management Sciences for Health (MSH)’s System for Improved Access to Pharmaceuticals and Services (SIAPS) and USAID’s Global Health Supply Chain Program – Procurement and Supply Management (GHSC/PSM) have each been partly funded by GHET and aim to strengthen the national pharmaceutical logistics management system. PSM provides effective and environmentally benign treatment and disposal of Unusable Medical and Pharmaceutical Products (UMPP) at Guinean warehousing facilities and supports the design of a long-term strategy to improve the country’s ability to properly manage hazardous waste. The Pillar II portion of the activity is mainly focused on identifying expired drugs at all levels of the health system. PSM developed a manual with standard operating procedures for the medical supplies distribution system with their main counterpart, the Inspector-General of the Ministry of Health, as well as the Ministry of the Environment and the National Directorate of Pharmacy (Global Health Ebola Team, 2016). At the same time, Management Sciences for Health’s (MSH) SIAPS activity trained 10 World Food Programme (WFP) logisticians at the airport warehouse on good storage and distribution practices. These logisticians will work with Central Medical Stores (Pharmacie Centrale de Guinée, or PCG). SIAPS also developed a roadmap to build and implement an electronic logistics management information system, including a national list of commodities and logistics indicators (GHET, 2016). In addition, SIAPS completed a National Pharmaceutical Management System Assessment in February 2017 and participated in the development of a new pharmaceutical law (Nfor, E, and C. Bahati, L. Camara. 2017). Achievements from Abt Associates’ Health Finance and Governance activity (HFG) include mentorship of members of the Legislative Assembly Health Commission and capacity building in health budgeting. The HFG program in Guinea is a 2-year, $6 million program that aims to provide institutional support to increase MOH capacity to better manage the health sector and respond to health emergencies such as the recent Ebola crisis. The HFG scope of work targets four areas of health systems strengthening – institutional strengthening, governance, financing, and Human Resources for Health (HRH) – all of which are building blocks of a well-functioning health system. The activity is scheduled to be implemented between June 2016 and June 2018. HFG’s institutional strengthening component included a functional audit of the health system, management and capacity building workshops, and efforts to improve health sector coordination, research capacity, and financial management. HFG supported the Ministry of Health (MOH) in human resource management to strengthen the National HRH Platform, roll out the human resources information system iHRIS, and develop a national in-service training plan for health workers, allowing the health sector to mobilize human resources in a timely manner to address the next emerging threats. HFG also supports training institutions in charge of continuing professional development to build their capacity in coaching and mentorship. In terms of health governance, HFG provides targeted support to the Standing Committee on Health to improve oversight of health programming and build support for additional public investments in the health sector. To address the lack of public trust in the public health sector, HFG helps the MOH develop and begin to implement a strategic communication plan. The plan will cover both internal and external communications, with particular emphasis on budget transparency and resource utilization (HFG, 2017). ICT Reporting of health data during the EVD outbreak was, in most cases, neither timely nor accurate. MEASURE was awarded Pillar II resources to strengthen the Health Management Information System (HMIS) to increase availability and quality of health service delivery data and to institutionalize a culture of data driven decision making. According to the Guinea report by the Global M–18 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Health Ebola Team in July 2016, this activity had achieved the following: (1) Consensus was reached with all relevant national health information systems partners on the way forward; (2) Creation of reference list of health facilities nationwide and a draft manual of health data management procedures for the central, regional, and district levels was completed; (3) Pre-testing of health information system monitoring tools was conducted at the regional, district and health training levels; (4) Needs and capacity assessment of the Strategic Planning Unit and personnel was completed; (5) Health Management Information System Strategy was finalized; (6); an MOH M&E Plan was developed; (7) DHIS2 was implemented in every hospital and clinic in every prefecture; and (8) Data was entered from 2015-2017 and the system would be ready to go live with real-time data in January 2018. Pillar II activities provided customization training for statistics officers on Guinea DHIS2 utilization for data entry, analysis and reporting; and strengthened system integration and data warehousing. In addition to MEASURE’s work, HFG helps the MOH to continue the roll-out and institutionalization of the human resources information system (iHRIS) especially through data capture of health workers (HFG, 2017). Pillar II supported health innovations as mHero, a two-way, mobile phone-based communications system connected the Ministry of Health with healthcare workers. Another innovation is CommCare, an open￾source mobile platform to locate and trace people who may have been exposed to the Ebola virus (community-based surveillance) (USAID Guinea 2016). SBCC In addition to its pivotal role in controlling the EVD outbreak, community engagement is integral to health system resilience and ownership. It was therefore a critical part of the Pillar II strategy for health system recovery. Johns Hopkins University Health Communi￾cation Capacity Collaborative (JHU HC3) worked with local NGOs to increase the demand for health services through community health outreach activities. Hundreds of community health workers were trained using the Bridges of Hope community toolkit to promote overall wellbeing and health habits at home. Community dialogues about the quality of services and perceptions of care took place in health facilities with health providers, elected officials, religious lead￾ers, community leaders and women’s groups (Johns Hopkins Center for Communication Programs, November 2017). The ETP&SS pro￾gram strengthens critical health services for survivors through direct assistance to 14 public health facilities in N’Zérékoré, Kindia, and Conakry that serve a high volume of survivors. A critical component of the program is community-based surveillance, which monitors survivors and their close contacts to track their health status, to ensure access to services, and to make sure that early signs of EVD are detected and action taken to prevent or mitigate a new outbreak. The community-based surveillance is implemented using sentinel sites in the 21 prefectures and five urban settings with the largest number of survivors (JSI, 2018). Pillar II revitalized community radio in Guinea and community health workers harmonized their messages with radio spots and local radio to raise awareness. Institutional Enhancements Pillar II efforts contributed to USAID Guinea’s goal of increasing the utilization of quality health services through institutional enhancements by four activities, which often worked together or at least reinforced each other through simultaneous and/or complimentary efforts. As part of Jhpiego’s HSD activity, a baseline assessment of 83 facilities in the HSD intervention zone was fielded and analyzed during Project Year 2. A detailed report was published in October 2017. An assessment of IPC specifically was carried out in 166 facilities in Boké, Kankan, Labé, and Mamou regions in order to identify needs for improving IPC performance (CERREGUI, October 2017). MSH/SIAPS also conducted an assessment of the status of storage conditions and management of IPC materials, finding that improvement was needed (SIAPS, 2016). Another Pillar II USAID-funded Jhpiego activity, MCSP/ HSS, participated in the mapping of incinerators, and other IPC equipment and supplies, at hospitals and high-volume health centers as part of an initiative of the IPC cluster working group to address medical waste management. HSD aims to ensure that an essential and integrated care package (EICP) for maternal, neonatal child health (MNCH) and family planning (FP) is provided in a consistent and high-quality manner in health facilities and surrounding communities in the regions of Boké, Conakry, Kindia, Mamou, Faranah, Kankan and Labé. The integrated package is meant to be implemented through the household-to-hospital continuum of care, where the activity’s key interventions will ensure that high-quality care is available at each level, as well as access to information and referral for serious illness, obstetric fistula (OF) and long-acting and permanent contraception. HSD also provided facilities with materials, minor equipment, and consumables necessary to support the provision of the EICP based on needs identified in the baseline study. Over the course of the year, HSD helped minor rehabilitation accompany the donation of medical materials and instruments in five facilities. Guinea’s healthcare infrastructure is quite old and poorly maintained, with many facilities struggling with inconsistent availability of water and electricity from municipal sources. Therefore, minor improvements such as increased water storage capacity and solar powered back-up light can make a major difference (HSD, December 2017). A major focus of JHU/HC3’s activities in Guinea was to rehabilitate existing clinics and promote their rehabilitation to the community. HC3 Guinea worked with 64 health facilities in five regions severely impacted by Ebola – Conakry, Kindia, Boké, N’Zérékoré, and Faranah. In collaboration with MCSP HSS and the MOH, HC3 revamped a quality brand – entitled “Etoile d’Or,” or Gold Star – promoted nationally and regionally through a mass-media campaign that includes radio, television, billboards and community events. The campaign aims to build confidence in as well as increase the use of health services. Once accredited, a health facility receives a “gold star,” which is prominently displayed both inside and outside the ANNEX M. DESK REVIEWS – GUINEA n M–19 facility. During the activity, 37 facilities across the country received a gold star (Johns Hopkins Center for Communication Programs, November 2017) (Figure G2). OUTPUTS AND OUTCOMES Available documents on health activities that were fully or partially funded with Pillar II resources indicate that the activities were able to produce the majority of planned outputs (e.g., assessments completed, plans developed, HCWs trained, HFs strengthened). With respect to outcomes, the endline health services perception and intention to access survey of the Health Communication Capacity Collaborative (HC3) activity found that one-third of the 3,000 respondents reported improvements in the quality of their local health facility services in five regions severely impacted by Ebola – Conakry, Kindia, Boké, N’Zérékoré, and Faranah (JHUCCP, 2017). The survey was conducted by GeoPoll in mid￾2017 via a computer-assisted telephone survey and the sample comprised 2,000 women and 1,000 men aged 18-49 years. In terms of exposure to the Gold Star campaign, around one-third of respondents reported having seen or heard information about improving the quality of health services in the past month. The most common message recalled was: “They are improving facilities to help Guinea be healthy.” Overall, attitudes to health services were overwhelmingly positive, with women more likely to be positive than men, as demonstrated in Table G4 (Johns Hopkins Center for Communication Programs, November 2017). Health Service Delivery (HSD) activity reports indicate that quality of service assessment scores above 50% in 84 health centers increased from 11% in early 2014 (before Ebola outbreak), to 85% in 2016. Patient consultations in healthcare facilities increased by 57%, from 116,545 in the first half of 2015 to 183,149 in the first half of 2016 (USAID, 2016). The USAID Guinea end-of-2016 health portfolio review included the following achievements by HSD (USAID, 2016): ■ The number of women who received uterotonics at delivery in five regions (Kindia, Faranah, Kankan, Boké, and Mamou) increased almost from 17,375 in 2015 to 47,133 in 2016 (171% increase); ■ 2,259 newborns who were not breathing were resuscitated across the five regions (baseline unknown); and ■ The number of assisted deliveries in N’Zérékoré region increased from 22,462 in the period Oct 14-Sept 15 to 35,881 in Oct 15- Sept 16 (an increase of 60%). A key outcome reported by the Abt’s HFG activity was a significant increase in the government health budget from 2.5% of government national expenditure pre-Ebola to 8.2% in 2017, with a commitment to reach 15% by 2020 (KII, Conakry, September 28, 2017). However, according to the latest update for the fiscal year (FY) 2018, the health budget ratio against the national budget actually decreased to 6%. This is mostly due to the huge increase Table M–G4. Differences in attitudes about health services and intention to access services, by sex, Guinea (N=3,000) Statement Males n (%) Females n (%) Total n (%) Agrees that the local state hospital/health center is clean. 797 (79.7%) 1,707 (85.4%) 2,504 (83.5%) Agrees that the local state hospital/health center will keep people safe and free from infection. 786 (78.6%) 1,685 (84.3%) 2,471 (82.4%) Agrees that if he/she were to attend the local state hospital or health center, he/she would receive a warm welcome from the health provider. 798 (79.8%) 1,686 (84.3%) 2,484 (82.8%) Agrees that if he/she were to attend the local state hospital or health center, he/she trusts the health provider would meet his/her health care needs. 851 (85.1%) 1,748 (87.4%) 2,599 (86.6%) Agrees that health center has improved in the past six months. 721 (72.1%) 1,541 (77.1%) 2,262 (75.4%) Intends to go to a health provider the next time health care needed. 956 (95.6%) 1,925 (96.3%) 2,881 (96.3%) Figure M–G2. Gold Star award for service quality M–20 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES of the government national budget from16,219,228,580,000 Guinean Francs (GFN) in FY 2017 to 21,137,042, 269,000 GFN in FY 2018 (despite a nominal health budget increase from 1,141, 782,758,000 GFN in FY 2017 to 1,261,029,932,000 GFN in FY 2018. This indicates that despite the efforts of the Health Commission to advocate for a much higher ratio of health budget (and if not at least maintain the 8.2% in 2017), the budgetary arbitration at higher level was not done in favor of health budget (email communication, USAID/Guinea, March 10, 2018). MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Sustainable outcomes are more likely for those longer-term activities that were “topped up” with Pillar II funds, such as Health Service Delivery (12/15-12/20) the Global Health Supply Chain Program (2012-2020), and System for Improved Access to Pharmaceuticals and Services (10/11-9/17). Other Pillar II activities, such as HC3 and the Maternal and Child Survival Program, were of 2 years duration and were difficult to adequately build capacity for sustainability. Nevertheless, both activities achieved broad coverage and contributed to significant improvement in the quality of health services in numerous facilities. While the Ebola epidemic was extremely disruptive, especially in the health sector, it has created opportunities to build stronger systems than those in place prior to the epidemic. With the establishment of an effective coordination mechanism, the capacity to respond to health security threats in a more effective and timely manner has been strengthened. Agriculture and Food Security (AFS) PROBLEM STATEMENT The negative impact of the Ebola outbreak on household incomes and markets led to food insecurity due to lack of access to food. Economic activities were seriously disrupted as a result of restrictions that were imposed on movement and trade in order to limit the transmission of the disease. This economic slowdown has hit all areas of Guinea irrespective of the local magnitude of the epidemic. Although overall food availability was not significantly reduced compared to normal years, areas severely hit by the Ebola outbreak faced serious food shortages at the start of 2015 when food stocks from the current harvest were exhausted. This was particularly the case for N’Zérékoré region where rice harvests were well below average and the movement of goods and people was restricted. The most affected were households directly hit by Ebola and casual and agricultural workers. SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II Initially, Pillar II (through FFP) funded Catholic Relief Services to improve the food security situation for vulnerable households (HH) in Ebola-affected areas of Guinea (February 24, 2015-December 30, 2016). This activity focused on two prefectures in the Forest region. While USAID funds seven agriculture and food security activities in Guinea through the Feed the Future Initiative, post-Ebola recovery funds support only one agriculture activity – the Improved Livelihoods and Agriculture Development Program (personal communication, USAID Guinea, October 2017). This activity was classified into the Economic Crisis Mitigation thematic area but will be covered in this section since it relates to both AFS and ECM. Both activities covered in this section fall into the Social Protection intervention type. Social Protection through Food Vouchers In response to demonstrated food insecurity during the EVD outbreak, CRS was awarded a USAID/Food for Peace (FFP)- funded program, totaling USD $3,223,000 for the period of 2015-2016. The Emergency Access to Food for EVD-Affected Guineans (EFSP) aims to improve the food security situation for vulnerable households (HH) in Ebola-affected areas of Guinea. The activity provided food vouchers and nutrition promotion to 7,000 vulnerable households in Macenta and N’Zérékoré prefectures in the Forest region. Routine monitoring data indicated that the activity supported 7,008 HHs (initial target was 4,000 HHs), reaching nearly twice the number of beneficiaries it set out to support initially after successfully applying for more funding from USAID. The activity was also able to improve access to food for 97 % of targeted HHs. Social Protection through Agricultural Inputs The Improved Livelihoods and Agriculture Development Program (ILADP), implemented by OICI, mitigates second-order economic impacts of Ebola in Boffa prefecture. The secondary objective is to create more economic resiliency in/around Guinea’s mining communities. These objectives will be achieved by building human and social capital in agricultural productivity, entrepreneurship (focused on women), food marketing, and vocational skills necessary to increase employment opportunities and increase vibrancy in commercial oriented agricultural market systems. Farmers were taught improvements in rice harvesting and milling using less wood. Social Protection through Access to Credit and Cooperative Selling ILADP also facilitated access to credit for women and provided support to women selling rice to merchants from Conakry by forming trader cooperatives to negotiate a fair price. As a result, women are now getting around 25% more for their rice (Personal Communication, Conakry, October 2017). ANNEX M. DESK REVIEWS – GUINEA n M–21 SBCC ESFP provided nutrition promotion activities to all 7,000 recipients of its food voucher program. In addition to ILADP, USAID Guinea supports seven agriculture and food security activities through the Feed the Future Initiative, with a total budget of USD $34.5 million. These activities will no doubt complement ILADP in reducing food insecurity in Guinea. OUTPUTS AND OUTCOMES Both activities reported outputs that achieved the majority of their targets, or surpassed them, like the EFSP Food Voucher Program. Both activities appear to have achieved their objectives. Household Dietary Diversity Score (HDDS) and Household Food Insecurity Access Scale (HFIAS) were assessed prior to the first food voucher distribution by CRS. The results of the baseline survey indicated that 95% (596/628) of respondents were severely food insecure while 32% of households had only medium to low dietary diversity (Emergency Access to Food for EVD-Affected Guineans, March 2017). Before the end of the activity, a final evaluation was organized to assess the activity’s impact on the HDDS and HFIAS scores. Overall, the results showed improvements in all categories, particularly in reducing food insecurity from 95% to 57% and increasing the proportion of households with a high HDDS (food group greater than or equal to six) from 68% to 82%. The proportion of households in the categories of “food secure and mildly food insecure” increased from 1% to 32% and respondent households reported increased diversity in their family diets and a significant increase in their family’s health status. The activity was also able to improve access to food for 97 % of targeted HHs. In the ILADP activity, support was provided to women selling rice to merchants from Conakry by forming trader cooperatives to negotiate a fair price. As a result, women are now getting around 25% more for their rice. Both rice producers and consumers are benefiting from the stabilization of the rice price. The market price for rice has dropped from 9,000 GNF per kilo in 2015 to around 7,000 GNF in 2017. This compares with the price of rice imported from Bangladesh at 7,500 GNF. The price received by producers has risen from around 4,000 GNF to 5,000 GNF per kilo. The main success story is the stabilization of rice prices through collective action by women who have been empowered by the activity. The price they receive increased from 4,000 GNF per kilo to 5,000-5,500 per kilo after the activity. MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Though other USAID-funded AFS activities are ongoing in Guinea, only one agriculture activity was funded under Pillar II past 2016. This may have been due to the short-term nature of Pillar II funding and the long approvals reported for activating funding and the fact that other, already-in-place activities were better suited to address AFS needs in the time of recovery. Undoubtedly one of the most impressive aspects of the voucher component was the coordination between NGOs and government. To mitigate beneficiary selection bias and risk to beneficiary selection process, AFS IPs verified 10% of the registered participants and validated the final beneficiary lists by local committees (EFSP, 2017). To support an effective voucher exchange system between vendors and activity beneficiaries, Pillar II AFS activities identified opportunity to select vendors through a competitive selection process to avoid collusion and market distortion. Governance PROBLEM STATEMENT For more than five decades, poor performance of Guinea’s health and other critical sectors has crippled the well-being and growth of the country. Severely weak governance, at the central, regional and local levels, contributes to failures in the social sector in Guinea as evidenced by the 2016 Human Development Report that ranked Guinea at 183 out of 188 countries and territories worldwide (UNDP, 2016). The Ebola epidemic in Guinea illuminated and exacerbated weak governance structures and institutions. There was extensive mistrust of all branches of government, which contributed to widespread social resistance to the response. There was a political dimension to resistance with local ethnicities in the Forest region (sometimes referred to as ‘Aboriginal’) and the Mandingo Savannah empire conflicting with Conakry-centered political networks with extensions in the north and west of the country (Cenciarelli et al, 2015). In some prefectures, especially Forecariah, there was considerable support for the previous president and resentment and suspicion of the current government. For local populations, epidemic containment activities in health facilities and health messaging campaigns in local communities were closely aligned with previous experiences of political and social repression. Moreover, this exacerbated tensions related to the long-postponed local legislative elections (Cenciarelli et al, 2015). USAID RESPONSE The second USAID Guinea Country Development Cooperation Strategy [CDCS] (2015-2020) development objective (DO2) is Democratic Governance and Economic Processes Strengthened. This CDCS goal is aligned with Pillar I of the GoG’s Poverty Reduction Strategy – Governance and Reinforcement of Institutional and Human Capacities. The expected immediate results (IR) are as follows: M–22 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES IR 2.1: Transparent, Competitive, Accountable Governance Strengthened IR 2.2: Conflict Prevention and Mitigation Promoted in the Forest Region IR 2.3: Agricultural Education and Rural Microenterprises Strengthened Pillar II Guinea activities in Governance fell mostly into the man￾agement, coordination, and partnerships (MCP) intervention type. MCP The overall goal of the Consortium for Elections and Political Processes Strengthening (CEPPS) (August 2015-February 2018), led by the National Democratic Institute, is to support public dialogue and engagement around Guinea’s upcoming municipal elections in order to promote transparent, participatory, peaceful and credible elections. To achieve this goal, CEPPS has the following objectives: (1) Support confidence building in Guinea’s electoral processes through facilitation of dialogue and consensus among political actors; (2) Reinforce political party capacities and processes for more inclusive policies and peaceful electoral strategies; (3) Extend geographic reach and enhance outreach capacity and sustainability of existing civic and voter education infrastructure; (4) Develop large-scale civic and voter education campaign in synergy with the CENI; (5) Strengthen the capacities of political parties to train and deploy poll watchers to observe Guinea’s presidential and local elections; (6) Help Guinean citizen organizations hone their technical and management skills to independently and credibly monitor electoral processes; (7) Enhance election officials’ capacity to manage the upcoming electoral process by providing technical elections expertise (CEPPS, 2017). The Quarter 2 report in 2017 noted the following achievements: (1) A total of 646 political party and civil society representatives, including 228 women and 280 youth debated with potential candidates on public concerns in seven regional public forums in January. Twenty-five political parties and 179 CSOs were represented. (2) Twenty civil society representatives, including 10 women, from 10 CSOs learned how to convene and moderate focus groups to solicit citizen priorities. (3) In the five communes of Conakry, 180 citizens, including 68 women, discussed their socio-economic priorities and perceptions of local governance during focus group discussions. (4) In total, 67 women from 15 political parties, 17 CSOs, three political associations, four government ministries, all seven administrative regions of Guinea and Conakry exchanged experiences during a two-day “Women in Politics” event. (5) A total of 77 political party youth from 16 different parties, including 31 women, learned about effective leadership and communication skills in political organizing during trainings in N’Zérékoré, Kankan and Labe. The USAID Guinea 2016 year-end governance portfolio review reported the following outputs for CEPPS: 1. Individuals receiving civic and voter education through USG￾assisted programs jumped from 2,509,664 in 2015 to 9,437,284 in 2016. 2. Enhanced civic engagement through the seven Civic Education Centers is contributing to improved trust and communication across diverse communities. 3. 14,714 individuals instead of the targeted 250 received political party training (12,661 men and 2,063 women) to build capacities in poll administration. 4. The cross-party Working Group of Women and Girls of Guinean Political Parties (Cadre de Concertation des Files/ Femmes des Parties Politiques de Guinée), representing 12 political parties developed an Action Plan to engage more women to increase the number of women candidates for local elections. They also began advocating for party leaders to abide by the 30% gender quota law. 5. Production and broadcasting of intercultural tolerance media programming (148 Nimba magazines, 192 Wontanara episodes programs and 48 call-in show programs). 6. 15 civic engagement and peace skits produced in at-risk communities, reaching approximately 7,508 people (USAID, 2016b). The Search for Common Ground activity, Support to Local Election Reconciliation and National Unity in Guinea (Sep 2012-Sep 2017) received supplementary funding from Pillar II in 2015. The goal is to promote peace and reconciliation, directly supporting the Reflection Commission. Activities include community, regional and national interventions to promote participatory and inclusive dialogues to address the root of community-based conflict, especially in the Forest region. The goal of FHI 360’s Citizens Engagement in Health Governance is to improve citizens’ understanding of and participation in Guinea’s health system reforms, and in the process, build public trust in Guinea’s health sector and governance. This will be achieved through expanding opportunities for elected and appointed officials and citizens to engage in constructive dialogue and improving the quality of public discussion on health issues. This activity runs between June 2017 and June 2020. An office was established in June 2017 and implementation began in October. It is too early to report on the performance of this activity (personal communication, USAID, Conakry, October 2017). SBCC All three activities in the Governance sector also engaged in SBCC interventions in the form of radio programs and community dialogues and discussions to promote trust and consensus among diverse political parties and community groups. CEPPS supported confidence building in Guinea’s electoral processes through facilitation of dialogue and consensus among political actors through its Code of Conduct awareness campaign, which utilized community forums and radio spots. Search for Common Ground (SFCG) promotes community dialogue to get at the root of ANNEX M. DESK REVIEWS – GUINEA n M–23 conflict, and FHI 360 has provided opportunities for communities to dialogue about public health and health services to build more trust in the health system. OUTPUTS AND OUTCOMES SFCG released a final report (2012-2017) in February 2017, before the activity was extended until December. The evaluation findings showed that the activity: 1) Increased citizens’ knowledge about the sources of violence and its consequences and above all the ways in which violence can be avoided; 2) Enhanced citizens’ capacities and opportunities to talk about their differences through dialogue sessions; 3) Overall, 72% of the respondents said that the media productions had made a notable change on the behavior of populations. Overall, throughout the communities where the activity was being implemented, there had been a reduction in the level of violence between the communities aside from Lola where animosity between belligerents around the mosque’s management is still present. However, there has not been any outburst of violence recently (Search for Common Ground, 2017). Besides, through the modified agreement starting in September 2015 with Pillar II funds, the evaluation findings show that the activity: 1) Significantly increased citizens’ information about the election process (67% of respondents), mostly through the media (source of information in this period of 87% of respondents); 2) More than 60% of respondents found the information reliable around the election, which guided them in their voting decisions. The evaluation also found that media were capacitated by SFCG in the framework of the activity in two ways: 1) Through the partnerships and through the capacitation of the media professionals in order to produce quality programs that promote peace and justice for the citizens; 2) Throughout the communities where the activity was being implemented, there has been a reduction in the level of violence between the communities. MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA CEPPS reported several challenges in implementing governance activities in Guinea, namely, a lack of consensus among actors in the electoral process and the lack of clarity around the date of local elections impacted the planning of election-related activities. CEPPS/NDI learned that not all parties who had participated in the 2014 Win with Women assessment had implemented their action plans, which was due in part to lack of financial means and political will. Innovation, Technology, and Partnerships PROBLEM STATEMENT The key issues in Guinea’s ability to take advantage of Innovation, Technology, and Partnerships were the need to train health workers on health information systems, provide opportunities for public￾private partnerships, and allow innovators a place to test ideas. USAID RESPONSE Pillar II funded 14 innovations through the regionally focused Fighting Ebola Grand Challenge initiative out of the Center for Innovation and Impact in the Global Health Bureau (CII). This business-minded approach has helped CII to spark new solutions to serious global health challenges from malaria to HIV to maternal, newborn, and child health. Including this challenge, CII has leveraged over $300 million in outside capital to develop and test 150 innovations in 35 countries, of which 25 were scaling or transitioning to scale in 2018. These challenges jumpstart innovative product developments by providing seed funding for research, development and testing. The Fighting Ebola challenge was issued via a Broad Agency Announcement (BAA) mechanism to innovators to “develop new practical and cost-effective solutions to improve infection treatment and control that can be rapidly deployed (1) to help healthcare workers provide better care and (2) to transform our ability to combat Ebola” (USAID, 2014). The original challenge received 1500 ideas. With input from the White House Office of Science and Technology Policy (OSTP), the DOD, the CDC, OFDA, Offices in the Bureau of Global Health, the Global Development Lab, the Africa Bureau, and experts from USAID missions, which informed the team about current conditions in West Africa, the CII winnowed the 1500 to 30 semi￾finalists and finally 14 winners (see Table G5). Twelve of the 14 innovations have been field-tested in West Africa. The testing in Guinea included: ■ 3D Family Productions, working with the Ministry of Health and MSF, held a national song-writing competition that increased awareness of and trust in Guinea’s restored health services; ■ Columbia University tested its colorized bleach solution and also demonstrated the innovation in local Guinean hospitals; ■ After JHU field-tested its PPE innovation in Liberia, it also conducted a scoping trip in Guinea and was slated to begin user testing in both Guinea and Liberia in January 2016. The team found no information on the outcomes of those country applications; and ■ mHero, a two-way mobile phone-based communication innovation developed by IntraHealth International, reached an estimated 7,729 health workers regionally by the end of the third quarter of FY 2016. IntraHealth received one of the largest Grand Challenge grants to develop and field-test an ICT innovation for Ebola response and recovery efforts. The mHero innovation for health worker mobile communications on health was piloted in Liberia, adapted M–24 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–G5. Fighting Ebola “Grand Challenge” innovations Innovator Product Award (nearest 000) Duration Objective met? Comments Johns Hopkins Univ. Improved Personal Protective Equipment 794 6/15-6/17 Yes Improves visibility of and for HCW; Improved heat management; Simplified doffing (reduced from 8 to 4 pieces); improving HCW safety and confidence. IntraHealth Int. Scale up mHero, mobile phone-basd platform for MOH – HCW communication about EVD/other health issues 700 6/15-9/16 Yes Refined the architecture and user support for mHero, to strengthen the health systems in all 3 countries. Enabled mHero to go to national scale in Liberia, and mHero has been integrated into the MoH HMIS in Liberia and Guinea. SPR Advanced Tech Inc. Improved antiviral protective coating 656 5/15-8/16 No Innovative approach to longer lasting protection, but additional R&D is required to resolve technical shortfalls. Columbia University – Kinnos “Highlight” powdered color additive to improve EVD decontamination w. bleach 649 6/15-7/16 Yes Enables “visualization” of decontamination, improving HCW safety and confidence. Field tested in all 3 countries; now widely in use. Scripps Health “STAMP” – wearable sensor and personalized analytics￾monitor patient (Px) vitals 632 5/15-5/16 Yes Field tested by IMC in Sierra Leone. Potential for multiple uses; monitoring heat stress in HCW . Baylor College of Med. Develop light weight, re-usable, transportable Emergency Smart Pod 614 6/15-6/17 Yes Collapsible, movable ETU requiring only 30 minutes training to set up, and with estimated production cost of $50K-$150K, and “shelf life” of 10-15 years. TOMI Environmental Solutions SteraMistTM Mobile Decontamination chambers 559 5/15-2/16 Yes SteraMist is FDA approved; Expanding testing to Philippines, Malaysia, Mexico, and Panama; SteraMist will be pre-positioned in the US for future outbreaks. SteraMist is also EPA and FDA registered, included in the WHO Compendium of Innovative Health Technologies for Low Income Countries, and distributed worldwide. IBM Research Africa EPIC platform & analytics for front line HCW decision￾making 527 7/15-3/17 Yes EPIC gathers and integrates data from stove￾piped information sources (DHIS-2, IP data sources, etc.) to provide both health and context data relevant to district level health services decision-making. Successful test in one district in Sierra Leone. Modula S Modula-S Rapidly Deployable Emergency Medical Treatment Unit - Antimicrobial Copper CuVerro Materials 500 5/15- 12/15 Yes Hard-shell popup ETU building, autonomous & solar self-powered with direct contact kill antimicrobial CuVerro copper interior surfaces. Wall mounted beds, and solid floor surface pitched to drains to facilitate easy and thorough cleaning. Third generation design, now available, one third the shipping size and ISO container building modules start at $500,000. Makerere University “Epi-tent” – a next generation hospital tent with improved air flow and heat exchange 482 6/15-4/16 Yes Production in Uganda; In use in refugee settlements; Included in WHO Compendium of Innovative Health Technologies for Low Income Countries. 4 design patents filed, new housing use case design optimization. ANNEX M. DESK REVIEWS – GUINEA n M–25 Table M–G5. Fighting Ebola “Grand Challenge” innovations International Personal Protection Test 3 concepts/designs to improve PPE 324 5/15-8/16 Yes Participated in expert group; tested 3 models and identified preferred. Overall cost still too high ($85/unit). Shift Labs “Drip Assist” precision infusion monitor 319 5/15-6/16 Yes In collaboration with Z/Map, Drip Assist scaled throughout West Africa. In use in 18 developing countries Dimagi Inc. Adapt CommCare platform for EVD surveillance and buid capacity 299 5/15-5/16 Yes Developed 7 templates based on CommCare, Dimagi’s open-source mobile platform that is in use in 50 countries. It allows tracking of individuals over time, so it suited the need to track symptoms of suspected EVD cases for 21 days. 3D Family Productions Africa Stop Ebola campaign in Guinea 269 5/15-8/15 Yes Implementing partner – MSF. and further developed in Guinea and Sierra Leone, and has been incorporated into broader health information strengthening efforts (IntraHealth, 2018). There were noted challenges in all three countries in implementing mHero, including an initial misunderstanding about the tool’s open-source nature in Guinea (USAID Office of Inspector General 2016 (b)). OUTPUTS AND OUTCOMES Pillar II activities in this area achieved most targets in terms of outputs. The mHero system was incorporated into the MOH’s HMIS strategic plan and may have contributed to lasting change in the connection between healthcare workers and their Ministries of Health. SUMMARY OF KEY FINDINGS IN FOUR DOMAINS Performance USAID Pillar II activities were of short duration (30 months or less), and the actual implementation period was even shorter due to the needed time to work with government and communities on introduction and start-up activities. Outcome data are limited, but the activities for which output data are available have achieved all or most of their intended outputs. Key lessons learned are as follows: 1. No one activity alone can solve a crisis; rather it is the combination of multi-sectoral efforts that will provide the most impact. AFS and health interventions mutually supported each other. For example, the EFSP in the Forest region had a significant impact on household food security and dietary diversity among the 7,000 households targeted but this may be short-term. The sole agriculture activity funded by Pillar II seems to have contributed to stabilization of the rice price for consumers and an increase in the income earned by women rice vendors in rural markets (personal communication, OICI, Conakry, October 3, 2017). This created income for vendors, who could then purchase food and other essentials in the market. Assistance to market traders helped ensure that there were products to purchase. Vouchers provided resources for households to buy a greater diversity of food, hence boosting nutrition, and hygiene commodities such as soap. In the health sector, it was obvious that multiple aspects of service delivery needed to be addressed (e.g., Healthcare Workers (HCWs), water, electricity, equipment, supplies and drugs). On the demand side, there was a need for more information on Reproductive, Maternal, Neonatal, and Child Health (RMNCH), as well as a need to re-engage community members and promote accountable and trust. Individuals will be more willing to take time away from their farms to seek health services if they have confidence in the availability and quality of services. Improvements in the HIS and other information systems will improve epidemiological forecasting, allocation of staff and budget, and increase accountability at the health facility, prefecture, regional, and central levels. 2. The array of activities funded with Pillar II resources were on target to address the main challenges of EVD recovery, but they were constrained by the short timeframe of the Pillar II funding rules. While there are limited primary data available to us concerning outcomes, we were able to find evidence of significant outputs in the form of a number of policy and strategy documents and standard operating procedures that should contribute to improved population health. M–26 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES 3. Building trust, both in the health system and the government more broadly, must be a priority in any public health crisis. While fewer in number than the health sector, there is evidence that Pillar II supported governance activities have contributed to increased participation by the population in the electoral process, with a focus on women, and effective conflict prevention in the sensitive Forest region (USAID, 2016b). One positive indicator is that in the February 2018 municipal elections, 23% of candidates were women (the goal was 30%). 4. Community investment, inclusion, and feedback are vital to the success of recovery efforts. Social mobilization is paramount. Pillar II activities enhanced trust, boosting community interactions with health facilities, opposing political groups, and the government and building a collaborative environment between actors to reduce conflict. The HC3 endline survey indicates a significant increase in confidence by the population in the quality of health services and that more than 80% of people would seek care at local health facilities in the case of illness. This is a transformative outcome. 5. Good data availability, management, and sharing allow for more successful system improvements and consistent improvement over time. Pillar II activities worked with MOH to finalize policy on health management information systems, and provided customization training for statistics officers on Guinea District Health Information System 2 (DHIS2) utilization for data entry, analysis and reporting; and strengthened system integration and data warehousing. 6. Building on existing activities is key to getting recovery activities going quickly in a crisis. Reports from HSD, which expanded its existing activity with Pillar II funds, indicate that quality of service assessment scores above 50% in 84 health centers increased from 11% in 2014 to 85% in 2016 (health centers scoring more than 75% increased from 1% to 35%). Quality of service assessment scores above 50% in 139 hospitals increased from 22% in 2014 to 89% in 2016 (hospitals scoring more than 75% increased from 4% to 60%). Other significant outcomes reported by the Guinea mission include significant increases in patient consultations at healthcare facilities, quality of care assessment scores in health centers and hospitals, women having assisted deliveries, women receiving uterotonics at delivery, and newborns being resuscitated (USAID, 2016). 7. Crises, as difficult as they are, may provide opportunities for improving and strengthening systems that prove more difficult outside of a crisis. One important (indirect) outcome may be the contribution by HFG to a promise of a threefold increase in the government health budget since 2015 (personal communication, Abt Associates, Conakry, September 28, 2017). This created the opportunity to create a larger discussion around health budgeting, though it ultimately did not get approved at higher levels. Resilience The three dimensions of resilience examined in this review are the absorptive capacity to mitigate shocks and stresses, adaptive capacity to restore services and rebuild systems, and transformative interventions that bring about fundamental changes in the society that reduce vulnerability and strengthen it to withstand shocks. At this stage, we could surmise that the health sector activities funded by Pillar II have bolstered the absorptive and adaptive capacity of the country by strengthening health service quality and restoring trust in the health system. This is evidenced by USAID support for a number of important policy initiatives in the areas of health information management, human resource management, financial management, pharmaceuticals, and health communications. In addition, data provided by USAID Guinea suggest that overall health service utilization has increased, including assisted deliveries and perinatal care. Likewise, in food security, ILADP has contributed to adaptive capacity through the stabilization of the rice price nationally and increased income by rice farmers and female market vendors. One could argue that the mentorship by HFG of the Legislative Assembly’s Health Commission has led to a promise of more than threefold increase in the government health budget, which could be a transformative intervention, if it comes to pass. In addition, Support to Local Election Reconciliation and National Unity in Guinea, implemented by Search for Common Ground has been transformative by mitigating potential conflicts in the Forest region, and thus contributing to relative peace in what has been a troubled region in the past. CEPPS has also taken steps toward transformation by significantly mobilizing the participation of women in the electoral process. Sustainability The sustainability framework developed by the ET for the overall Ebola Pillar II evaluation outlines key sustainability determinants: (1) Policy, (2) Finances (3) Accountability, (4) Institutional Support (5) Technical Choices, (6) Human Dimension, and (7) External Threats and Enabling Environments (See Box 1). 1. Policy—Activities were designed to support national policies and work with government to develop them where they either did not exist or needed revision. For example, the HFG has worked ANNEX M. DESK REVIEWS – GUINEA n M–27 BOX 1. DEFINITIONS: KEY DETERMINANTS THAT CONTRIBUTE TO SUSTAINABILITY IBTCI utilizes seven potential categories of distal and proximate determinants that enhance or impede sustainability: policy, finances, institutional support, technical choices, accountability, human dimension, external threats and enabling environment. Below are the definitions for each determinant. 1. Policies, Strategies, and Plans—Policies are predicated by procedures, rules, and allocation mechanisms to provide services and/or to set priorities and guide programs. Policies may be implemented through four types of instruments: laws, regulations, assignment of responsibilities to institutions operating in the sector, and economic incentives (i.e., subsidies) for a given sector to operate properly. Hence, a determinant labeled “policy” will indicate both how it is manifested and also how it is implemented. IBTCI submits a broad definition for policy: Policy is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. Policies with sound strategies are more likely to be adopted. Likewise, strategies need comprehensive and clear plans in order to be implemented. Thus, policies, strategies, and plans are subsumed under one overarching determinant, as they should be inextricably linked in order for the effect of the policies to be realized. Therefore, the IBTCI definition for the first determinant is expanded: There is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. There are also extant strategies and plans for policy implementation. Policies, strategies, and plans create a framework for the implementation of any of the activities associated with post￾EVD recovery. An example might be a health policy or an agricultural development policy that was conceived either as the result of USAID funding and/or influenced or shaped by the Pillar II interventions that USAID supports. 2. Finances—As noted, policies and strategies are operationalized through plans. The implementation of plans, however, requires budget allocations and access to funds. In regard to finances, IBTCI defines this determinant as: Allocation of financial resources for the implementation of plans is in place, with a determination of how much funding is needed, the source of those funds, and how the funds will be apportioned and used. 3. Accountability—Accountability describes the mechanisms and procedures through which decision-makers, policy makers, and implementers justify and assume responsibility for the actions, decisions, services, and products that they undertake. Accountability is best proven through written documentation. There are different types of accountability. Further defined, the salient aspects of accountability follow (UNDP Water Governance Facility/UNICEF, 2015): a. Financial accountability means that institutions and individuals must document the intended and actual use of financial resources allocated to them. A clear demonstration of how the funds were expended should be explicit and without ambiguity. b. Political accountability occurs when constituents request their representatives to justify their actions. If there is political accountability, there are no questions as to the legitimacy of the authority or position of a politician. There may be vertical channels of accountability that link citizens to their government. Horizontal accountability refers when partners of equal standing request their counterparts to account for their actions (i.e., when one local government requests another to justify and explain their actions. c. Social accountability are actions taken by civil society, the media and individuals to incite or implore decision-makers to account for decisions and actions undertaken. d. Administrative accountability is exemplified when one level of an administration asks another to account for their decisions and actions and is documented in an organized manner. Guidance and protocols need to be in situ to ensure accountability. Governmental and other institutional and organizational systems are put into place to decide who gets what, when, and how. These mechanisms should be clearly delineated, and expectations should be communicated to relevant stakeholders. These may include incentives for high performance or disincentives/sanctions for inadequate or irresponsible performance. In sum, this determinant is defined as: Systems and/or mechanisms are in place to ensure appropriate use of financial resources and rights holders’ and duty bearers’ access to information (e.g., on performance and results). These systems and/ or mechanisms safeguard the responsible and constructive use of resources and have transparent processes that are manifested by the populace’s respect for the respective institution. 4. Institutional Support—This determinant refers to organizations that are responsible for implementing actions in support of a policy, strategy, plan, or program. Institutional support examines the actions that are implemented as part of that support and how it may be coordinated among the actors involved. Further down this determinate’s pathway to sustainability, exit and transition strategies are established. Institutional support is thus defined by IBTCI as: Organizational arrangements and/or mechanisms are in place that a) operationalize agreed-upon inputs, roles and responsibilities for M–28 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES implementing policies and interventions and b) produce exit and transition strategies when feasible and appropriate. 5. Technical Choices— Assuming resources are constant, a list of technical choices can be infinite. Hence, the Team will measure a technical choice based on the following criteria: a) appropriateness and relevance to the problem at hand and whether it will solve that problem in an efficient and effective fashion at best value; b) timeliness; and c) demand on resources (human, capital, physical). Thus, the IBTCI definition of the technical choices determinant is: The best technical approach has been selected to lead to implementa￾tion decisions determining whether interventions are “fit for purpose” (effective in addressing the identified problem), feasible given time and resource constraints, and technically relevant. 6. Human Dimension—The importance of the human dimension in the sustainability of development programs is multi-faceted. The human dimension might be as straight-forward as having a person available whose job is to maintain a newly refurbished school or health facility, or a cadre of information technology experts to maintain a fiber optic cable around Monrovia. On the other hand, the human dimension might include more oblique characteristics, such as cultural or institutional norms. Norms present challenges to discussion or systematic analysis as their salient components may be difficult to define. Questions to ask might include, “Is there stigma and discrimination and if so, to what degree?” The Team is exploring the degree to which Pillar II funding ameliorated the consequences of negative norms and/or directed their activities to change them. Transparent and equitable decision-making may be a harbinger for successful implementation of an activity or project, the uptake of services, or changes in social behaviors. This particular determinant stretches across all intervention typologies. Interventions need involvement from a human being to sustain it and serve as its champion. Arguably, this is a somewhat oblique determinant that is open to interpretation and further definition. Thus, for the purposes of evaluation, IBTCI defines human dimensions as follows: Any situation whereby a human being’s input is needed to sustain the intervention. Identify the human dimension— describe it and define it—and the level at which it is available in the appropriate quantity and quality. These nuances are then examined qualitatively to identify activities poised for success and sustainability. 7. External Threats and Enabling Environments—Complex environments, such as those areas in which EVD was most palpable, are influenced by external threats and/or buoyed by enabling environments. This category of determinants is focused on external threats and factors that influence the intervention outcome. They revolve around socio-political stability, geo-politics, climatic conditions, and population migration/internal displacement. For example, roads are needed for the transportation of goods and services to and from villages (e.g., the cold chain for vaccines). Torrential rains or natural disasters may affect the lifespan of roads, thus increasing the need for frequent maintenance or replacement. Climate then is a contextual factor that affects the sustainability of road access and use. Other threats may include internal conflicts between territories, elections, or natural disasters. External threats and enabling environments as a determinant are thus defined: Those factors that have a direct or indirect influence on the ability of an intervention to be sustained, even when all other determinants are supportive of a sustainable outcome. Sustained Results Outbreak Thematic Areas of Response Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology Institutional Enhancements Social & Behavior Change Communications Intervention Types Policies, Strategies, and Plans Finances External Threats and Enabling Environments Human Dimensions Determinants of Sustainability Accountability Institutional Support Technical Choices Immediate Results Ebola Pillar II Response Challenge: Mitigate second-order impacts of the outbreak at the individual, community, and institutional level Survivor Programs Figure M–G3. Analytical framework to measure sustainability ANNEX M. DESK REVIEWS – GUINEA n M–29 effectively in partnership with the MOH Office of Strategy and Development, beginning with a three-day team-building workshop with the MOH Cabinet, including the minister (personal communication, Abt Associates, Conakry, September 28, 2017). This was the start of an effective strategic planning process and the development of a national health sector recovery plan. 2. Finances—Sustainable outcomes are more likely for those longer-term activities that were “topped up” with Pillar II funds, such as Health Service Delivery (12/15-12/20) the Global Health Supply Chain Program (2012-2020), and SIAPS (10/11- 9/17). Other Pillar II activities, such as HC3 and MCSP, were of 2 years duration and therefore difficult to adequately build capacity for sustainability. 3. Institutional Support—Pillar II activities infused GoG efforts with state-of-the-art information, tools, and approaches. When formal guidance did not exist, activities worked closely with GoG counterparts to make that guidance available, and they often funded the development of manuals and job aids to roll out the policies and incorporate them into training. HFG improved and updated the curricula and tools available for pre￾service training of clinical personnel, and it is capacitating local faculty and preceptors from pre-service training institutions. JHU HC3 did the same for community volunteers. 4. Technical Choices—Pillar II activities centered on approaches with proven effectiveness. Nonetheless, specific choices regarding intervention modalities have implications in terms of program effectiveness and, ultimately, sustainability. As an illustration, among Pillar II IPs implementing health-sector activities, strategic decisions in relation to how the activitie addressed human capital gaps (e.g., embedment of technical experts with GoG structures, intensive training and capacity building of existing GoG staff) has implications in terms of sustainability. Embedment can meet immediate needs but with removal of embedded staff, there is the risk that gains will be lost. 5. Accountability—HC3’s “Gold Star” program and community dialogues about health service provision provide an illustration of deliberate measures taken under the auspices of Pillar II to promote accountability in the health sector, while CEPPS and SFCG’s election programs allow for communities to get involved in holding government officials accountable through transparent election processes. These programs may contribute to sustainability if their accountability gains are upheld over time. 6. The Human Dimension—Interventions fostered ownership. Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services, implemented by JSI, has fostered ownership both by communities (through sentinel surveillance) and the national association of Ebola survivors. Community ownership of HC3 activity activities was critical to sustaining activity achievements. Despite the short timeframe, HC3 was committed to fostering this ownership through investment in community-centered approaches that engaged community members in planning, implementation and celebrating achieved results, as well as strengthening relationships of trust and accountability between all parties. 7. Other Contextual Considerations—Pillar II efforts focused on recovery and enhancements to aid preparedness for future emergencies. Efforts in Guinea also explicitly addressed external threats such as sociopolitical instability, a factor that can thwart progress and/or undo past gains. Opportunities and Gaps While the Ebola epidemic was extremely disruptive, especially in the health sector, it has created opportunities to build stronger systems than those in place prior to the epidemic. With the establishment of an effective coordination mechanism through ANSS, the capacity to respond to health security threats in a more effective and timely manner has been strengthened. The realization that inadequate community engagement was a key factor in the early spread of Ebola has led to a heightened awareness of social mobilization as a key element in health development. The Forest region has a history of ethnocentrism, political exclusion and inter-ethnic tensions, a potential bias or risk for beneficiary selection for the Pillar II AFS activities existed. To mitigate beneficiary selection bias and risk to beneficiary selection process, AFS IPs verified 10% of the registered participants and validated the final beneficiary lists by local committees (EFSP, 2017). To support an effective voucher exchange system between vendors and activity beneficiaries, Pillar II AFS activities identified an opportunity to select vendors through a competitive selection process to avoid collusion and market distortion. Unaccustomed to voucher exchange activities, vendors were reluctant to hand their merchandise over to activity beneficiaries using a coupon system as normal, day-to-day exchanges occurred in cash at the time of purchase. To resolve this issue, IP enlisted the support of the local governmental authorities to validate the activity’s strategy and certify the vendor payment methods used by the EFSP activity (EFSP, 2017). Political tensions between government and opposition parties in Guinea were on the rise in March-April 2015 over disagreements regarding the electoral calendar and the sequencing of elections. A particular gap identified by governance activities was the controversy and misinterpretations surrounding the composition of the Independent National Electoral Commission (Commission Electorale Nationale Indépendante-CENI). In addition, political parties and other political actors had limited familiarity with the country’s legal and electoral procedures and institutions. With this political context, the three Pillar II governance activities aimed to improve citizens’ understanding and participation in health governance, promote consensus building and strengthen elections M–30 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES and political processes, and promote peace and reconciliation by directly supporting the election commission. At activity startup, CEPPS conducted consultations with all Guinean stakeholders to assess the feasibility of the creation of the inter-party working group. CEPPS held meetings with party leaders from the ruling party and from the opposition, as well as the CENI. Based on the assessment findings, the CEPPS activity determined that the best approach was to support the existing consultative inter-party working group established by the CENI (CEPPS workplan 2015). Local CSOs required additional support in order to conduct civic and voter education activities in their communities. CEPPS focused on supporting local CSOs in building consensus and promoting informed and peaceful citizens’ participation in the electoral process (CEPPS Quarterly Report 4, 2017). EFPS activity conducted a livelihoods assessment in January 2015 to identify the local livelihood related gaps in Macenta and N’Zérékoré so that the interventions can be tailored to the target population livelihood practices. This assessment result was used to tailor interventions to provide local varieties of crop seeds, and tools for agricultural production including gardening, processing (palm oil, rice, groundnut) petty trade, skilled labor and casual labor. As the activity progressed, other gaps were found such as lack of access to capital and lack of training and technical support in vocational & business skills. To address these gaps, EFSP collaborated with the Department of Agriculture (DPA) to provide the technical training and counsel to activity beneficiaries (EFSP, 2017). Collaboration with the DPA provided an opportunity to ensure the quality of agricultural inputs as per the Ministry of Agriculture’s standards and their certification (for example, germination and varietal purity tests on selected seeds provided at the livelihood fairs organized by EFPS). However, there were missed opportunities. The eight USAID￾funded activities in the health sector partly or fully funded by Pillar II have all been closely aligned with the GoG’s health sector recovery strategy and cover a broad range of priority areas within the CDCS. While this review was unable to assess development outcomes with primary data, there is evidence of effectiveness, as described above. The main barrier to longer-term effectiveness and sustainability has been the short duration of some activities funded by Pillar II. This is exemplified by HC3 which closed after two years even though improved health communications should be a major priority given its failure early in the Ebola epidemic. Likewise, the closure of the Strengthening Data Availability and Use in Guinea activity would be a missed opportunity as the country embarks on real-time use of DHIS2 in 2018. The livelihoods activity ended in December 2016. The final evaluation report found that beneficiaries were already starting to run out of stock of seeds by February 2017 (EFSP, 2017). Management Issues It is not possible to comment on the basis of activity documents alone. In KIIs with IPs and GOG, discussed in further detail in the IBTCI Pillar II Final Performance Evaluation 1 Report, most stakeholders felt that USAID was very consultative and supportive with partners and government stakeholders and praised USAID for its coordination with government and other partners. The supportive role of the mission’s M&E section was highlighted. A number of stakeholders felt that the Pillar II activities were too short in duration and, therefore, unable to build sustainability. Some respondents, especially outside Conakry, felt that there had been inadequate consultation with government stakeholders in the planning of activities. Others felt that investments in health were at the expense of other sectors. ANNEX M. DESK REVIEWS – GUINEA n M–31 SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED – GUINEA African Development Bank Group. Guinea Economic Outlook. Retrieved from https://www.afdb.org/en/countries/west-africa/ guinea/guinea-economic-outlook/. Accessed February 13, 2018. Cenciarelli Orlando, Pietrpaoli Stefano, Malizia Andrea, Carestia Mariachiara, D’Amico Fabrizio, et al (2015). Ebola Virus Disease 2013-2014 Outbreak in West Africa: An Analysis of the Epidemic Spread and Response. International J of Microbiology. 2015; Article ID 769121, 12 pages. CEPPS. Quarterly Report: January 1, 2017-March 31, 2017. CERREGUI (Cellule de recherché en santé de la reproduction en Guinée). Evaluation de base des structures sanitaires cibles du Projet HSD Disponibilité et Capacité Opérationelle des services de santé. Demographic Health Survey and Multiple Indicator Survey. Final Report (French). https://dhsprogram.com/pubs/pdf/FR280/ FR280.pdf. Accessed February 13, 2018. EFSP, Emergency Access to Food for EVD Affected Guineans. Final Program Report, March 2017. Emrey, B. (2016). Preparing for future shocks: Building resilient health systems. In J. Charles, T. Ifafore, & K. Greene (Eds.). USAID Global Health Mini-University. European Commission (2015). EU pledges €450 million to Ebola affected countries. July 2015. Retrieved from http://europa.eu/ rapid/press-release_IP-15-5338_en.htm. Accessed December 18, 2017. Evans David K, Goldstein Markus, Popova Anna (2015). “Health￾care worker mortality and the legacy of the Ebola epidemic.” The Lancet Global Health 3 (8): e439–e440. Retrieved from http://www.thelancet.com/journals/langlo/article/PIIS2214- 109X(15)00065-0/fulltext Global Fund (2017). Global Fund Grants in the Republic of Guinea. August 25, 2017. Geneva, Switzerland. Global Health Ebola Team (2016). Guinea TDY Debrief. Conakry, July 14, 2016. Government of Guinea (2015). Post-Ebola Socioeconomic Recovery and Resilience Strategy 2015- 2017. Unofficial translation. https://papersmart.unmeetings.org/media2/7650686/guinea￾recovery-strategy-summary-non-official-translation-en-3-.pdf Health Governance and Finance (HGF). Guinea Annual Performance Monitoring Report. October 1, 2016–September 30, 2017. Health Services Delivery project (HSD). Quarterly Report and Annual Summary. July 1–September 30, 2017. Conakry, December 31, 2017. HFG Guinea. Health Governance and Finance (HGF). Guinea Annual Performance Monitoring Report. October 1, 2016-September 30, 2017. IntraHealth 2018. Retrieved from IntraHealth website: https:// www.intrahealth.org/news/intrahealth-receives-grand￾challenge-award-for-ebola-response. Accessed March 19, 2018 International Federation of Red Cross and Red Crescent Societies (2015). Red Cross Red Crescent denounces continued violence against volunteers working to stop the spread of Ebola. February, 2015. Retrieved from http://www.ifrc.org/en/news-and￾media/press-releases/africa/guinea/red-cross-red-crescent￾denounces-continued-violence-against-volunteers-working￾to-stop-the-spread-of-ebola/ Johns Hopkins Center for Communication Programs. HC3 Guinea Country Program Research Report. Findings from an Endline Quantitative Phone Survey. Baltimore. November, 2017. John Snow International (JSI). Ebola Transmission Prevention and Survivor Services. Program Overview: Guinea.https://www. advancingpartners.org/sites/default/files/sites/default/files/ resources/apc_guinea_one_pager_a4_web.pdf. Accessed February 13, 2018. MCSP Guinea. Ebola Response and Recovery Workplan. Quarterly report, April-June 2017. MEASURE Evaluation Phase IV. Excerpted Quarterly Report: Ebola Funded Activities. Year 3, Quarter 4. July 1–September 30, 2017. Nfor, E, and C. Bahati, L. Camara (2017). Guinea National Pharmaceutical management system. Assessment. Parpia AS, Ndeffo-Mbah ML, Wenzel NS, Galvani AP (2016). Effects of Response to 2014–2015 Ebola Outbreak on Deaths from Malaria, HIV/AIDS, and Tuberculosis, West Africa. Emerg Infect Dis. 2016;22(3):433-441. https://dx.doi.org/10.3201/ eid2203.150977 Search for Common Ground. (2017). Rebuilding Together: Community-Driven Reconciliation and Enhanced Communication in Guinea Forestière. Final report. June 30, 2017. SIAPS (2016). SIAPS-Guinea Monitoring and Evaluation Plan for the Ebola Supplemental Funds Work Plan. Conakry, August 2016. UNDP. Human Development Report 2016. Retrieved from http:// hdr.undp.org/sites/default/files/2016_human_development_ report.pdf M–32 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES USAID (2014). USAID Development Innovation Accelerator Broad Agency Announcement for Fighting Ebola: A Grand Challenge for Development. Nov 2014 http://pdf.usaid.gov/pdf_docs/ PBAAB273.pdf. USAID (2015). Pillar II: Mitigating Second-Order Impacts of the Ebola Virus Disease (EVD) Epidemic. A Strategic Framework for Liberia, Guinea, Sierra Leone. Washington, 2015. USAID Guinea (2015). Country Development Cooperation Strategy (2015-2020). Conakry, 2015. https://www.usaid.gov/sites/ default/files/documents/1860/Guinea_CDCS_May_2020.pdf USAID Guinea (2016).Year-end Portfolio Reviews: Health. Conakry. November, 2016. USAID Guinea (2016b). Year-end Portfolio Reviews: Governance. Conakry. November, 2016. USAID Guinea (2016c). Year-end Portfolio Reviews: Agriculture and Food Security. Conakry. November, 2016. USAID/IBTCI (2017). Ebola Pillar II Monitoring, Evaluation, and Learning Project: Performance Evaluation Inception Report. Revised July 19, 2017. USAID. (ND). The Resilience Agenda: Measuring Resilience in USAID. http://bit.ly/2DId88N USAID Office of Inspector General. (2015). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, First Quarter. Washington, DC. World Health Organization, (WHO 2014). Ground zero in Guinea: the Ebola outbreak smoulders – undetected – for more than 3 months: A retrospective on the first cases of the outbreak. September 2014. http://www.who.int/csr/disease/ebola/ebola￾6-months/guinea/en/ Accessed December 18, 2017. WHO (2016). Ebola Situation Report. January 20, 2016. http:// apps.who.int/ebola/current-situation/ebola-situation-report￾20-january-2016. Accessed December 18, 2017. World Bank (2016). 2014-2015 West Africa Ebola Crisis: Impact Update. May 10, 2016. http://pubdocs.worldbank.org/ en/297531463677588074/Ebola-Economic-Impact-and￾Lessons-Paper-short-version.pdf World Bank (2015). Summary on the Ebola Recovery Plan: Guinea. April 16, 2015. http://www.worldbank.org/en/topic/ebola/ brief/summary-on-the-ebola-recovery-plan-guinea World Bank (2017). The World Bank in Guinea: Overview. October, 2017. ANNEX M. DESK REVIEWS – GUINEA n M–33 SUB-ANNEX B. PILLAR II ACTIVITIES BY SECTOR AND USAID DEVELOPMENT OBJECTIVE Table M–AB Activities by sector and USAID development objective – Guinea Title Prime IP Sector Dates Funding type Cost (USD) USAID Development Objective (DO) and Immediate Results (IR) Health HC3: Initiative for improving the quality and increasing the demand for reproductive, maternal , neonatal and child health services (RMNCH) Johns Hopkins Univ. Health 9/12-9/17 Ebola 5,500,000 DO1: Utilization of Quality Health Services Increased IR 1.1: Delivery of Quality Health Services Improved IR 1.2: Healthy Behaviors and Demand for Quality Health Services Improved Maternal and Child Survival Program (MCSP) – Health Systems Strengthening Jhpiego Health 4/16- 12/17 Ebola 5,150,000 ($2.75m Ebola) DO1: Utilization of Quality Health Services Increased IR 1.1: Delivery of Quality Health Services Improved Sub-IR 1.1.1 : Availability of Integrated Quality Maternal, Neonatal and Child Health Services Increased Health Service Delivery (HSD) Jhpiego Health 12/15- 12/20 Ebola, MCH, FP 24 million (Ebola $2.9m) DO1: Utilization of Quality Health Services Increased IR 1.3: Health Systems Strengthened Strengthening Data Availability and Use in Guinea MEASURE Evaluation Health 10/15- 3/18 Ebola 2,500,000 DO1: Utilization of Quality Health Services Increased IR 1.3: Health Systems Strengthened Sub-IR 1.3.4: Availability of Quality Strategic Information for Evidence-Based Decision Making Improved Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services JSI Health 6/16-6/18 Ebola 6,000,000 DO1: Utilization of Quality Health Services Increased IR 1.1: Delivery of Quality Health Services Improved Sub-IR 1.1.3: Availability of Prevention, Care, and Treatment Services Increased USAID Global Health Supply Chain Program – Procurement and Supply Management Chemonics Health 2012- 2020 Ebola, PMI 1,250,000 DO1: Utilization of Quality Health Services Increased IR 1.3 Health Systems Strengthened Sub-IR 1.3.3: Health Commodity and Essential Drug Availability Increased Comprehensive Health Systems Strengthening Activity for Guinea (Health Finance and Governance – HFG) Abt Health 6/16-6/18 Ebola 6,000,000 DO1: Utilization of Quality Health Services Increased IR 1.3 Health Systems Strengthened Sub-IR 1.3.1: Policy, Planning, and Transparent Governance Strengthened System for Improved Access to Pharmaceuticals and Services (SIAPS) MSH Health 10/11- 9/17 PMI, Ebola 10,304,800 (Ebola $2.5m) DO1: Utilization of Quality Health Services Increased IR 1.3 Health Systems Strengthened Sub-IR 1.3.3: Health Commodity and Essential Drug Availability Increased M–34 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AB Activities by sector and USAID development objective – Guinea Title Prime IP Sector Dates Funding type Cost (USD) USAID Development Objective (DO) and Immediate Results (IR) Agriculture and Food Security, Economic Crisis Mitigation Emergency Access to Food for EVD-affected Guineans (EFSP) CRS AFS 2/15 – 12/16 FFP/ Ebola $3.3 million DO2: Democratic Governance and Economic Processes Strengthened (Agricultural Productivity) Improved Livelihoods and Agriculture Development Program (ILADP) OICI ECM 3/17 – 12/19 Ebola $1.5 million DO2: Democratic Governance and Economic Processes Strengthened (Agricultural Productivity) IR 2.3: Agricultural Education and Rural Microenterprises Strengthened Governance Citizens Involvement in Health Governance FHI 360 Govnc 6/17-6/20 Ebola/DG $12.17 million DO2: Democratic Governance and Economic Processes Strengthened (Governance and Conflict) IR 2.1: Transparent, Competitive, Accountable Governance Strengthened Consortium for Elections and Political Processes Strengthening (CEPPS) National Dem. Institute Govnc 8/15-2/18 Ebola/DG $4,100,000 DO2: Democratic Governance and Economic Processes Strengthened (Governance and Conflict) IR 2.1: Transparent, Competitive, Accountable Governance Strengthened Sub-IR 2.1.1: Electoral Processes Strengthened Support to Local Election Reconciliation and National Unity in Guinea Search for Common Ground Govnc 9/12-3/17 (ext’d) Ebola, CMM $3,550,969 DO2: Democratic Governance and Economic Processes Strengthened (Governance and Conflict) IR 2.2: Conflict Prevention and Mitigation Promoted in the Forest region Sub-IR 2.2.2: Shared Values Developed ANNEX M. DESK REVIEWS – GUINEA n M–35 Table M–AC. Documents consulted for Pillar II activities – Guinea Title Prime IP Sector Dates Funding type Cost (USD) Documents Reviewed Health HC3: Initiative for improving the quality and increasing the demand for reproductive, maternal , neonatal and child health services (RMNCH) Johns Hopkins Univ. Health 6/15-9/17 Ebola 5,500,000 Johns Hopkins Center for Communication Programs. HC3 Guinea Country Program Research Report. Findings from an Endline Quantitative Phone Survey. Baltimore. November 2017. Maternal and Child Survival Program (MCSP) – Health Systems Strengthening Jhpiego Health 6/15- 12/17 Ebola 5,150,000 (2.75m Ebola) MCSP Guinea. Ebola Response and Recovery Workplan. Health Systems Strengthening. Quarterly Report, April – June 2017. Health Service Delivery (HSD) Jhpiego Health 12/15- 12/20 Ebola, MCH, FP 24 million (2.9m Ebola) Health Services Delivery Project (HSD). Quarterly Report and Annual Summary. J July 1–September 30, 2017. Conakry, December 31, 2017. CERREGUI (Cellule de recherche en santé de la reproduction en Guinée). Evaluation de base des structures sanitaires cibles du Projet HSD Disponibilité et Capacité opérationnelle des services de santé. October 2017. Kindia site visit report, Sep 2016 Monitoring and Evaluation Manual Feedback on the HSD M&E Plan DQA Report, September 2016. Strengthening Data Availability and Use in Guinea MEASURE Evaluation JSI Health 10/15- 3/18 Ebola 2,500,000 MEASURE Evaluation Phase IV. Excerpted Quarterly Report: Ebola Funded Activities. Year 3, Quarter 4. July 1–September 30, 2017. Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services JSI Health 6/16-6/18 Ebola 6,000,000 Advancing Partners & Communities Results Framework, undated. USAID Global Health Supply Chain Program – Procurement and Supply Management Chemonics Health 2012- 2020 Ebola, PMI 1,250,000 n/a Comprehensive Health Systems Strengthening Activity for Guinea (Health Finance and Governance – HFG) Abt Health 6/16-6/18 Ebola 7,000,000 Health Governance and Finance (HGF). Guinea Annual Performance Monitoring Report. October 1, 2016 – September 30, 2017. HFG M&E Plan, January 2017 Mission comments on M&E Plan. System for Improved Access to Pharmaceuticals and Services (SIAPS) MSH Health 10/11- 9/17 PMI, Ebola 10,304,800 (2.5m Ebola) Guinea National Pharmaceutical Supply System Assessment. February 2017 SIAPS-Guinea. Monitoring and Evaluation Plan For the Ebola Supplemental Funds Work Plan. August 2016. DQA Report, Sep 2016. SUB-ANNEX C. DOCUMENTS CONSULTED FOR PILLAR II ACTIVITIES – GUINEA M–36 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AC. Documents consulted for Pillar II activities – Guinea Citizens Involvement in Health Governance FHI 360 Govnc 6/17-6/20 Ebola/DG 12,170,000 million n/a Title Prime IP Sector Dates Funding type Cost (USD) Documents Reviewed Consortium for Elections and Political Processes Strengthening (CEPPS) National Dem. Institute Govnc 8/15-2/18 Ebola/DG 4,100,000 (Ebola 1.5m) CEPPS Quarterly Report: January 1, 2017-March 31, 2017 CEPPS Quarterly Report: October 01, 2016-December 31, 2016. GUINEA: SUPPORT GUINEA 2015 LOCAL AND PRESIDENTIAL ELECTIONS Workplan: August 26, 2015 – August 31, 2016. Support to Local Election Reconciliation and National Unity in Guinea Search for Common Ground Govnc 9/12-3/17 (ext’d) Ebola, CMM 3,550,969 (500k Ebola) DQA Final Action Plan M&E Plan Final Report, June 30, 2017 Quarterly report, January 2016 Quarterly report, April 2016 Quarterly report, July 2016 Quarterly report, October 2016 SFCG Success Stories x 2 Data collection and analysis tool sheets M&E standard operating procedures Performance indicator reference sheet Emergency Access to Food for EVD-Affected Guineans (EFSP) CRS Food Security 2/15 – 12/16 FFP/Ebola 3.3 million Emergency Access to Food for EVD Affected Guineans (EFSP). Final Program Report - March 2017 Guinea EFSP: Baseline Report - April/May 2015. Emergency Access to Food for EVD Affected Guineans (EFSP). Annual Results Report FY16 Improved Livelihoods and Agriculture Development Program (ILADP) OICI Food security/ agriculture 3/17 – 12/19 Ebola 1.5 million n/a ANNEX M. DESK REVIEWS – LIBERIA n M–37 Photo by USAID LIBERIA This report was made possible by the support of the American people through the United States Agency for International Development (USAID). It was prepared by Donna Espeut and Barbara De Zalduondo of International Business and Technical Consultants, Inc. (IBTCI). M–38 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES LIBERIA TABLE OF CONTENTS ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–40 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–42 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–42 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–42 Insights on the Four Evaluation Question Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–43 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–44 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–44 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–45 Overview of the Epidemic and its Impact on Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–45 Major Social and Economic Impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–46 Recovery Response 2015–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–46 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–48 The USAID Ebola Pillar II MEL Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–48 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–48 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–52 Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–52 Basic Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–55 Agriculture and Food Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–55 Governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–58 Economic Crisis Mitigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–59 Innovation, Technology and Partnerships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–60 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–64 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–64 Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–64 Opportunities and Gaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–68 Management Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–69 RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–70 SUB- ANNEX A. REFERENCES AND SOURCES CONSULTED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–71 SUB-ANNEX B. Liberia Pillar II Activities with information in the SBU file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–78 SUB-ANNEX C. Population size and Household Density of priority areas, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–88 ANNEX M. DESK REVIEWS – LIBERIA n M–39 Contents TABLES Table M–L1. Selected human development indicators, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–45 Table M–L2. GOL Response: Illustrative changes in the policy landscape that are relevant to Ebola recovery efforts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–47 Table M–L3. Liberia Pillar II activities included in this desk review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–50 Table M–L4. Pillar II activities in the health sector, according to activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–52 Table M–L5. Pillar II interventions and implementers in the food security sector, Liberia . . . . . . . . . . . . . . . . . . . . . . M–56 Table M–L6. Unconditional cash transfer programs and distribution type, number of households and counties, by implementer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–56 Table M–L7. Fighting Ebola “Grand Challenge” innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–61 Table M–AB. Activities with information in the SBU files, Liberia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–78 Table M–AC.1. Population by county urban/rural (from LISGIS 2008 updated, via for PE1 sampling plan) . . . . . . . . . M–88 Table M–AC.2. Households per county . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–88 FIGURES Figure M–L1. Distribution of EVD cases and deaths, 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–46 Figure M–L2. Jhpiego RHS: Three performance indicators from supported facilities at baseline, end-of-performance year (PY) 1, PY 2, and LOP target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–54 Figure M–L3. Changes in HDDS between baseline and endline by three Pillar II AFS activities . . . . . . . . . . . . . . . . . . M–58 Figure M–L4. Changes in the percent of HHs experiencing moderate or severe hunger between baseline and endline by three Pillar II activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–58 Figure M–L5. Number of counties where mSTAR is operational for MoH employees . . . . . . . . . . . . . . . . . . . . . . . . M–63 Figure M–L6. Number of MoH employees enrolled in Mobile Money Salary Payments, mSTAR . . . . . . . . . . . . . . . . . M–63 Figure M–L7. Capacity building and engagement results from IREX’s CSML activity that promoted accountabilitiy . M–65 Figure M–L8. Additional accountability results from the CSML activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–65 Figure M–L9. Analytical framework to measure sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–67 M–40 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ACF Action Contre la Faim AEU Africa Ebola Unit AFR Africa Bureau AFS Agriculture and Food Security ANC Antenatal Care APS Annual Programming Statement APC ETP&SS Advancing Partners and Communities – Ebola Transmission Prevention and Survivor Services BMGF Bill & Melinda Gates Foundation CaLP Cash Learning Partnership CDC United States Centers for Disease Control and Prevention CDCS Country Development Cooperation Strategy CfW Cash for Work CHV Community Health Volunteer CLTS Community Led Total Sanitation COR Contracting Officer Representative CSH Collaborative Support for Health CSML Civil Society and Media Leadership CSO Civil Society Organization CT Cash Transfer CTWG Cash Transfer Working Group DfID Department for International Development (United Kingdom) DEC Development Experience Clearinghouse DHIS-2 District Health Information System 2 DHS Demographic and Health Survey DO Development Objective ECM Economic Crisis Mitigation ECRL Education Crisis Response in Liberia EPHS Essential Package of Health Services ERSP Economic Recovery and Stabilization Plan ET Evaluation Team EU European Union EVD Ebola Virus Disease FAO Food and Agriculture Organization FEWS NET Famine Early Warning Systems Network FFP Food for Peace FY Fiscal Year gCHV General Community Health Volunteer GBV Gender-based Violence GDP Gross Domestic Product GEMS Governance and Economic Management Support GHET Global Health Ebola Team GoL Government of Liberia GSA General Services Agency JHU HC3 Johns Hopkins University Health Communication Capacity Collaborative HCW Healthcare Worker HDDS Household Dietary Diversity Score HF Health Facility HH Household HIS Health Information System HIV Human Immunodeficiency Virus IBTCI International Business and Technical Consultants, Inc. IE Institutional Enhancements ICT Information and Communication Technology IP Implementing Partner IPC Infection Prevention and Control IMAM Integrated Management of Acute Malnutrition ITP Innovation, Technology and Partnerships JSI John Snow, Inc. LDP+ Leadership Development Program LIPA Liberia Institute for Public Administration LMWP Liberia Municipal Water Project LOP Life of Project LSA Liberia Strategic Analysis M&E Monitoring and Evaluation MACs Ministries, Agencies and Committees MCP Management, Coordination and Partnerships MCSP Maternal and Child Survival Program MDG Millennium Development Goal MEL Monitoring, Evaluation, & Learning MoAg Ministry of Agriculture MoGCSP Ministry of Gender, Children, and Social Protection MoH Ministry of Health MoHSW Ministry of Health and Social Welfare MoFDP Ministry of Finance and Development Planning MOU Memorandum of Understanding MSH Management Sciences for Health NDS National Drug Service NGO Non-Governmental Organization FFP Office of Food for Peace (USAID) OU Operating Unit PACS Partnership for Advancing Community-Based Services ACRONYMS ANNEX M. DESK REVIEWS – LIBERIA n M–41 PHC Primary Health Care PHU Peripheral Health Unit PTAs Parent-Teacher Associations PY Performance Year RMNCH Reproductive, Maternal, Newborn, and Child Health SAM Severe Acute Malnutrition SBCC Social and Behavior Change Communication SBU Sensitive but Unclassified SO Strategic Objective SOW Scope of Work STC Save the Children ToC Theory of Change UK United Kingdom UN United Nations UNICEF United Nations Children’s Fund UNMEER UN Mission for Ebola Emergency Response USAID United States Agency for International Development USG United States Government WASH Water, Sanitation, and Hygiene WFP World Food Programme WHO World Health Organization M–42 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES EXECUTIVE SUMMARY In 2014 and 2015, Liberia, a country of under five million people, was struck by the largest outbreak of Ebola Virus Disease (EVD) in history. The epidemic paralyzed economic growth and reversed many of the nation’s recent development gains. This report is based on a desk review of key efforts funded either partially or entirely by the United States Agency for International Development (USAID) between March 2015 and March 2017 to help the people of Liberia recover from the EVD crisis and “build back better.” Methodology The desk review is part of a broader performance evaluation of USAID’s Ebola Pillar II response. Based on activity documents provided by USAID and additional information found in the public domain, information is presented according to six thematic areas: (1) Health (Non-Ebola Health Services and Health Systems Recovery and Survivor Programs); (2) Basic Education; (3) Agriculture and Food Security (AFS); (4) Governance; (5) Economic Crisis Mitigation (ECM); and (6) Innovation, Technology and Partnerships (ITP). Based on activity mapping conducted in 2017, the Evaluation Team (ET) identified 44 activities in Liberia that were classified by USAID as Pillar II￾supported efforts. However, one key limitation of the desk review was the ET’s limited access to Pillar II activity documentation. This report is based on a review of documents made available for 25 Pillar II activities in Liberia, including four Pillar II ‘regional’ activities whose purview included, but was not limited, to Liberia. To the extent relevant information was available, the desk review provides insight into four sets of evaluation questions: (1) Performance (How are the Pillar II Ebola Recovery activities in Liberia contributing to the achievement of the mission and Ebola Pillar II strategic framework’s objectives?); (2) Sustainability (What Pillar II accomplishments are still observable after the activities have closed, and why?); (3) Gaps and opportunities (Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and missed?); and (4) Management (What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency?) Findings Health: The main Pillar II health activities supported efforts to enhance care and support for EVD survivors, addressing training needs of service providers and working with relevant national bodies (National Ebola Survivors Network of Liberia, National Ebola Survivor Secretariat); addressed frontline worker support (primarily management and technical training); management, coordination and partnerships (MCP); health information system (HIS) strengthening, mobile money salary payments to MoH employees (ICT); social and behavior change communication (SBCC) to re-ignite and enhance demand for formal-sector health services, and institutional enhancements (e.g., minor health facility upgrades and renovations). In general, the Pillar II health activities were able to produce the majority of planned outputs (e.g., assessments completed, plans developed, health workers trained, facilities strengthened) and have documented some noteworthy outcomes related to health service recovery and improved performance in areas such as infection prevention and control (IPC). Basic Education: Pillar II interventions in this thematic area, implemented largely through UNICEF’s Education Crisis Response in Liberia (ECRL) activity, sought to restore and boost educational outcomes through improving social protection support (e.g., providing primary school learners with school supplies), frontline worker support (limited teacher training on issues such as ‘psychological first aid’ for children who have experienced trauma), SBCC (e.g., to promote hygiene practices) and institutional enhancements (e.g., refurbishing and resupplying schools; improving school-based IPC and water, sanitation and hygiene (WASH) facilities). AFS: Social protection was a major thrust of Pillar II-supported AFS activities, with four activities implementing activities such as cash transfer programs (CTPs) through direct cash and mobile money modalities. Activities with baseline and endline data documented major improvements in food security outcomes such as household dietary diversity and household hunger scores, not just in the reach of their social protection interventions. Governance: Pillar II-supported activities in this thematic area focused on MCP interventions, as well as some SBCC interventions, to strengthen governance systems within the Government of Liberia (GoL), and to engage civil society organizations (CSOs) including women’s and media organizations to increase civic participation in local and national governance. The activities addressed human capacity, institutional capacity, and the enabling environment for good governance, transparency and accountability. ECM: Pillar II AFS activities contributed to economic strengthening of vulnerable individuals, households and communities and/or those most affected by EVD. Other ECM activities also took place. However, activity documentation was not made available to the ET for this desk review. ANNEX M. DESK REVIEWS – LIBERIA n M–43 ITP: Pillar II-supported efforts in this thematic area have addressed both hardware and ‘software’ (e.g., training, quality assurance, improved interoperability of different information systems/ platforms). In health, ICT activities receiving Pillar II funds include mHero, implemented by IntraHealth International, and mSTAR (Mobile Solutions Technical assistance and Research Program), implemented by FHI360. mHero involves a platform to allow two-way communication using mobiles to both gather data from health workers and support decision making. Pillar II funds have helped expand mSTAR, a successful mobile money salary payment platform in the education sector, to the health sector. Pillar II funds have also been allocated to Tetra Tech, a private sector company, to improve piped water systems and water management capacity in four municipalities. Insights on the Four Evaluation Question Domains Performance: Pillar II activities were of short duration (30 months or shorter), and the actual implementation period was often shorter than the official award period due to the time needed to work with GoL and communities on activity introduction and start-up activities. Outcome data are limited, but the activities for which output data have been made available have achieved all or most of their intended outputs. Sustainability: Pillar II activities have supported national policies and/or worked with GoL to develop them where they either did not previously exist or needed revision. Some Pillar II activities have also promoted sustainable financing within their thematic areas; for example, to cover operating expenses and maintenance related to WASH infrastructure. A myriad of Pillar II activities strengthened institutional support for particular activities by supporting GoL efforts with state-of-the-art information, tools, and approaches, as well as capacity building. Although Pillar II activities centered on approaches with proven effectiveness, specific technical choices (e.g., provision of cash transfers using direct cash distribution via banks versus mobile money platforms) have had implications in terms of program effectiveness and, ultimately, sustainability. Technical choices on how human capacity needs were addressed (e.g., embedment of technical experts with GoL structures, intensive training and capacity building of existing GoL staff), also have bearing on sustainability. Enhanced accountability, which can also be a determinant of sustainability, was observed for a few activities. In considering how the ‘human dimension’ influences sustainability, community engagement is a common feature of Pillar II efforts—not just consultation on priorities and needs when implementation plans were being developed, but also the fostering of local ownership in implementation of specific strategies (e.g., in the education sector, parent-teacher associations were engaged in devising strategies to continue WASH activities in schools). Other contextual considerations such as climate change or sociopolitical instability may also affect sustainability. However, there is limited evidence on contingency planning in light of those risks/threats that affect the enabling environment. Opportunities and Gaps: Allocation of Pillar II funds to private sector players such as Innovation Lab, Google and Coca Cola has brought some unconventional players in development programming into the fold. Opportunities related to ICT abound, even though telecommunications network infrastructure is limited. There are also noted gaps such as the need for a clearer analytic focus and measurement strategy for tracking community-level effects of the EVD crisis and monitoring the full range of second￾order impacts of the EVD outbreak. Management: Pillar II implementation experiences have demonstrated that timing is everything. The four months between March and August 2014 – a mere moment in usual international development assistance time – was enough to enable the virus to spread throughout Liberia, and for rumors and fears to gain traction. These fears had profound impacts on the individual and organizational behaviors that constituted the main second-order impacts of the EVD outbreak. Delays in release of funding for Pillar II activities while, again, very short by normal development standards, could lead to major inefficiencies. For example, Project Concern International’s first-round of agricultural input vouchers did not reach farmers until after the planting season. While the seeds could be kept for the following year and were not wasted, they didn’t help farm families get through that first lean season. Another management-related insight concerns how to integrate new, urgent programmatic action with pre-existing, pre-arranged, and pre-agreed plans. There are lessons learned by Pillar II activities on community engagement in identifying vulnerable households for targeted interventions. Strong, multi-component monitoring systems, which include adequate staff and budget with strong, empowering normative and communication components, improve transparency, accountability and acceptability. USAID programing is constrained in many ways by funding earmarks and conditions, and it is understandable that program management is organized accordingly. However, the diversity of funding units, with money linked to specific operating units, complicated the planning and implementation of Pillar II activities in Liberia. M–44 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Lessons Learned Key lessons learned are as follows: 1. No one intervention can stand alone; the various types of interventions across the different thematic areas (e.g., AFS and health) mutually supported each other. 2. The array of activities funded with Pillar II resources were on target to address the main challenges of EVD recovery, but they were constrained by the short timeframe of the Pillar II funding rules. 3. Financial and social safety nets are invaluable. Short-term infusions of cash, seeds, equipment, drugs, and infrastructure are necessary to restart and rebuild livelihoods and health services, but resilience and sustainable change comes through creating safety nets, which are vital to building resilience. 4. Building human capacity is a powerful investment. Pillar II activities enhanced human capacity, boosting management capacity in central government agencies, training multiple levels of healthcare providers, supporting community- and district￾level forums for health and education ministries to listen to the people, and training women and girls to actively contribute to such discussions. All of the above contribute to resilience and have the potential to endure and grow beyond Pillar II investments. 5. Pillar II interventions supported small steps toward transformative change, most notably in introducing or revitalizing frameworks and decision-making platforms in which communities can participate and hold the government accountable, as well as make provisions for women’s inclusion as beneficiaries and project implementers. Recommendations Illustrative recommendations are as follows: 1. Redouble M&E efforts to support sound, rapid, evidence-based decision making during and after a crisis. 2. As part of resilience building, invest/prepare poor households for the inevitable annual “lean season.” 3. Continue to invest in trust-building activities with both government and constituents. 4. Invest in comprehensive and agile disease surveillance. 5. Address persistent stigma against EVD survivors, orphans and dependents. ANNEX M. DESK REVIEWS – LIBERIA n M–45 INTRODUCTION In 2014 and 2015, Liberia, a West African country of under five million people (see Table 1), was struck by the largest outbreak of Ebola Virus Disease (EVD) in history. By the time the World Health Organization (WHO) declared that Liberia’s Ebola epidemic had ended in January 2016, there were 10,678 suspected, probable, and confirmed cases of Ebola in Liberia, with 4,810 deaths (WHO, 2016a). The epidemic paralyzed economic growth and reversed many of the nation’s recent development gains (Government of Liberia, GoL), 2015a). With concerted effort and investment, the GoL, the United States Government (USG), international health and development agencies, and affected communities succeeded in containing the epidemic. This report summarizes key efforts by the United States Agency for International Development (USAID) between March 2015 and March 2017 to support Liberia’s recovery from the EVD crisis and to “build back better” (USAID Africa Bureau, 2015). It also describes the context in which USAID efforts were implemented, and it presents preliminary conclusions and recommendations. Overview of the Epidemic and its Impact on Health The first cases of Ebola in Liberia were identified in the northwest county of Lofa in March 2014, but the seriousness of the outbreak was not immediately recognized. By the time President Johnson Sirleaf declared a state of emergency on August 6, 2014, the virus had spread throughout the country (see Figure 1 for EVD hotspots), and over 1,000 people had died in the region (Nyanfore, 2016). In the initial stages, fluid population movement within Liberia, as well as between Liberia, Guinea, and Sierra Leone, contributed to both the scale and longevity of Liberia’s EVD epidemic (Ladner et al., 2015). The emergence of new cases in July 2015 underscored the need for continued vigilance, particularly given the sheer number of persons affected by the initial wave of infections in 2014, and the Ebola virus’s ability to persist within a population (WHO, 2015). Factors such as health workforce shortages impeded the country’s Ebola response (Shoman et al., 2017, Weah et al., 2016). Healthcare workers (HCWs) were also highly affected by the epidemic: there were 378 confirmed HCW EVD cases, of which 192 died, accounting for 12% of all EVD cases and 4% of all EVD-related deaths in the country (GoL, 2016, Nyenswah et al., 2016). Coupled with health service delivery challenges, public fears that health facilities (HFs) were playing a role in propagating Ebola infections also contributed to reduced healthcare use (Ly et al., 2016). There was a documented decline in the use of primary healthcare (PHC) services such as antenatal care (ANC), and specialized care such as HIV treatment (GoL, 2016; Hira and Piot, 2016; Parpia et al., 2016). Table M–L1. Selected human development indicators, Liberia Indicator Estimate Population Size 4.5 million Median Age 18.6 % of Population Living in Urban Areas 49.7% Human Development Index Ranking 177 Gross National Income Per Capita (2017) $683 Life Expectancy at Birth 61.2 years Adult Literacy Rate 47.6% Poverty Headcount (% of population @ $1.90/day 2011 PPP) 68.6% Maternal Mortality Ratio 725 * Rate obtained using DHS module, involving a reading test, and tends to be lower than simple reports of literacy. Source: United Nations Development Programs (UNDP) Human Development Report 2016: Country Data for Liberia M–46 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Major Social and Economic Impacts Prior to the epidemic, the country set out a lofty vision for 2030 (GoL, 2012). In an ambitious Agenda for Transformation (GoL, 2015), the GoL laid out a costed plan for inclusive economic growth and social development. EVD struck at a time when Liberia was recovering from 14 years of civil war, which had taken over 250,000 lives and disrupted education and health services for a generation (Liu, 2017). The EVD crisis exposed weaknesses in Liberia’s economic, social service, and health infrastructure, systems, and institutions (GoL 2015; UNECA, 2015; Parshley, 2016; USAID/Liberia, 2016). Economic disruptions sparked by the Ebola epidemic led the World Bank to reduce the 2014 projected Gross Domestic Product (GDP) growth from 5.9% to less than 1% (World Bank, 2015a). During the 18-month outbreak, commerce halted, major farming and ore industries slowed or ceased operations, and a large fraction of the private sector and development assistance workforce left the country. Although EVD was not evenly distributed across the country (Figure 1), second-order impacts were national in scope. Government tax and non-tax revenues in 2015 were expected to amount to USD $401 million, over $100 million less than had been projected before the epidemic, while the GoL estimated that responding to the epidemic increased expenditures by USD $76.2 million, contributing to deterioration in the fiscal balance (World Bank, 2015a). Progress in rebuilding trust in the GoL services was also compromised (GoL Ministry of Health and Social Welfare [MoHSW], 2015; USAID/Liberia 2016). The estimated fiscal impact of Liberia’s EVD crisis is $300 million (World Bank, 2016). There was a 40% decrease in the percentage of persons working since the onset of the epidemic – a phenomenon that was particularly high among women employed prior to the Ebola crisis (World Bank, 2016). Notably, however, a 2017 report by the USAID-supported Famine Early Warning Systems Network (FEWS NET) documented that, with the exception of households (HHs) with working-age HH members who died from Ebola, losses related to HH food or income do not appear to be permanent (FEWS NET, 2017a). Recovery Response 2015–2017 GOL RESPONSE Since initial delays in acknowledging the epidemic’s severity, the GoL has demonstrated leadership in the country’s recovery. It established an incident management system to direct and oversee the multi-sectoral response to the epidemic (and future crises). In addition, intensive reviews were conducted in multiple sectors to understand the immediate and root causes of the fragility of the systems and public institutions that were responsible for protecting and serving the Liberian people. In consultation with a breadth of development partners, the GoL developed Liberia’s Economic Stabilization and Recovery Plan (GoL, 2015). Other policies (see Table 2) also contribute to an enabling environment for recovery. USAID’S SUPPORT FOR EBOLA RECOVERY In March 2015, USAID established the Africa Ebola Unit (AEU) in Washington to oversee coordination of Ebola-related activities within USAID, across agencies, and with the wider international community. In collaboration with other USAID units, the AEU provides support to USAID missions in affected countries to implement a robust set of development programs to address secondary impacts of the outbreak and ensure that Guinea, Sierra Leone, and Liberia are prepared to prevent, detect, and respond to future outbreaks. On September 30, 2016, AEU’s responsibilities were transferred to the Africa Bureau’s (AFR) Office of West African Affairs, which now oversees the USG’s “all of government” Ebola recovery. That effort has four pillars: (I) Controlling the epidemic at its source; (II) Mitigating second￾order impacts including blunting the economic, social and political consequences in the region (i.e., health services and health systems; basic education and food security; governance and economic crisis mitigation; and innovation, technology and public-private partnerships); (III) Engaging and coordinating with a broader global audience; and (IV) Fortifying health security infrastructure in the region and beyond. Pillar II, which is the focus of this desk review, has three strategic objectives (SOs): (1) Prevent the loss of development gains; (2) Recover and strengthen existing institutions and infrastructures whose weaknesses enabled the rapid spread of Ebola or slowed the response; and (3) Build sustained systems through public-private partnerships, innovation, and capacity building. Some Pillar II inputs were designed and funded at the mission level, and others Figure M–L1. Distribution of EVD cases and deaths, 2014 by USAID operating units (OUs) in Washington. ANNEX M. DESK REVIEWS – LIBERIA n M–47 OTHER SUPPORT A myriad of private, non-governmental organizations (NGO), bilateral and multilateral development partners stepped up to assist the GoL in its Ebola recovery efforts (MoHSW, 2015). In 2014 and 2015, large aid flows from the USG and others accounted for almost 19% of Liberia’s GDP (World Bank, 2016). The USG has led Liberia’s response and recovery efforts, with other partners such as the British Government working alongside USG implementers (Department for International Development – United Kingdom [DfiD/UK], 2016). For example, through agencies such as Oxfam, the UK supported social mobilization; distribution of hygiene items; and water, sanitation and hygiene (WASH) improvements and rehabilitation at schools and peripheral health units (PHUs) in some counties (Adams et al, 2015). The European Union (EU) has been supporting other needs such as infrastructure strengthening (e.g., Red light to Gbarnga Road Project), provided targeted support in the agricultural and education sectors, and invested in health-sector efforts to reduce maternal mortality and morbidity (EU, 2017). From September 2016-October 2017, it funded Pro-Resilience Action, implemented by Action Contre la Faim (ACF), to revitalize and build resilience of rural food systems in EVD-affected areas (EU, 2017). Table M–L2. GOL Response: Illustrative changes in the policy landscape that are relevant to Ebola recovery efforts Sector/Lead Ministry Policy Documents/Developments Before the onset of the Ebola crisis During the outbreak After the Ebola crisis Min. of Agriculture (MoAg) National Food Security & Nutrition Strategy (2008) Min. of Finance and Development Planning (MoFDP) Agenda for Transformation (2013) MoFDP Strategic Plan, 2015–2018 Min. of Gender, Children & Social Protection (MoGCSP) National Social Protection Policy & Strategy (2013) National Gender Policy (2009) MoH/MoHSW National Nutrition Policy (2008) Essential Package of Health Services (EPHS), Phase I (2011) National Health & Social Welfare Policy and Plan, 2011–2021 National Health & Social Welfare Financing Policy & Plan, 2011–2021 National Human Resources Policy & Plan for Health & Social Welfare, 2011–2021 National Community Health Services Strategy (revised 2011) Investment Plan for Building a Resilient Health Systems, 2015–2021 (2016) Mental Health Policy & Strategic Plan 2016–2021 Liberian National Ebola Survivor Secretariat (launched 2017) Liberia Ebola Survivors Clinical Care Guidelines (2016) Revised National Community Health Services Policy (2016) National Community Health Assistant Program (launched 2016) Liberia Telecom. Authority Liberia Information, Communication & Technology (ICT) Policy, 2017–21 CROSSCUTTING/ Multisectoral Liberia RISING 2030 (2012) National Policy on Decentralization & Local Governance (2012) Integrated Water Resources Management Policy (2008) Water, Sanitation, & Hygiene Sector Strategic Plan 2012–2017 Water Supply & Sanitation Policy (2009) National Ebola Response Strategy: Scaling up to the challenge (2014) EVD Survivors Care & Support National Policy (2016) GoL references for the above appear in the List of References (SUB-ANNEX A). M–48 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES During the crisis, unified United Nations (UN) efforts took place under the auspices of the UN Mission for Ebola Emergency Response (UNMEER). UNMEER closed in July 2015, transitioning recovery and development responsibilities back to individual UN agencies (UN, ND). The following are illustrative UN agency recovery efforts: (1) The Food and Agriculture Organization (FAO) is addressing economic impact, livelihoods and food security in Bong and Lofa, and Nimba; (2) The World Food Programme (WFP) is addressing food security, with an emphasis on young children and EVD-affected individuals and HHs in former EVD hotspots, as well as GoL emergency preparedness (UN, ND); (3) WHO developed a global toolkit for achieving health service resilience (WHO, 2017a), and it is supporting Liberia’s MoH on infection prevention and control (IPC) training and preparedness (WHO, 2017b). The World Bank is the largest financier of the GoL’s investment plan for “Building a Resilient Health System” (World Bank, 2017). It is financing procurement of ambulances and other vehicles, essential medical equipment and supplies, medical education (laboratory, pre-clinical and graduate residency programs), development of triage and isolation facilities in five counties, and improved housing duplexes for HCWs in hard-to-reach areas. In addition to health, a portion of the roughly USD $400 million mobilized by the World Bank for Liberia’s Ebola response and recovery efforts is invested in the education sector (World Bank, 2016a). For example, since October 2015, its Education Sector Ebola Recovery and Reconstruction Project has supported the safe reopening of schools, promotion of school attendance and enhanced quality of learning (Darvas, 2017). Since 2016, the World Bank has also invested in targeted Ebola recovery livelihood support for females aged 18–27 years in Grand Bassa, Montserrado and Margibi counties, via MoGCSP (World Bank, 2016b). Given the multiplicity of actors supporting recovery, and overlapping themes across efforts (e.g., health system recovery), complementarity of effort is paramount. When examining USAID’s contributions, it will be important to consider concurrent efforts on the part of other development actors in Liberia, particularly in the same geographic areas, and with the same target groups. Also, the aforementioned efforts did not occur independent of USAID. The World Bank and UN agencies such as UNICEF, WFP and WHO also received Pillar II funds to support some of their recovery efforts (USAID Office of Inspector General, 2015). APPROACH TO DESK REVIEW The USAID Ebola Pillar II MEL Project In October 2016, International Business & Technical Consultants, Inc. (IBTCI) was awarded a three-year project – Ebola Pillar II Monitoring, Evaluation, & Learning (MEL) – to conduct performance and impact evaluations and monitoring activities to examine USAID-funded Ebola recovery interventions in Guinea, Liberia, and Sierra Leone from March 2014 through December 2019. There are four key evaluation questions: (1) Performance: How are the Pillar II Ebola Recovery activities in Liberia contributing to the achievement of the mission and Ebola Pillar II Strategic Framework’s objectives? (2) Sustainability: What Pillar II accomplishments are still observable after the projects have closed, and why? (3) Gaps and Opportunities: Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and what was missed? (4) Management: What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency? Methodology PURPOSE The desk review is intended to collate available information on USAID’s Pillar II investments in Liberia, serving as one of several methods of data collection and analysis. It summarizes findings from documents from activities in Liberia that were funded partially or entirely with USAID Pillar II funds. KEY COMPONENTS The desk review began with a call for documents (with assistance from USAID/Washington and USAID/Liberia). Based on the set of activity documents provided by USAID, and supplemented by additional information gleaned from Implementing Partner (IP) documents and other sources that the Evaluation Team (ET) found in the public domain, the ET extracted information according to the following six thematic areas: (1) Health (Non-Ebola Essential Health Services and Health Systems Recovery and Survivor Programs); (2) Basic Education; (3) Agriculture and Food Security (AFS); (4) Governance; (5) Economic Crisis Mitigation (ECM); and (6) Innovation, Technology and Partnerships (ITP). Based on that evidence base, the ET: (a) Assessed activity contributions to USAID Ebola Pillar II strategy and the GoL Agenda for Transformation, using Ebola Pillar II Theory of Change (ToC); (b) Analyzed contributions of the interventions to dimensions of resilience and sustainability; and (c) formulated recommendations ANNEX M. DESK REVIEWS – LIBERIA n M–49 for the next phase of recovery from the EVD crisis. In exploring resilience, the team used USAID‘s resilience definition: “The ability of people, households, communities, countries and systems to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.” (USAID, ND). Focusing on health systems, Emrey (2016) suggested that resilience consists of: “Capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons during the crisis, reorganise if conditions require it.” USAID defines sustainability as: “The ability of a local system to produce desired outcomes over time, including ability to be both resilient and adaptive in the face of changing circumstances.” This definition emphasizes the persistence of project effects after interventions have ended. The ET developed an analytical framework for examining sustainability determinants such as policy, financing, accountability, institutional support, the human dimension, and the enabling environment. To the extent that relevant information exists, determinants have been explored under the auspices of this desk review. Actual measurement of sustainability and resilience, and assessment of outcomes, will be addressed in greater depth in the Pillar II MEL Project Performance Evaluation 2 at the end of 2018. SAMPLING Through activity mapping in 2017, the ET identified a total of 44 activities in Liberia that were classified by USAID as Pillar II efforts, and were expected to have M&E plans, regular results reports, and baseline, midline and/or end of activity evaluation reports. Information received from the mission and the Liberia Strategic Analysis (LSA) project in Phase 1 indicate that the LSA project tracks the activities’ progress against their M&E plans. However, the ET was unable to obtain documentation on a large number of Pillar II activities. In the spring of 2017, IBTCI reached out to the mission and the IPs through the activity’s Contracting Officer Representative (COR) to request activity documentation. Second and third data calls were submitted in October and December 2017, respectively. Some of the documents are publicly available, but many were classified Sensitive but Unclassified (SBU) and have only been reviewed by staff who have signed Non￾Disclosure Agreements with USAID/W and USAID/Liberia. This desk review report is based on available documents from USAID/AFR and other Washington OUs, USAID/Liberia and/ or by IPs at the mission’s request, the USAID Development Experience Clearinghouse (DEC), and other documentation/data found in the public domain (e.g., organizational websites, peer￾review journals) for 25 Pillar II activities. Annex A contains the list of documents reviewed, and Annex B contains basic information on the 25 activities. Table L3 classifies each activity according to the six thematic areas, as well as the following six intervention type(s): (1) Social Protection for EVD Survivors and Other Vulnerable Groups; (2) Frontline Worker Support; (3) Management, Coordination and Partnerships (MCP); (4) Information and Communication Technology (ICT); (5) Social and Behavior Change Communication (SBCC); and (6) Institutional Enhancements (IE). LIMITATIONS The desk review is not the sole source of evidence in answering the main evaluation questions. It represents IBTCI’s first contribution to compiling information and distilling insights on USAID’s contribution at a macro level. The following are specific limitations: (1) Measuring outcomes is challenged by the short duration of the interventions, the lack of baseline data and/or the poor quality of monitoring data. (2) The ET had very limited access to summative/endline reports, or other documents that presented cumulative results since activity inception. (3) Because the team did not have access to routine monitoring reports, this limited the ability to collate results and examine time-series data. (4) Most activity outputs are reported as “numerator” only (i.e., IP reports note the number of people or health facilities impacted, and no denominators to assess how much of the need was addressed). (5) Beyond indicating county/ies of operation, few reports offer detailed geographic information on their service delivery area. (6) There are a number of activities for which no data or documents were provided by USAID. M–50 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–L3. Liberia Pillar II activities included in this desk review IP and Activity Thematic Areas of Focus Predominant Intervention Types Health Edcn AFS Govnc ECM ITP Soc Protn Ftln Wkr MCP ICT SBCC IE ACDI/VOCA: Ebola Recovery and Resilience Program (ERRP) ✔ ✔ ACF: Cash Learning Partnership (CaLP) ✔ ✔ Carolina Population Center: MEASURE Eval. ✔ ✔ Chemonics: Famine Early Warning Systems Network (FEWSNET) ✔ ✔ FHI 360: Mobile Solutions Technical Assistance and Research (mSTAR) ✔ ✔ Global Environment and Technology Foundation, with The Coca-Cola Company, The Coca-Cola Foundation, Global Fund to Fight AIDS, TB and Malaria USAID and BMGF: Partnership with Project Last Mile ✔ ✔ ✔ ✔ IBI International: Governance and Econ. Management Support (GEMS)* ✔ ✔ International Rescue Committee (IRC): Partnership for Advancing Community￾Based Services (PACS) ✔ ✔ ✔ ✔ ✔ ✔ Inveneo: Ebola Recovery Connectivity Initiative (ERCI 2.0) ✔ IREX: Civil Society and Media Leadership Program (CSML) ✔ ✔ JHPIEGO: Restoration of Health Services in Liberia ✔ ✔ ✔ ✔ JHPIEGO: Maternal and Child Survival Program/Liberia Human Resources for Health ✔ ✔ JSI: DELIVER ✔ ✔ JSI/Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services (APC￾ETP&SS) ✔ ✔ ✔ Johns Hopkins Univ. Center for Communication Programs (JHU CCP): Health Communication Capacity Collaborative (HC3) ✔ ✔ ANNEX M. DESK REVIEWS – LIBERIA n M–51 Table M–L3. Liberia Pillar II activities included in this desk review IP and Activity Thematic Areas of Focus Predominant Intervention Types Health Edcn AFS Govnc ECM ITP Soc Protn Ftln Wkr MCP ICT SBCC IE Mercy Corps: Economic Recovery from Ebola for Liberia (EREL) ✔ ✔ Management Sciences for Health: Collaborative Support for Health ✔ ✔ Project Concern International: Protect and Empower for Ebola Resilience (PEER) ✔ ✔ Save the Children: Emergency Food Assist. for Ebola Affected Families ✔ ✔ Tetra Tech: Liberia Municipal Water Program (LMWP) II ✔ ✔ UNICEF: Screening, referral, treatment for children with SAM, RUTF ✔ ✔ ✔ UNICEF: Education Crisis Response in Liberia - WASH in schools ✔ ✔ UNICEF: Strengthening Routine Immunization in Liberia ✔ ✔ ✔ WFP: Emergency Response Operation Project (EMOP)— Refugee Protracted Relief and Recovery Operation ✔ ✔ WFP: UN WFP emergency school feeding in EVD affected areas ✔ ✔ * This activity was not funded under Pillar II but has been included in the desk review by the author. A similar activity, which picked up where GEMS left off and was implemented by Chemonics and IBI, was later funded by Pillar II with many of the same goals and components and the document review of GEMS activities may provide valuable insight into the types of activities funded under governance in Liberia. M–52 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES PROGRESS WITHIN THEMATIC AREAS Health PROBLEM STATEMENT At the start of the Ebola outbreak, Liberia’s healthcare system was still recovering from the devastations of the civil war but was on the path to “build back better” under the National Health and Social Welfare Policy 2011–2021 (GoL MoHSW, 2010). Liberia achieved its Millennium Development Goal 4 targets by 2012, showing improvements in indicators such as skilled birth attendance–from 40% (2007) to 61.6% by 2013 (GoL MoH, 2015, citing 2013 Demographic and Health Survey [DHS] data). The GoL health-sector appropriation more than tripled between Fiscal Year (FY) 2007/2008 and 2014/2015, from $18.7 million to $63 million (GoL MoH, 2015). This equates to 12.6%, which is short of the Abuja target (15%) but indicates a GoL commitment to health (MoFDP, 2015). The National Health Policy and Plan 2011–2021, which outlines the Essential Package of Health Services (EPHS) according to Liberia’s four-tiered system (Central, County, District and Community levels). However, healthcare access remains a challenge for large segments of the population. Liberia has very low health workforce density (per 1,000 population) compared to other African countries (total health workforce density=0.86 and 2.22, respectively; density for nursing and midwifery personnel=0.16 and 1.22, respectively), with health workers heavily concentrated in Monrovia (Jhpiego, ND). Ebola placed further demands on Liberia’s already weak pharmaceutical supply system. Outpatient attendance fell by 61%, and declines in ANC-1 visits, institutional deliveries and vaccinations were observed (GoL MoH, 2015). The outbreak also highlighted health information system (HIS) weaknesses. While the District Health Information System 2 (DHIS-2) was functioning in Liberia, there were multiple, parallel information systems (e.g., focused on logistics, human resources) (GoL MoH, 2015). The GoL undertook a health system review in 2015, culminating in its Investment Plan for Building a Resilient Health System, Liberia 2015–2021 (GoL MoH, 2015). The strategy focused on Table M–L4. Pillar II activities in the health sector, according to activity Interventions, Liberia Pillar II Activity CSH PACS HC3 MSCP/PST MSCP/RHS DELIVER APC ETP&SS Social Protection for EVD survivors and other Vulnerable Groups Addressing needs of EVD survivors ✔ ✔ Frontline Worker Support Health worker training ✔ ✔ ✔ ✔ ✔ ✔ MCP Leadership/management support ✔ ✔ ✔ Supply chain management ✔ ✔ ✔ ✔ ICT Health information systems ✔ ✔ ✔ SBCC Community engagement ✔ ✔ Other SBCC activities ✔ ✔ Institutional Enhancements WASH ✔ ✔ ✔ Facility infrastructure upgrades ✔ ✔ ANNEX M. DESK REVIEWS – LIBERIA n M–53 universal EPHS access, improved accountability and community participation, and enhanced health leadership and management (MoH, 2015). It also moved social welfare functions and the Essential Package of Social Services out of the MoH and into the MoGCSP. Community Health Volunteers (CHVs) and General Community Health Volunteers (gCHVs) became recognized parts of the health system. Two national bodies exist to address the needs of EVD survivors: (1) National Ebola Survivors Network of Liberia, which is a civil society organization (CSO) that involves elected representatives from each county; and (2) National Ebola Survivor Secretariat, a team of four technical specialists seconded to the MoH to coordinate care and support activities for EVD survivors (John Snow, Inc. [JSI], 2017). SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II USAID/Liberia worked closely with the MoH to mobilize its existing IPs, in alignment with the 2015 Investment Plan for Building a Resilient Health System, Liberia 2015–2021(GoL MoH, 2015). The main health activities funded with Pillar II resources were: (1) Management Sciences for Health (MSH)/ Collaborative Support for Health (CSH); (2) IRC/PACS; (3) JHU CCP/HC3; (4) Jhpiego/Human Resources for Health Maternal and Child Survival Program (MCSP)/HRH (MCSP/PST); (5) Jhpiego/MCSP/Restoration of Health Services (MCSP/RHS); (6) JSI/DELIVER; and (7) JSI APC ETP&SS. Smaller awards were made to entities such as UNICEF to restore routine immunization services, to MEASURE to assess and support planning for HIS strengthening, and to Chemonics for Procurement and Supply Chain Management. Public-private partnership efforts led to co￾funding with Chevron for a Center of Excellence for Infectious Disease Control (CEIDC) at the JFK Hospital in Monrovia; and with the Coca Cola Company and its Foundation, the Global Fund to Fight AIDS, Tuberculosis and Malaria and the Bill & Melinda Gates Foundation (BMGF)to fund the Private Sector Engagement: Partnership with Project Last Mile to build MoH capacity in pharmaceutical supply-chain management (USAID, ND). The principal interventions in the health sector are shown in Table L4. Addressing Needs of EVD Survivors A Complex Issue Requiring Multiple Types of Interventions: Available documentation on Pillar II activities suggests that addressing the needs of EVD survivors requires a mix of both supply-side and demand-side activities. Pillar II supported frontline worker support AND SBCC and MCP activities, to address EVD survivor needs. All USAID-supported activities that have included pre-service and in-service HCW training (e.g., CSH, PACS) and Ebola-related communications (e.g., HC3, CSML), have sought to combat Ebola-related stigma, and they addressed HCW responsibilities to provide quality health services to EVD survivors and their families without prejudice or discrimination. JSI’s APC ETP&SS activity began in October 2016 and was funded to review and strengthen specialized health services including mental health services, ophthalmology and rheumatology needed by some survivors, and to strengthen the engagement and participation of survivors in their healthcare. With respect to MCP, APC ETP&SS has provided technical assistance to the two national EVD survivor bodies, and it has funded organizational development, leadership training, and core administrative functions for the CSO EVD survivor network. Frontline Worker Support Pillar II resources supported pre- and in-service training of community volunteers, nurses, midwives, and laboratory personnel. Topics were based on district priorities and documented gaps. PACS built the capacity of gCHVs and CSOs in health communications and social mobilization on WASH and reproductive, maternal and newborn and child health (RMNCH) using various approaches (e.g., “self-driven performance improvement,” Natural Leader training, “Community-led total sanitation” (CLTS)). Other activities (e.g., Jhpiego’s MCSP/HRH activity) also supported HCW training and are making progress toward LOP targets. In addition to HCW training activities, USAID dedicated Pillar II resources to regional efforts such as mSTAR (further described in the ITP section) to adapt a successful mobile money salary payment system in the education sector for use in paying MoH employees. MCP MSH’s CSH activity, (2/27/15 – 2/27/19) builds on MSH’s previous Rebuilding Basic Health Services activity in Liberia, working with and embedding staff in the MoH to strengthen the building blocks of an effective health system at central and county levels. CSH’s leadership development program (LDP+) has strengthened County Health Boards and provided technical support to national regulatory and training bodies including the Liberian Board of Nurses and Midwives, the Liberia Medical and Dental Council and the Liberia Pharmacy Board. CSH has also provided technical support and management training to the Ministry of Public Works, which is responsible for WASH services. ICT Reporting of health data during the EVD outbreak was, in most cases, neither timely nor accurate (Fast and Waugaman, 2016). The Investment Plan called for increased investment in the HIS at central and subnational levels, and for integrating the parallel systems for human resource and supply chain management (GoL MoH, 2015). In 2016, MEASURE was awarded $500,000 in Pillar II resources to assess the HIS system and to develop a strategic plan to improve it. Although MEASURE Evaluation did not have previously funded project activities in Liberia, a key MEASURE Evaluation partner, JSI, did have pre-existing health recovery experience in the country, as well as relationships with MoH officials (Hart et al., 2017). The Pillar II-funded activity embedded two M&E experts in the MoH for two months, during which they conducted participatory assessments of both the HIS and the ICT infrastructure, at central and county levels M–54 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES (Hart et al., 2017). Formulation of the HIS strategic plan was followed by six months of virtual technical support. One noted implementation challenge was the availability of MoH personnel and other stakeholders during the time of crisis, which led to shifts in timelines, ultimately leading to a shortening of the duration that virtual technical assistance was provided (Hart et al., 2017). SBCC In addition to its pivotal role in controlling the EVD outbreak, community engagement is integral to health system resilience and ownership, and thus was a critical part of the Pillar II strategy for health system recovery. Pillar II activity approaches to rebuild linkages between and trust among communities and HFs, and to increase service use included: (1) mass media and community dialogues on RMNCH, (2) district and community health committees, (3) HF makeovers according to local priorities, including provision of basic equipment, and (4) strengthening the enabling environment for health promotion. JHU HC3 provided technical support and training to the MoH’s National Health Promotion Division and Community Health Services Division; developed distance-learning radio programs for gCHVs and listeners; and supported the national mass media campaign, Healthy Life, creating an MoH brand for health messages and services that met MoH standards. HC3 also implemented community engagement activities in Lofa, Nimba and Grand Bassa, in/around 60 of the 77 HFs supported by MCSP/RHS. Institutional Enhancements Pillar II resources contributed to the MoH’s agenda to renovate district and community HFs to meet the basic standards required for IPC and triage, and ensure both client and HCW safety. By September 2017, the MCSP/RHS activity had brought 74 of its 77 HFs up to MoH EPHS standards. In addition, the JHU HC3 activity supported light facility repairs/refreshing (no construction or provision of supplies and drugs). As an SBCC-focused activity, HC3’s approach to institutional enhancements involved both community and health facility engagement. A Makeover Committee (MC) was selected to provide oversight and the forum for all stakeholders to discuss challenges and issues they experience in relation to the HF, help the community and HF staff overcome misunderstandings and conflicts, and promote cooperation to reach their common health goals. JSI/DELIVER supported the MoH and National Drug Service (NDS) in the design, implementation and management of a revamped supply chain system for health commodities. Key interventions included: (1) technical assistance to the MoH to develop a “20-year supply chain master plan,” (2) capacity building in the NDS and Supply Chain Management units of the MoH, (3) development of a committee to harmonize measurement and coordinate procurement and supply plans, (4) development and implementation of a new “top-up distribution system” to improve accountability of drug deliveries, and (5) advocacy and policy dialogue to create a more-enabling environment for commodity security. Working in five focus counties that include 70% of Liberia’s population, DELIVER coordinated with the Global Fund for AIDS, TB and malaria to cover the other 10 counties, so that all 657 HFs in Liberia were included in the revamped supply chain system. Stockout rates in USAID-supported HFs declined from 93% (first round) to 5% (2016). Expiration of drugs declined from 7% to 2% (JSI, ND). OUTPUTS AND OUTCOMES Available documents on health activities that were fully or partially funded with Pillar II resources indicate that the activities were able to produce the majority of planned outputs (e.g., assessments completed, plans developed, HCWs trained, HFs strengthened). With respect to outcomes, efforts under Jhpiego’s RHS Pillar II activity have yielded noteworthy outcomes related to health service recovery (see Figure L2; Data Sources: Quarterly Reports (PY 2 (FY17), Quarters 3 and 4). At the end of the activity’s Performance Year (PY) 2, the overwhelming majority of supported HFs achieved standards in IPC readiness (78%) and delivery care readiness (using the MgSO4 availability, an essential drug for preventing/ treating eclampsia, as a proxy). The activity achieved its target of ensuring that 100% of supported HFs are open and providing basic PHC and emergency services, a noteworthy achievement over the baseline situation (in which only 57% of targeted HFs were open and providing such care). MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Within USAID, the Global Health Ebola Team (GHET), Food for Peace (FFP), and Democracy and Governance contributed to the Ebola Pillar II response. Other USG agencies including the Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) also have health activities in the country. There has also been extensive interest and contributions by international NGOs, such as the CHAI, the Figure M–L2. Jhpiego RHS: Three performance indicators from supported facilities at baseline, end-of-performance year (PY) 1, PY 2, and LOP target 46% 60% 57% 65% 41% 100% 78% 73% 100% 100% 100% 100% 0% 20% 40% 60% 80% 100% 120% % meeting 80% of IPC minimum standards % offering delivery services where MgSO4 is available in delivery room % open & providing basic primary health care and emergency services Baseline Actual: Performance Yr 1 Actual: Performance Yr 2 Life of Project Target ANNEX M. DESK REVIEWS – LIBERIA n M–55 Carter Center, faith-based organizations, and the Global Fund for HIV TB and Malaria. The principal mechanisms for maintaining coherence within USAID are the mission’s Country Development Cooperation Strategy (CDCS) and the universal commitment of USAID-funded entities to align with and contribute to the GoL’s Agenda for Transformation and the MoH’s Investment Plan 2015- 2021. Coordination among partners occurs in regular technical working group meeting. USAID/Liberia funded LSA/Social Impact to provide M&E and reporting support for the mission’s whole portfolio. In a mid-term evaluation of the CSH activity, evaluators noted that the “shifting financial and technical landscape” necessitates “continuous coordination with other partners, and frequent modification of the specific activities defined in the original SOW” (LSA, 2017:17). CSH proposed signing a memorandum of understanding (MOU) with the MoH to clarify roles and responsibilities, but USAID wanted an MOU for its “entire portfolio,” which was initiated in 2015 but never finalized (LSA, 2017). Basic Education PROBLEM STATEMENT After 14 years of civil war, Liberia had seen the pervasive impact that school closures can have on individuals and families, many of whom never “caught up” for the lost education time (USAID/ Liberia 2016a; Liu, 2017). This reality was exacerbated by the EVD outbreak. By government order, schools were closed between August 2014 and March 2015. A decrease in educational outcomes, and consequent impact on Liberia’s economic and social development, was identified as one of the seven main challenges facing the country in the aftermath of the epidemic (Navarro, 2014). Thus, there was a strong motivation to get schooling back on track, both at the primary and higher education levels. SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II Pillar II interventions sought to restore and boost educational outcomes by refurbishing and resupplying schools, providing primary school learners with school supplies, improving schools’ water and hygiene facilities to meet WASH standards, and improving the system for paying teachers’ salaries through an e-payment system (GoL, 2015b). USAID Pillar II funds were allocated to UNICEF for the Education Crisis Response in Liberia (ECRL) activity, which supported the Ministry of Education (MoE) to create safe, protective and proactive learning environments for schools across Liberia. The activity involves three primary interventions: (1) Procurement and distribution of TLM to 4,460 schools; (2) Training of 10,000 teachers and 5,000 PTA members in pedagogy and PSS; and (3) IPC through WASH in Schools (WinS) interventions (UNICEF Liberia, 2016). OUTPUTS AND OUTCOMES In mid-2017, of 14 indicators in the activity’s M&E plan, three were met; nine were reported on track, and two were delayed. Over the course of the activity, UNICEF delivered Teaching and Learning Materials to 4,460 schools across the country. The activity reached 517,249 primary school children (5% over its target) with “school in a box” “kits” delivered to their schools. Since 2015, 10,064 teachers have been trained in “psychological first aid,” preparing them to handle children affected by EVD and other traumas, including Gender-based Violence (GBV), and a Teachers’ Code of Conduct (UNICEF Liberia, 2017). Agriculture and Food Security (AFS) PROBLEM STATEMENT The vast majority of Liberians, around 70%, depend on agriculture for their livelihoods. However, Liberia is considered an import￾dependent country, with much of the food supply, including staples such as rice, being imported (World Bank, 2017). Liberia’s Ebola outbreak negatively impacted agricultural production. The peak months of the epidemic (August–October 2014) coincided with the country’s harvest period (Murphy et al., 2016). The most affected products were imported rice, palm oil, cocoa, bush meat, minerals, and rubber. Northwest Liberia – the main food￾producing area of the country – was adversely affected. (FEWS Net, 2017a). Road maintenance, crucial during the six-month rainy season (May – November), had collapsed, leaving whole communities cut off from markets and other resources. Quarantine efforts that closed international borders and limited internal movement were linked to substantial food price increases. Rural, agriculture-dominant areas were not the only affected areas. In urban slum areas, the Ebola outbreak was linked to secondary crises such as increased food prices, reduced food consumption, reduced livelihoods and social unrest (Y Care International, 2016). By the first quarter of 2015, market functioning began to improve, albeit slowly, with most bans and quarantines lifted. However, counties with the highest levels of food insecurity prior to the 2014-15 Ebola outbreak experienced deteriorating food-security conditions in 2015 (Murphy et al., 2016). M–56 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II FFP partnered with ACDI/VOCA, Mercy Corps, PCI, Save the Children (STC), WFP and UNICEF to address AFS. Social protection was a prominent intervention type. Table L5 summarizes the activities and their geographical focus and strategies. Interventions focused on EVD-affected and other vulnerable HHs (e.g., child-headed, female-headed, high￾dependency ratios, caring for OVC). WFP distributed food to refugees from Côte D’Ivoire in three refugee camps in Nimba, Grand Geddeh and Maryland counties. 1. This point was stressed in several group meetings held with Food for Peace partners in Liberia during the MEL Project Situation Analysis visit. Social Protection through Cash Transfers (CTs) USAID established a Cash Transfer Working Group (CTWG) that met regularly in early 2016. IPs cited it as a useful forum for coordination and for sharing lessons learned.1 Four Pillar II AFS activities implemented unconditional cash transfers (CTs) in priority districts (based on EVD burden, poverty, infant and child mortality, malnutrition among children, and food insecurity) of USAID’s priority counties. Table 6 summarizes the delivery mechanism, service provider, amount per transfer, number of households targeted, and the activity county for the IPs implementing CTs. Three activities attempted to distribute cash through mobile money solutions. While showing great potential, these efforts were hampered by the tendency of CT agents to be Table M–L5. Pillar II interventions and implementers in the food security sector, Liberia Intervention Strategies, Liberia Implementing Partner and Target Counties ACDI/VOCA (Bong, Nimba) Mercy Corps (Lofa, Margibi, Montserrado) PCI (Bomi, Grand Cape Mount) STC (Margibi, Bong) WFP* UNICEF In-kind food assistance ✔ ✔ Unconditional Cash Transfers ✔ ✔ ✔ ✔ Agricultural inputs ✔ ✔ ✔ ✔ Cash for Work ✔ ✔ Training ✔ ✔ ✔ ✔ Food Supplements ✔ ✔ Market Analysis ✔ * World Food Programme areas included Bomi, Gbarpolu, Grand Bassa, River Cess, Grand Geddeh, Sinoe, River Gee, Grand Kru, Maryland, Nimba counties. Table M–L6. Unconditional cash transfer programs and distribution type, number of households and counties, by implementer Implementing Partner Mode of distribution or delivery Service Provider UCT Amount No of HHs Counties Direct Cash E-Transfer ACDI/VOCA ✔(92%) ✔(8%) Ecobank, Lonestar, MTN $50 9,895 Bong, Nimba Mercy Corps ✔(98%) ✔(2%) Ecobank, Cellcom, Lonestar $42 30,077 Lofa, Margibi, Montserrado PCI ✔ n/a $50 6,918 Bomi, Grand Cape Mount Save the Children ✔ Lonestar, MTN $50 5,000 Bong, Margibi Adapted from Table 3 of Cash Transfers for Food Security in Epidemics: A review of FFP response to the Ebola Crisis in Liberia and Sierra Leone (Radice, 2017) ANNEX M. DESK REVIEWS – LIBERIA n M–57 located in urban areas, and their limited liquidity (CaLP, 2016; Mercy Corps, 2016, STC, ND). Overcoming these constraints required considerable time and diplomacy on the part of USAID/ Liberia and the IPs. Save the Children produced a written case study based on its experiences in Bong and Margibi, highlighting the link between the health of the banking sector and the feasibility of administering CTs through a mobile money platform (STC, ND). While a significant level of effort was required to manage and coordinate mobile money agents, as well as facilitate logistics of cashouts, the activity also highlighted the added value of mobile money for mobile network operators. It documented that 18% of beneficiaries who did not own mobile handsets at the start of the program purchased handsets over the life of the program. This equates to over 700 new customers for Lonestar, the activity’s mobile network operator. Beneficiaries directly affected by Ebola were readily identified; they were essentially Ebola survivors, orphans, or those that had lost the head of households to Ebola. The MoH issued Ebola survivor certificates when people were discharged from HFs, which helped to identify EVD survivors. Many communities also autonomously created “survivor associations.” In addition, each activity worked with local leaders and EVD survivors to reach consensus on HHs most in need of support. This was viewed as a critical step in ensuring community buy-in, and in promoting the security of implementers and recipients (Radice, 2017; PCI, 2016). Social Protection through Agricultural Inputs The following agricultural inputs, aiming to boost immediate and long-term food production and HH income were provided: (1) Seed vouchers, (2) Agronomy training, (3) Post-harvest management training, and (4) Logistical support to MoAg. Seed vouchers enabled selected farming HHs to replenish seed stores that had been consumed to offset hunger during the crisis and/ or to improve seed quality. They also included seeds for additional crops such as maize, okra, eggplant, hot pepper, cucumber and bitter ball, which had the potential both to increase the Household Dietary Diversity Score (HDDS) and to be sold at market. The seeds provided were tested by the IP and the MoAg to ensure viability. This proved a crucial step for rebuilding trust in government services. In addition, technical assistance accompanied the seed kits, in the form of training, demonstrations, and training of “Lead Farmers” who provided follow-on, post-distribution support and interacted more effectively with MoAg extension agents. Since baseline assessments had shown a very high level of post-harvest crop loss, the agricultural inputs included training in improved crop storage and management. Techniques ranging from raised vegetable beds to post-harvest drying techniques were introduced, according to observed needs (STC, 2016). Social Protection through Food Supplements FFP supported UNICEF to resume screening, referral, and treatment for children with severe acute malnutrition (SAM) (services were suspended during the height of the Ebola crisis) and provided UNICEF with in-kind, ready-to-use therapeutic food for SAM treatment. UNICEF trained CHWs on use of national protocols for Integrated Management of Acute Malnutrition (IMAM). They were able to treat 3,349, and to release 96% of them cured (UNICEF/Liberia, 2016). MCP At the outset of the crisis, FFP immediately leveraged its existing technical capacity within FEWS NET to make timely and informed decisions for food assistance response (FEWS NET, 2017c). The activity also commissioned special studies that provide valuable overviews of the impact of the EVD crisis on food security and on livelihoods in Liberia (FEWS NET, 2017a and 2017b). SBCC Training and demonstrations were supported by various implementers to provide education and hands-on learning to improve the nutrition status of the target HHs and guide decision making on the use of the cash. Generally, training participants were the same as the participants in CTs and seed distribution. They focused on the value of dietary diversity, and on the essentials of infant and child nutrition. OUTPUTS AND OUTCOMES All programs reported outputs that achieved the majority of their targets, although there were delays in start-up and, at times, in implementation. The combination of interventions – unconditional CTs and agricultural inputs, with or without additional elements of Cash for Work (CfW), nutrition, health and budget training, market monitoring, and food supplementation, appear to have achieved their objectives. Levels of food insecurity in the country were low to moderate before EVD, and the EVD crisis, which devastated the national economy and government systems, did not lead to widespread serious household hunger or nutritional deficiencies (FEWS NET, 2017a). National levels of chronic malnutrition remained “relatively stable at 12 percent severely stunted and 19 percent moderately stunted” (FEWS NET, 2017c: 7). Activity-specific data reveal noteworthy outcomes in catchment areas. STC’s activity documented an increase in average HDDS from 4.5 in November 2015 to 5.8 by the end of Fiscal Year 2016 (STC, 2016). Between baseline and endline, EREL reported that the average HDDS increased from 4.7 to 8.6 (Mercy Corps, 2017), and PEER reported an increase from 3.8 to 5.6 (PCI, 2016). There were also substantial reductions in the percentage of HHs experiencing moderate or severe hunger since baseline (e.g., for EREL, from 58% to 8%; for SC, from 62% to 30%). Figure L3 provides data from selected Pillar II AFS activities demonstrating HHDS improvements between baseline and endline. M–58 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Activities also documented improvements in reducing the percentage of households experiencing moderate or severe hunger (see Figure L4). STC – the only IP to report their outcomes broken down by the gender of the HH head – found the greatest food security progress was observed in HHs with both a male and female adult present, and the least was in female-headed HHs with no adult male (SC, 2016). MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Some of the applications for these activities took several months to be approved and required some iteration on plans to align them with USAID/FFP’s objectives and meet their quality standards. These delays resulted in activities starting later than initially envisioned. Some activities’ CTs first reached beneficiaries in June 2015, after the peak of the EVD crisis, while others were delayed until early 2016. Although the intention was to cover food insecurity during the lean season, many of the early transfers did not reach beneficiaries until after that most challenging time (Radice, 2017). CTs delivered through mobile money solutions have the potential to enable the GoL to adjust cash amounts in line with market prices, so that their purchasing power remains constant. Reaching the rural poor, who were the majority targeted by Pillar II, requires more widely available communication networks, disbursement agents, and liquidity outside county capitals. The IPs had to negotiate these crucial details with two of the country’s banks, and to come up with creative solutions, including increasing incentives to cashout agents which raised the cost of the services per beneficiary. Undoubtedly one of the most impressive aspects of the CT component was the coordination between NGOs and government. This was driven by USAID/FFP, which made it a requirement of their partners. Coordination mechanisms guided CT Pillar II implementers, and the CTWG was the primary platform for sharing experience, skills and information. This helped implementers to coordinate and facilitate beneficiary selection, cash disbursement process and messaging (Radice, 2017). Governance PROBLEM STATEMENT Before the EVD outbreak, the GoL was progressing in strengthening public sector management, accountability and transparency, and it was implementing its 2010 National Capacity Development Strategy (IBI International, 2016). However, those efforts did not sufficiently buttress the public sector to withstand the stress of the 2014-15 EVD outbreak. During the outbreak, public services were interrupted as the central government was either focused on EVD control or virtually shut down. However, out of that negative shock came a renewed will to strengthen governance functions. USAID RESPONSE USAID’s strategy involved a two-pronged approach: (1) Strengthening governance systems within the GoL and strengthening CSOs (including strengthening women’s and media organizations) to increase civic participation in local and national governance, and 2) To increase the responsiveness, transparency, and accountability of the government to the people. 0 2 4 6 8 10 Mercy Corps EREL PCI PEER STC 5.8 4.5 5.6 3.9 8.6 4.7 BASELINE ENDLINE Figure M–L3. Changes in HDDS between baseline and endline by three Pillar II AFS activities 0 10 20 30 40 50 60 70 80 Mercy Corps EREL PCI PEER STC 8.6 58% BASELINE ENDLINE 8% 76% 62% 52% 30% Figure M–L4. Changes in the percent of HHs experiencing moderate or severe hunger between baseline and endline by three Pillar II activities *Sources for Fig. 3/4: (1) Mercy Corps EREL Final Program Report; PCI PEER Final Results Report and Final Evaluation Report of PEER Activity; and STC FY 2016 Annual Results Report ANNEX M. DESK REVIEWS – LIBERIA n M–59 MCP USAID’s pre-existing portfolio of Liberia activities such as its Governance and Economic Management Support (GEMS) activity were built upon to support Pillar II. An activity with Chemonics and IBI International was funded by Pillar II and seeks to provide eGovernance programming to address specific weaknesses to effectively prevent, detect, and respond to future outbreaks. The scale and breadth of the GEMS activity’s contributions to strengthening the public sector in Liberia contributed to overall recovery and institution strengthening, despite not being under the Pillar II funding stream. GEMS activities addressed issues such as M&E, financial and economic management, human resource management and ICT. GEMS used a tailored version of the USAID Human and Institutional Capacity Development framework and implemented a four-phase process to strengthen services in 17 GoL institutions: (1) Assess and identify gaps and establish baselines; (2) Develop plans to fill gaps; (3) Implement planned solutions; and (4) Sustain improvements in behavior and performance (IBI International, 2014). SBCC To strengthen civil society demand for and participation in good governance in Ebola recovery, USAID funded a costed extension to the Civil Society and Media Leadership (CSML) Search for Common Ground (SCG) activity, implemented by IREX. The CSML costed extension, “Recovery Together,” combatted stigmatization of EVD survivors and their families nationally, and through partner CSOs in six counties: Margibi, Bong, Lofa, Nimba, Grand Cape Mount, and Montserrado. A total of 953 people attended 18 CSML-facilitated forums to express community concerns. The most commonly raised concerns in these forums included care of Ebola orphans, poor school quality, and the need for memorialization of people lost to EVD. Based on this input, CSML/SCG trained the project managers of their partner CSOs, along with 21 station managers of community radio groups, to discuss reconciliation and memorialization of people lost to EVD, and develop memorialization activities, to help heal the psychosocial scars that had been largely neglected by health￾focused EVD recovery activities (IREX, 2016b). CSML also trained 99 journalists from Monrovia, and community radios in the 15 counties, on: “How to report on the impact of Ebola funds and resources earmarked for Ebola” (IREX, 2016b). The activity created an online platform, www.trackingaidliberia. org, with an interactive database of EVD funds. In addition, 20 field researchers in the 15 counties and 19 Monrovia reporters filed investigative reports on Ebola funding on YouTube and Facebook pages, including reports on the ETUs that were never used. To extend its impact, CSML built on its Community Leaders Forums approach, with cascade training through 13 partners, reaching 688 community leaders in 41 communities in 11 counties. This involved recasting the questioning of traditional and government leaders, from disrespecting or undermining them, to a source of information that officials could use to improve services. Unfortunately, these efforts did not move the needle on trust in government in the expected direction. The percentage of the population who responds positively to attitudinal surveys regarding perceptions of government declined from 35% at baseline to 21% at endline (IREX, 2016). OUTPUTS AND OUTCOMES The pre-Pillar II activity GEMS focused on departments that could roll out learning and systems improvements to other Ministries, Agencies and Commissions (MACs), namely, the Civil Service Agency, the General Services Agency (GSA) and the Liberia Institute for Public Administration (LIPA). It guided development of policies, manuals, guidelines and training programs that can increase the clarity and efficiency and consistency of practices long after the activity is over – if they are used. For example, for the GSA, it delivered 30 documents and tools, ranging from Building Management Guidelines to an Institutional Fleet Register spreadsheet to keep track of all GoL vehicles. The percent of MACs with updated Asset Registers increased from 0 (2012) to 88.9% in December 2015 (IBI International, 2016:57). Of the GEMS activity’s 33 indicators, 90% were exceeded, achieved or substantially achieved. Of 27 indicators reported by CSML, 12 were exceeded, 10 met, 2 substantially met, and 3 not met. Both of these activities were regarded as very successful programs. However, there is a larger question regarding the extent to which there is sustainability of the improvements achieved, now that the activity’s embedded staff and added resources are no longer available. The GEMS activity final report indicated serious concerns that the gains would be lost without continued support and investment (IBI International, 2016). However, further review of documentation of the follow-on activity with Chemonics and IBI may provide additional information on the sustainability of governance gains. Economic Crisis Mitigation PROBLEM STATEMENT After almost a decade of progress and growth, the EVD crisis led to a dramatic scaleback in trade and both investments in Liberia. Weakness of the GoL’s financial management capacity had a potentiating effect on other sectors, as HCWs and teachers abandoned their posts for lack of pay. In the Agenda for Transformation, the GoL had a strong framework for mobilizing and coordinating domestic and international partners to recover from the crisis and to “build back better” (GoL, 2015). The World Bank and GoL also developed the Economic Recovery and Stabilization Plan (ERSP), which is aligned with the Agenda for Transformation (GoL, 2014). M–60 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Strengthening resilience and reducing vulnerability is one of the ERSP’s objectives, calling for government activities in four key sectors: (1) Health and Social Welfare, to increase access to, and utilization of, quality health and social welfare services and offer a comprehensive package of interventions of proven effectiveness; (2) Education, to ensure equitable access to free basic education for all children; (3) Social Protection, to support the poorest and most vulnerable in the recovery from the Ebola crisis and to improve resilience through future shocks; and (4) Security and Rule of Law, to maintain the current security situation, ensure their sustainability and to increase the accountability and legitimacy of national security institutions and the public’s confidence in them. USAID RESPONSE In Liberia, Pillar II resources support advocacy, technical support, and policy efforts to restore and improve the enabling environment (see sustainability section for a discussion of the enabling environment as a sustainability determinant for Pillar II activities) for investment and trade, and to create a social safety net for the most vulnerable. Please refer to the section on the AFS thematic area, which describes social protection measures undertaken by Pillar II AFS activities, as well as their outputs and outcomes. Documentation on other Pillar II ECM activities were not made available to the ET. However, according to activity mapping, non￾AFS ECM activities included ICT, institutional enhancement and MCP interventions. For example, the World Bank received Pillar II support to conduct a social protection impact evaluation and support a database for government assistance. Innovation, Technology and Partnerships PROBLEM STATEMENT While internet access in Liberia is extremely poor and only 20% of Liberians have a bank account, mobile phones are a part of daily existence. Ninety percent of the country has mobile coverage, and there are 1.3 million unique cellphone subscribers (FHI 360, 2016). Deficits in communication between individuals and private and government service providers at the central, county and district levels played a key role in the explosion of EVD into a national crisis in 2014 and 2015. The deficits included limitations in the physical infrastructure (telephone, cell phone and internet systems, electrical power), weaknesses in information systems required for sharing and reporting crucial information; and the prevalence of communication norms and expectations that fostered fragmentation rather than integrated systems in which community, district and county teams are linked to central/national levels and see themselves as parts of a whole. Both the GoL’s Agenda for Transformation and the MoH’s Investment Plan recognize the need for expansion of the ICT sector. The two agendas also recognize the vital role of the private sector as a source of financial and technical resources (GoL, 2015; GoL MoH, 2015). USAID RESPONSE NetHope received Pillar II funds for the following: (1) Technical experts providing direct support to GoL and USAID to implement ICT infrastructure and policy improvements; (2) Program to enable GoL ICT capacity to better respond to future outbreaks; and (3) Support for Liberian telecommunications regulator to deploy equipment to address connectivity breakdowns (with FCC). W3C also received funding to conduct a workshop on ICT in Liberia to develop PPP policy recommendations for responding to future outbreaks, and to provide technical assistance to GoL MoPT to modernize the National ICT and Telecom Policy to strengthen communications systems and networks. USAID did not provide the ET with activity documentation on either of the above activities. Inveneo’s Ebola Recovery Connectivity Initiative (ERCI 2.0) will prioritize connecting institutions such as HFs with broadband internet. A consultative, participatory approach will be used to select rural sites for internet connection (Inveneo, 2017). Other ICT activities include mHero, implemented by IntraHealth International, and mSTAR (Mobile Solutions Technical assistance and Research Program), implemented by FHI360. USAID/Liberia looks to both mHero and mSTAR for their “potential to increase financial inclusion and empowerment (especially among women and vulnerable populations) reduce time away from employment required to collect salaries, increase transparency, reduce corruption and ghost workers, and address cost-efficiency and safety challenges faced in transporting cash to rural locations” (USAID Global Development Lab, 2015). Pillar II funded 14 innovations through the regionally focused Fighting Ebola Grand Challenge initiative out of the Center for Innovation and Impact in the Global Health Bureau (CII). This business-minded approach has helped CII to spark new solutions to serious global health challenges from malaria to HIV to maternal, newborn, and child health. Including this challenge, CII has leveraged over $300 million in outside capital to develop and test 150 innovations in 35 countries, of which 25 were scaling or transitioning to scale in 2018. These challenges jumpstart innovative product developments by providing seed funding for research, development and testing. The Fighting Ebola challenge was issued via a Broad Agency Announcement (BAA) mechanism to innovators to “develop new practical and cost-effective solutions to improve infection treatment and control that can be rapidly deployed (1) to help healthcare workers provide better care and (2) to transform our ability to combat Ebola” (USAID, 2014). The original challenge received 1500 ideas. With input from the White House Office of Science and Technology Policy (OSTP), the DOD, the CDC, OFDA, Offices in the Bureau of Global Health, ANNEX M. DESK REVIEWS – LIBERIA n M–61 Table M–L7. Fighting Ebola “Grand Challenge” innovations Innovator Product Award (nearest 000) Duration Objective met? Comments Johns Hopkins Univ. Improved Personal Protective Equipment 794 6/15-6/17 Yes Improves visibility of and for HCW; Improved heat management; Simplified doffing (reduced from 8 to 4 pieces); improving HCW safety and confidence. IntraHealth Int. Scale up mHero, mobile phone-basd platform for MOH – HCW communication about EVD/other health issues 700 6/15-9/16 Yes Refined the architecture and user support for mHero, to strengthen the health systems in all 3 countries. Enabled mHero to go to national scale in Liberia, and mHero has been integrated into the MoH HMIS in Liberia and Guinea. SPR Advanced Tech Inc. Improved antiviral protective coating 656 5/15-8/16 No Innovative approach to longer lasting protection, but additional R&D is required to resolve technical shortfalls. Columbia University – Kinnos “Highlight” powdered color additive to improve EVD decontamination w. bleach 649 6/15-7/16 Yes Enables “visualization” of decontamination, improving HCW safety and confidence. Field tested in all 3 countries; now widely in use. Scripps Health “STAMP” – wearable sensor and personalized analytics￾monitor patient (Px) vitals 632 5/15-5/16 Yes Field tested by IMC in Sierra Leone. Potential for multiple uses; monitoring heat stress in HCW . Baylor College of Med. Develop light weight, re-usable, transportable Emergency Smart Pod 614 6/15-6/17 Yes Collapsible, movable ETU requiring only 30 minutes training to set up, and with estimated production cost of $50K-$150K, and “shelf life” of 10-15 years. TOMI Environmental Solutions SteraMistTM Mobile Decontamination chambers 559 5/15-2/16 Yes SteraMist is FDA approved; Expanding testing to Philippines, Malaysia, Mexico, and Panama; SteraMist will be pre-positioned in the US for future outbreaks. SteraMist is also EPA and FDA registered, included in the WHO Compendium of Innovative Health Technologies for Low Income Countries, and distributed worldwide. IBM Research Africa EPIC platform & analytics for front line HCW decision￾making 527 7/15-3/17 Yes EPIC gathers and integrates data from stove￾piped information sources (DHIS-2, IP data sources, etc.) to provide both health and context data relevant to district level health services decision-making. Successful test in one district in Sierra Leone. Modula S Modula-S Rapidly Deployable Emergency Medical Treatment Unit - Antimicrobial Copper CuVerro Materials 500 5/15- 12/15 Yes Hard-shell popup ETU building, autonomous & solar self-powered with direct contact kill antimicrobial CuVerro copper interior surfaces. Wall mounted beds, and solid floor surface pitched to drains to facilitate easy and thorough cleaning. Third generation design, now available, one third the shipping size and ISO container building modules start at $500,000. Makerere University “Epi-tent” – a next generation hospital tent with improved air flow and heat exchange 482 6/15-4/16 Yes Production in Uganda; In use in refugee settlements; Included in WHO Compendium of Innovative Health Technologies for Low Income Countries. 4 design patents filed, new housing use case design optimization. M–62 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES the Global Development Lab, the Africa Bureau, and experts from USAID missions, which informed the team about current conditions in West Africa, the CII winnowed the 1500 to 30 semi￾finalists and finally 14 winners (See Table 10). Twelve of the 14 innovations have been field-tested in West Africa. Liberia, where the burden of disease was highest, was often targeted for field applications of the Grand Challenge innovations. These included (Table L7): ■ In November 2015, TOMI™ Environmental Solutions field￾tested its decontamination chambers and bleach alternative in collaboration with PCI. ■ Columbia University tested its colorized bleach solution, Highlight, in Liberia and Guinea with PCI and IMC, respectively. ■ JHU field-tested its redesigned PPE suit in Liberia in partnership with Jhpiego. In addition, Baylor’s emergency pod, Shift Lab’s Drip Assist and Dimagi’s CommCare platform were both tested in Liberia. mHero’s communication platform was also implemented. mHero allows two-way communication using cellphones to gather data from healthcare workers, and to provide them with information, reminders, and other communications. IntraHealth’s award is one of the 14 Grand Challenge Awards made by USAID to spur innovation in response to the EVD crisis. The award included: (1) Support for global level investment in the mHero website to improve users’ access to content; (2) In collaboration with the MoH in Liberia, the design of “workflows” on family planning (FP) to enable the MoH to gather FP/RMNCH information from health workers, to aid in planning activities to increase contraceptive use; and (3) A baseline assessment phone survey with 335 health workers, which covered mobile phone use, “market saturation with health-related text messages” and exposure to mHero. The activity embedded two full-time staff in the MoH to support development of the Human Resource Information System (HRIS). This entailed “cleaning” MoH personnel records, which had lapsed during the EVD crisis. The activity also developed eight “use cases” (reaching 1,500 recipients), which addressed HCW validation (to confirm or update personnel records); ICT electrical assessment at HFs; ICT infrastructure at HFs; mental health services and beneficiaries; nutrition/anthropometric tools in HFs; HCW status in prisons; IPC and other stock levels in prison HFs; and identification of focal persons on County Health Teams for warehouse management (IntraHealth, 2015). The mSTAR activity was already operating in Liberia’s education sector. USAID Pillar II resources were used to plan and pilot use of the system to pay MoH employees. Experiences with mobile money in the AFS CT activities have shown that quite a number of obstacles must be overcome before ePayments work smoothly for vulnerable households in rural areas. For example, most end￾users have to convert their ePayments into cash, (use of credit cards is virtually non-existent) which must be sourced from banks. However, 51of the 87 banks in Liberia are in Montserrado County (Frontier Research U and UNDP, 2016). Half of the counties in Liberia do not have a commercial bank branch (FHI 360, 2016). OUTPUTS AND OUTCOMES As shown in Figures L5 and L6, over the life of Pillar II funding, the activity is expected to provide mobile phone salary payments to a target of 3,600 MoH employees across nine counties. In addition to ICT-based activities, Pillar II resources were also given to private sector entities. For example, Pillar II support to Tetra Tech, a private sector company, to resume training and infrastructure investments to rehabilitate the piped water system in up to four county capitals: Voinjama, Robertsport, and Saniquellie. The ET received limited activity documentation on Tetra Tech’s Liberia Municipal Water Program (LMWP). The LMWP II Combined Year 1 and Quarterly Report 4 (dated October 30, Table M–L7. Fighting Ebola “Grand Challenge” innovations International Personal Protection Test 3 concepts/designs to improve PPE 324 5/15-8/16 Yes Participated in expert group; tested 3 models and identified preferred. Overall cost still too high ($85/unit). Shift Labs “Drip Assist” precision infusion monitor 319 5/15-6/16 Yes In collaboration with Z/Map, Drip Assist scaled throughout West Africa. In use in 18 developing countries Dimagi Inc. Adapt CommCare platform for EVD surveillance and buid capacity 299 5/15-5/16 Yes Developed 7 templates based on CommCare, Dimagi’s open-source mobile platform that is in use in 50 countries. It allows tracking of individuals over time, so it suited the need to track symptoms of suspected EVD cases for 21 days. 3D Family Productions Africa Stop Ebola campaign in Guinea 269 5/15-8/15 Yes Implementing partner – MSF. ANNEX M. DESK REVIEWS – LIBERIA n M–63 2017) provided to the ET presents indicator targets but no data on actual LMWP II Year 1 achievements vis-à-vis performance indicators. Nevertheless, based on reported Year 1 progress, LMWP is making progress in relation to sustainable financing. The activity generally exceeded its Year 1 targets related to the non￾subsidized WASH infrastructure operating expenses. For example, the reported percentage of operating expenses covered by customer charges without external subsidies was 51% at the end of PY 1, exceeding the PY1 target of 47%. In addition, there have been a number of process achievements (e.g., Institutional Framework Review and development and finalization of a Roadmap Plan) related to sustainable management and cost recovery. In addition, the number of action plans implemented for water security, integrated water resource management, and/or water source protection as a result of USG assistance increased from 0 at baseline to 6 by the end of Year 1 (Tetra Tech, 2017). 0 2 4 6 8 10 FY17 Q1 0 9 4 22 FY17 Q2 FY17 Q3 FY17 Cumulative Target Figure M–L5. Number of counties where mSTAR is operational for MoH employees 0 500 1000 1500 2000 2500 3000 3500 4000 FY17 Q1 0 3600 FY17 Q2 FY17 Q3 FY17 Cumulative Target 0 119 119 Figure M–L6. Number of MoH employees enrolled in Mobile Money Salary Payments, mSTAR M–64 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SUMMARY OF KEY FINDINGS IN FOUR DOMAINS Performance USAID Pillar II activities were of short duration (often 30 months or shorter), and the actual implementation period was even shorter due to the needed time to work with government and communities on introduction and start-up activities. Outcome data are limited, but the activities for which output data are available have achieved all or most of their intended outputs. Key lessons learned are as follows: 1. No one intervention can stand alone. AFS and health interventions mutually supported each other. For example, CTs allowed beneficiaries to buy products from traders who could, in turn, hire labor. This created income for the laborers, who could then purchase food and other essentials in the market. Assistance to market traders helped ensure that there were products to purchase. The cash received through CTs provided resources to buy a greater diversity of food, hence boosting nutrition, and hygiene commodities such as soap. In the health sector, it was obvious that multiple aspects of service delivery needed to be addressed (e.g., HCWs, water, electricity, equipment, supplies and drugs). On the demand side, there was a need for more information on RMNCH, as well a need to re-engage community members and promote accountability and trust. Individuals will be more willing to take time away from their farms to seek health services if they have confidence in the availability and quality of services. Improvements in the HIS and other information systems will improve epidemiological forecasting, allocation of staff and budget, and increase accountability at the health facility, district, county and central levels. 2. The array of activities funded with Pillar II resources were on target to address the main challenges of EVD recovery, but they were constrained by the short timeframe of the Pillar II funding rules. For example, CTs in the AFS thematic area targeted the most important cause of livelihood threat, which was the lack of available cash due to job losses across the board (not just among farmers or due directly to EVD deaths), rather than decreased food availability. While there have been strides in food security, relaunching and expanding job opportunities is a longer-term endeavor. 3. Financial and social safety nets are invaluable. Short-term infusions of cash, seeds, equipment, drugs, and infrastructure are necessary to restart and rebuild livelihoods and health services, but resilience and sustainable change comes through creating safety nets. Cash and training to improve agricultural production and increase the profitability of small business practices contribute to financial safety nets as increased sales and profits produce savings, and a small financial cushion. Other important financial safety nets are the village savings, loan associations and seed banks that were re-capitalized and provide a source of credit to smooth out ups and downs in income. These same financial groups also build social capital and social safety nets. Other kinds of social networks and support are being built through the community health committees, Parent-Teacher Associations (PTAs), beneficiary selection committees, Lead Farmers groups, and Community Leaders Forums and EVD survivor associations that were supported by Pillar II activities. Having such structures in place could potentially contribute to individuals and communities being able to more quickly adapt to another crisis. 4. Building human capacity is a powerful investment. Pillar II activities enhanced human capacity, boosting management capacity in GoL agencies, training HCWs, supporting community- and district-level forums for health and education ministries to listen to the people, and training women and girls to actively contribute to such discussions. They conveyed new ways of managing agricultural outputs and supported new business practices among traders, among others. These interventions also fostered a sense of optimism and self-efficacy, which can contribute to resilience and endure beyond Pillar II inputs. Resilience AFS activities successfully bolstered the absorptive and adaptive capacity of people, households and communities where they worked. The interventions addressed the poor welfare and health instigated by EVD, and the related loss of agricultural productivity and markets by reducing hunger. They also restored services and rebuilt systems – helping communities to adapt – by restocking and revitalizing markets, linking people with sources of financial and social support, and providing the needed inputs to restart agricultural activity. The training provided information and new capacities to plan for stresses with financial plans and better understanding of nutrition and how diets can be diversified. Health sector interventions were heavily concentrated in building adaptive capacity by building up multiple components of the health system and restoring basic health services. Pillar II interventions also supported small steps toward transformative change, most notably in introducing or revitalizing frameworks and groups for communities to participate in decision making and hold the government accountable, and in making provisions for women’s inclusion as beneficiaries and activity implementers. While the acknowledgment that Ebola disproportionally affected women is a significant step, several activities went much further. Examples of this include the extensive community consultation ANNEX M. DESK REVIEWS – LIBERIA n M–65 on who should receive CTs and what form they should take, and the facility management committees that help set priorities for the health facility and monitor the performance of the unit. Activities strived for gender balance in management committees, ensuring that service delivery plans took women’s workload and schedules into account. The CSML activity reported success in bringing more women into journalism, including women “in front of the camera,” but they were disappointed in the lack of change in the coverage of gender issues and the uptake of gender equity as a goal. Sustainability The sustainability framework developed by the ET for the overall Ebola Pillar II evaluation outlines key sustainability determinants: (1) Policy, (2) Finances (3) Accountability, (4) Institutional Support, (5) Technical Choices, (6) Human Dimension, and (7) External Threats and Enabling Environments (see Box 1). 1. Policy—Liberia has a relatively rich policy landscape, with a plethora of policies, strategies, and plans contributing to an enabling environment for sustainability. Liberia’s Pillar II activities and others supported national policies and/or worked with GoL to develop them where they either did not previously exist or needed revision (e.g., the HIS-related efforts of MEASURE Evaluation, mHero and GEMS activities; JSI’s APC ETP&SS activity’s contributions to the development of the National Ebola Survivors Care and Support Policy; JSI/Deliver’s advocacy and policy dialogue related to commodity security). 2. Finances—As described earlier, some Pillar II activities are promoting sustainable financing within their thematic areas. For example, Tetra Tech’s current WASH activity, LMWP II, has introduced cost-recovery measures to cover operating expenses and maintenance. There is also evidence of Pillar II IPs supporting GoL counterparts to address financing in the health sector (e.g., MSH’s CSH activity). 3. Accountability—IREX’s CSML program illustrates the deliberate measures taken under the auspices of Pillar II to promote accountability. As shown in Figures L7 and L8, some of CSML’s performance indicators relate to: (a) approaches that introduce or strengthen mechanisms and skills to promote social accountability and/or (b) manifestations of those program inputs (IREX, 2016). 4. Institutional Support—Pillar II activities infused GoL efforts with state-of-the-art information, tools, and approaches. When formal guidance did not exist, activities worked closely with GoL counterparts to make that guidance available, and they often funded the development of manuals and job aids to roll out the policies and incorporate them into training. MCSP/HRH improved and updated the curricula and tools available for pre￾service training of clinical personnel, and it is capacitating local faculty and preceptors from pre-service training institutions. PACS did the same for community volunteers, helping the MOH to revise the National Community Health Services Policy, develop the curriculum for Community Health Assistants and conduct training of trainers for national and regional trainers. All Pillar II health activities conducted their training activities in collaboration with the relevant County and District Health Management Teams using MoH-approved curricula and training materials. Supportive supervision, on-the-job training, and mentorship – which had been abandoned during the EVD crisis – were reinstituted in HFs supported by MCSP, CSH, and 0 5 10 15 20 25 30 35 32 # female journalists participating in accountability reporting 30 2 8 120 MALE (actual) 221 FEMALE (actual) TARGET # CSO/CBO staff trained in conflict-sensitive engagement # CSO/CBO staff trained in advocacy 2 8 Figure M–L7. Capacity building and engagement results from IREX’s CSML activity that promoted accountabilitiy Figure M–L8. Additional accountability results from the CSML activity 0 50 100 150 200 250 38 # communities participating in/undertaking at least 2 events or actions related to accountability over the previous 4 months 41 38 41 120 221 TARGET ACTUAL # USG-assisted communities engaging in advocacy and watchdog functions at least 2 times in the last 4 monthly reports # news stories on accountability produced by Monrovia and Community Radio journalists M–66 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES BOX 1. DEFINITIONS: KEY DETERMINANTS THAT CONTRIBUTE TO SUSTAINABILITY IBTCI utilizes seven potential categories of distal and proximate determinants that enhance or impede sustainability: policy, finances, institutional support, technical choices, accountability, human dimension, external threats and enabling environment. Below are the definitions for each determinant. 1. Policies, Strategies, and Plans—Policies are predicated by procedures, rules, and allocation mechanisms to provide services and/or to set priorities and guide programs. Policies may be implemented through four types of instruments: laws, regulations, assignment of responsibilities to institutions operating in the sector, and economic incentives (i.e., subsidies) for a given sector to operate properly. Hence, a determinant labeled “policy” will indicate both how it is manifested and also how it is implemented. IBTCI submits a broad definition for policy: Policy is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. Policies with sound strategies are more likely to be adopted. Likewise, strategies need comprehensive and clear plans in order to be implemented. Thus, policies, strategies, and plans are subsumed under one overarching determinant, as they should be inextricably linked in order for the effect of the policies to be realized. Therefore, the IBTCI definition for the first determinant is expanded: There is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. There are also extant strategies and plans for policy implementation. Policies, strategies, and plans create a framework for the implementation of any of the activities associated with post￾EVD recovery. An example might be a health policy or an agricultural development policy that was conceived either as the result of USAID funding and/or influenced or shaped by the Pillar II interventions that USAID supports. 2. Finances—As noted, policies and strategies are operationalized through plans. The implementation of plans, however, requires budget allocations and access to funds. In regard to finances, IBTCI defines this determinant as: Allocation of financial resources for the implementation of plans is in place, with a determination of how much funding is needed, the source of those funds, and how the funds will be apportioned and used. 3. Accountability—Accountability describes the mechanisms and procedures through which decision-makers, policy makers, and implementers justify and assume responsibility for the actions, decisions, services, and products that they undertake. Accountability is best proven through written documentation. There are different types of accountability. Further defined, the salient aspects of accountability follow (UNDP Water Governance Facility/UNICEF, 2015): a. Financial accountability means that institutions and individuals must document the intended and actual use of financial resources allocated to them. A clear demonstration of how the funds were expended should be explicit and without ambiguity. b. Political accountability occurs when constituents request their representatives to justify their actions. If there is political accountability, there are no questions as to the legitimacy of the authority or position of a politician. There may be vertical channels of accountability that link citizens to their government. Horizontal accountability refers when partners of equal standing request their counterparts to account for their actions (i.e., when one local government requests another to justify and explain their actions. c. Social accountability are actions taken by civil society, the media and individuals to incite or implore decision-makers to account for decisions and actions undertaken. d. Administrative accountability is exemplified when one level of an administration asks another to account for their decisions and actions and is documented in an organized manner. Guidance and protocols need to be in situ to ensure accountability. Governmental and other institutional and organizational systems are put into place to decide who gets what, when, and how. These mechanisms should be clearly delineated, and expectations should be communicated to relevant stakeholders. These may include incentives for high performance or disincentives/sanctions for inadequate or irresponsible performance. In sum, this determinant is defined as: Systems and/or mechanisms are in place to ensure appropriate use of financial resources and rights holders’ and duty bearers’ access to information (e.g., on performance and results). These systems and/ or mechanisms safeguard the responsible and constructive use of resources and have transparent processes that are manifested by the populace’s respect for the respective institution. 4. Institutional Support—This determinant refers to organizations that are responsible for implementing actions in support of a policy, strategy, plan, or program. Institutional support examines the actions that are implemented as part of that support and how it may be coordinated among the actors involved. Further down this determinate’s pathway to sustainability, exit and transition strategies are established. Institutional support is thus defined by IBTCI as: Organizational arrangements and/or mechanisms are in place that a) operationalize agreed-upon inputs, roles and responsibilities for ANNEX M. DESK REVIEWS – LIBERIA n M–67 implementing policies and interventions and b) produce exit and transition strategies when feasible and appropriate. 1. Technical Choices— Assuming resources are constant, a list of technical choices can be infinite. Hence, the Team will measure a technical choice based on the following criteria: a) appropriateness and relevance to the problem at hand and whether it will solve that problem in an efficient and effective fashion at best value; b) timeliness; and c) demand on resources (human, capital, physical). Thus, the IBTCI definition of the technical choices determinant is: The best technical approach has been selected to lead to implementa￾tion decisions determining whether interventions are “fit for purpose” (effective in addressing the identified problem), feasible given time and resource constraints, and technically relevant. 2. Human Dimension—The importance of the human dimension in the sustainability of development programs is multi-faceted. The human dimension might be as straight-forward as having a person available whose job is to maintain a newly refurbished school or health facility, or a cadre of information technology experts to maintain a fiber optic cable around Monrovia. On the other hand, the human dimension might include more oblique characteristics, such as cultural or institutional norms. Norms present challenges to discussion or systematic analysis as their salient components may be difficult to define. Questions to ask might include, “Is there stigma and discrimination and if so, to what degree?” The Team is exploring the degree to which Pillar II funding ameliorated the consequences of negative norms and/or directed their activities to change them. Transparent and equitable decision-making may be a harbinger for successful implementation of an activity or project, the uptake of services, or changes in social behaviors. This particular determinant stretches across all intervention typologies. Interventions need involvement from a human being to sustain it and serve as its champion. Arguably, this is a somewhat oblique determinant that is open to interpretation and further definition. Thus, for the purposes of evaluation, IBTCI defines human dimensions as follows: Any situation whereby a human being’s input is needed to sustain the intervention. Identify the human dimension— describe it and define it—and the level at which it is available in the appropriate quantity and quality. These nuances are then examined qualitatively to identify activities poised for success and sustainability. 3. External Threats and Enabling Environments—Complex environments, such as those areas in which EVD was most palpable, are influenced by external threats and/or buoyed by enabling environments. This category of determinants is focused on external threats and factors that influence the intervention outcome. They revolve around socio-political stability, geo-politics, climatic conditions, and population migration/internal displacement. For example, roads are needed for the transportation of goods and services to and from villages (e.g., the cold chain for vaccines). Torrential rains or natural disasters may affect the lifespan of roads, thus increasing the need for frequent maintenance or replacement. Climate then is a contextual factor that affects the sustainability of road access and use. Other threats may include internal conflicts between territories, elections, or natural disasters. External threats and enabling environments as a determinant are thus defined: Those factors that have a direct or indirect influence on the ability of an intervention to be sustained, even when all other determinants are supportive of a sustainable outcome. Sustained Results Outbreak Thematic Areas of Response Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology Institutional Enhancements Social & Behavior Change Communications Intervention Types Policies, Strategies, and Plans Finances External Threats and Enabling Environments Human Dimensions Determinants of Sustainability Accountability Institutional Support Technical Choices Immediate Results Ebola Pillar II Response Challenge: Mitigate second-order impacts of the outbreak at the individual, community, and institutional level Survivor Programs Figure M–L9. Analytical framework to measure sustainability M–68 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES PACS. Tetra Tech’s LMWP ensured that attention was given, and skills were imparted, to permit upkeep and oversight of WASH infrastructure. All of the above have the potential for long-lasting effects. 5. Technical Choices—Pillar II activities centered on approaches with proven effectiveness. Nonetheless, specific choices regarding intervention modalities have implications in terms of program effectiveness and, ultimately, sustainability. This is certainly the case when one examines different approaches undertaken with social protection strategies pursued by different Pillar II AFS activities (e.g., direct cash distribution via banks versus mobile money platforms). As another illustration, among Pillar II IPs implementing health-sector activities, strategic decisions in relation to how the activity addressed human capital gaps (e.g., embedment of technical experts with GoL structures, intensive training and capacity building of existing GoL staff) has implications in terms of sustainability. Embedment can meet immediate needs but with removal of embedded staff, there is the risk that gains will be lost. 6. Human Dimension—Community engagement was the lynch￾pin of Liberia’s success in controlling the EVD outbreak (GoL MoH, 2015; Koyabashi et al., 2015; Gillespie et al., 2016). In Pillar II activities, stakeholders were consulted on their priori￾ties and needs before implementation plans were set in stone. Community engagement in health was also the central focus of IRC’s PACS activity. Despite the short timeframe, HC3 was committed to fostering community ownership through invest￾ment in community-centered approaches that engaged commu￾nity members in planning, implementation and celebration of achieved results, as well as strengthening relationships of trust and accountability between all parties. UNICEF’s ECRL activ￾ity also promoted community engagement and sustainability of the improvements through PTAs. Community engagement interventions have also served as plat￾forms for reconciliation. IREX reported the example of Gbolo￾kai Ta community, where 100 people died of EVD. There had been internal schisms about whether to bury the dead or to alert the authorities and await their arrival. For the first time, the Liberia Media for Democratic Initiatives activity held an LMDI dialogue that permitted both sides of the debate to present their reasons and concerns and heal a serious rift (IREX, 2016b). Because social and behavior change is not a short-term endeavor, activities did not always document widespread improvements in ultimate behavioral outcomes. However, each activity documented achievements vis-à-vis important precursors/success factors for sustained behavior change. For example, PACS found that when CHWs were recognized by clinical staff, chiefdom leaders, and community members, they were able to fulfill their roles and responsibilities to promote improved HF quality, which encouraged individuals to seek care (IRC, 2016a). 7. External Threats and Enabling Environments—Although all Pillar II efforts focused on recovery and enhancements to aid preparedness for future emergencies, they generally did not explicitly address external threats such as climate change or sociopolitical instability, two factors that can thwart progress and/or undo past gains. Opportunities and Gaps USAID Pillar II Ebola activities support the GoL prosperity agenda in several ways: (1) By improving and increasing the utilization of health services; (2) By specifically strengthening RMNCH/FP, continuing to a slowing of fertility; (3) By expanding community WASH access; (4) By bolstering small-scale agricultural production, which provides livelihoods in rural areas and contributes to reducing migration to the cities; (5) Through a substantial focus on participation of women as beneficiaries of the food security programs, emphasis on women’s health, and attention to gender parity in the community engagement activities and community management structures; and (6) In the emphasis on strengthening information systems, which will contribute to better planning, targeting of resources, and M&E of the Agenda for Transformation. The involvement of the Global Development Lab has brought USD $30 million and private-sector panache to the mission’s support for ECM. Public-private partnerships with Google and Coca Cola have the opportunity to bring additional international attention and investment into the country. The following is an itemization of opportunities and gaps, as gleaned from desk review documents: ■ There is a need for a clearer analytic focus and measurement strategy for tracking community-level effects of the EVD crisis, and the recovery. The activities looked at second-order impacts, and some looked at households and EVD survivors, but none measured or captured community-level effects. Thus, despite many comments on the importance of community-level factors such as social cohesion, there was no evidence to assess these effects. ■ Monitoring second-order impacts of the EVD outbreak in the southeastern counties was raised as a gap. In a Phase 1 KII, an AFS IP commented that they expected to find that EVD stigma would be highest in most-affected counties but found the opposite. Fear was highest in least exposed counties (in the southeast region), yet those counties were only touched directly by the few interventions that had national scope. ■ Communities in six focus counties emphasized concerns about care for “Ebola orphans,” poor school quality, and need for memorialization (IREX,2016). Memorialization events in 18 ANNEX M. DESK REVIEWS – LIBERIA n M–69 communities adapted to the cultural and religious traditions of the community. “Our spirit is at peace now because we were finally able to lay to rest our loved ones who died as a result of the Ebola crisis.” (Imam in Grand Cape Mount county, cited in IREX, 2016b). ■ Despite a vibrant Pillar II-supported activity in the governance sector (CSML) and non-Pillar II activities on the critical issue of land rights, the impact of the EVD crisis on the legal system did not get much attention. Rebuilding, and building back better, especially in rural areas, brought forward long-festering conflicts over land rights. ■ CSML cited the need for continued work with GoL officials on transparency and contact with the public and their advocates on accountability issues. ■ There is a need for greater efforts to integrate youths into the farming economy who otherwise are likely to move to urban areas and mining (FEWSNET, 2017a). ■ CfW may have considerable potential for continuation and scale-up. Leveraging newly acquired tools and knowledge, there can be community-led maintenance of the roads that are essential to accessing everything from healthcare to economic growth opportunities. ■ Use of mobile tools/platforms for beneficiary registration and verification and performance monitoring had many benefits and can be expanded in tandem with mobile connectivity. Management Issues Below are key management issues emerging from the desk review. Timing is everything. The four months between March and August 2014 – a mere moment in usual international development assistance time – was enough to enable the virus to spread throughout Liberia, and for rumors and fears to gain traction. These fears had profound impacts on the individual and organizational behaviors that constituted the main second-order impacts of the EVD outbreak. Delays in release of funding for Pillar II activities, while again, very short by normal development standards, could lead to major inefficiencies. For example, the PCI activity’s first round of agricultural input vouchers did not reach farmers until after the planting season. While the seeds could be kept for the following year and were not wasted, they didn’t help farm families get through that first lean season. Another timing issue concerns how to integrate new, urgent programmatic action with pre-existing, pre-arranged, and pre￾agreed plans. It might appear efficient to wait until pre-arranged activities are completed before launching something new, but in emergencies, this can be extremely costly. Perceived fairness depends on process as well as content. Lessons learned in identifying vulnerable HHs for targeted interventions have broad application, especially if and when the resource envelope for international development support shrinks. The CT and peace-building activities have methodologies and personnel that can be leveraged. Strong, multi-component monitoring systems. Thanks to adequate staff and budget, and strong empowering normative and communication components, transparency, accountability and acceptability were improved. For example, PCI included beneficiary verification, issued IDs, provided toll-free phone for complaints, conducted regular post-distribution monitoring, and systematically included empowerment messages. They also publicized and required staff to sign an Ethical Code of Conduct pledge (PCI, 2016). Where possible, fit the framework to the problem, not the program structure. USAID programing is constrained in many ways by funding earmarks and conditions, and it is understandable that program management is organized accordingly. However, the diversity of funding units, with money linked to OU, complicated the planning and implementation of Pillar II activities in Liberia. It was important in this analysis to distinguish USAID management units (e.g., DG and GH) from substantive material and expertise relevant to the development challenge. The collapse of Liberia’s health system was not exclusively a health issue. Governance shortcomings were at play, yet Pillar II governance interventions were delayed and sparsely funded. M–70 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES RECOMMENDATIONS 1. M&E staff need to be reminded and supported that their functions are critical to enable sound and rapid, evidence￾based decision making during a crisis, on everything ranging from informing government and citizens, to raising funds, to allocation of staff and health commodities. 2. As part of resilience building, prepare poor households for the annual “lean season” (FEWS NET, 2017b). 3. Suggest early messaging and modeling of the limits of contagion. It is possible that the extended impact of fear of EVD on the rural agricultural markets might have been reduced if the population had been informed early that it was safe to buy and consume food from EVD-affected areas and HHs. 4. Continue to invest in trust-building activities. A rarified sensitivity to politics pervades civil society and government, especially around the time of elections. However, that sensitivity does not seem to have traction on improving accountability and transparency of GoL services. USAID-supported activities have created foundations for dialogue, improving the contacts that CSOs and communities have with officials. This endeavor is worth expanding. 5. Disaster contingency planning differs from “risk assessment,” which is covered in USAID planning, design and oversight guidelines. Every USAID mission and every IP should review and reinforce its disaster contingency plans in an appreciative, forward-looking manner, perhaps first internally, then in a safe (“no harm, no foul”) context where others can help and contribute. 6. Comprehensive and agile disease surveillance requires investment. Nigeria had the human capacity and infrastructure to identify and contain EVD in Lagos and Port Harcourt in August 2014 (Vaz et al., 2016). 7. Address persistent stigma against EVD survivors, orphans and dependents, which is slowing down recovery for those beneficiaries relative to rest of the population (FEWSNET, 2017a). ANNEX M. DESK REVIEWS – LIBERIA n M–71 SUB- ANNEX A. REFERENCES AND SOURCES CONSULTED ACAPS. (2015). Sierra Leone Multi-sector Needs Assessment Report. ACAPS. (2016). Beyond a public health emergency: Potential secondary humanitarian impacts of a large-scale Ebola outbreak. Retrieved from https://www.acaps.org/special-report/beyond￾public-health-emergency-potential-secondary-humanitarian￾impacts-large-scale ACDI/VOCA. (2016). Quarterly Program Performance Report. Ebola Recovery and Resilience Program (ERRP). FY2016/Quarter 3/April 1 - June 30, 2016. Action Against Hunger - US. (2016). Annual Results Report - FFP - G - 16-00054. UNDG West and Central Africa. (2015). Socio-Economic Impact of Ebola Virus Disease in West African Countries. A call for national and regional containment, recovery and prevention. https:// undg.org/wp-content/uploads/2015/07/Ebola.png Adams, J., Lloyd, A., and Miller, C. (2015). The Oxfam Ebola Response in Liberia and Sierra Leone: An evaluation report for the Disasters Emergency Committee. Oxford, UK. Advancing Partners and Communities/ JSI Research & Training Institute, Inc. (2017). Ebola Transmission and Prevention Services Program Overview: Liberia. Retrieved from https://www. advancingpartners.org/sites/default/files/sites/default/files/ resources/apc_liberia_pne_pager_a4_508.pdf Ansumana, R., Keitell, S., Roberts, G. M., Ntoumi, F., Petersen, E., Ippolito, G., & Zumla, A. (2017). Impact of infectious disease epidemics on tuberculosis diagnostic, management, and prevention services: experiences and lessons from the 2014-2015 Ebola virus disease outbreak in West Africa. Int J Infect Dis, 56, 101-104. doi:10.1016/j.ijid.2016.10.010 Barry, A., Bardon O’Fallon, J., Hazerjian, J., & Brodish, P. (2015). A rapid assessment of Ebola-related implications for RMNCH service delivery and utilization in Guinea. Boima, T (Director--Community Health Services Division/ Liberian MoH). (2017). Overview of the National Community Health Assistant Program [PowerPoint slides]. Retrieved from http://moh.gov.lr/wp-content/uploads/2017/12/Comunity￾Health-Presentation-Tamba.pdf Brolin Ribacke, K. J., Saulnier, D. D., Eriksson, A., & von Schreeb, J. (2016). Effects of the West Africa Ebola Virus Disease on Health-Care Utilization - A Systematic Review. Front Public Health, 4, 222. doi:10.3389/fpubh.2016.00222 Carolina Population Center. (2017). July-September 2017 Quarterly Report for Monitoring and Evaluation to Assess and Use Results (MEASURE) Evaluation Phase IV Project for Ebola-funded Activities. Chapel Hill, NC. Cash Learning Partnership (CaLP). (2016a). Learning event report. CaLP. Liberia. Cash Learning Partnership (CaLP). (2016b). Scope of Work for Food for Peace (FFP) Ebola Research Documentation Report. Scope of Work. Action Against Hunger. Cash Learning Partnership (CaLP). (2017). USAID/FFP Quarterly Report, Q2 2017. Cooper, C., Fisher, D., Gupta, N., MaCauley, R., & Pessoa-Silva, C. L. (2016). Infection prevention and control of the Ebola outbreak in Liberia, 2014–2015: key challenges and successes. BMC Medicine, 14, 2. http://doi.org/10.1186/s12916-015- 0548-4 Dahlmia, S. (October 24, 2014). How foreign aid screwed up Liberia’s ability to fight Ebola. The Week. Darvas, Peter. 2017. Liberia - Education Sector - Ebola Recovery and Reconstruction Project: P154880 - Implementation Status Results Report: Sequence 02 (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/ curated/en/171961488217795401/Liberia-Education-Sector￾Ebola-Recovery-and-Reconstruction-Project-P154880- Implementation-Status-Results-Report-Sequence-02 Department for International Development (DfID/United Kingdom). (2016, January 15). The end of the Ebola epidemic: The World Health Organisation declares the widespread Ebola crisis in West Africa is over. Retrieved from https://www.gov.uk/ government/news/the-end-of-the-ebola-outbreak Ellis, S. (2005). The Mask of Anarchy. The Destruction of Liberia and the Religious Dimension of an African Civil War. Second Edition. New York: New York University Press. European Union (2017). Pro-Resilience Action 2015: Supporting the poor and food and nutrition insecure to react to crises and strengthen resilience in Liberia. Website accessed on January 31, 2018. https://ec.europa.eu/europeaid/projects/pro-resilience￾action-2015-supporting-poor-and-food-and-nutrition￾insecure-react-crises-0_en. Emrey, B. (2016). Preparing for future shocks: Building resilient health systems. In J. Charles, T. Ifafore, & K. Greene (Eds.). USAID Global Health Mini-University. Fast, L., & Waugaman, A. (2016). Fighting Ebola with Information: Learning from the Use of Data, Information, and Digital Technologies in the West Africa Ebola Outbreak Response. Retrieved from www.globalinnovationexchange.org/ fighting-ebola-information M–72 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES FEWS NET. (2017a). Ebola and Livelihoods. The effect of the Ebola virus on household livelihoods in Guinea, Liberia, and Sierra Leone. Washington, DC: FEWS NET. FEWS NET. (2017b). Famine Early Warning Systems Network (FEWS Net III) TO 3 and 4. Quarterly Performance Report. January - March 2017. FEWS NET. (2017c). Assessment of Chronic Food Insecurity in Liberia. FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 1 (10/1/2016-12/31/2016). FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 2 (1/1/2017-03/30/2017). FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 3 (4/1/2017-6/30/2017). FHI 360. (2016). Understanding the Potential Role of Mobile Money in Civil Servant Payments in Liberia: A Contextual Analysis. Focus1000. (2014). Folayan, M. O., Haire, B. G., & Brown, B. (2016). Critical role of ethics in clinical management and public health response to the West Africa Ebola epidemic. Risk Management and Healthcare Policy, 9, 55-65. doi:10.2147/RMHP.S83907 Gillespie, A. M., Obregon, R., El Asawi, R., Richey, C., Manoncourt, E., Joshi, K., . . . Quereshi, S. (2016). Social Mobilization and Community Engagement Central to the Ebola Response in West Africa: Lessons for Future Public Health Emergencies. Global Health Science Practice, 4(4), 626-646. doi:10.9745/GHSP-D-16-00226 Government of Liberia (GoL). (2017). National ICT Policy. GoL, (2015). Liberia Economic Stabilization and Recovery Plan (April 2015). Monrovia, Liberia. GoL, (2015). WASH in schools. Liberia’s first step to recovery from Ebola. Monrovia, Liberia. GoL, (2012). Liberia Vision 2030. Monrovia: Government of Liberia. GoL, (2014). National Ebola Response Strategy. Retrieved from https://www.unicef.org/emergencies/ebola/files/National_ Ebola_Response_Strategy_Sept_2014.pdf GoL, (2015). Agenda for Transformation. Steps toward Liberia rising 2030. Monrovia: Government of Liberia. GoL, (2016). Investment Case for Reproductive, Maternal, New￾Born, Child, and Adolescent Health. 2016–2020. http://bit. ly/2ncglnV GoL, Ministry of Health (MoH). (2016). Mental Health Policy and Strategic Plan 2016–2021. Retrieved from http://www.afro.who. int/sites/default/files/2017-06/Mental%20Health%20Policy%20 and%20Strategic%20Plan%20for%20Liberia%202016%20-%20 2021%20.pdf GoL, MoH. ND. The National Health Policy (2011-2021). Monrovia, Liberia: Government of Liberia. GoL, MoH. (2015). Investment Plan for Building a Resilient Health System. Liberia 2015-2021. Monrovia, Liberia: Government of Liberia. GoL, MoH. (2016). EVD survivors care and support national policy. Monrovia, Liberia: Government of Liberia. GoL Ministry of Gender and Development. (2013). National Social Protection Strategy and Policy. Retrieved from http:// fscluster.org/sites/default/files/documents/SP%20strategy%20 and%20policy_%20final%20_30Oct13.pdf GoL, Ministry of Health and Social Welfare. (2010). National health and social welfare policy; National health and social welfare plan 2011-2022. Monrovia: Government of Liberia Retrieved from http://www.nationalplanningcycles.org/sites/default/files/ country_docs/Liberia/ndp_liberia.pdf. GoL, Ministry of Health and Social Welfare. (2008). National Nutrition Policy. Retrieved from http://liberiamohsw.org/ Policies%20&%20Plans/National%20Nutrition%20Policy.pdf Halsey, E. S. (2016). An Outbreak of Fearsome Photos and Headlines: Ebola and Local Newspapers in West Africa. Am J Trop Med Hyg, 95(5), 988-992. doi:10.4269/ajtmh.16-0245 Hart, L., Street, D., & Kulatilaka, H. (2017). Building capacity for resilient health systems. Lessons learned from Sierra Leone, Guinea and Liberia in the time of Ebola. Washington, DC: Hira, S., & Piot, P. (2016). The counter effects of the Ebola epidemic on control and treatment of HIV/AIDS, tuberculosis, and malaria in West Africa. AIDS, 30(16), 2555-2559. doi:10.1097/QAD.0000000000001231 Health Communication Capacity Collaborative (HC3). (2017). Social Mobilization Lessons Learned: The Ebola Response in Liberia. Baltimore, Maryland: Johns Hopkins Center for Communication Programs. Retrieved from http://pdf.usaid.gov/ pdf_docs/pa00mvks.pdf House of Commons (UK) International Development Committee (2016). Recovery and Development in Sierra Leone and Liberia: Sixth Report of Session 2014-2015. IBI International. (2016). USAID/Liberia Governance and Economic Management Support (USAID-GEMS) Project. ANNEX M. DESK REVIEWS – LIBERIA n M–73 Final Report: July 2011- June 2016. Washington DC: IBI International. (2014). USAID/Liberia Governance and Economic Management Support Project (USAID-GEMS) Annual Report: October 2013–September 2014 (Fiscal Year 2014). Washington, DC. ICT Works. (2017). How We Fought Ebola with Information. Inveneo, Inc. (2017). Ebola Recovery Connectivity Initiative (ERCI 2.0) Work Plan for FY 2017-18 (Submitted September 6, 2017) International Business and Technical Consulting Inc. (IBTCI). (2017a). USAID Africa Bureau Ebola Pillar II Recovery Efforts to Mitigate Second-Order Impacts of the Epidemic in Guinea, Sierra Leone, and Liberia. Sierra Leone Desk Review. Washington, DC. International Business and Technical Consulting Inc. (IBTCI). (2017b). Ebola Pillar II Monitoring, Evaluation, and Learning Project: Performance Evaluation Inception Report. Washington, DC. International Rescue Committee (IRC). (2015). Partnership for advancing community-based services, Liberia. Knowledge, attitudes and practices (KAP) survey among caregivers of children under five years of age in the counties of Bong, Lofa, and Nimba in Liberia. Report. Liberia. International Rescue Committee (IRC). (2016a). Partnership for Advancing Community-Based Services (PACS) Quarterly report. April 1, 2016 to June 30, 2016. International Rescue Committee (IRC). (2016b). Indicator matrix: Targets Achieved (PACS) IntraHealth. (2015). USAID’s Fighting Ebola Grand Challenge: mHero. Quarterly Report. June 6, 2015 - September 5, 2015. Quarterly Report. IntraHealth International. IREX. (2016a). Liberia Civil Society and Media Leadership Program (CSML) (Vol. 2017, pp. Project overview, covering broad range of activities, from developing a CSO media plan to strengthening the Traditional Council, to creating model radio stations.) IREX. IREX. (2016b). Trusted Voices in Challenging Times: Civil Society and Media Leadership (CSML) Program. Final Report. Washington DC: Jeans, H., Castillo, G. E., & Thomas, S. (2017). The Future is a Choice. Absorb, adapt, transform. Resilience Capacities. Retrieved from Oxfam Policy and Practice website: https://policy-practice. oxfam.org.uk/publications/absorb-adapt-transform-resilience￾capacities-620178 JHPEIGO. (2016). Maternal and Child Survival Program (MCSP): Human Resources for Health in Liberia. Quarterly Report. Q1 July 1, 2016-September 30, 2016 (Fiscal Year 2016). JHPEIGO. (2017a). Non GHET Indicators. JHPEIGO. (2017b). MCSP Restoration of Health Services (RHS) in Liberia. GHET Quarterly Report. July-September 2017. Washington DC: JHPEIGO. (ND). MCSP Liberia: Human Resources for Health. Strengthening pre-service education. Registered midwives (RMs) and medical lab technicians (MLTs). Timeframe April 2016- 2018. (pp. 18): JHPEIGO. John Snow International (JSI). (2017a). ETP&SS Monitoring and evaluation strategy: October 2017. Updated July 2017. John Snow International (JSI). (2017b). Advancing Partners and Communities baseline ebola survivor assessment. John Snow International (JSI). (ND). USAID | DELIVER project. Final country report. Liberia. Johns Hopkins Center for Communication Programs (JHU/ CCP). (2016). Pillar II Liberia JHU Indicator Report, Q2. Baltimore, MD Johns Hopkins University Center for Communication Programs (JHU/CCP). (ND). PII Liberia JHU Indicator Report [HC3 Project]. Kaplan, Z. A., Kyle, P., Shugart, C., & Moody, A. (2012). Developing Public Private Partnerships in Liberia. Retrieved from http://dx.doi.org/10.1596/978-0-8213-9480-9 Kastelic, Kristen Himelein; Kastelic, Jonathan G; 2015. The socio-economic impacts of Ebola in Liberia: results from a high frequency cell phone survey round five (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/ curated/en/162381468179333776/The-socio-economic￾impacts-of-Ebola-in-Liberia-results-from-a-high-frequency￾cell-phone-survey-round-five Kobayashi, M., Beer, K. D., Bjork, A., Chatham-Stephens, K., Cherry, C. C., Arzoaquoi, S., . . . Nyenswah, T. G. (2015). Community Knowledge, Attitudes, and Practices Regarding Ebola Virus Disease - Five Counties, Liberia, September￾October, 2014. MMWR Morb Mortal Wkly Rep, 64(26), 714- 718. Koch, T. (2016). Ebola in West Africa: lessons we may have learned. Int J Epidemiol, 45(1), 5-12. doi:10.1093/ije/dyv324 Ladner, J. T., Wiley, M. R., Mate, S., Dudas, G., Prieto, K., Lovett, S., … Palacios, G. (2015). Evolution and spread of Ebola virus in Liberia, 2014–2015. Cell Host & Microbe, 18(6), 659–669. http://doi.org/10.1016/j.chom.2015.11.008 Liberia Strategic Analysis (LSA). (2017). USAID/Liberia Collaborative Support for Health (CSH): Midterm performance evaluation report. M–74 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Liberia Ministry of Finance and Development Planning (MoFDP), 2015. MoFDP Strategic Plan 2015-2018. Liberia Ministry of Health. (2015). National Knowledge, Attitudes and Practices Study on Ebola Virus Disease. Monrovia, Liberia Liu, S. (2017). Wartime educational loss and attitudes toward democratic institutions. Working Paper No. 175. Liu, S. Q., Rayner, S., & Zhang, B. (2015). How Ebola has been evolving in West Africa. Trends Microbiol, 23(7), 387-388. doi:10.1016/j.tim.2015.05.003 Ly J, Sathananthan V, Griffiths T, Kanjee Z, Kenny A, Gordon N, et al. (2016) Facility-Based Delivery during the Ebola Virus Disease Epidemic in Rural Liberia: Analysis from a Cross￾Sectional, Population-Based Household Survey. PLoS Med 13(8): e1002096. https://doi.org/10.1371/journal.pmed.1002096 Management Sciences for Health (MSH). (2016). Liberia. Collaborative Support for Health (CSH) program. Annual summary report for fiscal year 2016 (October 1, 2015 - September 30, 2016). Manguvo, A., & Mafuvadze, B. (2015). The impact of traditional and religious practices on the spread of Ebola in West Africa: time for a strategic shift. Pan Afr Med J, 22 Suppl 1, 9. doi:10.11694/pamj.supp.2015.22.1.6190 Mercy Corps. (2016). Economic Recovery from Ebola for Liberia (EREL) Program. unded by USAID/Food for Peace (FFP). Fiscal Year (FY) 2017 Quarter 1 Report October 1 - December 31, 2016. Modarres, N., Berg, K. (2016). Qualitative assessment on health system trust and health service utilization in Liberia. Health Communication Capacity Collaborative, Baltimore: Johns Hopkins Bloomberg School of Public Health, Center for Communication Programs. Mohammed, E., Bedri, A., & Demeke, F. (2016). Liberia. Emergency Food Security and Ebola Response Program. Final Evaluation Report (Draft). Final Results Report, draft. PCI. Murphy, Emmet; Erickson, Kali; and Tubman, Macon. (2016). USAID Office of Food for Peace Food Security Desk Review for Liberia, 2016–2020. Washington, DC: FHI 360/FANTA. Navaro, R. (2014). Pillar II: Mitigating second-order impacts of the Ebola Virus Disease (EVD) Epidemic. A strategic framework for Liberia, Guinea and Sierra Leone. USAID/Africa Bureau, Africa Ebola Unit. Unpublished. NetHope. (2014). Information and Communications Technology Response to the Liberia Ebola Crisis: Desk Review and Recommendations for Private Sector Engagement. http://ebolacommunicationnetwork.org/wp-content/ uploads/2014/10/USAID_LiberiaICTDeskReview_Final_ Oct2914-2.pdf Nyanfore, D. K., II. (2016). History of Ebola in Liberia: A Case of Under-Development. The Patriotic Vanguard. Nyenswah, T. G., Kateh, F., Bawo, L., Massaquoi, M., Gbanyan, M., Fallah, M....De Cock, K. M. (2016). Ebola and Its Control in Liberia, 2014–2015. Emerging Infectious Diseases, 22(2), 169- 177. https://dx.doi.org/10.3201/eid2202.151456. Parpia, A. S. M., Ndeffo-Mbah, M. L., Wenzel, N. S., & Galvani, A. P. (2016). Impact of the 2014-2015 Ebola Outbreak on Malaria, HIV, and Tuberculosis in West Africa. Emerging Infectious Diseases (in press). Emerging Infectious Diseases, 22(3), 433-441. doi:10.3201/eid2203.150977 Parshley, L. (2016). The Aftermath of Ebola in Liberia. The Atlantic (July/August Issue), 17. PCI. (2016). Emergency Food Security and Ebola Response Program. Liberia. Protect and Empower for Ebola Resilience (PEER). Final Report. PCI. (2015). Protect and Empower for Ebola Resilience (PEER). Bomi and Grand Cape Mount Counties. Baseline Evaluation Report. Radice, H. (2017). Cash Transfers for Food Security in Epidemics: A review of the Food for Peace Response to the Ebola crisis in Liberia and Sierra Leone. The Cash Learning Partnership/USAID. Risso-Gill, I., & Finnegan, L. (2015). Children’s Ebola Recovery Assessment (CERA) in Sierra Leone. . Santermans, E., Robesyn, E., Ganyani, T., Sudre, B., Faes, C., Quinten, C., . . . Plachouras, D. (2016). Spatiotemporal Evolution of Ebola Virus Disease at Sub-National Level during the 2014 West Africa Epidemic: Model Scrutiny and Data Meagerness. PLoS One, 11(1), e0147172. doi:10.1371/journal. pone.0147172 Save the Children (SC). (2016). Emergency Food Assistance for Ebola Affected Families in Liberia - Margibi and Bong Counties. Fiscal Year 2016 Annual Results Report. October 1, 2015 - September 30, 2016. Save the Children. (ND). Digital Cash Transfers in Liberia: A Case Study from Save the Children’s Emergency Food Security Program (2015–2016). Shoman, H., Karafillakis, E., & Rawaf, S. (2017). The link between the West African Ebola outbreak and health systems in Guinea, Liberia and Sierra Leone: a systematic review. Globalization and Health, 13, 1. http://doi.org/10.1186/s12992- 016-0224-2 Shultz, J. M., Cooper, J. L., Baingana, F., Oquendo, M. A., Espinel, Z., Althouse, B. M., . . . Rechkemmer, A. (2016). The Role of Fear-Related Behaviors in the 2013-2016 West Africa Ebola Virus Disease Outbreak. Curr Psychiatry Rep, 18(11), 104. doi:10.1007/s11920-016-0741-y ANNEX M. DESK REVIEWS – LIBERIA n M–75 Sridhar, S., Brouqui, P., Fontaine, J., Perivier, I., Ruscassier, P., Gautret, P., & Regner, I. (2016). Risk perceptions of MSF healthcare workers on the recent Ebola epidemic in West Africa. New Microbes New Infect, 12, 61-68. doi:10.1016/j. nmni.2016.04.010 STEPS Centre. (2015). The social life of infectious diseases. STEPS Impact Stories. Retrieved from https://medium.com/ steps-impact-stories/the-social-life-of-infectious-diseases￾d694d99114d4#.lib3e958g Strong, A., & Schwartz, D. A. (2016). Sociocultural aspects of risk to pregnant women during the 2013-2015 multinational Ebola virus outbreak in West Africa. Health Care Women Int, 37(8), 922-942. doi:10.1080/07399332.2016.1167896 Tearfund and World Vision UK. (2017) Analysis of the post-Ebola recovery period of Sierra Leone and Liberia with lessons for future health emergencies. Retrieved from http://www.tearfund. org/~/media/files/tilz/research/2017-wv-tearfund-after-the￾outbreak-en-revised.pdf Tetra Tech. (2017). Liberia Municipal Water Project (LMWP) II Combined Year 1 and Quarterly Report 4 (submitted October 31, 2017; revised November 20, 2017). The Lancet Infectious Diseases. (2016). Infectious disease emergencies: taking the long-term view. Lancet Infect Dis, 16(12), 1305. doi:10.1016/S1473-3099(16)30483-2 The Mitchell Group. (2014). Liberia monitoring and evaluation program (L-MEP) Year Four annual report (final). The World Bank. (2015, April 16, 2015). Summary on the Ebola Recovery Plan: Liberia - Economic Stabilization and Recovery Plan (ESRP). Understanding Poverty. Retrieved from http:// www.worldbank.org/en/topic/ebola/brief/summary-on-the￾ebola-recovery-plan-liberia-economic-stabilization-and￾recovery-plan-esrp The World Bank. (2015). West African Health Ministers and Development Partners Exchange Best Practices on Ebola Prevention (October 20, 2016). Retrieved from News (http://web.worldbank.org/WBSITE/EXTERNAL/ NEWS/0,,pagePK:34382~piPK:34439~theSitePK:4607,00. html) The World Bank. (2015). Net ODA received per capita (current USD $). Retrieved from https://data.worldbank.org/indicator/ DT.ODA.ODAT.PC.ZS?locations=LR The World Bank. (2016). World Bank Group Ebola Response Fact Sheet. Retrieved from http://www.worldbank.org/en/topic/ health/brief/world-bank-group-ebola-fact-sheet The World Bank. (2017, October 16, 2017). The World Bank in Liberia. Retrieved from http://www.worldbank.org/en/country/ liberia/overview Tremblay, N., Musa, E., Cooper, C., Van den Bergh, R., Owiti, P., Baller, A., … Gasasira, A. (2017). Infection prevention and control in health facilities in post-Ebola Liberia: don’t forget the private sector! Public Health Action, 7(Suppl 1), S94–S99. http:// doi.org/10.5588/pha.16.0098 Frontier Research U & UNDP. (2016). Liberia: Financial Diagnostic Update and Feasibility Study for the Possible Entry of a Third-Party Cash Management Partner or Agent Aggregator. Retrieved from Monrovia, Liberia: UNDG West and Central Africa. (2015). Socio-Economic Impact of Ebola Virus Disease in West African Countries. A call for national and regional containment, recovery and prevention. Retrieved from https://undg.org/wp-content/uploads/2015/07/ Ebola.png UNECA. (2015). Socio-economic Impacts of Ebola on Africa. Revised Edition Retrieved from Addis Ababa, Ethiopia: UNICEF Liberia. (2016). Education Crisis Response for Liberia (ECRL). Revised Proposal with Reprogrammed Activities for USAID, February 2016. UNICEF Liberia. (2016). Monthly Updates - August 2016. UNICEF Liberia. (2015). Contribution Summary: RUTF in-kind support to Liberia. Retrieved from Monrovia, Liberia UNICEF Liberia (2017). ECRL Fifth Progress Report Submitted to USAID., June 2017. United Nations. (ND). Global Ebola Response webpage on Liberia. http://ebolaresponse.un.org/liberia United Nations Development Programme (UNDP) (2016). Human Development Report 2016 Briefing Note for Liberia. Retrieved from: http://hdr.undp.org/sites/all/themes/hdr_ theme/country-notes/LBR.pdf United Nations Economic Commission for Africa. (2015). Socio￾economic Impacts of Ebola on Africa. Addis Ababa, Ethiopia. United Nations High Commission for Refugees (UNHCR). (2015, 18 December, 2015). Liberia, Côte D’Ivoire, UNHCR facilitate voluntary repatriation of Ivorian refugees after interruption by Ebola outbreak. Retrieved from https://unmil. unmissions.org/liberia-cote-d’ivoire-unhcr-facilitate-voluntary￾repatriation-ivorian-refugees-after-interruption USAID. (2015). mHero: Leveraging HIS and Mobile Phones to Support Frontline Health Workers AMID the Ebola Outbreak in West Africa (BAA-EBOLA-2014 Addendum 1). Washington, DC. USAID. (2015). (SBU) Modification of Assistance. PACS project. Modification to PACS award. USAID. (2015). Strategic Program Review Health, Pillar II Health. Washington, DC. M–76 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES USAID. (2016). Feed the Future Global Performance Evaluation. Final Evaluation Report Annexes. USAID. (2016). USAID Ebola Fact Sheet 12 west_africa_ebola_ fs12_09-30-2016. USAID. (2016). MCSP Liberia: Restoration of health services. Midline assessment findings - in brief. USAID. (2016). Strategic Program Review Health. Pillar IV – Global Health Security Agenda & Ebola Preparedness USAID Africa Bureau. (2015). SBU AEU Strategic Program Review 10-28-2015. USAID Bureau of Food for Peace (FFP). (2016, December 2016). Ebola Response in Liberia. USAID Bureau of Global Health. (2015). SBU GHB slides Strategic Program Review July 2015_6.30_FINAL. USAID Center for Accelerating Innovation and Impact (CII). (2015). SBU Fighting Ebola. A grand challenge for development. (Internal Strategic Review 10.1). USAID Global Development Lab. (2015). The Lab’s Contribution to USAID’s Ebola Efforts. Strategic Program Review. Washington, DC. USAID Global Development Lab. (ND). Lab activities in Liberia. Internal Mission portfolio review. FY 2016/2017. Monrovia, Liberia. USAID Office of Inspector General. (2015). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, First Quarter. Washington, DC. USAID Office of Inspector General. (2016). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, Second Quarter. Washington, DC. USAID/Bureau of Global Health. (ND). MCSP GHET Indicators 1a, 1b and 2. USAID. (ND). The Resilience Agenda: Measuring Resilience in USAID. http://bit.ly/2DId88N USAID. (ND). Project Last Mile-Liberia/Sierra Leone. Retrieved January 5, 2018, from https://partnerships.usaid.gov/ partnership/project-last-mile-liberiasierra-leone. USAID/Liberia. (2012). USAID Liberia Country Development Cooperation Strategy, 2013-2017. USAID. USAID/Liberia. (2015). Program Description. Mobile Money Roll-Out in the Liberian Health Sector. USAID/Liberia draft for Ebola Supplemental Funding under mSTAR. USAID/Liberia. (2016). USAID Liberia Country Development Cooperation Strategy, 2013-2018. Revision 2. USAID. USAID/Liberia. (2016). USAID/Liberia. Performance Management Plan (FY 2013 - FY 2017). 26. USAID/Liberia. (2016). Liberia: Recovering from Ebola and Preparing for the Future. Retrieved from Washington, DC. USAID/Liberia. (2017). Lab Activities in Liberia. Internal Mission Portfolio Review. FY2016/17. Monrovia, Liberia. USAID/Liberia. (ND). MCSP Liberia: Human Resources for Health. Strengthening pre-service education. Registered midwives (RMs) and medical lab technicians (MLTs). Timeframe: April 2016-June 2018. Vaz, R. G., Mkanda, P., Banda, R., Komkech, W., Ekundare￾Famiyesin, O. O., Onyibe, R., … Tegegne, S. G. (2016). The Role of the Polio Program Infrastructure in Response to Ebola Virus Disease Outbreak in Nigeria 2014. The Journal of Infectious Diseases, 213(Suppl 3), S140–S146. http://doi.org/10.1093/ infdis/jiv581. Weah, V. D., Doedeh, J. S., Wiah, S. Q., Nyema, E., Lombeh, S., & Naiene, J. (2017). Enhancing Ebola Virus Disease Surveillance and Prevention in Counties Without Confirmed Cases in Rural Liberia: Experiences from Sinoe County During the Flare-up in Monrovia, April to June, 2016. PLoS Currents, 9, ecurrents.outbreaks.2b7f352af0866accbd7e5a82f165432a. http://doi.org/10.1371/currents. outbreaks.2b7f352af0866accbd7e5a82f165432a World Bank (2017, June 17). After Ebola, Liberia’s Health System on Path to Recovery. Retrieved from http://www.worldbank.org/en/ news/feature/2017/06/07/after-ebola-liberias-health-system￾on-path-to-recovery World Bank (2016). 2014-2015 West Africa Ebola Crisis: Impact Update. http://pubdocs.worldbank.org/ en/297531463677588074/Ebola-Economic-Impact-and￾Lessons-Paper-short-version.pdf World Bank (2016b). Project Information Documents for initial and follow-on funding for Girls Ebola Recovery Livelihood Support (GERLS) Project. Retrieved from documents. worldbank.org World Food Program (WFP). (2016). APS USAID FFP/202(e). Final results report form. PRRO 200550. World Food Program. (2015). Ebola response from crisis to recovery. Retrieved from http://documents.wfp.org/stellent/ groups/public/documents/communications/wfp276313.pdf World Health Organization (WHO). (2000). World Health Report. Geneva. World Health Organization (WHO). (2008). International Health Partnership and related Initiatives (IHP+) Harmonization of Health in Africa (HHA). Retrieved from Lusaka http://www. who.int/healthsystems/FINAL_IHP_LusakaWayForward.pdf ANNEX M. DESK REVIEWS – LIBERIA n M–77 WHO). (2015. Ebola Situation Report. 11 November, 2015. http://apps.who.int/ebola/current-situation/ebola-situation￾report-11-november-2015 WHO. (2016a). Latest Ebola outbreak over in Liberia; West Africa is at zero, but new flare-ups are likely to occur. Retrieved from http://www.who.int/mediacentre/news/releases/2016/ebola￾zero-liberia/en/ WHO. (2016b, June 6, 2016). End of the most recent Ebola virus disease outbreak in Liberia. Retrieved from http://www.who. int/mediacentre/news/releases/2016/ebola-liberia/en/ WHO. (2016c, June 6, 2016). Ebola Outbreak 2014-2015. Retrieved from http://www.who.int/csr/disease/ebola/en/ WHO (2016). Ebola Response Phase 3: framework for achieving and sustaining a resilient zero. Retrieved from http://apps.who. int/iris/bitstream/10665/184693/1/ebola_resilientzero_eng. pdf?ua=1. WHO. (2017a). Recovery Toolkit: Supporting countries to achieve health service resilience: A library of tools & resources”) available during the recovery period of a public health emergency. Geneva. Retrieved from http://apps.who.int/iris/ bitstream/10665/205944/1/WHO_HIS_SDS_2016.2_eng. pdf?ua=1 WHO. (2017b). The Ministry of Health and World Health Organization conclude a 3-day advanced infection prevention control (IPC) follow up training workshop for county and hospital IPC focal persons [Press release]. Retrieved from http:// www.afro.who.int/news/ministry-health-and-world-health￾organization-conclude-3-day-advanced-infection-prevention Wright, J., Health Finance & Governance Project. (2015). Essential Package of Health Services Country Snapshot: Liberia. Bethesda, MD: Health Finance & Governance Project, Abt Associates Inc. Y Care International. (2016). Emergency Ebola Response Evaluation and Learning Summary, February 2016. Retrieved from http://1c8puy1lylrov7ssf1oz3o22-wpengine.netdna-ssl. com/wp-content/uploads/2016/02/YCI_Ebola-Response￾Evaluation-Summary.pdf. Zielinski, C., Kebede, D., Mbondji, P. E., Sanou, I., Kouvividila, W., & Lusamba-Dikassa, P.-S. (2014). Knowledge systems in health in sub-Saharan Africa: results of a questionnaire-based survey. Journal of the Royal Society of Medicine, 107(1S), 6. doi:10.1177/0141076813518525 M–78 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SUB-ANNEX B. LIBERIA PILLAR II ACTIVITIES WITH INFORMATION IN THE SBU FILE Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Agriculture and Food Security Ebola Recovery and Resilience Program (ERRP) ACDI/ VOCA 7/8/15– 9/9/16 (14 mos.) ACDI/VOCA Quarterly Program Performance Report. EERP/Liberia, FY2016 Q3 (April 1–June 30, 2016) Bong and Nimba counties. - Unconditional CTs reached 9,895 direct beneficiaries (97% of planned target). - 8% of these through Lonestar Cell MTN platform. - Cash for Work (CfW) ($4/day for max of 15 days) – 2,409 participants (99% of target); completed 300km roads (100% of target), at Cash Payment Points (Ecobank offsite banking) across 25 cluster communities. - Incorporated EVD messaging, and IPC prevention training, and “provision of sanitation and hygiene materials to households and places of business” into the ongoing ACDI/ VOCA activities (USAID FFP Fact Sheet, December 2016). NOTES: Refers to 2016 Annual Report, but that has virtually nothing on Liberia, and nothing on ERRP. Mentions financial services to 5000 farmers, through a different activity. Refers to Liberia Agriculture Upgrading, Nutrition and Child Health. FFP Fact sheet (Dec 2016) states UCTs, CfW and Agricultural input vouchers benefited “nearly 102,000 people.” The Cash Learning Partnership (CaLP) ACF CaLP. (2016a). Learning event report; CaLP. (2016b). Scope of Work for Food for Peace (FFP) Ebola Research Documentation Report. Scope of Work. Action Against Hunger. Cash Learning Partnership (CaLP). (2017). USAID/FFP Quarterly Report, Q2 2017. Regional Activity NOTES: Liberia Livelihood Improvement for Farming Enterprises III activity facilitated financial services to 5,000 farmers; provided loans worth $500,000 to community members. FEWSNET Chemonics 9/15-3/17 FEWS NET. (2017a). Ebola and Livelihoods. The effect of the Ebola virus on household livelihoods in Guinea, Liberia, and Sierra Leone. Washington, DC: FEWS NET. FEWS NET. (2017b). Famine Early Warning Systems Network (FEWS Net III) to 3 and 4. Quarterly Performance Report. January-March 2017. FEWS NET. (2017c). Assessment of Chronic Food Insecurity in Liberia Multi-country scope; includes but is not limited to Ebola. According to Q3-4 Progress Report, the activity’s Task Order 3 (awarded in late 2015) pertained to focused support to Guinea, Liberia and Sierra Leone. The above quarterly report makes reference to the activity’s TO-3 closeout (technical, financial and administrative) plan NOTES: Available documentation provides data generated/analyzed under the auspices of FEWSNET; not many particulars re activity implementation Economic Recovery from Ebola for Liberia (EREL) Mercy Corps 1/15- 12/16 Mercy Corps. Final Program Report (9 January 2015-31 December 2016), submitted March 31, 2017. Lofa, Montserrado, Margibi Third Mercy Corps program –emergency food security program (Jan 2015- Dec 2016) targeting 30000 HHs in 3 counties. Cash transfers to 10,000 HHs also got 7.2 million USD. (Phase 1 KII) Baseline survey – September 2016 NOTES: EREL targeted 30,000 HHs (150,000 individuals) in ten districts in Margibi, Montserrado, and Lofa counties with a combination of cash transfers and agricultural inputs vouchers. The number of beneficiaries reached by the program exceeded the original target of 30,000 HHs, bringing cash transfers to 30,077 HHs (estimated at 179,980 individuals). Of these, 602 (3,602 individuals) received cash through the mobile money cash transfers modality, while the remaining 29,475 HHs (176,378 individuals) received direct cash. The total value of cash spent by the program under Objective 1 was USD $7,263,564. Under Objective 2, the program targeted 10,000 farming HHs with agricultural inputs vouchers, complemented by basic agronomic training and follow-up support, to help families resume and maintain production in the wake of the epidemic. By the end of the program, EREL had distributed vouchers to 9,931 farming HHs (49,655 individuals). A total of 59,468 vouchers (99% of the goal) were distributed during the lifespan of the EREL program, with a value of USD $ 594,680 out of the target of USD $600,000. The percentage of HHs experiencing moderate hunger had decreased from 54% in the first quarter of 2016 to 8 % in December 2016. An increase in the diversity of HH diets was also noted over the course of program implementation, with the average dietary diversity score improving from 4.7 (medium dietary diversity) during the baseline survey conducted among newly registered beneficiaries in March of 2016 to 8.6 (high dietary diversity) during the endline ANNEX M. DESK REVIEWS – LIBERIA n M–79 Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Protect and Empower for Ebola Resilience (PEER) PCI 2/15-9/16 Baseline Evaluation. Mohammed, E, Bedri, Abdunaser, and Demeke, F. 2016. Emergency Food Security and Ebola Response Program. Liberia. Final Evaluation Report (Draft). September 2016 Project and Empower for Ebola Resilience (PEER) Final Report. Two Counties: Bomi, Grand Cape Mount. Targeted districts: Tewor, Porkpa and Golakonneh for GCM; Senjeh, Klay, Dowein and Suehn Mecca in Bomi [p 31 of PCI final report). Interventions: (1) Unconditional cash transfers (UCT): reached 6,918 vulnerable HHs (target was 6,718), with USD $46.88/mo.; (2) Cash for Work (rehabilitated 187 km community roads in 48 communities, exceeding target of 121 km); paid $77.1 to 2,406 beneficiaries (1,677 male and 842 female) for unskilled labor, and $110 to 113 for skilled labor; (3) Agricultural Input Vouchers provided to 5,847 farmers (3,461 males; 2386 females, i.e., 49%) in 96 communities – 110% of target. Activity provided seeds (beans, corn, pepper, okra, cabbage, cucumber, bitterball, eggplant). Tools supplied included cutlass, tapeline, shovel, hoes, axes, watering cans, rakes and digger. Technical assistance – training manual and training sessions on seed management and horticultural practice. Cascade model: Trained 264 Lead Farmers (66 Female) who shared skills with 4,560 farmers (2,582 females). Training re: vegetable production, farming as a business, and leadership. NOTES: Goal: “to reduce food insecurity among households with pregnant or lactating women, children under two, and EVD affected households” in Bomi and Grand Cape Mount counties. Two objectives: (1) To increase food access for EVD affected and other vulnerable households by increasing HH income through UCTs; (2) To increase food availability for EVD-affected and other vulnerable HHs by increasing agricultural production of smallhold farmers through targeted distribution of agricultural input vouchers. Baseline study has interesting comparison of “EVD-infected and non-infected households” re: vulnerability and food security; HH dietary diversity, HH expenditure, market access and agriculture. At baseline - largest HH expenditure: food (average 23% of expenditure), followed by education (18%) and clothing (10%). HHs grow more than 30% of their food in 4 of the 6 districts surveyed. Collaboration with MoAg. Population-based HH survey of 646 HHs in the two counties; “Participatory Impact Assessment/ FGDs in 12 communities with beneficiaries. Focus of assistance was vulnerable HHs, including those affected by EVD and also, prioritized female/child headed HHs, elderly people with disabilities & pregnant and lactating women. Formed Beneficiary Selection Committees to review criteria and identify eligible beneficiaries. Training in vegetable production translated into both improved food availability and improved dietary diversity. As of July 2016, small farmers aided by the activity reported 435,941 kg harvest and $205,619 additional income. Outcomes from the HH survey: - Average HH dietary diversity increased from 3.89 to 5.59 (95% CI=5.34-5.84). - Percent of HH with moderate or severe hunger declined from 75% to 62.2% (CI 58.3-65.9). - Mean number of food groups consumed by women 15-49 years old, increased from 2.8 at baseline to 4.58 (CI 4.45 – 4.71). Emphasis on inclusion of women and girls was carried through the activity: prioritized vulnerable women and girls; Accommodated women’s workload in CFW activities; Included training on gender and GBV and sexual harassment for activity staff and beneficiaries. Achieved 33.4% females in the CFW activity. Seeds were received too late for planning in the first year but could be saved for the next planting season. Funding was insufficient to cover all families that met vulnerability criteria; PCI and village BCSs distributed the interventions across more families rather than providing smaller number of HH with all 3 interventions. Emergency Food Assistance for Ebola Affected Families in Liberia STC 9/15- 11/16 (15 mos.) Emergency Food Assistance for Ebola Affected Families in Liberia – Margibi and Bong Counties. Fiscal Year 2016 Annual Results Report. October 1, 2015-September 30, 2016. 5,000 HH in 133 communities in Margibi, Bong counties, most heavily affected by EVD. Interventions: (1) UCT, cashed out through LoneStar mobile money service; (2) Ag. inputs: 6 varieties of seeds, with agronomy training, and follow-up support from “Lead Farmers”; (3) Market analysis: tracking prices of basic commodities “which ordinary Liberians usually buy: rice, fish, cassava, charcoal, beans, pepper, onions, okra, palm oil, vegetable oil, potato greens, and bitter balls. UCT Target: 5000 HHs in Bong (3,500) and Margibi (1,500) Registered 5,000 HHs, including 3,254 female-headed HHs, 45 child-headed HHs; only 733 had both male and female adult. Net population served: 24,978 individuals. NOTES: Aim: To increase access to food through UCTs and increased agricultural production for HHs directly and indirectly affected by Ebola. Conducted “time zero survey” in Nov. 2015. Close collaboration with MoGCSP. Community Committees were formed to select HHs according to criteria established by MoGCSP (affected by EVD, small land holding <1 acre; high dependency ratio; female headed; members with disabilities; accommodating OVC; elderly member or unable to work). Registration was accomplished in less than 1 month (Oct. 23-Nov 17, 2015). Accomplished six transfers of USD $50 (equivalent in Liberian dollars) to 5,000 HHs between Dec 2015-Sept 2016. Provided 6 mos. of monthly transfers/ vouchers to 27,021 individuals in the 5000 HHS; Amount was intended to cover “a portion of the household’s monthly food basket.” Post-distribution monitoring found 51% to 67% of UCTs were used for food, with peak in the lean season. Provided a subset of 4,000 HHs (selected by Community Committee in the 133 villages) with a seed kit including 6 seed varieties, selected for their potential use both for HH consumption and for sale and vetted by MoAg: maize, okra, eggplant, hot pepper, cucumber and bitter ball. Market monitoring in FY2016 found prices “remain in par with historical data.” (i.e., markets had recovered by 2016). Outcomes: - HH Hunger: moderate or severe hunger, fell from 51.9% (Nov 2015) to 30.5%, with greatest progress in HHs with both male and female adult, and least progress in HHs with female and no male adult. - HDDS: from Nov 2015 level of 4.54 to end-of-project, 5.83 (out of 12). - Coping Strategy Index (CSI): from 22.55 to 5.70 (again, greatest improvement in male and female HHs, and least in female-only HHs). Challenges: Limited LoneStar MTN agent capacity and liquidity were a problem. STC worked with banks to improve agent liquidity. STC had to negotiate with mobile money agents – usually based in towns – to travel to distribution points that could serve the rural communities targeted by the activity. Reported “actual average cost per activity participant” as USD $147.44. M–80 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Screening, Referral and treatment for children with severe acute malnutrition (SAM), and ready￾to-use therapeutic food (RUTF) for treatment UNICEF 6/15- 12/16 Mislabeled document – on pager This activity includes but is not limited to Liberia. Liberia focus counties: Bomi, Bong, Grand Cape Mount, Lofa, Margibi, Montserrado, and Nimba. Interventions: Training in IMAM screening for CHWs, intake and treatment of SAM, feeding with locally sourced, Ready to Use Food packets (RUTF). NOTES: 96% (3,205 out of 3,349) of the severely malnourished children treated and discharged from Integrated Management of Acute Malnutrition (IMAM) in the 7 counties (Bomi, Bong, Grand Cape Mount, Lofa, Margibi, Montserrado, and Nimba) affected by Ebola from August to December 2015 were cured. 3,472 (67%) out of the 8,696 cartons ready to use therapeutic food (RUTF) sent to the country in July 2015 were released and sent to the 7 counties by end of December 2015. Funds received in November 2015 will be utilized starting January 2016 to provide technical support and training to: (1) community health volunteers on active nutrition screening, and (2) health workers on the post-Ebola IMAM guidelines with emphasis on continued adherence to infection prevention and control since the modified Ebola guidelines on IMAM is no longer applicable in the current context. The funds will also be used to ensure regular supportive supervision to service providers, and monitoring of the program activities in accordance with the national protocols. PRRO - World Food Programme emergency food assistance to support Ivorian refugees and host families using local and regional procurement WFP 12/14- 7/16 World Food Program (WFP). (2016). APS USAID FFP/202(e). Final results report form. PRRO 200550. Retrieved from INFO MISSING 3 government-designated camps: Bahn refugee camp (Nimba), PTP camp (Grand Gedeh) and Little Wlebbo refugee camp (Maryland). NOTES: Planned to reach 30,000; In 2015 Q4 reached 29,558 (99%). Cost per beneficiary $USD 96.71. Activity promoted inclusion of women and girls by requiring at least 50% females on food management. committees; and food transfers handed to female HH members when possible. UN World Food Programme emergency school feeding in EVD￾affected areas WFP 4/15- 10/16 World Food Program. (2015). Ebola response from crisis to recovery. Retrieved from http://documents.wfp. org/stellent/groups/public/documents/ communications/wfp276313.pdf Bomi, Gbarpolu, Grand Bassa, River Cess, Grand Gedeh, Sinoe, River Gee, Grand Kru, Maryland, Nimba. NOTES: “WFP provided daily hot meals to school children, and take-home rations of vegetable oil to girls, both to meet emergency food security needs and to incentivize parents to send their children, especially girls, back to recently reopened schools.” (FFP Fact Sheet, Dec 2016) “In Liberia, WFP resumed its school meals programme in April 2015, providing more than 100,000 schoolchildren with a nutritious meal each day. By the end of 2015, it aims to reach 127,000 children from 720 schools. WFP also plans to provide take-home food rations to 5,000 schoolgirls.” (WFP, 2015) ANNEX M. DESK REVIEWS – LIBERIA n M–81 Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Health MEASURE Carolina Population Center/ UNC 7/14– 6/19 Hart, L, Street, D and Kulatilaka, H, 2017. Building Capacity for Resilient Health Systems. Lessons Learned from Sierra Leone, Guinea and Liberia during the Ebola outbreak. July-Sept 2017 (Year 3 Quarter 4) Quarterly report for Monitoring and Evaluation to Assess and Use Results (MEASURE) Phase IV Projects for Ebola￾funded Activities. For 3 months, 2 embedded M&E experts followed by 6 months remote technical support. The activity was “regional” in scope and was implemented in Liberia, Guinea and Sierra Leone. In Liberia, DHIS 2 was already a functional platform when Ebola hit. Pillar II funds were allocated to ensure that the DHIS 2-based HIS was resilient enough to accommodate the sheer scale and scope of the Ebola outbreak. NOTES: Objective: to assist the MoH to (a) assess the HIS system; (b) develop a strategic plan to improve the HIS. [Due to the short timeframe of the activity, it did not support implementation of the plan]. Very weak summary paper – virtually content free. They implemented assessments of the HIS, and the ICT infrastructure, adapting and building on existing tools and frameworks, through a participatory approach. Based on the assessments, they developed “strategic objectives,” recommending “changes to their resource use, developing and disseminating policies, establishing infrastructure, establishing working groups, data sources and indicators,” data quality (along with its management, dissemination and use), and HIS strategic plan M&E. PACS (Partnership for Advancing Community-Based Services) IRC, with Global Comm; PSI, national Planned Parenthood Association of Liberia; and YMCA 7/15- 6/17 Partnership for Advancing Community￾Based Services (PACS) Quarterly Report, April 1, 2016-June 30, 2016. Mod adding 7 million Ebola supplemental, raising total to $31,800,000. Indicator Sheet for Year 2, Quarter 4, Jul-Sept 2016 Bong, Lofa and Nimba, 2000 HHs; age >= 18 caregivers of children <5 (CU5). 34 sub-results under 4 broad results, to launch or strengthen health management structures (CHTs, coordination meetings, embedding BCC staff); train and support cadres of CHWs, measure and promote health behavior change, specifically around WASH and malaria prevention, community mobilization for Community-Led Total Sanitation (CLTS). NOTES: 5-Year activity. Bilateral. 2/2015 – 2/2020. Consortium “to build cap of Ministry and CSOs to manage and deliver quality community-based health services, support the implementation of effective health communication strategies, and improving access to safe WASH services in Liberia. Focus: MoH, CHTs, CSOs, and community organisations “comprehensive and fully participatory approach to institutional strengthening” to “go beyond knowledge transfer and address context-specific factors… which have been overlooked or underfunded in past interventions.” (PACS KAP Survey 2015) Used “a coordinated technical assistance (TA) model that blends embedded TA with targeted short-term TA (STTA) … integrated into day-to-day work.” Focus – skills in health communications, on WASH, maternal health, and child health. For CSOs – “self-driven steps for performance improvement and eventual graduation to direct extended management.” Activity 3.1.6c includes extending Natural Leader training (WASH); GIS monitoring of coverage, and establishing a Center of Excellence in health promotion in “the additional 3” counties (Margibi, Grand Bassa, rural Montserrado). CLTS – Community-led total sanitation (community mobilization intervention for WASH). KAP Survey doc cited previous projects: Community-Led Total Sanitation (CLTS) and Natural Leader Network (NLN). KAP Finding Highlights: 70% caregivers were aged between 18-34; 50% no formal education; 35% primary education. <1% of respondents “recall hearing or seeing a specific USG supported FP/SRH message” (KAP Survey Report Table 2). Over 2/3 never treat their water, but 70% have heard of WaterGuard; only 9% reported using it. 37% of HHs have HH latrine. Handwashing: 60.7% “always” wash hands before preparing food etc. 60% said at least one gCHV resided in their community. March 2015 modification added $7m EVD funds; added 3 counties (Margibi, Grand Bassa & rural Montserrado). HRH MSCP/PST – Human Resources for Health: Maternal and Child Survival Program – Pre￾Service Training Jhpiego 4/16-6/18 Human Resources for Health Liberia Quarterly Report FY 2016 Q1 (July￾September 2016). PowerPoint: MCSP Liberia: Human Resources for Health. Strengthening Pre-service Education Registered Midwifes (RMs) and Medical Lab Technicians (MLTs) [date unspecified]. GHPro 2017: Midterm Evaluation Maternal And Child Survival Program (MCSP) August 2017. Evaluation Assignment Number: 341. A flagship program of USAID’s Bureau of Global health. Priority counties are Bong, Grand Gedeh, Lofa, Montserrado. Supported schools in Grand Gedeh, Bong, Lofa, Montserrado, [and Margibi? MPCCH SUMU). 70% of funding (x all countries) is from field support. NOTES: Aims to “strengthen the capability and resilience of Liberia’s frontline health workforce to address second-order impacts from the Ebola crisis and provide safe, quality and respectful services.” 7 key activities: Rapid needs assessment; Upgrading teaching and learning curricula incorporating post-Ebola; Strengthen skills, computer and science labs; Use PSE quality improvement standards; Technical updates and teaching skills-building for faculty and preceptors; Facilitating linkages with MoH, professional/regulatory bodies, and clinical sites; Development of leadership and management skills. HRH “mirrors the MoH plan” – to achieve “a fit-for-purpose productive and motivated health workforce providing quality care.” Objective 1: “Increase the quality of instruction at targeted pre-service training (PST) institutions [upgrading technical and teaching competencies; and strengthen curricula, course materials, and deliver. Objective 2: “Strengthen the learning environment at targeted PST institutions and teaching sites” through improved instructional resources, equipment and technology. Proposed a phased HR plan with expected minimum and maximum staffing in each of 18 cadres at Health Centers and District Hospitals in Phase 1 (1-3 years) and Phase 2 (4-10 years). Expected results include: Increased sustainability of investment for the health workforce production pipeline through improvements in management and administrative capacity of training institutions. Identified challenges with infrastructure, commodities, accessibility [poor road and bridge conditions], staffing; linking communities to facilities where PACS is not covering; connectivity – lack of electricity and cell phone coverage. M–82 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope MCSP/RHS – Restoration of Health Services in Liberia Jhpiego 8/15-6/18 MCSP/RHS Quarterly Report July-Sept 2017. 60 facilities in Lofa, Grand Bassa and Nimba. 54 public and 23 private health facilities. Activity in Jul-Sept 07: - Paid incentives to 124 staff in 3 counties “ensuring continuity of high quality MCH services in supported health facilities.” - Continued infrastructure work in 43 facilities - Provided equipment and IPC supplies - In-service training to 200 HCW in INMCI and Maternal & Newborn Death Surveillance and Response - Supportive supervision with MoH in the 77 supported HCFs - M&E and learning: WASH study, and studied integrated EPI-FP approach in 12 intervention and 4 control sites NOTES: Objective 1 [Prevention at facilities]: IPC practices at 77 health facilities are strengthened through training, intensive supportive supervision, triage, improvement of waste management and planning and management of essential IPC commodities and supplies Objective 2 [Increase MCH Service Demand and Utilization through Restored Service Delivery]: Demand is generated and delivery of quality primary healthcare services is restored through the implementation of integrated reproductive, maternal, newborn, child, and adolescent health as part of the Essential Package of Health Services in 77 facilities. APC ETP&SS: Advancing Partners and Communities – Ebola Transmission Prevention and Survivor Services JSI 2016- 6/2018 ETP&SSS Strategy: October 2016; updated July 2017. (Contains logframe with list of indicators & modes of verification). JSI. Advancing Partners &n Communities. Baseline Ebola Survivor Assessment, February 2017. 4 Counties with highest concentration of EVD Survivors: Montserrado, Lofa, Bong, Margibi Partners: ELWA Hospital; Liberian College of Physicians and Surgeons; and Carter Center Three Objectives: (1) Strengthen coordination and management of EVD survivor support activities per national policy; (2) Improve access for EVD survivors to needed services and meds, and combat stigma and discrimination by providers; (3) Build capacity in public sector to provide specialized services, especially mental health/psychiatry/clinical psych; rheumatology; and ophthalmology. Activities include: - Grant to ELWA Hospital to fund EVD survivor access to free health care [with limits]; - Capacity building to EVD Survivor Network and staff seconded to Secretariat/MoH; -Training & equipment for rheumatology, ophthalmology. & mental health; -Training in stigma reduction and improved IPC; - Targeted infrastructure and equipment support to facilities serving large EVD survivor populations. Baseline assessment: 433 EVDS (205 male, 228 female) over age 18, identified through HCFs. NOTES: GHET-funded activity to support MoH’s Ebola Survivors Care and Support National Policy “to address the risk of resurgence associated with possible extended Ebola virus persistence in survivors; ensure the effective delivery of healthcare psychosocial services [...per EVDS needs]” and direct assistance to clinical and mental health services within the public health system.” Aims to strengthen National Secretariat of EVD Survivors (GoL) and National Ebola Survivors Network (CSO). Planned for “beneficiary information channel that would allow for questions to be asked, concerns to be raised, and corruption to be reported by program beneficiaries.” Also plan on Organizational Capacity Assessment at baseline and endline. Baseline assessment found that the principal barriers to care are transport costs and distance to health facilities. Costs of treatment (especially drugs) and wait times were also reported. “Stigma was not listed as a major barrier to care, despite 33 percent of respondents claiming two or more occurrences of stigmatization by health facility staff in the past six months.” (Baseline assessment p. 5). Only 26.9% of male respondents reported regular condom use with all sexual partners. DELIVER (TO #4 - SCM) JSI 9/10-2/17 Deliver Project. 2016. Final Country Report. Liberia. Arlington, VA: USAID |Deliver Project, Task Orders 4 and 7]. Five focus counties Four objectives: (1) Support MoH quantification and supply planning; (2) Work with National Drug Store (NDS) to improve warehousing and storage; (3) Procure malaria and FP commodities and essential medicines; (4) Operate a “top-up distribution system” in the NDS in five focus counties. Key interventions: (1) TA to MoH to develop “20-year Supply Chain Master Plan” (revised after an assessment in 2015); (2) Capacity building at NDS and MoH Supply Chain Management Unit; (3) Develop “national quantification committee” to harmonize & coordinate procurement & supply plans; (4) Design & implement new “top up distribution system (the “Interim Approach) to improve accountability of deliveries. Advocacy and policy dialogue to create more enabling environment for “commodity security”, with NMCP, USAID, CHAI, SCMU, NDS, OFDA Supply Chain Technical Working Group for FP/RH and malaria. ANNEX M. DESK REVIEWS – LIBERIA n M–83 Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope NOTES: DELIVER PROJECT works in partnership with MoH and other organizations “to improve health outcomes by increasing the availability of health supplies”. USAID invested in Liberia’s health programs by supporting the design, implementation and management of a new supply chain system for health commodities. “End-to-end support from the central level to the last mile.” (p.9) Contributed to MoH “reproductive health commodity security strategy.” Provided 3.2 CYP; With top-up system, restored reliable supplies of contraceptives (cited 35.7% unmet need). Joined Logistics Cluster in 2014 and included forecasting, storage and distribution support for EVD commodities. Rolled out the Interim Approach (vendor-managed top up system) in five focus counties (where 70% of Liberian population), and GFATM covered the other 10 counties. This arrangement provides national coverage (657 facilities in all 15 counties). 2015 assessment of failures of the supply-chain management program led government to outsource NDS and SCM for a period of time. Stockout rates declined from max. of 93% in first round to 5% in 9th round in 2016 in USAID-supported counties. [In KII- ask JSI what caused the WORD MISSING. Expiry of drugs in NDS reduced from 7% to <2% (p 18). Established multi-partner supply chain TWG, provided members training in quantification and supported annual quantification exercise. HC3 – Returning to RMNCH Services: Rebuilding Trust and Increasing Demand through SBCC JHU-CCP 9/12- 9/17 Modarres, N., Berg, K. (2016). Qualitative assessment on health system trust and health service utilization in Liberia. Health Communication Capacity Collaborative, Baltimore: JHU CCP. 2 Indicator tables HC3 launched in Liberia in Oct. 2014. Played a key role in GoL’s Incident Management System. Collaborates with and supports National Health Promotion Division and Community Health Services Division. Some activities are national in scope; others concentrate on counties where partner IPs are strengthening health services (e.g., MCSP – strengthening HFs in Lofa, Nimba and Grand Bassa). NOTES: Provides key inputs for research, M&E, messages and materials development, training, media and documentation, radio, country-level coordination, and community engagement. Serves as chair of Health Promotion TWG, and serves on Community Health Services TWG. Supports stronger “health promotion presence” in County Health Teams as they “better links the county, district, health facility and community levels to competent behavioral expertise, effective messaging and materials, and improved skills to implement, monitor and evaluate community activities focused on demand creation and promotion of healthy behaviors.” (HC3, 2015, SOW, p. 3) Produces series of 30-minute distance learning modules for gCHVs, based on updated curricula developed by NHPD, PACS, CSH, etc., with primary audience, CHVs, but secondary audience is communities, learning what to expect, and to demand. Promotes trust and demand for EPHS through national, multi-channel, multi-phase mass media Healthy Life campaign. “There were no major indications that community members do not trust healthcare providers or that they are scared to return to health facilities since the time of Ebola.” (HC3 Qualitative Trust Assessment). Barriers to use of HFs: “Barriers include access to clinics, fees to be seen by a HCW, and time lost if there is no medicine available”. Also concerns about attitudes of HCWs, esp. younger/recently trained – tend to be impatient. “They need to learn how to talk to people.” Mothers first try to treat <5 children at home w/ store-bought drugs (e.g., paracetamol); they go to HF if child’s condition worsens. Same for care seeking during pregnancy. Women go to hospital for emergencies; they may wait until too late. Collaborative Support for Health (CSH) MSH 2/15– 2/18 USAID/Liberia CSH Mid-term Performance Evaluation report (by LSA) June 2017 (MTPE 2017) $21.5 million, with two option years (to Feb. 2020) for $10.9 million County Focus: Bong, Lofa, Nimba. Seven components: Leadership and governance; water supply mgmt.; QA/ QI; HRH mgmt.; supply chain mgmt.; healthcare financing and health mgmt. information. Official launch was May 2015. USAID “provisionally approved CSH’s 1st work plan on July 9, 2015, “but it continued to evolve during the first year.” LSA review of workplans identified 126 minimum required activities for the 7 objectives and 28 sub-objectives; expanded (in response to demand) to 182 activities for FY16 and 180 for FY17 NOTES: Builds on previous Rebuilding Basic Health Services (RBHS) activity (2008-2015) “Project start-up was affected by the Ebola epidemic, which modified the context for and affected the pace of execution.” Foundation – WHO’s 6 building blocks of HSS (1) deliver essential services; (2) health workforce; (3) HIS; (4) access to essential medicines. LSA/CSH evaluation found: “Assessment of management capacity shows that progress has been neither uniformly positive in the CSH counties nor markedly different from the non-CSH counties.” (p. 11) LSA/CSH concludes the Health Care Finance, Human Resources, and HIS components of CSH have been most successful. Value added of CSH (to FARA) is strengthening the “fiduciary capabilities” of MoH to ensure CHTs “utilize Govt. funds properly and transparently for their intended purpose” and account for govt. funds as required. MTE KIIs identified “35 priority interventions” where CSH made improvements, but more at county than at central levels. LSA cites CSH as “process driven” and demand driven, rather than results driven. Too much responsiveness to MoH needs and USAID demands. Cites “tendency to over-promise (based on its annual work plans) and under deliver (based on its available resources).” (MTE p. 14) Recommends redesign with “more focused objectives and fewer activities” and “better accountability for results (including clearer agreement on measures of project success).” (p. 14) Also recommends: -Identifying and planning for “short, medium, and long-term gains” which will enable better sequencing. -“Must increase its engagement with MoH’s Senior Management Team in approving the annual work plan and in communicating the results of the activities.” LSA found problems with M&E (lack of data for 30% of program indicators; concerns about quality of QA/QI data, when some data was reanalyzed on the dashboard. Also, concerns about the M&E framework - disconnect with ToC which wasn’t changed when indicators were changed to align with Mission RF. Findings on capacity development used RBHS tool so could compare 2012, 2014 and 2017. But poor results; and “CSH does not adequately define capacity at the county level” – nor show how improvements at central connect with changes at county level. M–84 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Strengthening Routine Immunization in Liberia￾under Ebola Non-Emergency Health Services/ Immunizations UNICEF 7/15– 12/16 Final SOW All 15 counties targeted. Aims to: (1) provide support for the strengthening of immunization services at all levels through printing and distribution of EPI data tools – child health cards, ledgers etc.; (2) ensure availability of vaccines of assured quality in adequate quantities at all levels; (3) strengthen coordination and supportive supervision, monitoring and evaluation at all levels; (4) support effective communication in support of EPI services including introduction of new vaccines. (IPV, Rota, HPV) NOTES: BAbout 160,840 children aged 0-11 months will receive all routine vaccines such as BCG, Polio, Penta, yellow fever, measles and about 160,840 pregnancy women will receive TT vaccine. - Rumor issues addressed and the demand creation of vaccination is increased. - Regular routine services continue to be provided. Mobile Solutions Technical Assistance and Research Program (mSTAR) – Liberia Mobile Money FHI 360 9/12– 9/17 FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 1 (10/1/2016-12/31/2016). FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 2 (1/1/2017-03/30/2017). FHI 360. (2017). Mobile Solutions Technical Assistance and Research (mSTAR), Liberia. Quarterly Report, FY 2017, Quarter 3 (4/1/2017-6/30/2017). FHI 360. (2016). Understanding the Potential Role of Mobile Money in Civil Servant Payments in Liberia: A Contextual Analysis. Pre-Ebola activity; received Pillar II funds to extend mobile money salary payments (operational for MoE employees) to MoH employees. Target counties: Bomi, Bong, Gbarpolu, Grand Bassa, Grand Gedeh, Grand Cape Mount, Lofa, Maryland, Nimba, River Cess. NOTES: Through Q3 FY17, operational for MoH employees in four counties. 119 out of target of 3,600 MoH employees enrolled in the mobile salary payment service. Innovation, Partnerships & Technology Partnership with Project Last Mile (PLM) Global Env. and Tech. Fndn., with The Coca￾Cola Co., The Coca￾Cola Fndn., Global Fund to Fight AIDS, TB and Malaria USAID and BMGF 6/14– 6/19 No documents available; information in this table obtained from USAID website. Focus is on supporting the MoH; thus the scope and “impact potential” is national. Providing technical assistance and capacity building support to the Liberian MOH in order to build long-term resilience of the system. PLM developed a Detailed Consultant Implementation Plan (Inception Report) that was consistent with the broad Transition implementation strategy and as developed and mutually agreed with the MOH and USAID NOTES: USAID. (ND). Project Last Mile-Liberia/Sierra Leone. Retrieved January 5, 2018, from https://partnerships.usaid.gov/partnership/project-last-mile-liberiasierra-leone. “This partnership will harness The Coca-Cola Company’s proprietary supply chain expertise to help the Ministry of Health improve its ability to deliver medicines to rural health facilities in partnership with USAID’s flagship supply chain program. This partnership will improve management of medical supply distribution in Liberia & Sierra Leone.” -USG Investment $ 3,000,000 Resource Partner: The CocaCola Company, The Coca-Cola Foundation, Global Fund to Fight AIDS, TB and Malaria, and BMGF. ANNEX M. DESK REVIEWS – LIBERIA n M–85 Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Ebola Recovery Connectivity Initiative (ERCI 2.0) Inveneo 10/16– 10/18 ERCI 2.0 FY 2017-18 Workplan (submitted September 6, 2017). According to the ERCI workplan, the activity is implemented by a private sector entity (Inveneo) and “prioritizes connecting institutions with broadband internet that serve the public good.” (p. 6) A participatory approach is supposed to be used to select rural sites for internet connection. NOTES: There are two main objectives: (1) Expand ability to utilize connectivity to communicate potential health threats; (2) Strengthen capacity of ICT market. A number of qualitative and quantitative studies and assessments are planned over the life of the activity (see Evaluation and Research Plan included in the workplan, p. 20). Governance Governance and Economic Mgmt. Suppt. (GEMS) (not Pillar II, though another activity with this IP and Chemonics built on this activity) IBI Intl 6/11– 6/16 USAID/Liberia Governance and Economic Management Support (USAID-GEMS) Project. Final Report: July 2011-June 2016. GEMS provided TA and capacity building in 17 GoL Ministries, Agencies and Commissions (MACs) to improve their ability to perform their public-sector functions. Focused on and developed a three-year strategic plan for the Civil Service Agency (CSA), to establish a core agency that could sustain and drive improvements in other MACs. Relative to 2012 baselines, values on GEMS indicators in 2016 EOP – 25% exceeded; 47% met; 11% significantly met; 7% fairly met; 2% had “poor performance” and 8% were not met. GEMS reported the 8% included building management and maintenance indicators in Assets Management (under GSA), and noted that GSA had been focused on other aspects due to and after the EVD crisis. In TA to the Civil Service Agency, a competency assessment found 55% of employees have “the qualifications, skills and competencies required for the job they hold. In one key directorate, only 33% of staff had the requisite requirements and in one, none did. This will continue to limit the ability of the CSA to undertake internal reforms…” (GEMS final report, p. 27) NOTES: Excellent lessons learned section, pp. 25-30, with some issues specific to Liberia, and others of general relevance. e.g., - Liberia - GEMS trained the CSA (Civil Service Agency) with emphasis on writing clear job descriptions. But, the people trained in CSA were not the staff in the Employment Services Directorate (ESD) that does job descriptions; The capacity of ESD staff was “far below what was required for the exercises.” - General: “Patience, persistence, and perseverance matter in dealing with the public sector.” (p. 25) e.g., senior staff initially avoided GEMS trainings, but “Because the advisors persisted, the MAC staff eventually made themselves available to learn new approaches to delivering their work. (p. 25) Also – the importance of ownership and trust of senior leaders, which is founded on developing genuine in-depth understanding of the problem. “This process takes an enormous amount of consultant time and willingness to persist.” (p. 25). And “In transformational change reform initiatives, it must be acceptable to build the foundations and proof of concept for others to continue with the details, because these could take decades to implement and are beyond the scope of a single project.” (p. 26) And in multiple sectors, and multiple MACs, GEMS found a mismatch between the staff capacities and the resourcing, and the responsibilities of the positions or agencies. e.g., efforts toward e-government, but internet service is extremely expensive – ($700-$1000/mgbit/Month). Civil Society and Media Ldrshp. Program (CSML)--$2.2 million cost extension (CE) for Acctby. for Ebola IREX 2/10– 11/15 11 counties, and Monrovia. CSML is designed to strengthen CSOs and the independent media as a means to include and engage the Liberian population, thereby contributing to the creation of rapid, inclusive, peaceful, and sustainable democratic development. IREX Search for Common Ground (SFCG) used CSML-CE money for “Recovery Together” initiative, combatting stigmatization of EVDS and their families, in Montserrado , Margibi (Liberia Youth Network (LIYONET), Bong (Catholic Justice and Peace Commission CJPS), Lofa (Community Safety Initiative, CSI), Nimba (Effective Advocacy to Restore Stability for the Masses – EARS), and Grand Cape Mount (Transitional Justice Working Group). It trained project managers of these projects, along with 21 station managers of community radio groups, to discuss reconciliation and memorialization of people lost to EVD, and develop memorialization activities. 953 people attended the 18 forums to express community concerns. The most commonly raised concerns: care of Ebola orphans, poor school quality, and the need for memorialization. M–86 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope NOTES: IREX defines accountability as “the obligation of the government of Liberia and other stakeholders, including donors and INGOS that make commitments, to the people of Liberia to account for their activities, accept responsibility for them, and to disclose the results in a transparent way.” (CSLM Final Report: 80) IREX trained 99 journalists from Monrovia and community radios x the 15 counties, on “how to report on the impact of Ebola funds and resources earmarked for Ebola, including developing story ideas, making connections with organizations who could explain the uses of the funds, and fact-checking before reporting. Created www.trackingaidliberia.org platform, with database of EVD funds. 20 field researchers in the 15 counties & 19 Monrovia reporters filed reports on YouTube & Facebook pages. They raised questions about the $ spent on ETUs that were never used. The activity extended the Community Leaders Forums approach, with cascade training through 13 partners, reaching 688 community leaders in 41 communities in 11 counties, “to understand that accountability of the government to its citizens is considered a basic human right.” This involved recasting questioning leaders from disrespecting or undermining them, to simple exercise of their rights, and a way to improve services. Officials questioned included development superintendents, district commissioners, paramount chiefs, mayors, elected representatives to the legislature and heads of Ebola Task Forces. Material and technical support to 19 community radio stations raised coverage from 43% to 94% of the Liberian population (p. 95). Of 27 indicators reported for the cost extension, 12 were exceeded, 10 were met, 2 substantially met, and 3 were not met. This was a very successful program. Areas where “work is required” - Little progress on gender balance and gender responsive content, although more women’s voices were included; - More emphasis on assisting government officials to permit if not welcome questioning from constituents; Recognizing the culture change this entails (from disrespect to citizens’ rights). Basic Education Education Crisis Response in Liberia (ECRL) UNICEF 2/15– 7/17 UNICEF. (2017). ECRL Fifth Progress Report Submitted to USAID, June 2017. UNICEF Liberia. (2016). Education Crisis Response for Liberia (ECRL). Revised Proposal with Reprogrammed Activities for USAID, February 2016. The broad objective of the ECRL program is to support MoE to effectively and systematically provide safe, protective and proactive learning environments for schools in all 15 counties of Liberia. There are more specific objectives, as well. Objectives 1 – 2 are national in scope. Objective 3, WASH in Schools – is in 120 schools in most affected counties: 25 schools in Bong, 25 in Grand Cape Mount, 25 in Lofa, 25 in Margibi, and 20 in Montserrado). Education Section carried out rapid assessments of County Ed. Officers (CEOs) and District Ed. Officers (DEOs) in the 15 counties and 98 education districts to assess their capacity in monitoring, supervision and ensuring sustainability of completed facilities. Education Section is preparing a training module to include WASH and interventions are being designed to build their capacity on monitoring, evaluation and reporting. Document notes that training of CEOs and DEOs to be conducted in Jul–Sept 2017. NOTES: Three primary interventions: (1) procurement and distribution of TLM to 4,460 schools; (2) training of 10,000 teachers and 5,000 PTA members in pedagogy and PSS; (3) IPC through WASH in Schools (WinS) interventions. Delivered Teaching and Learning Materials to 4460 Liberian schools across the country. Reached 517,249 primary school children (5% over-target) with “school in a box” “kits” delivered to the school. Since 2015, 10,064 teachers have been trained on “psychological first aid,” preparing them to handle children affected by EVD and other trauma, including GBV. In 2015 and 2016, ECRL directly trained 4,729 chairs of PTAs on the roles and responsibilities of a PTA, the Teachers’ Code of Conduct and psychological first aid. Of 14 indicators in the activity’s M&E plan, in mid-2017, 3 were met; 9 were reported On Track, and 2 were Delayed (re: IR5 – training and accreditation of Community and District Education Officers in M&E of “WinS”. ANNEX M. DESK REVIEWS – LIBERIA n M–87 Table M–AB. Activities with information in the SBU files, Liberia Title Prime IP Dates Documents Scope Economic Crisis Mitigation Liberia Municipal Water Program (LMWP) TetraTech 2 actvies 2012-16 active called LMWPII LMWP II Combined Year 1 and Quarterly Report 4 (dated October 30, 2017). LMWP has worked in three urban municipalities: The activity aims to rehabilitate the piped water system in up to four county capitals: Voinjama, Robertsport, and Saniquellie. NOTES: The annual report presents indicator targets but no data on actual LMWP II Year 1 achievements vis-à-vis performance indicators. Nevertheless, based on reported Year 1 progress, LMWP is making progress in relation to sustainable financing (see Figure X on previous page). The activity generally exceeded its Year 1 targets related to the non-subsidized WASH infrastructure operating expenses. In addition, there have been a number of process achievements (e.g., Institutional Framework Review and development and finalization of a Roadmap Plan) related to sustainable management and cost recovery (Tetra Tech, 2017). In addition, the number of action plans implemented for water security, integrated water resource management, and/or water source protection as a result of USG assistance increased from 0 at baseline to 6 by the end of Year 1. Grand Challenge mHero: Fighting Ebola Grand Challenge: mHero – Leveraging HIS and Mobile Phones to support Frontline Health workers amid the Ebola Outbreak in West Africa Intra Health Intl. 6/15– 6/16 USAID’s Fighting Ebola Grand Challenge: mHero. Quarterly Report, June 6-Sept. 5, 2015. Regional activity, but intensive activities in Liberia. National/central-level in scope. Baseline assessment – Oct 2015. Data were collected from MoH staff in Monrovia and nationwide. NOTES: mHero is a regional activity that supports Ministries of Health to communicate with front line field workers using mobile phone technology. It is one of the 14 Grand Challenge awards selected by USAID’s Innovation Lab. The award included: - Support for global level investment in the mHero website, to improve users’ access to content - With MoH/Liberia design of “workflows” on Family Planning to enable MoH to gather FP/RH information from health workers, to aid in planning activities to increase contraceptive use. (workflows- on Family Planning Counseling, Method Mix for Family planning; Family Planning Training, and Barriers to Family Planning). - Baseline assessment – phone survey with health workers, re mobile phone use, “market saturation with health-related text messages” and exposure to mHero. - Embedded 2 FT staff in MoH to support iHRIS, including cleaning MoH personnel records – which are key to mHero system, and which lapsed during the EVD crisis. - Baseline assessment was based on 335 interviews with HCW and in-depth phone survey with 25 of these, and survey with MoH staff. - 8 “use cases” were initiated, reaching 1,500 recipients. They addressed: health worker validation; IT electrical assessment at HCF; IT infrastructure at HCF; Mental health services and beneficiaries; Nutrition/anthropometric tools in HCF; HCW status in prisons; IPC and other stock levels in prison HCFs; Identification of focal persons for warehouse management in each Country Health Team. The system is designed for “back end interoperability” to enable linkage with other existing systems (e.g., Dimagi’s CommCare; HISP on DHIS-2) and gleaning and sharing of relevant content. MoH has adopted mHero, including it in the Health Information Systems, Research and Communication Systems Annual Workplan for 2016-2017, Liberia. M–88 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–AC.1. Population by county urban/rural (from LISGIS 2008 updated, via for PE1 sampling plan) Rural Urban Code County Name Male Female Total Male Female Total 03 Bomi 34,626 33,062 67,688 8,314 8,117 16,431 09 Grand Bassa 82,060 80,970 163,030 28,853 29,810 58,663 21 Lofa 120,494 129,760 250,254 13,117 13,492 26,609 24 Margibi 61,854 58,701 120,555 43,986 45,382 89,368 30 Montserrado 41,478 40,636 82,114 508,255 527,872 1,036,127 33 Nimba 202,232 202,865 405,097 27,881 29,048 56,929 SUB-ANNEX C. POPULATION SIZE AND HOUSEHOLD DENSITY OF PRIORITY AREAS, LIBERIA Table M–AC.2. Households per county Code County Name Counter 03 Bomi 20,508 06 Bong 69,810 09 Grand Bassa 47,440 12 Grand Cape Mount 23,950 15 Grand Gedeh 18,143 18 Grand Kru 8,969 21 Lofa 49,642 24 Margibi 45,095 27 Maryland 19,254 30 Montserrado 232,585 33 Nimba 80,734 36 River Cess 13,981 39 Sinoe 15,829 42 River Gee 9,822 45 Gbarpolu 14,533 TOTAL 670,295 ANNEX M. DESK REVIEWS – SIERRA LEONE n M–89 Photo by USAID SIERRA LEONE This report was made possible by the support of the American people through the United States Agency for International Development (USAID). It was prepared by Suzanne Essama-Bibi and Naomi Rutenberg of International Business and Technical Consultants, Inc. (IBTCI). M–90 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES SIERRA LEONE TABLE OF CONTENTS ACRONYMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–92 EXECUTIVE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–94 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–94 Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–94 Insights on the Four Evaluation Question Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–95 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–95 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–95 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–96 Overview of the Epidemic and its Impact on Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–96 Major Social and Economic Impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–97 Recovery Response 2015–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–98 APPROACH TO DESK REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–100 The USAID Ebola Pillar II MEL Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–100 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–100 PROGRESS WITHIN THEMATIC AREAS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–102 Health . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–102 Agriculture and Food Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–108 Governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–113 Innovation, Technology and Partnerships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–114 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–117 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–117 Resilience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–118 Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–118 Opportunities and Gaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–121 Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–121 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–122 SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–123 ANNEX M. DESK REVIEWS – SIERRA LEONE n M–91 TABLES Table M–SL1. Selected human development indicators, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–96 Table M–SL2. Sierra Leone Pillar II activities included in this desk review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–101 Table M–SL3. Pillar II activities in the health sector, according to activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–103 Table M–SL4. Pillar II activities in the agriculture and food security sector, according to activity . . . . . . . . . . . . . . . M–108 Table M–SL5. Changes in selected outcome indicators of household food security in the RESSNER program districts, Sierra Leone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–111 Table M–SL6. Distribution of key economic activities by gender of household head (HHH) . . . . . . . . . . . . . . . . . . . M–111 Table M–SL7. Fighting Ebola “Grand Challenge” innovations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–115 FIGURES Figure M–SL1. Distribution of EVD cases and deaths, 2014 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–97 Figure M–SL2. Analytical framework to measure sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M–120 M–92 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ACDI/VOCA The Agricultural Cooperative Development International & Volunteers Overseas Cooperative Assistance AEU Africa Ebola Unit AFDB African Development Bank AfP Agenda for Prosperity AFR Africa Bureau AFS Agriculture and Food Security APC Advancing Partners and Communities APS Annual Programming Statement CARE Cooperative for Assistance and Relief Everywhere CCT Conditional Cash Transfer CDC U.S. Centers for Disease Control and Prevention CFSVA Comprehensive Food Security and Vulnerability Analysis CIC Community Identification Committee CLA Collaborate, Learning, and Adapting CHO Community Health Officer CHP Community Health Post CHW Community Health Worker COMAHS College of Medicine and Allied Health Sciences COR Contracting Officer Representative CPES Comprehensive Program for EVD Survivors CRS Catholic Relief Services CSO Civil Society Organization CT Cash Transfers CTP Cash Transfer Program CWG Cash Working Group DDI Dietary Diversity Index DEC Development Experience Clearinghouse DFID UK Department for International Development DH District Health DHA WHO Digital Atlas DHMT District Health Management Team DOHC Directorate of Primary Healthcare DQA Data Quality Assurance EBODAC Ebola Vaccine Deployment Acceptance and Compliance [Project] ECM Economic Crisis Mitigation EFSP Emergency Food Security Project ESCC Ebola Survivor Care Consortium ET Evaluation Team ETP&SS Ebola Transmission Prevention and Survivor Services ETU Ebola Treatment Unit EVD Ebola Virus Disease FAO Food and Agriculture Organization FCS Food Consumption Score FEEDS Food for Emergency Ebola Virus Disease Support FFP Office of Food for Peace FHCI Free Healthcare Initiative FMCs Facility Management Committees FEWS Net Famine Early Warning System FSP Financial Services Providers GoSL Government of Sierra Leone HC3 Health Communication Capacity Collaborative [Project] HDDS Household Dietary Diversity Score HED Health Education Division HH Household HHS Household Hunger Scale HIS Health Information System HMIS Health Management Information System IBTCI International Business and Technical Consultants, Inc. ICT Information and Communication Technology ICCM Integrated Community Case Management iHRIS Integrated Human Resource Information System IMNCI Integrated Management of Childhood and Newborn Illnesses IOM International Organization on Migration IP Implementing Partners IPC Infection Prevention and Control ITP Innovation, Technology and Partnership IU Implementation Unit JSI John Snow Institute KII Key Informant Interview KM Knowledge Management KSLP King’s Sierra Leone Partnership MADAM Mankind Activities for Development Accreditation Movement MAFFS Ministry of Agriculture, Forestry, and Food Security MAM Moderate Acute Malnutrition MCC Millennium Challenge Corporation MCH Maternal and Child Health MDA Ministry, Department and Agency M&E Monitoring & Evaluation MEL Monitoring, Evaluation, & Learning MCHP Maternal and Child Health Post ACRONYMS ANNEX M. DESK REVIEWS – SIERRA LEONE n M–93 MCP Management, Coordination and Partnership MHN Mental Health Nurse MoFED Ministry of Finance and Economic Development MoU Memorandum of Understanding MOHS Ministry of Health and Sanitation MSWGCA Ministry of Social Welfare, Gender and Children’s Affairs NaCSA National Commission for Social Action NGOs Non-Governmental Organizations NHIS Sierra Leonean National Health Information System NSPP National Social Protection Program OPHI Oxford Poverty and Human Development Initiative OU Operating Unit PDM Post-Distribution Monitoring PDT Presidential Delivery Team PE Performance Evaluation PHEIC Public Health Emergency of International Concern PHU Peripheral Health Unit PSP Private Service Provider PUDR Progress Update and Disbursement Request rCSI Reduced Coping Strategy Index RESSNER Rapid Ebola Social Safety Net and Economic Recovery [Program] RMNCH Reproductive, Maternal, Newborn, and Child Health RM Routine Monitoring ROSCA Rotating Savings and Credit Association SBCC Social and Behavior Change Communication SECHNs State-Enrolled Community Health Nurses SFCG Search for Common Ground SIAPS Systems for Improved Access to Pharmaceuticals and Services SLAES Sierra Leone Association of Ebola Survivors SME Small and Medium-Sized Enterprises SNAP Sustainable Nutrition and Agriculture Promotion [Project] SO Strategic Objective SSN Social Safety Net STC Save the Children UCT Unconditional Cash Transfer UN United Nations UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government VSLA Village Savings and Loan Association WASH Water, Sanitation, and Hygiene WAGs Women in Agriculture Groups WFP World Food Programme WHO World Health Organization M–94 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES EXECUTIVE SUMMARY In 2014 and 2015, Sierra Leone, one of the poorest countries in the world, experienced the largest outbreak of Ebola virus disease (EVD) in history, with an estimated 14,122 confirmed, probable and suspected EVD cases and 3,955 deaths as of November 11, 2015 (WHO, November 2015). The EVD outbreak crippled economic growth, and reversed many of the country’s development gains, particularly in the agricultural, health, education, and economic sectors. This report is based on a desk review of key efforts funded, either partially or entirely, by the United States Agency for International Development (USAID) between March 2015 and March 2017 to help the people of Sierra Leone recover from the EVD crisis and “build back better.” Methodology The desk review is part of a broader performance evaluation of USAID’s Ebola Pillar II response. Based on activity documents provided by USAID and additional information found in the public domain, information is presented according to four thematic areas: (1) Health (Non-Ebola Essential Health Services and Health Systems Recovery and Survivor Programs); (2) Agriculture and Food Security (AFS); (3) Governance; and (4) Innovation, Technology and Partnerships (ITP). Based on activity mapping conducted in 2017, the evaluation team (ET) identified 25 activities in Sierra Leone that were classified by USAID as Pillar II-supported efforts. However, one key limitation of the desk review was the ET’s limited access to Pillar II activity documentation. This report is based on a review of documents made available for 14 Pillar II activities in Sierra Leone, including three Pillar II ‘regional’ activities whose purview included, but was not limited to, Sierra Leone. To the extent that relevant information was available, the desk review provides insights on four sets of evaluation questions: (1) Performance (How are the Pillar II Ebola Recovery activities in Sierra Leone contributing to the achievement of the mission and Ebola Pillar II Strategic Framework’s objectives?); (2) Sustainability (What Pillar II accomplishments are still observable after the activities have closed, and why?); (3) Gaps and opportunities (Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and missed?); and (4) Management (What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency?) Findings Health: The main Pillar II health activities addressed efforts to enhance care and support for EVD survivors, addressing training needs of service providers and working with relevant national bodies (Sierra Leone Association of Ebola Survivors (SLAES), frontline worker support (primarily management and technical training), management, coordination and partnerships (MCP), health information system (HIS) strengthening, mobile money salary payments to MoH employees (ICT), social and behavior change communication (SBCC) to re-ignite and enhance demand for formal-sector health services, and institutional enhancements (e.g., minor health facility upgrades and renovations). In general, the Pillar II health activities were able to produce the majority of planned outputs (e.g., assessments completed, plans developed, health workers trained, facilities strengthened) and have documented some noteworthy outcomes related to health service recovery and improved performance in areas such as infection prevention and control (IPC). AFS: Social protection was a major thrust of Pillar II-supported AFS activities, with five activities implementing activities such as cash transfer programs (CTPs) through direct cash and mobile money modalities. Activities with baseline and endline data documented major improvements in food security outcomes such as household dietary diversity and household hunger scores, not just in the reach of their social protection interventions. Governance: Pillar II-supported activities in this thematic area focused on MCP and health governance interventions, as well as some SBCC interventions, to strengthen governance. Community engagement in health services monitoring has been encouraged through community dialogues. Facility Maintenance Committees serve as an interface between the health facilities and communities, and a Community Engagement Strategy and Toolkit have been developed and are used to engage local communities. Community￾based organizations, Village Saving Loan Associations (VSLAs) and advocacy groups have been supported to empower women through economic and social support activities and many food security and health recovery activities have used gender-sensitive approaches (such as targeting female-headed households to identify potential program beneficiaries. ITP: Pillar II-supported efforts in this thematic area have addressed both hardware and ‘software’ (e.g., training, quality assurance, improved interoperability of different information systems/ platforms). In health, ICT activities receiving Pillar II funds include mHero, implemented by IntraHealth International, and MicroSave ANNEX M. DESK REVIEWS – SIERRA LEONE n M–95 (a activity to strengthen financial inclusion through agent network strengthening), implemented by SIA. mHero involves a platform to allow two-way communication using mobiles to both gather data from health workers and support decision making. Additionally, the Government of Sierra Leone (GoSL) has established a Digital Financial Services Working Group to strengthen digital financial inclusion in Sierra Leone. Insights on the Four Evaluation Question Domains Performance: Pillar II activities were of short duration (30 months or shorter), and the actual implementation period was often shorter than the official award period, due to the time needed to work with GoSL and communities on activity introduction and start-up activities. Outcome data are limited, but the activities for which output data have been made available have achieved all or most of their intended outputs. Sustainability: Pillar II activities have supported national policies and/or worked with GoSL to develop them where they either did not previously exist or needed revision. Myriad Pillar II activities strengthened institutional support for particular activities by supporting GoSL efforts with state-of-the-art information, tools, and approaches, as well as capacity building. Although Pillar II activities centered on approaches with proven effectiveness, specific technical choices (e.g., provision of cash transfers (CTs) using direct cash distribution via banks versus mobile money platforms) have had implications in terms of program effectiveness and, ultimately, sustainability. Technical choices regarding how supply chain needs (e.g., procurement of necessary medicines and building supply chain mechanisms) were addressed also have a bearing on sustainability. Community engagement in health services monitoring through community dialogues, radio listening groups, and participation in health promotion activities have increased accountability at the community level. In considering how the ‘human dimension’ influences sustainability, community engagement is a common feature of Pillar II efforts, not just consultation on priorities and needs when implementation plans were being developed, but also the fostering of local ownership in implementation of specific strategies. Other contextual considerations such as climate change or sociopolitical instability may also affect sustainability, but there is limited evidence on contingency planning in light of those risks/threats that affect the enabling environment. Opportunities and Gaps: Improvements in post-harvest management practices are an opportunity to achieve sustainable food security, while opportunities for involving local private sector institutions in capacity building and productive sectors (e.g., in agricultural production, microenterprise development) were not fully explored. Management: The President’s Delivery Team on Transition and Recovery (PDTTR) coordinated recovery priorities outlined by the government and provided technical support in monitoring to all involved ministries, departments, and agencies, where they embedded its members. USAID’s efforts and those of other donors were coordinated through this mechanism. USAID’s FFP activities worked very closely with the National Commission for Social Action (NaCSA), a semi-autonomous agency in charge of coordinating and implementing all social protection interventions. Cash transfer working groups were used by FFP IPs to coordinate many of their activities. In the health sector, IPs made substantial efforts to leverage synergies, technical expertise, and other resources to avoid duplication and maximize effectiveness of activities on the ground. District councils were the main mechanism for coordination of donor-supported efforts at the district level. These councils organized district forums to review progress of program interventions and resolve issues. Lessons Learned 1. Multi-sectoral, integrated programming of cash transfer programs enhances their effectiveness. 2. Financial and social safety nets are essential for building resilience. 3. Building human capacity contributes to resilience and promotes sustainability. 4. When given proper guidance and support, Facility Management Committees (FMCs) can bridge the communication gap between local health services and local communities and improve healthcare utilization. 5. VSLAs have been an effective complementary strategy used in Sierra Leone for building the resilience of vulnerable households receiving cash transfers through the SNAP+ and other CT programs. Recommendations 1. Continued support to community engagement activities is essential for sustainability. 2. Targeted training activities designed to build specific skills are an essential ingredient for building individual and community resilience to future emergencies and should be included in any cash-based response to a humanitarian crisis. 3. Protecting access of the most vulnerable households to land and other critical resources. M–96 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES INTRODUCTION In 2014 and 2015, Sierra Leone, one of the poorest countries in the world (See Table 1), experienced the largest outbreak of Ebola virus disease (EVD) in history, with an estimated 14,122 confirmed, probable and suspected EVD cases and 3,955 deaths as of November 11, 2015 (WHO, November 2015). The EVD outbreak crippled economic growth, and reversed many of the country’s development gains, particularly in the agricultural, health, education, and economic sectors. With support from bilateral and multilateral donors, non-governmental organizations and civil society organizations, the Government of Sierra Leone (GoSL) and affected communities mounted an effective response to the outbreak. On November 7, 2015, Sierra Leone was declared EVD-free by WHO (WHO, November 2015.), and the outbreak is now eradicated. However, the challenge now is to sustain zero EVD cases, while helping Sierra Leone recover and rebuild its economy and health system. This report summarizes key efforts by the United States Agency for International Development (USAID) from March 2015-March 2017 to support Sierra Leone’s recovery from the EVD crisis and to “build back better” (USAID Africa Bureau, 2015). It also describes the context in which USAID efforts were implemented, and it presents preliminary conclusions and recommendations. Overview of the Epidemic and its Impact on Health The first documented case of EVD in Sierra Leone during the 2014-2015 EVD outbreak was on May 24, 2014, in Kenema town (the third largest town in the country), located in the Kenema district. Despite its initially slow pace of transmission, the virus began to spread quickly across communities, and the GoSL declared a state of emergency in Kailahun on June 12, 2014, which resulted in the closing of schools and other places of public gathering, and the setting-up of checkpoints along the border between Sierra Leone and Guinea. The first confirmed case of EVD in the capital Freetown was documented on June 23, 2014, and the GoSL declared a national state of emergency at the end of July 2014 (WHO, n.d., Dalan Development Consultants and Forcier Consulting, May 2016). Although the outbreak originated in rural Guinea, it hit Liberia and Sierra Leone harder, partly because it spread to urban areas in the two countries, a factor that differentiated this outbreak in the two countries from previous outbreaks of EVD in other settings (World Bank, October 2014). As part of the EVD surveillance strategy, members of the public were allowed to report any cases of illness or death that they suspected could be related to EVD. Blood samples or oral swabs were collected from any individual with clinical symptoms compatible with EVD, and from any dead person aged 5 years or older who died within 14 days of the onset of symptoms and for whom a cause of death had not been determined. The majority of the EVD cases and deaths in Sierra Leone were reported between August and December 2014, as EVD case incidence began to decline in Sierra Leone (as well as in Guinea and Liberia) following the rapid scale–up of the treatment and isolation of EVD cases and the implementation of safe burial practices in the three countries. This fall in EVD incidence marked a transition to another phase (Phase 3) of the epidemic, characterized by limited transmission across small geographical areas (WHO, November 2015). By the time the Public Health Emergency of International Concern (PHEIC) related to EVD in West Africa was lifted on 29 March Table M–SL1. Selected human development indicators, Sierra Leone Indicator Estimate Population Size 6.5 million Median Age 18.5 % of Population Living in Urban Areas 39.9% Human Development Index Ranking 179 Gross National Income Per Capita $1529 Life Expectancy at Birth 51.3 years Adult Literacy Rate 48.1% Poverty Headcount (% of population @ $1.90/day 2011 PPP) 52.3% Maternal Mortality Ratio (per 100,000 live births) 1,360 * Rate obtained using DHS module, involving a reading test, and tends to be lower than simple reports of literacy. Source: United Nations Development Programs (UNDP) Human Development Report 2016: Country Data for Sierra Leone ANNEX M. DESK REVIEWS – SIERRA LEONE n M–97 2016, a total of 28,616 confirmed, probable and suspected cases had been reported in Guinea, Liberia and Sierra Leone, with 11,310 deaths (WHO, June 2016). The impact of the EVD epidemic on the country’s economy manifested itself through many pathways, including the effects of EVD and EVD-related mortality on the labor force, its direct effect on the cost of healthcare, and the changes in behavior arising from the fear of contracting the disease. During the outbreak in Sierra Leone, borders were closed, farming was interrupted, health services for conditions other than EVD were disrupted, and schools were closed for about nine months. Seven percent of the country’s doctors, nurse and midwives were killed by EVD, and in many communities, residents avoided health services out of fear of contracting the disease. As a result, the country saw a 23 percent decrease in health service delivery, a surge in poverty, and increased illness and disease. Major Social and Economic Impacts EVD struck at a time when Sierra Leone’s economic indicators were on an upward trajectory. In 2013 the country had launched its Agenda for Prosperity (AfP), which projected that Sierra Leone in 2035 would be an inclusive, green, middle-income country (GoSL, n.d.). Growth rates in real GDP stood at 20.1 percent in 2013, up from 15.2 percent in 2012. Poverty had dropped from 70 percent in 2003 to 52 percent in 2014. In 2015, Sierra Leone was severely negatively affected by both the human cost of the EVD epidemic, as well as the impact of the epidemic on economic activity and budgetary resources. Commerce was limited, major ore industries stopped their activities, and the private sector lost about half of its workforce. The dramatic fall of iron ore prices in 2014 led to the cessation of its production and exports in April 2015, and to a significant loss of revenue from iron ore mining. The revenue shortfall coupled with higher than budgeted spending related to the EVD epidemic, contributed to deterioration in the fiscal balance (World Bank, 2015). The short-term (2014) impact of the EVD outbreak on output was estimated to be on the order of 3.3 percentage points of GDP in Sierra Leone, reducing growth from 11.3 percent to 8.0 percent; and the short-term fiscal impact was also large, estimated at USD $95 million (2.1 percent) for Sierra Leone (World Bank, October 2014). A Comprehensive Food Security and Vulnerability Analysis (CFSVA) co-sponsored by the World Food Programme, the Food and Agriculture Organization, the World Bank Group, the European Union, and the African Development Bank revealed a significant deterioration of Sierra Leone’s socio-economic and food security situation since the last CFSVA was conducted in 2010. The 2015 CFSVA was conducted between September and October 2015, at the end of the “lean season,” and at a time when the EVD epidemic had started to recede in Sierra Leone. The CFSVA results indicated that i) Incomes among rural people were significantly reduced during the EVD outbreak, and ii) A lack of access to food led farming households to frequently eat the seeds that they had intended for the planting season. 49.8 percent of households were categorized as food insecure, based on their food consumption score, and this represented a nearly 5 percent increase in food insecurity compared to the levels reported by the 2010 CFSVA (45.0 percent), and 52.9 percent of households reported that they had experienced a decrease in their income levels as a result of the EVD epidemic (WFP, FAO, World Bank, EU, and ADB. 2015). Figure M–SL1. Distribution of EVD cases and deaths, 2014 M–98 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Recovery Response 2015–2017 GOSL RESPONSE The EVD crisis exposed weaknesses and gaps in Sierra Leone’s infrastructure, systems, institutions, and services. In March 2015, as the epidemic came under control, the GoSL introduced an initial 6-9-month Early Recovery Plan focused on four priority areas: 1) Restoring basic access to healthcare; 2) Getting students back to school; 3) Protecting vulnerable populations; and 4) Restoring growth through the private sector and agriculture. The Ministry of Finance and Economic Development (MoFED) was mandated by the government to work with other ministries, departments and agencies (MDAs) to identify priority areas and consolidate those areas into a national Early Recovery Program. As the planning process progressed, the leadership and coordination responsibility were relocated from MoFED to the Office of the Chief of Staff, Office of the President, in part to strengthen interface between the program development team and the Presidency (Dalan Development Consultants - Sierra Leone, and Forcier Consultants, May 2016). The implementation of the Early Ebola Recovery Plan ended on March 31, 2016, and an independent assessment was commissioned by the GoSL to 1) Assess progress in implementing the priorities outlined in the plan, and 2) Document the Plan’s achievements, the challenges encountered, and lessons learned and best practices from the perspectives of the beneficiaries, the service providers, and other stakeholders. The assessment findings were taken into account in planning the recovery priorities and activities during the second phase of the recovery plan, covering the period from April 2016-June 2017 (with some activities extended until December 2017). In July 2017, the recovery program came to an end, and the focus has now turned to the strategies laid out in the Agenda for Prosperity Plan, which provides a roadmap for sustainable development for the country. USAID’S SUPPORT FOR EBOLA RECOVERY In March 2015, USAID established the Africa Ebola Unit (AEU) in Washington to oversee coordination of EVD-related activities within USAID, across agencies, and with the wider international community. In collaboration with other USAID units, the AEU provides support to USAID missions in affected countries to implement a robust set of development programs to address secondary impacts of the outbreak and ensure that Guinea, Sierra Leone, and Liberia are prepared to prevent, detect, and respond to future outbreaks. On September 30, 2016, AEU’s responsibilities were transferred to the Africa Bureau’s (AFR) Office of West African Affairs, which now oversees the USG’s “all of government” EVD recovery. That effort has four pillars: (I) Controlling the epidemic at its source; (II) Mitigating second-order impacts including blunting the economic, social and political consequences in the region (i.e., health services and health systems, basic education, agriculture and food security, governance and economic crisis mitigation, and innovation, technology and public-private partnerships); (III) Engaging and coordinating with a broader global audience; and (IV) Fortifying health security infrastructure in the region and beyond. Pillar II, which is the focus of this desk review, has three strategic objectives (SOs): (1) Prevent the loss of development gains; (2) Recover and strengthen existing institutions and infrastructures whose weaknesses enabled the rapid spread of EVD or slowed the response; and (3) Build sustained systems through public-private partnerships, innovation, and capacity building. Pillar II Ebola recovery objectives in Sierra Leone are aligned with those of Sierra Leone’s National Ebola Recovery Strategy (July 2015-June 2017), which are: i) Maintaining zero EVD cases through enhanced EVD prevention and survivor services; ii) Restoring access to quality basic health services; iii) Strengthening social protection services for vulnerable populations, and iii) Enhancing private sector development (USAID (a). n.d.) The recovery interventions mainly focused on health services and health systems, agriculture and food security, communication and technology, and citizen participation in health governance and oversight of drug distribution at national, district and local levels. The activities were designed to move from the response to the EVD outbreak (emergency response) to the national recovery strategy, which was implemented in two phases: 1) The early recovery phase, covering the initial six to nine months post-EVD (2015-2016), and 2) The President’s Recovery Program, covering a period of 10 to 24 months following the early phase of recovery. Some Pillar II inputs were designed and funded at the mission level, and others by USAID operating units (OUs) in Washington. OTHER SUPPORT The UK’s Department for International Development (DFID) was the first international donor to respond to the EVD crisis in Sierra Leone. The U.S. Government (through USAID) joined DFID, the World Health Organization (WHO) and other UN agencies, and a number of NGOs and charitable organizations in both the emergency response and recovery efforts to control EVD and prevent future outbreaks of the disease. DFID helped set up rapid response teams across the country to respond to the emergency, worked with Public Health England to establish three laboratories in the country, provided support for the establishment and continued operation of four Ebola Treatment Units (ETUs), and supported WHO in training CHWs in identification of potential new cases, and hundreds of healthcare workers in Infection Prevention and Control (IPC). DFID provides limited direct financial support to the GoSL, and channels most of its funding through multilateral organizations, private sector firms, and civil society organizations (CSOs) who are contracted directly by DFID to provide services or technical expertise to the GoSL in specific areas (DFID, Sierra Leone. n.d.). The USAID recovery activities in the health sector are coordinating their activities (including ANNEX M. DESK REVIEWS – SIERRA LEONE n M–99 technical assistance) with DFID to strengthen health systems and in-country capacity to prevent and respond to future outbreaks (USAID (b), n.d.). The World Bank is working with the WHO, other UN agencies, bilateral partners, civil society, and private sector partners to support the response and recovery efforts in Sierra Leone (and in the other two EVD-affected countries in West Africa). Top priorities for the World Bank (in all three countries) are to build a strong and well-trained health workforce and build resilient health systems able to deliver quality health services, even in remote rural areas, implement effective disease surveillance, and detect and respond to future disease outbreaks. The World Bank has established an Ebola Recovery and Reconstruction Trust Fund to address the economic and social impact of the EVD crisis in the West Africa region, and mobilized USD $318 million in financing for Sierra Leone (as of December 2015) for EVD response and recovery efforts. This funding is providing budget support to help the GoSL cope with the economic impact of the outbreak and finance the scale-up of social safety net programs. It is also helping to train health workers, and to pay for essential supplies of drugs, medical equipment, and other essential health commodities (including IPC materials) (World Bank, April 2016). The World Bank is also supporting private sector development through matching grants for small and medium-sized enterprises (SMEs), and (along with DFID, the African Development Bank (AFDB), UNICEF, and the Millennium Challenge Corporation) supporting the rehabilitation of infrastructure of the water supply and sanitation system (USAID (b), n.d.). The United Nations Children’s Fund (UNICEF), the Centers for Disease Control and Prevention (CDC), and the NGOs consortium are supporting the MOHS’ efforts to develop and institutionalize IPC policies, standards, and practices in healthcare facilities through training of health personnel from PHUs in IPC practices, the development of IPC policies, guidelines, and standards, and technical assistance, and supportive supervision activities (USAID (b), n.d.). The World Food Programme’s recovery operations support the GoSL’s efforts to restore livelihoods and incomes by providing targeted food and cash assistance to the most food insecure populations in the country. WFP also supports the government’s efforts to expand access to markets, strengthen the supply chain for specialized nutrition products for the prevention of chronic malnutrition and the treatment of Moderate and Acute Malnutrition (MAM) in young children, and improve the capabilities of health and disaster management authorities as well as other relevant government institutions to respond to emergencies (World Food Programme. n.d.). Given the multiplicity of actors supporting recovery, and overlapping themes across efforts (e.g., health system recovery), complementarity of effort is paramount. When examining USAID’s contributions, it will be important to consider concurrent efforts on the part of other development actors in Sierra Leone, particularly in the same geographic areas, and with the same target groups. In addition, the aforementioned efforts did not occur independent of USAID. The World Bank and UN agencies such as UNICEF, WFP and WHO also received Pillar II funds to support some of their recovery efforts (USAID Office of Inspector General, 2015). M–100 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES APPROACH TO DESK REVIEW The USAID Ebola Pillar II MEL Project In October 2016, International Business & Technical Consultants, Inc. (IBTCI) was awarded a three-year project – Ebola Pillar II Monitoring, Evaluation, & Learning (MEL) – to conduct performance and impact evaluations and monitoring activities to examine USAID-funded EVD recovery interventions in Guinea, Liberia, and Sierra Leone from March 2014 through December 2019. There are four key evaluation questions: (1) Performance: How are the Pillar II Ebola Recovery activities in Sierra Leone contributing to the achievements of the mission and Ebola Pillar II Strategic Framework’s objectives? (2) Sustainability: What Pillar II accomplishments are still observable after the activities have closed, and why? (3) Gaps and Opportunities: Were there unanticipated barriers or consequences of Pillar II investments? What were the opportunities seized and what was missed? (4) Management: What lessons have been learned regarding how to use the strengths of USAID’s development mechanisms in a complex emergency? Methodology PURPOSE The desk review is intended to collate available information on USAID’s Pillar II investments in Sierra Leone, serving as one of several methods of data collection and analysis. It summarizes findings from documents from activities in Sierra Leone that were funded partially or entirely with USAID Pillar II funds. KEY COMPONENTS The desk review began with a call for documents (with assistance from USAID/Washington and USAID/Sierra Leone). Based on the set of activity documents provided by USAID, and supplemented by additional information gleaned from implementing partner (IP) documents and other sources that the Evaluation Team (ET) found in the public domain, the ET extracted information according to the following six thematic areas: (1) Health (Non-Ebola Essential Health Services and Health Systems Recovery and Survivor Programs); (2) Basic Education; (3) Agriculture and Food Security (AFS); (4) Governance; (5) Economic Crisis Mitigation (ECM); and (6) Innovation, Technology and Partnerships (ITP). Based on that evidence base, the ET: (a) Assessed activity contributions to USAID Ebola Pillar II strategy and the GoSL National Ebola Recovery Strategy, using Ebola Pillar II Theory of Change; (b) Analyzed contributions of the interventions to dimensions of resilience and sustainability; and (c) Formulated recommendations for the next phase of recovery from the EVD crisis. In exploring resilience, the team used USAID‘s resilience definition: “The ability of people, households, communities, countries and systems to mitigate, adapt to and recover from shocks and stresses in a manner that reduces chronic vulnerability and facilitates inclusive growth.” (USAID, ND). Focusing on health systems, Emrey (2016) suggested that resilience consists of: “Capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons during the crisis, reorganise if conditions require it.” USAID defines sustainability as: “The ability of a local system to produce desired outcomes over time, including ability to be both resilient and adaptive in the face of changing circumstances.” This definition emphasizes the persistence of activity effects after interventions have ended. The ET developed an analytical framework for examining sustainability determinants such as policy, finances, accountability, institutional support, technical choices, the human dimension, and external threats and enabling environments. To the extent relevant information exists, determinants have been explored under the auspices of this desk review. Actual measurement of sustainability and resilience, and assessment of outcomes, will be addressed in greater depth in the Pillar II MEL Project Performance Evaluation 2 at the end of 2018. SAMPLING Through activity mapping in 2017, the ET identified a total of 24 activities in Sierra Leone that were classified by USAID as Pillar II efforts, and were expected to have M&E plans, regular results reports, and baseline, midline and/or end of activity evaluation reports. However, the ET was unable to obtain documentation on a large number of Pillar II activities. In the spring of 2017, IBTCI reached out to the mission and the IPs through the activity’s Con￾tracting Officer Representative (COR) to request activity documen￾tation. Second and third data calls were submitted in October and December 2017, respectively. Some of the documents are publicly available, but many were classified Sensitive but Unclassified (SBU) and have only been reviewed by staff who have signed Non-Disclo￾sure Agreements with USAID/W and USAID/Sierra Leone. This desk review report is based on available documents from USAID/AFR and other Washington OUs, USAID/Sierra Leone and/or by IPs at the mission’s request, the USAID Development Experience Clearinghouse (DEC), and other documentation/data found on the public domain (e.g., organizational websites, peer￾review journals) for 14 Pillar II activities. Sub-annex A contains the list of documents reviewed. ANNEX M. DESK REVIEWS – SIERRA LEONE n M–101 Table M–SL2. Sierra Leone Pillar II activities included in this desk review IP and Activity Thematic Areas of Focus* Predominant Intervention Types Health AFS Soc Protn Ftln Wkr MCP ICT SBCC IE ACDI/VOCA: SNAP+/EFSP ✔ ✔ CARE: Rapid Ebola Social Safety Network and Economic Recovery (RESSNER) ✔ ✔ Catholic Relief Services (CRS): EFSP in Kenema District ✔ ✔ International Organization on Migration (IOM): IPC Phase II Training pre-training ✔ ✔ IntraHealth: mHero ✔ ✔ ✔ JSI/Advancing Partners and Communities: Strengthening Primary and Community Health Services ✔ ✔ ✔ JSI/Advancing Partners and Communities: Ebola Transmission Prevention and Survivor Services (APC-ETP&SS) ✔ ✔ ✔ ✔ ✔ Johns Hopkins Univ. Center for Communication Programs (JHU CCP): Health Communication Capacity Collaborative (HC3) ✔ ✔ Management Sciences for Health (MSH): Systems for Improving Access to Pharmaceuticals and Services (SIAPS) ✔ ✔ Save the Children: Kailahun Food for Emergency Ebola Virus Disease Support (FEEDS) ✔ ✔ SIA: Agent Network Strengthening (MicroSave) ✔ ✔ UNICEF: Screening, referral, treatment for children with SAM, RUTF ✔ ✔ ✔ UNICEF: Supporting Health Information Systems and mHealth** ✔ ✔ ✔ World Vision: EFSP in Port Loko ✔ ✔ *Only AFS and health-focused activities are reviewed in this Desk Review, though achievements in Governance and ITP are discussed. **This activity was included in the desk review because it was considered to be Pillar II by the author, though it does not appear in IBTCI’s PE1 activity mapping. This is most likely because the activity started in September 2017. Table SL2 classifies each activity according to thematic areas as well as the following six intervention type(s): (1) Social Protection for EVD Survivors and Other Vulnerable Groups; (2) Frontline Worker Support; (3) Management, Coordination and Partnerships (MCP); (4) Information and Communication Technology (ICT); (5) Social and Behavior Change Communication (SBCC); and (6) Institutional Enhancements (IE). LIMITATIONS The desk review is not the sole source of evidence in answering the main evaluation questions. It represents IBTCI’s first contribution to compiling information and distilling insights on USAID’s contribution at a macro level. The following are specific limitations: (1) Measuring outcomes is challenged by the short duration of the interventions, the lack of baseline data and/or the poor quality of monitoring data. (2) The ET had very limited access to summative/endline reports, or other documents that presented cumulative results since activity inception. (3) Because the team did not have access to routine monitoring reports, this limited the ability to collate results and examine time-series data. (4) Most activity outputs are reported as ‘numerator’ only (i.e., IP reports note the number of people or health facilities impacted, and no denominators to assess how much of the need was addressed). (5) Beyond indicating district(s) of operation, few reports offer detailed geographic information on their service delivery area. (6) There are a number of activities for which no data or documents were provided by USAID. M–102 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES PROGRESS WITHIN THEMATIC AREAS Health PROBLEM STATEMENT Since 2002, Sierra Leone had been on the path of recovery from the protracted civil war when the EVD hit the country in May 2014. The GoSL was rebuilding and rehabilitating its health infrastructure, which had deteriorated during the civil war. The EVD epidemic exposed prior weaknesses in the healthcare system in Sierra Leone and underscored the urgent need to address them in order to better prepare the GoSL and the country to prevent and respond to future disease outbreaks. In 2014, UNICEF conducted a survey of healthcare facilities across the country and identified gaps in four major areas: I) Inadequate training of health facility personnel; ii) Lack of necessary equipment; iii) Weak diagnostic capability at the facility level; and iv) Stockouts of essential medicines at peripheral health units (PHUs) (UNICEF, December, 2014). Other major weaknesses of the health system include 1) An insufficient number of health workers (with only 0.2 doctor and 3 nurses per 10,000 people, compared with the 23 doctors, nurses, and midwives per 10,000 inhabitants that are necessary to achieve 80% coverage in essential health services) (WHO, n.d., UNICEF, 2014); 2) Poor distribution of health human resources across health facilities in the country (caused in part by the heavy concentration of health workers in urban areas), leaving many rural areas under-resourced and ill-equipped to provide basic health care; 3) Gaps in health information system (HIS), related challenges in reporting of health data, and major imbalances in the distribution of basic health services across the country. The government launched the national Free Healthcare Initiative (FHCI) in April 2010. This initiative provides free services to pregnant and breastfeeding women, as well as children under 5 years of age seen at public healthcare facilities. Through the FHCI, the MOHS also distributes twenty essential drugs to the various health facilities in the country. While the FHCI has helped improve the availability of essential drugs for the management of childhood illnesses in health facilities, and increased access to services for women and children, many gaps remain in the healthcare system (UNICEF, 2014). In addition, about 3,500 of the estimated 9,000 people who were infected by the EVD survived the disease (JSI Research and Training Institute, Technical Brief (b). n.d.). Many EVD survivors have been stigmatized, and some are experiencing medical complications from EVD, and/or psychological problems such as depression, suicidal feelings, anxiety and difficulty sleeping, which require specialized medical services (JSI Research & Training Institute, Technical Brief (a). n.d., JSI Research & Training Institute, September 2016). EVD survivors need comprehensive services to address the medical and psychosocial challenges they face, and to minimize the risk of continued EVD transmission. Unfortunately, many survivors have difficulty accessing specialized medical services, because healthcare providers lack the necessary skills and experience to treat the medical complications of EVD, and also because of the risk of transmission. Health officials are concerned about the persistence of the EVD in the bodily fluids of survivors (including breast milk, semen, and spinal column) (JSI Research & Training Institute, Inc., September 2016). These weaknesses needed to be addressed with some urgency, in order to restore patients’ trust in the public health system and prepare the country to prevent and respond to future health emergencies. SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II Pillar II Ebola recovery interventions in the health sector were designed to restore basic health services with a focus on primary healthcare facilities (peripheral health units – PHUs), promote healthy and health-seeking behaviors, build the capacity of the health workforce in critical areas, and strengthen health information systems and supply chain management systems. Additional investments were made in strengthening local governance and accountability in healthcare and ensuring that EVD survivors have access to routine and specialized medical and mental health services that allow them to lead productive lives. The main health activities funded with Pillar II resources were: (1) JSI/APC Strengthening Primary and Community Health Services (SPCHS); (2) JHU CCP/HC3; (3) IOM/IPC Phase II; (4) MSH/ SIAPS; (5) UNICEF/Supporting HIS and mHealth; and (6) JSI/ APC ETP&SS. Smaller awards were made to entities such as UNICEF to procure essential medications, to UNCDF, Jhpiego, and mPowering Frontline workers to assess and support digital payments and digital health efforts, and to WHO and MEASURE to embed experts in the MOHS. Public-private partnership efforts were implemented with Aspen Innovation Partnership for Health for community health systems strengthening, and Project Last Mile for supply chain management. For many of these smaller awards, scarce information about activities was provided by USAID and searches in the public domain were not fruitful. The principal interventions in the health sector are shown in Table SL3. Addressing Needs of EVD Survivors: A Complex Issue Requiring Multiple Types of Interventions The GoSL is committed to ensuring the health and well-being of all EVD survivors in the post-EVD recovery period, by making sure that appropriate medical care and mental health services are available to EVD survivors who need them. With the support of development partners, the MOHS and the Ministry of Social ANNEX M. DESK REVIEWS – SIERRA LEONE n M–103 Welfare, Gender and Children’s Affairs (MSWGCA) designed the Comprehensive Program for EVD Survivors (CPES) with the following objectives: i) Support EVD survivors in the recovery of functional capacity through effective delivery of healthcare and psychosocial services in response to survivor needs; ii) Support the recovery of survivor livelihoods; iii) Support the re-integration of survivors into their communities; and iv) Address the risk of resurgence associated with extended Ebola viral persistence in survivors. Since 2016, the CPES program has received support from a number of funding agencies, including i) DFID (through the Ebola Survivor Care Consortium (ESCC), led by GOAL); and ii) USAID through the Ebola Transmission Prevention and Survivor Services (ETP&SS) activity, launched in July 2016 and implemented in Sierra Leone by JSI, under the Advancing Partners & Communities (APC) activity. The assistance provided to CPES by DFID through the Consortium supports activities in 13 of Sierra Leone’s 14 districts (excluding Bonthe district which has no EVD survivors recorded). USAID support provided to CPES through the ETP&SS activity is strengthening the capacities of the health system to provide specialized medical services and mental health services for EVD survivors. Currently, the mental health professional staff in Sierra Leone consists of only two psychiatrists and 20 Mental Health Nurses (MHNs) (JSI Research and Training Institute, Inc. Technical Brief (a). n.d.). The ETP&SS activity is working with the MOHS, the WHO, King’s Sierra Leone Partnership (KSLP), and the War Trauma Foundation to train health workers at all levels of the healthcare system to provide mental health services to EVD survivors who need them, consistent with the level of care of the facility. At the community level, the ETP&SS activity works in the six USAID priority districts (Bombali, Kailahun, Kenema, Port Loko, Western Area Rural and Western Area Urban) and implement community healing dialogues. The dialogues are facilitated by trained Community Health Officers (CHOs) and social workers and are held once a week with 15 to 18 community members. They are designed to help communities identify resources and appropriate coping strategies to address psychosocial problems that are identified. At the primary healthcare facility level, the activity is working with the MOHS, the WHO, King’s Sierra Leone Partnership (KSLP), and the War Trauma Foundation to train HCWs at the PHUs in psychological first aid, i.e., how to recognize distress symptoms in patients and refer them to the district MHNs. The activity is also training mid- and higher-level health care providers (MHNs, CHOs, and medical doctors) to enable them to provide more specialized mental health services (i.e., being able to assess clients for mental disorders, assign a diagnosis, and identify possible treatment options). In addition, Table M–SL3. Pillar II activities in the health sector, according to activity Interventions, Sierra Leone Pillar II Activity APC SPCHS HC3 IPC Phase II SIAPS APC ETP&SS Social Protection for EVD survivors and other Vulnerable Groups Addressing needs of EVD survivors ✔ Frontline Worker Support Health worker training ✔ ✔ ✔ ✔ ✔ MCP Leadership/management support ✔ Supply chain management ✔ ✔ ICT Health information systems ✔ SBCC Community engagement ✔ Other SBCC activities ✔ ✔ ✔ Institutional Enhancements WASH ✔ Facility infrastructure upgrades ✔ ✔ ✔ M–104 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES MHNs working at district- and tertiary-level hospitals receive continuous professional development training and mentoring, to ensure the services they provide at hospital mental health clinics and in the community meet established quality standards. To ensure greater access to specialized mental health services in the country, the activity plans to support the development and institutionalization of a Mental Health Diploma at the College of Medicine and Allied Health Sciences (COMAHS) (JSI Research and Training Institute, Inc. Technical Brief (a). n.d.). The ETP&SS activity is also helping the Sierra Leone Association of Ebola Survivors (SLAES) to become an effective advocacy organization for EVD survivors. The activity is building the capacity of SLAES’ leadership teams at the national level and in the districts in organizational development, advocacy, management, policy, and use of data for planning and advocacy through training of the SLAES executive leadership and other staff in those areas, and by working with SLAES to develop a Strategic Advocacy and Communication Strategy and related workplans. The activity is also providing direct support for the implementation of two SLAES projects – an Adult Literacy Project to improve literacy among EVD survivors (currently implemented in Western Rural and Western Urban districts), and a Survivor to Survivor project that provides support and guidance to EVD survivor groups who want to start income-generating activities. (JSI Research and Training Institute, Inc. Technical Brief (b). n.d., JSI Research and Training Institute, Inc. APC, Sierra Leone. ETP&SS. n.d.). The ETP&SS activity was able to achieve the following outputs: ■ 680 in 30 communities across the six districts reached through Community Dialogues; ■ At the primary healthcare level, 350 HCWs from 125 PHUs across the six priority districts have received psychological first aid training; ■ A national Mental Health Network has been established, and mental health nurses have been deployed to every district; ■ A mental health clinic has been established in every government hospital; ■ The MOHS has drafted the National Mental Health Policy, with technical support from the activity; ■ SLAES has updated its Mission and Vision Statement and developed a five-year Strategic Plan; ■ The Adult Literacy program and the Survivor to Survivor activity have been launched and are ongoing (JSI Research and Training Institute, Inc. Technical Brief (a). n.d.; JSI Research and Training Institute, Inc. Technical Brief (b). n.d.). Frontline Worker Support Pillar II resources supported pre- and in-service training of community volunteers, nurses, midwives, and laboratory personnel. In addition to supporting health worker training in maternal and child health interventions and IPC under specific health service delivery interventions like JSI’s APC activity (JSI Research and Training Institute, October 2016), another significant investment made by USAID to the EVD recovery program, and in the long term to health systems strengthening in Sierra Leone has been its support for the development and introduction of training programs on IPC in selected academic institutions in Sierra Leone. USAID funded the International Organization on Migration (IOM) (USD $3 million) to establish infection prevention control short course departments and mobile training at the College of Medicine and Allied Health Services (COMAHS) in Freetown, and at Tonkolili College of Health Sciences. The project comprises three activities: i) Establishment of Infection Prevention and Control departments at COMAHS, and Tonkolili College of Health Sciences; ii) Mobile Infection Prevention and Control Certification Training; and iii) Renovation and equipment of COMAHS and Tonkolili College of Health Sciences IPC departments. IOM’s partners to this project are the MHS, COMAHS and Tonkolili College of Health Sciences, the US Centers for Disease Control and Prevention (CDC), WHO, DFID, UK Department of Health, and District Health Management teams (DHMTs) (IOM, n.d.). USAID’s investment (through its IOM support) in the GoSL’s effort to integrate and institutionalize IPC into the pre-service training program for health professionals has yielded positive results. More than 30 master trainers have completed their training in IPC and graduated from the program. The master trainers are expected to reach 5,000 university students (USAID Presentation. 4th October 2017). MCP The deployment of health personnel to the response to the EVD outbreak created a severe shortage of trained pharmaceutical staff at national, district, and peripheral levels. The country’s pharmaceutical management information system was weak, data recording (which mainly used paper-based tools), was inefficient, and reporting was irregular. The Systems for Improved Access to Pharmaceuticals and Services (SIAPS) Program (2011–2018) received two years of funding from USAID to assist the GoSL MOHS in its efforts to rebuild and strengthen pharmaceutical supply chain management and improve rational use of medicines use in the country. SIAPS was implemented by Management Sciences for Health (MSH). The activity has provided technical assistance to key government pharmaceutical entities, including the Directorate of Drugs and Medical Supplies (DDMS), the Pharmacy Board of Sierra Leone, DHMTs, and health facilities. A total of 17 pharmacists across the 13 districts have been trained as trainers and will conduct cascade training in the districts. SIAPS has assisted the MOHS in 1) Updating the National Medicines List to inform purchasing and selection of drugs; and 2) Implementing the continuous results monitoring and support system (CRMS). CRMS is used to track and monitor the factors that influence medicine availability and disease case management, in order to ensure availability, ANNEX M. DESK REVIEWS – SIERRA LEONE n M–105 inventory management, and rational use of pharmaceuticals. The CRMS had been implemented in all 13 districts of the country as of the end of 2016. On average, 70 percent of health facilities in each district participated in the CRMS exercise. SIAPS also supported the implementation of its electronic data management tools – Quantimed, Pipeline, and QuanTB, assisted in the revision of Sierra Leone’s Logistics Management Information System (LMIS) and the training of pharmacists from around the country. It also designed a revised treatment/pharmacy register for use in approximately 1,200 PHUs and 21 district hospitals. The revised treatment/pharmacy register captures data on key pharmaceuticals, including 13 essential products and selected EVD-related commodities. Continuous monitoring and support activities are ongoing in all 13 districts, covering 85% of PHUs (MSH, 2018). Additionally, other MCP interventions included focus on policy changes such as APC’s support for the review and revision of the national CHW policy, and the MCH guidelines for health post personnel (JSI Research and Training Institute, October 2016). ICT The weaknesses of the Sierra Leonean National Health Information System (NHIS) limited the ability of the MOHS to effectively plan an effective response to the EVD outbreak. It is reported that the outbreak was not immediately detected through the system, until the outbreak had spread to several districts, resulting in a large number of EVD cases and EVD-associated deaths in the country (GoSL (a). n.d.). To identify the deficiencies of the NHIS, the GoSL commissioned several assessments with support from various partners and a five-year Strategic Plan for the NHIS (2017–2021) was developed to improve the overall performance of the national HIS, improve the quality of the data it provides, and facilitate the use of the data at all levels. The MOHS requested assistance from USAID to improve the national HIS. USAID’s assistance was provided through UNICEF under the “Support Health Information Systems and mHealth Program.” This program aims “to strengthen the system and process to collect, manage and use health data for key decision making at all levels, using both existing means and innovative techniques. The activity has three specific objectives: i) To institutionalize and roll-out eHealth solutions in the country; ii) To improve data on CHWs and data reported by CHWs; and iii) To improve HMIS data quality and use for decision making). The program’s efforts to improve data on CHWs and data reported by CHWs were led by the MOHS through the CHWs’ Hub and the DPHC, in collaboration with UNICEF, WHO and the International Rescue Committee (IRC) (UNICEF, May 2017). To improve HMIS data quality and use for decision making, the program supported DHMTs to lead monthly review meetings with PHU In-charges and DHMT focal points during which facilitators helped PHU In-charges and DHMT focal points to address critical HMIS issues that arose from the previous month’s data review, including helping PHU In-charges to identify practical ways of ensuring the timely submission of HMIS forms to DHMTs. The program also provided support to stakeholders’ meetings at the chiefdom level, and to DHMTs to conduct HMIS-focused supportive supervision in selected PHUs, targeting low-performing PHUs as well as PHUs that are located in hard-to-reach areas. These activities, combined with DHIS tools such as the district action-points tracker and the PHU HMIS journal (which were introduced at the district and facility levels), were designed to promote and institutionalize data review practices at district, health facility, and chiefdom levels (UNICEF, May 2017). The main contributions of the USAID-funded “Support Health Information Systems and mHealth Program to efforts to strengthen HIS are: ■ 1,230 PHU managers (PHU In-Charges) received training on the use of RapidPro (an open-source mobile communication platform) and the role of the eHealth Coordination Hub. ■ A system for use by the MOHS to communicate with health workers via RapidPro has been set up and is currently used by the MOHS to communicate with health workers via by telephone or text message at no cost to the MOHS (USAID, October 2017). ■ The establishment of a national eHealth Coordination Hub to facilitate government-led eHealth initiatives at all levels and to coordinate and regulate eHealth activities in the country. The national eHealth Coordination Hub was officially launched on March 30, 2017 and is functioning well. It has a Steering Committee, a core strategy team (established within the MOHS, with representation from relevant departments of the MOHS, as well as UNICEF, and WHO), and a technical working group that meets regularly. ■ Providing technical support and coordination to a number of eHealth initiatives, including assistance to the Directorate of Primary Healthcare (DOHC) to provide mobile training and support to community health workers (CHWs) under the Ebola Vaccine Deployment Acceptance and Compliance (EBODAC) project, and assistance to the national malaria bed net distribution campaign with real-time reporting and feedback; and assistance to the national HIV/AIDS program in tracking HIV commodities and supplies. ■ Supporting the customization of the WHO Digital Atlas (DHA) that will be used to track and archive eHealth projects in Sierra Leone (UNICEF, May 2017). SBCC In addition to its pivotal role in controlling the EVD outbreak, community engagement is integral to health system resilience and ownership. It was therefore a critical part of the Pillar II strategy for health system recovery. Through the HC3 activity implemented by JHU, USAID has helped strengthen the capacity of the MOHS at national and district levels to plan, manage, coordinate and evaluate BCC and health promotion interventions. HC3 partnered at national level with the MOH’s Health Education M–106 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Division (within the Primary Healthcare Directorate) and its Reproductive and Child Health Directorate, and in the districts with DHMTs, local communities, local radio stations, and community-based structures. Other key implementing partners were GOAL (for community engagement activities), Search for Common Ground (SFCG) (for radio and campaign activities), Kyne Communications, Inc. (for campaign design), and Dalan (for research activities). The HC3 activity was implemented at the national level and in the five USAID priority districts (Bombali, Port Loko, Tonkolili, Western Area Urban, and Western Area Rural) (HC3, August 2017(a)). The activity also assisted in the development of national guidance documents such as the National Health Promotion Strategy (2017-2021) and five-year implementation plan) and community engagement for improved health practices and improving the quality and coordination of SBCC interventions in Sierra Leone. To build capacity in BCC and health promotion and sustain project activities and results at the national level, HC3 supported the training of 49 government staff in SBCC (double the target number of 25 set for the life of the activity). The increased number of trained staff provided a critical mass of technical staff skilled in SBCC who were capable of managing and coordinating health promotion activities in the country. In addition, HC3 contribution to the development of Sierra Leone’s National Health Promotion Strategy 2017-2021 provides a strategic framework and guidelines for the design of new SBCC interventions and the implementation, integration and coordination of existing health promotion activities across different health areas. At the district and community level, HC3 promoted and fostered a sense of community ownership of project activities by engaging community members (through community dialogues and facility makeovers) in planning, implementing and monitoring HC3 project activities in their respective communities, and highlighting their accomplishments. Through the facility makeover activities, communities were given the opportunity to identify and prioritize makeover activities that would encourage use of health services. In collaboration with the activity staff and health facility staff and FMCs, they were able to mobilize local labor and other resources to implement agreed upon makeover activities (HC3, August 2017 (a)). Another of the activity’s important roles is its contribution to SBCC knowledge management. HC3 and MHS HED jointly developed the Health Education Division Knowledge Gateway in October 2016. The gateway provides an online forum for the storage of resources to support the planning, implementation and evaluation of health promotion activities at all levels. Complementary HED Knowledge Sharing guidelines were also developed to support HED and gateway members to manage and use the system. HC3 provided an additional year of continued funding (April 2017 to April 2018) to support internet connectivity for the HED and DHMTs and the management of the HED Knowledge Gateway, and a total of 20 laptops to HED and DHMTs for the District Social Mobilization Coordinators (DSMCs) and some District Health Sisters. The gateway currently houses more than 592 documents; and all SBCC products and tools developed and supported by HC3 were uploaded to the Knowledge Gateway for the MHS HED’s future use (HC3, August 2017 (a)). Institutional Enhancements To support USAID’s Pillar II strategic objective of restoring and recovering basic essential services, the APC activity supported the MOHS’ post-EVD recovery efforts to improve access to and availability of quality reproductive, maternal, newborn, and child health (RMNCH) services in primary care facilities (PHUs) and at the community level in Sierra Leone. The populations targeted by this intervention are lactating women, pregnant women, mothers of young children (children under 5 years of age), and young children (under 5 years of age). The activities supported by the project included the rehabilitation of maternal and child health posts (MCHPs) and community health posts (CHPs) to enhance the quality, safety, and accessibility of health services (including construction of bore holes and wells to provide safe water, and installation of sanitation facilities and solar power systems), and equipping MCHPs and CHPs with lifesaving basic medical equipment for RMNCH services (JSI Research and Training Institute, October 2016). The APC activity supported efforts to rebuild trust in the healthcare system by establishing the Facility Management Committees (FMCs) or reactivating already existing ones and supporting the development of the Community Engagement (CE) Strategy and Toolkit (John Snow Research and Training Institute, Inc. n.d.). The CE Strategy and toolkit were used to engage local communities through the FMCs in the operation and maintenance of PHUs, and in so doing, improve relationships between community leaders and health facilities, and improve utilization of government health services. By December 2016, a total of 214 FMCs were established or reactivated in the APC activity districts, and more than 2,500 FMC members were trained to provide health education and mobilize support for maintaining the PHUs. The activity then worked with the MOHS to adapt the FMC strategy and toolkit to be scaled up nationally. The endline assessment of the activity found that 97 percent of PHUs surveyed met with the FMC at least once every three months, compared with 76 percent at baseline (John Snow Research and Training Institute, Inc. n.d. (e).). Efforts to rehabilitate PHUs yielded the following results: 1) 110 PHUs rehabilitated in the activity’s five priority districts; 2) 214 FMCs (consisting of 2,539 FMC members—903 women and 1,627 men) were trained and their skills were strengthened, as reflected by increased and sustained engagement of local communities in health facility maintenance operations and health activities (JSI Research and Training Institute, Technical Brief (c). n.d.); 3) assisted in the revision of the National Community Health Worker Policy (2016–2020) and supported the training of about 1,500 CHWs (JSI Research and Training Institute, Inc. n.d (e)), as well as the training of 950 clinical and non-clinical staff from 243 health facilities across the five districts on reproductive ANNEX M. DESK REVIEWS – SIERRA LEONE n M–107 and child health, and on IPC (JSI Research and Training Institute, Technical Brief (f). n.d. The activity also supported the completion and printing of the 2017 MOHS Standards & Guidelines for WASH Services in Health Facilities in Sierra Leone. The WASH standards were subsequently used in the rehabilitation of the 110 PHUs ((JSI Research and Training Institute, Technical Brief (g). n.d.). The activity was implemented by JSI, in collaboration with its partners—Action Against Hunger, Adventist Development and Relief Agency, GOAL, International Medical Corps, and Save the Children—and district health management teams, and in close coordination with MOHS agencies and departments at central level. OUTPUTS AND OUTCOMES Available documents on health activities that were fully or partially funded with Pillar II resources indicate that the activities were able to produce the majority of planned outputs or surpass them (e.g., assessments completed, plans developed, HCWs trained, HFs strengthened). Activity output and outcome monitoring data suggest that the HC3 community dialogues and health facility makeover activities improved the physical conditions and functionality of the PHUs in the five districts, and increased utilization of RMNCH services at those health facilities: 1) A total of 31 PHUs were renovated with activity (GHET) support in the target districts, exceeding the target of 25 PHUs; 2) 499 individuals completed training in interpersonal communication approaches; and 3) 49 government￾affiliated staff members were trained in SBCC at national and district levels; iv) 97.9 percent of survey participants interviewed in the first quarter of 2017 reported using a health facility during the past six months, compared to 81.3 percent in the first quarter of 2016 (HC3, August 2017 (b)). The findings from the endline survey and exit interviews show that: 1. Knowledge of specific services increased, notably: family planning (increased from 50 percent at baseline to 66 percent), infant and child health services (increased from 18 percent to 44 percent), services for newborns (increased from 26 percent to 76 percent) and immunization (increased from 51 percent to 73 percent). There was not a significant increase in knowledge regarding ANC (83 percent), postnatal care (53 percent) and facility delivery (57 percent). 2. Positive attitudes toward facility-based workers was high at baseline and remained high at endline, with 86 percent of clients (not a significant change from baseline) reporting a positive attitude. A smaller proportion of respondents expressed a positive attitude of community-based workers (59 percent), and this was also unchanged during the activity period. 3. More importantly, overall use of RMNCH services increased from 66 percent to 79 percent, use of maternal services increased from 37 percent to 63 percent, and use of infant care services increased from 43 percent to 57 percent. (HC3, August 2017 (a)). Key results of the UNICEF program’s support to GoSL’s efforts to improve HMIS data quality and use for decision making include: ■ Reported increases in overall reporting rates in the three program districts (Bombali, Port Loko, and Tonkolili) with sustained improvements in report completion rates reported in each district over a seven-month period (September 2016 to April 2017) – Bombali from 93 percent in September 2016 to 96 percent in April 2017; Port Loko from 97 percent to 99 percent; and Tonkolili from 99.6 percent to 100 percent in each of the last six consecutive months. ■ Reported increases in on-time reporting rates over the seven￾month period in two of the three districts – Bombali from 43 percent in September 2016 to 96 percent in April 2017; Port Loko: no change/maintained at 96 percent; and Tonkolili from 44 percent to 93 percent. ■ Supportive supervision visits focused on HMIS were conducted by DHMT members in 108 PHUs, including 12 visits to PHUs located in riverine and hard-to-reach areas. ■ Over 340 facility In-Charges (112 in Bombali; 114 in Port Loko; and 108 in Tonkolili) participated in data review forums (UNICEF, May 2017). MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Within USAID, the Global Health Ebola Team (GHET), Food for Peace (FFP), and bilateral funding contributed to the Ebola Pillar II response. Pillar II Ebola recovery activities in the health sector made substantial efforts to leverage synergies, technical expertise and other resources among activities in order to avoid duplication of efforts and maximize impact of project activities on the ground. For example, HC3 linked with the Management Sciences for Health (MSH) Systems for Improved Access to Pharmaceuticals and Services (SIAPS) activity to ensure makeover PHUs received essential storage equipment and packing support for facility drug stores when these needs could not be covered by the HC3 activity. HC3 also collaborated and coordinated closely with the APC activity, and HC3 implementing partner (GOAL) was required to have monthly activity partner meetings with JSI implementing partners in each district. UNICEF and USAID have developed an effective partnership in Sierra Leone in a number of areas. USAID implementing partners also coordinate and consult regularly with their counterparts in the MOHS at all levels. Program review meetings, as well as TWGs, are used by implementing and development partners and GoSL officials to review project activities and resolve implementation problems and issues. M–108 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Agriculture and Food Security (AFS) PROBLEM STATEMENT The vast majority of Sierra Leoneans depend on agricultural production for their livelihoods. The main crops grown are rice and cassava; maize, millet and sorghum also contribute a significant share to cereal production in the country. EVD struck Sierra Leone during the preparation of land for rice cultivation, and many farmers lost crops and seeds. Many petty traders lost their stock and had no capital to rebuild with after the EVD crisis. As a result, food prices increased, and households resorted to coping mechanisms such as borrowing money to pay for food and/ or skipping meals (Radice, June 2017). The 2015 CFSVA results discussed earlier highlighted the significant deterioration of Sierra Leone’s socio-economic and food security situation during the outbreak and the subsequent year, with more than half (52.9 percent) of households reporting that they had experienced a decrease in their income levels, and farming households reporting that they frequently consumed the seeds that were intended for the planting season. The situation began to normalize slowly in late 2014, and by the end of January 2015, the functioning of markets was reported to be improving slowly. (Radice, June 2017; FEWSNET, 2015). SOLUTIONS/INTERVENTIONS PROPOSED BY USAID PILLAR II The main Pillar 2 interventions in the agriculture and food security sector were all under the Social Protection intervention type: unconditional cash transfers (UCT), conditional cash transfers (CCT), agricultural inputs, training in agricultural production, health, nutrition, household resource management, and other relevant topics, and savings and loans. These interventions were implemented by Food for Peace (FFP) partner organizations (ACDI/VOCA, CARE, Catholic Relief Services, Save the Children, and World Vision) in six priority districts – Kailahun, Bombali, Tonkolili, Koinadugu, Kenema, and Port Loko. FFP also partnered with UNICEF to provide in-kind food assistance to households impacted by the EVD epidemic. Table SL4 summarizes the activities, their geographical focus and strategies. Agriculture and Food Security activities in Sierra Leone aimed to: Provide cash transfers to the most vulnerable households to purchase food in local markets; provide agricultural input vouchers (e.g., vouchers for seeds, fertilizers, and tools); to stimulate food production for household consumption; and provide cash grants to VSLAs and small-scale traders to stimulate market recovery. The agricultural inputs combined with the cash transfers and training in relevant areas (e.g., training in post-harvest handling and storage) were expected to restore agriculture markets, leading to increased food availability and consumption and improved food security. Social Protection through Unconditional Cash Transfers (UCT) UCT provide immediate access to food for food insecure households and help UCT recipients to avoid negative coping strategies such as selling household assets, skipping meals, begging, and child labor. Five USAID/FPP partners (ACDI/ VOCA, CARE, CRS, Save the Children, and World Vision) implemented unconditional cash transfer activities in FFP priority districts. The districts were selected in close consultation with the Ministry of Agriculture, Forestry and Food Security (MAFFS) and the National Commission for Social Action (NaCSA), based on specific criteria, including population density, poverty, infant and child mortality, malnutrition in young children, and food insecurity. NaCSA is the government agency with oversight for CT programs in the country. NaCSA is governed by a board of directors consisting of representatives of the government, donors and civil society (GoSL, 2008). The GoSL, with support from the Table M–SL4. Pillar II activities in the agriculture and food security sector, according to activity Intervention Strategies, Sierra Leone Pillar II Activity ACDI/ VOCA CARE CRS STC World Vision UNICEF Unconditional cash transfers ✔ ✔ ✔ ✔ ✔ Conditional cash transfers ✔ Agricultural inputs ✔ ✔ ✔ Health, nutrition, budget training ✔ ✔ ✔ Access to savings and credit ✔ Food supplements ✔ ANNEX M. DESK REVIEWS – SIERRA LEONE n M–109 World Bank and UNICEF, released the ‘EVD Response-Social Safety Net Cash Transfers Standard Operating Procedures’ in May 2015 (GoSL, 2015). This document has guided the GoSL’s Rapid Ebola Social Safety Net response through NaCSA and served as the basis for the USAID/FFP partners’ programs. Poverty criteria were used for selecting UCT beneficiaries because of EVD’s significant impact on the very poorest, and the evidence from earlier interventions in the response that targeting only EVD survivors could lead to stigmatization. The beneficiaries directly affected by EVD included EVD survivors, orphans, or individuals who had lost the head of their households to EVD. The categories of households that were indirectly affected by the EVD outbreak include: i) Quarantined and post-quarantined communities or households; ii) Vulnerable populations (e.g., chronically ill people, malnourished children, elderly people, and pregnant or lactating women); and iii) Households or communities that had suffered significant economic losses due to the outbreak. The identification and registration of UCT program beneficiaries was facilitated by Community Identification Committees (CICs). The CICs are composed of representatives of the community, including women leaders, elderly community members, religious leaders, and teachers, who volunteer to assist with the screening and registration of CT beneficiaries. CIC members were trained in the screening and registration procedures and worked with local government officials and community groups to identify the poorest households, using established poverty criteria (Morlai, June 2017; Inclusive Development and Research Consultancy, October 2017.) ACDI/VOCA implemented the recovery UCTs intervention in three districts – Bombali, Tonkolili and Kailahun districts, under the Sustainable Nutrition and Agriculture Promotion Plus (SNAP+) activity. SNAP+ targeted households that had been impacted by the EVD outbreak between 2014 and 2015. UCTs of USD $30 per month were given to 13,588 vulnerable households over a 12-month period to improve household food security. ACDI/VOCA provided UCTs to an additional 3,260 vulnerable households in Bombali district and 5,540 households in Tonkolili District under the follow-on Emergency Food Security Program (EFSP). The SNAP+/EFSP activity also provided modest recovery grants to financially stressed farmer producer groups and village savings and loan associations (VSLA) (Mbevi, November 2016). The CARE Rapid Ebola Social Safety Network and Economic Recovery (RESSNER) activity provided UCTs to 7,567 vulnerable households in Bombali and Tonkolili districts. This UCT intervention provided cash assistance to 28 percent of the CT program beneficiaries identified by NaCSA as extremely poor, enabling them to meet basic needs through the lean season (Inclusive Development and Research Consultancy, October 2017). CRS was the implementing partner for the Emergency Food Security Project (EFSP) in Kenema district (from August 2015 to December 2017). The CRS EFSP provided UCTs to 4,437 EVD-affected households across five chiefdoms in the district. Up to 80% of the beneficiary households were female￾headed. World Vision led the implementation of the EFSP in Port Loko district. The activity provided monthly UCTs to 5,324 vulnerable households in the district (Radice, June 2017). Save the Children’s Food for Emergency Ebola Virus Disease Support (FEEDS) activity provided UCTs to vulnerable households directly and indirectly affected by EVD in Kailahun District. FEEDS initially targeted 6,445 households in 2015 but the program received two consecutive cost extensions, with each extension resulting in resulting in an increase of the number of targeted households, thereby enabling the program to increase its target to 10,318 households (Radice, June 2017). Social Protection through Conditional Cash Transfers (CCT) Save the Children was the only USAID/FFP partner implementing CCTs in Sierra Leone (Radice, June 2017). CCTs were intended to restore the functioning of local markets which were disrupted during the outbreak as a result of quarantine measures, travel restrictions and people’s fear of being infected with EVD. To help stimulate the local markets in Kailahun district, Save the Children through FEEDS activity targeted 400 small traders who were selling foodstuffs in the local markets in the district. Most of these small traders were women. Each trader received two consecutive CCTs of USD $200, two to three months apart, to restock his/ her goods and sell them with an acceptable profit margin. To receive the second tranche of funds, the CCT recipients needed to attend basic business skills training, invest the first tranche of funds received in restocking their goods/supplies, and accurately record these transactions, as evidence of the business skills they have acquired (Radice, June 2017). The FEEDS activity also provided conditional grants to Women in Agriculture Groups (WAGs) to enable them to purchase agricultural inputs. The UCTs amounted to an equivalent of USD $50 per month per participant for two months. FEEDS monitoring data suggests that CCTs helped to stimulate local markets in Kailahun district. Almost all the CCT beneficiaries reported increased sales of their goods, compared to their pre-EVD levels, and the beneficiaries reported that the business concepts they had learned helped them to improve their sales (Radice, June 2017). Social Protection through Seed Voucher Interventions Seed vouchers were an important component of the food security programs implemented by CARE, ACDI/VOCA, and World Vision in their respectively program districts. ACDI/VOCA’s SNAP+/EFSP seed voucher intervention provided emergency seed recovery packages, primarily to provide an additional source income for beneficiaries, but some seeds were also provided for household use. The SNAP+/EFSP activity also gave seed grants to ‘apex’ farmers i.e., farmers who act as seed banks), as many of them did not have the capital needed to restart their seed banks once the EVD epidemic had ended. (ACDI/VOCA, 2017a; ACDI/VOCA, 2017b; ACDI/VOCA, 2018). M–110 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES The CARE RESSNER program provided seed vouchers to 900 farming households during the second phase of the program. This conditional support targeted household members who had the capacity to resume farming activities and diversify their agricultural production, and met the following conditions: i) From an EVD￾affected household, ii) Not a current recipient of seed vouchers from any other source, iii) Cannot access or afford the cost of quality seeds iv) Has access to farmland for cultivation, and v) Not a beneficiary of an unconditional cash transfer intervention. Households which met these selection criteria were trained on improved agronomic practices, post-harvest management and crop diversification. The participating farmers received the seed voucher after completing the training (CARE Sierra Leone, April 2018). World Vision provided seed vouchers to farmers across six chiefdoms in Port Loko. A total of 3,302 farmers were supplied with seed rice and trained in Post-Harvest Management, with a focus on post-harvest losses, handling, and storage (Radice, June 2017). Social Protection through Food Supplements As part of its food security support to the most vulnerable households, the SNAP+/EFSP activity supported the screening and complementary feeding of malnourished children. Children between the age of 2 and 59 months who were diagnosed with, or were at high risk of Moderate Acute Malnutrition (MAM), were provided with a locally manufactured nutritional supplement (Sierra Mix) to boost their intake of protein and other critical nutrients (Mbevi, November 2016; ACDI/VOCA, 2018). The USAID/FFP recovery program also provided support to the MOHS (through UNICEF) to resume the screening, referral, and treatment of children with severe acute malnutrition (SAM) (as these services were discontinued during the outbreak). USAID/ FFP program provided UNICEF with U.S.-sourced, ready-to-use therapeutic food products for the treatment of children with SAM (USAID (c). n.d.). Social Protection through Access to Credit and Savings Participation in Village Savings and Loan Associations (VSLA) helps individuals and small groups to pool their resources on a regular basis and to access credit for economic and social purposes. VSLAs also provide basic financial literacy for members as they learn to track their deposits and loans. Members gradually increase their contributions as they gain experience and confidence in their ability to make sound investment decisions. ACDI/VOCA provided small recovery grants of approximately USD$ 350 per group to 170 VSLAs in 96 locations in the program districts. These VSLAs experienced major financial losses during the EVD outbreak when their members were unable to make their repayments. The majority of the VSLA members (60-70 percent) were female, and the members were trained in bookkeeping (Mbevi, November 2016; Radice, June 2017). SNAP+/ESFP program results revealed that during the recovery period, many CT beneficiary households joined VSLAs that were operating prior to the EVD outbreak, and this was possible because of the cash support they had received from the cash transfer program. Reactivating VSLA activities has helped households and communities regain some of their resilience (Radice, June 2017; ACDI/VOCA, 2018). SBCC through Training in Health, Nutrition, and Household Resource Management Practical training and demonstrations were provided under various USAID/FFP agricultural support and food security recovery interventions. The training often had to be completed before a participant receives the support, viewed as essential in improving the knowledge and practices of program participants and guiding their decision making on the allocation of household resources. Some of the topics covered include food production, business start-up, financial management, health, infant feeding practices, household resource management, and post-harvest management. Generally, the training activities targeted the same individuals and households participating in CT and seed distribution activities, and the training sessions were usually conducted at cash distribution points and on the day of the cash distribution (Radice, June 2017). OUTPUTS AND OUTCOMES The available evidence suggests that Cash Transfer (CT) programs have improved the lives of program beneficiaries in targeted households and districts in Sierra Leone. Results from the performance evaluations of the RESSNER program indicate that the UCT interventions implemented under the program were effective in restoring household food security to vulnerable, food insecure households targeted by the program. UCT program beneficiaries were able to meet their basic food needs and avoid the negative coping strategies used in EVD-affected households in the two program districts. Other positive, unexpected outcomes included high dietary diversity, low household hunger, and improved economic activities in extremely poor households across the two districts. The RESSNER program was implemented in two phases – Phase 1 (August 2015-January 15, 2017), and Phase 2 (January 16, 2017-December 2017). A total of 7,567 extremely poor households were reached with UCT of $30 per household per month during the period 2015-2017. The RESSNER program also reached 900 poor households with seed vouchers implemented during the second phase. The impact of the RESSNER program is reflected in significant improvements from the baseline values in nearly all the key outcome indicators (see Table SL5) (Inclusive Development and Research Consultancy, October 2017). The main achievements of the ACDI/VOCA SNAP+/EFSP activity in Sierra Leone include: ■ CT payments to 124,220 beneficiaries were successfully completed in the four program districts (Kailahun, Bombali, Tonkolili and Koinadugu districts). ANNEX M. DESK REVIEWS – SIERRA LEONE n M–111 Table M–SL5. Changes in selected outcome indicators of household food security in the RESSNER program districts, Sierra Leone Outcomes Indicators Baseline values Target Endlline values z-value Statistical significance Reduced negative coping strategies among extremely poor households in EVD￾affected communities % HH beneficiaries reporting negative coping strategies 80% 30% 40% 7.7 Highly significant change p<0.05 Increased access to cash for purchase of basic food items by targeted household beneficiaries % HH beneficiaries with increased number of meals per day as compared to baseline 26% 70% 84% 12.6 Highly significant change p<0.05 % of HH beneficiaries reporting increased average daily income from current base income (Le 10,000) 33% 80% 66% 6.5 Highly significant change p<0.05 % HH beneficiaries reporting increased consumption of food groups (consumption of more than 6 food groups (Household Dietary Diversity) 41% 70% 94% 14.1 Highly significant change p<0.05 % of total average daily cash spent by HH beneficiaries on food items 47% 60% 59% 2.3 Significant change p<0.05 % of total average daily cash spent by HH beneficiaries on non-food items 35% 30% 40% 1.0 No significant change Improved ability of households to recover from the impacts of EVD % of HH beneficiaries reporting they have re-engaged in agricultural activities 9% 80% 89% 17.7 Highly significant change p<0.05 Number of HH beneficiaries who are able to save seeds for the next planting season 0% 75% 95% 17.7 Highly significant change p<0.05 % of beneficiary HHs engaged in savings activities (VSLAs/ROSCAs) 15% 40% 59% 8.4 Highly significant change p<0.05 Source: Inclusive Development and Research Consultancy (IDRC). Evaluation Report. Rapid Ebola Social Safety Net and Economic Recovery Activity in Bombali and Tonkolili Districts in Sierra Leone. October 2017) Table M–SL6. Distribution of key economic activities by gender of household head (HHH) Economic activity % HHs engaged in economic activity, baseline N=111 % HHs engaged in economic activity, endline N=526 Male HHH n=34 Female HHH n=77 Male HHH n=105 Female HHH n=421 Agriculture 97 91 90 88 VSLA 21 10 52 46 Petty trading 41 34 44 59 M–112 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES ■ A Cash Transfer Working Group was established and is functioning, enabling government, partners and donors to coordinate efforts and harmonize approaches. ■ 1,225 children with or at risk of MAM were identified, screened, and treated with food supplements. ■ 285 of the farming groups formed under the GoSL Smallholder Commercialization Program (consisting of approximately 23,800 farmers) were provided with seed packages through a voucher program to resume agricultural production. ■ 9,600 CT program beneficiaries were enrolled in activity￾supported VSLAs. ■ An agricultural grant program was established and is providing recovery (“Jump Start”) grants to small agribusiness enterprises heavily impacted by the EVD outbreak. The agribusinesses are now again able to provide inputs and other services to poor farmers and other individuals engaged in agricultural production activities (ACDI/VOCA, 2018). CRS conducted a midterm evaluation in June 2016, ten months after the first distribution of cash. A sample of 470 participants from 25 communities in the target chiefdoms responded to a structured questionnaire. The survey results show that: ■ The percentage of households that reported moderate or severe hunger decreased substantially from 68 percent at baseline to 27.5 percent at midterm. ■ The average dietary diversity score, which measures the number of different food groups consumed and captures the quality of diet, increased from 2.3 at baseline to 5.5 at midterm among participating households. ■ Supplementary (“top up”) payments (provided to mitigate the impact of high inflation on the participants’ purchasing power) were mostly used to buy various agricultural inputs: seeds (81.2%), payment for farm labor (62.0%) and buying other agricultural inputs (38.4%). ■ Almost half of the respondents (48.4%) reported that their farm land under cultivation has increased because they now have access to most farm inputs (CRS, 2016a; CRS, 2016b). Gender Considerations in Agriculture and Food Security Given that a higher proportion of poor households were headed by females, and the majority of VSLA members were women, some EVD recovery programs in Sierra Leone put extra emphasis on targeting women and female-headed households for their cash transfer interventions. From a gender perspective, the results were very encouraging. The RESSNER program’s final evaluation showed that more women were registered as program beneficiaries than men. The evaluation also revealed that while engagement in VSLA activities and petty trading had increased for both male- and women￾headed households in the program districts between the baseline and endline, the observed increase was significantly higher for female-headed households than for male-headed households. The proportion of female-headed households engaged in petty trading increased by 25 percentage points (from 34 percent to 59 percent) between the baseline and endline, while the proportion of male￾headed households only increased by 3 percentage points (from 41 percent to 44 percent) during the same period (see Table SL6) (Inclusive Development and Research Consultancy, October 2017). Analyses on decision making between male- and female-headed households revealed that 70 percent of female-headed households reported strong involvement of women in decision making on the use of cash resources and expenditure, compared to 25 percent of male-headed households reporting similar involvement by women in decision making regarding the use of cash resources and expenditure. This finding suggests that by targeting vulnerable female-headed households with cash transfers, the RESSNER program empowered more women to make independent decisions over the use of cash resources. (Inclusive Development and Research Consultancy (IDRC). October 2017). Educational Benefits of CT Programs during the Post-Ebola Recovery Period Program and evaluation results strongly suggest that CTP activity interventions had a positive impact on children’s access to education in Sierra Leone during the post-EVD recovery period. The Gender Impact Assessment of Cash Transfers Support to Vulnerable Households in Sierra Leone found that the SNAP+ CTP intervention had a significant impact on children’s access to education. Ninety percent of children in CTP recipient households indicated that they were able to return to school as a result of the program. This was a significant change, given that some of the children had been out of school for as long as two years. In contrast, 97 percent of children from the non-CTP households indicated that they had dropped out of school due to lack of money for education. It is also important to point out that most CTP participants could not afford to send all their children back to school with the amount received. However, their decision regarding which specific children to send to school was reportedly not based on the sex of the child, nor did it indicate a preference for boys’ education over girls’ education or vice versa. The participants indicated that their decisions were based on whether the child was considered a “high performer” or “was about to sit a final exam.” Both the parents and children of CTP and non-CTP participants reported that they viewed education as a safety net for them and a way out of poverty (Mbevi, November 2016). MANAGEMENT ISSUES WITHIN THIS THEMATIC AREA Program review and evaluation findings on a number of USAID/ FFP Pillar 2 recovery activities documented concerted efforts by IPs to coordinate activity activities with national and local government agencies, to ensure project activities were aligned with ANNEX M. DESK REVIEWS – SIERRA LEONE n M–113 GoSL’s recovery priorities and program objectives. The activity teams ensured that the CT program interventions were aligned with the National Standards and Policies on Social Safety Net. This was done through frequent consultations and collaboration with NaCSA, local government authorities, and community-based organizations in the planning, implementation and monitoring of project activities. NaCSA is a semi-autonomous government agency mandated (among other responsibilities) to “provide and… engage in social relief programs and to promote community￾based…and sustainable development activities leading to the alleviation of poverty…” (GoSL, 2008). It is the lead agency on social protection in Sierra Leone and maintains a database of all CT beneficiary households in the country. The commission worked closely with USAID/FFP activities in identification of potential CT beneficiaries, training of field staff on beneficiary selection processes, establishment of the cash transfer system, and external monitoring of activity implementation (Government Inclusive Development and Research Consultancy, October 2017). Cash Transfer Working Groups (CTWGs) were the primary platform for obtaining and providing input, sharing information, coordinating strategies, harmonizing approaches, and leveraging resources. CTWGs were used by USAID/FFP implementers to coordinate beneficiary selection, cash disbursement processes, and messaging (Radice, 2017). USAID/FFP representatives, along with implementing partners, and representatives of relevant government-line ministries and agencies, generally participated in the CTWG meetings. At district level, District Councils and DHMTs monitor and evaluate interventions in their respective districts and report to central MDAs. To ensure that district-level CT activities were aligned with district priorities and agreed-upon recovery development plans, District Councils periodically organized district forums to review progress on program interventions, and to discuss and resolve issues. Presidential working groups based in every district are also responsible for monitoring development interventions implemented at the district and local levels, including EVD recovery efforts (GoSL (b), n.d.). Additional coordination and program review mechanisms specific to the cash transfer programs were the Post Distribution Monitoring (PDM) activities and After-Action Reviews (AAR) The AAR usually brings together key stakeholders from the program communities, as well as district- and national-level government officials involved in the CT program. At chiefdom level, Paramount Chiefs and Village Headmen are responsible for monitoring and evaluating the implementation of project activities in their respective chiefdoms and villages. They provide their feedback to district councils and district MDAs (GoSL (b), n.d.; Inclusive Development and Research Consultancy, October 2017). Governance PROBLEM STATEMENT Weak governance systems and institutions, limited accountability and transparency, and limited civic participation of the population, have been serious challenges to development in Sierra Leone, even before the EVD outbreak. USAID investments in governance in Sierra Leone during the recovery period were aimed at strengthening accountability and transparency by government authorities in the management of public resources and the delivery of services, and encouraging citizen engagement in the planning, implementation and monitoring of project activities and services. The ultimate aim of USAID’s assistance to this sector in Sierra Leone was to strengthen accountability and transparency in the management of public health services at the national and local levels and increase public confidence in the public health system (USAID Presentation, October 3, 2017). USAID RESPONSE Community engagement in health services monitoring has been encouraged through community dialogues, radio listening groups, and participation in health promotion activities. Facility Maintenance Committees (FMCs) have been established under various recovery activities and are serving as an interface between the health facilities and communities, helping to facilitate communications about community needs and concerns, and to re-establish trust between the health facilities and the communities. In addition, a Community Engagement Strategy and Toolkit have been developed and are used to engage local communities (through the FMCs) in the operation and maintenance of primary healthcare facilities (JSI, n.d. (e)). Community-based organizations, VSLAs and advocacy groups have been helped to empower women through economic and social support activities. In addition, many food security and health recovery activities have used gender-sensitive approaches (such as targeting female-headed households to identify potential program beneficiaries). All these efforts are having a positive impact on women’s decisions regarding the use of household resources, and the lives of women and their families. The effects have been discussed under the food security and health sections of the report. M–114 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Innovation, Technology and Partnerships (ITP) PROBLEM STATEMENT The key challenges facing private entrepreneurs in Sierra Leone are limited access to capital, inadequate communications, information and technology infrastructure, and underutilized mobile technology for business transactions. USAID RESPONSE Pillar II funded 14 innovations through the regionally focused Fighting Ebola Grand Challenge initiative out of the Center for Innovation and Impact in the Global Health Bureau (CII). This business-minded approach has helped CII to spark new solutions to serious global health challenges from malaria to HIV to maternal, newborn, and child health. Including this challenge, CII has leveraged over $300 million in outside capital to develop and test 150 innovations in 35 countries, of which 25 were scaling or transitioning to scale in 2018. These challenges jumpstart innovative product developments by providing seed funding for research, development and testing. The Fighting Ebola challenge was issued via a Broad Agency Announcement (BAA) mechanism to innovators to “develop new practical and cost-effective solutions to improve infection treatment and control that can be rapidly deployed (1) to help healthcare workers provide better care and (2) to transform our ability to combat EVD” (USAID, 2014). The original challenge received 1500 ideas. With input from the White House Office of Science and Technology Policy (OSTP), the DOD, the CDC, OFDA, Offices in the Bureau of Global Health, the Global Development Lab, the Africa Bureau, and experts from USAID missions, which informed the team about current conditions in West Africa, the CII winnowed the 1500 to 30 semi-finalists and finally 14 winners (See Table SL7). Twelve of the 14 innovations have been field-tested in West Africa. Six of the 14 Fighting Ebola Grand Challenge innovations (see were field tested in Sierra Leone. These included two innovative devices to enhance patient care and HCW safety (Shift Lab’s Drip Assist, and STAMPS remote monitor of patients’ vital signs – see below), two improved designs for emergency health posts (Baylor’s Emergency Pod, Makerere University’s Epi Tent), and two ICT innovations to improve HCW access to timely health information in outbreak and routine circumstances (IBM EPIC platform and Dimagi’s CommCare platform). During the post-EVD recovery period, there has been a stronger desire in the country to engage the private sector more, and specific activities where progress has been made include: i) m-Hero, a mobile phone-based communication system used to link the MHS and healthcare providers through text messaging. It was developed by UNICEF and IntraHealth to facilitate communications within the health system during the EVD outbreak in Liberia. A communication tool (open source platform) called Rapid Pro has been developed to support data collection efforts and share data in real time. RapidPro is currently used in Sierra Leone by the MHS to send data and communication messages to HCWs. HCWs also use RapidPro to send data back to the healthcare facilities. This communication tool has been very effective and is now going to be turned over to the GoSL MHS; ii) E-payments: The Central Bank of Sierra Leone is making progress in putting in place regulations, policies, and procedures to facilitate e-payments. These measures are designed to address the lack of transparency and increase financial inclusion in the financial sector. An estimated 85 percent of the population of Sierra Leone does not have a bank account, and therefore they are not formally engaged in the national economy, let alone in the global financial sector. The e-payments and related policies provide opportunities for the population to get included in the national financial system and enjoy its benefits; iii) Digital Financial Inclusion: USAID has supported the establishment of the Digital Financial Services (DFS) Working Group, led by the Central Bank of Sierra Leone. In partnership with the UN Capital Development Fund (UNCDF), USAID is funding an activity called Digital Financial Inclusion in Fragile and Conflict States. This activity aims to extend financial inclusion of the general population through policy development, regulatory measures and technical assistance; iv) Global Money Agent Network Strengthening: USAID is providing technical assistance to two telecommunications companies – ORANGE and AFRICEL – under an activity called Agent Network Strengthening project (ANSP). The activity aims to extend financial inclusion. The focus is on global money agents who are hired by these companies to market their products in order to provide mobile financial products and services. The activity provides TA to the team of managers who manage these money agents; and v) E-Health Coordination Hub: In the health sector, USAID has helped establish an e-Health Coordination Hub, (see discussion under Health) (Gamma Roberts, USAID, Global Development Lab, personal communication, December 5, 2017). OUTPUTS AND OUTCOMES Many of the initiatives discussed above were launched in 2017, so it is too early to assess results. Achievements to date include: ■ With regards to e-payments, the Central Bank of Sierra Leone (BSL) has established Digital Financial Services and Financial Literacy Working Groups and drafted a policy framework to support innovative solutions from private sector Fin Tech firms (USAID, October 2017). ■ SIA/MicroSave is working with mobile network operators to strengthen services designed to increase financial inclusion. It has trained over 30 management staff of mobile network operators Africel and Airtel/Orange on a range of topics, including agent recruitment, training, compensation, management, fraud prevention, liquidity management, and marketing (USAID, October 2017). ANNEX M. DESK REVIEWS – SIERRA LEONE n M–115 Table M–SL7. Fighting Ebola “Grand Challenge” innovations Innovator Product Award (nearest 000) Duration Objective met? Comments Johns Hopkins Univ. Improved Personal Protective Equipment 794 6/15-6/17 Yes Improves visibility of and for HCW; Improved heat management; Simplified doffing (reduced from 8 to 4 pieces); improving HCW safety and confidence. IntraHealth Int. Scale up mHero, mobile phone-basd platform for MOH – HCW communication about EVD/other health issues 700 6/15-9/16 Yes Refined the architecture and user support for mHero, to strengthen the health systems in all 3 countries. Enabled mHero to go to national scale in Liberia, and mHero has been integrated into the MoH HMIS in Liberia and Guinea. SPR Advanced Tech Inc. Improved antiviral protective coating 656 5/15-8/16 No Innovative approach to longer lasting protection, but additional R&D is required to resolve technical shortfalls. Columbia University – Kinnos “Highlight” powdered color additive to improve EVD decontamination w. bleach 649 6/15-7/16 Yes Enables “visualization” of decontamination, improving HCW safety and confidence. Field tested in all 3 countries; now widely in use. Scripps Health “STAMP” – wearable sensor and personalized analytics￾monitor patient (Px) vitals 632 5/15-5/16 Yes Field tested by IMC in Sierra Leone. Potential for multiple uses; monitoring heat stress in HCW . Baylor College of Med. Develop light weight, re-usable, transportable Emergency Smart Pod 614 6/15-6/17 Yes Collapsible, movable ETU requiring only 30 minutes training to set up, and with estimated production cost of $50K-$150K, and “shelf life” of 10-15 years. TOMI Environmental Solutions SteraMistTM Mobile Decontamination chambers 559 5/15-2/16 Yes SteraMist is FDA approved; Expanding testing to Philippines, Malaysia, Mexico, and Panama; SteraMist will be pre-positioned in the US for future outbreaks. SteraMist is also EPA and FDA registered, included in the WHO Compendium of Innovative Health Technologies for Low Income Countries, and distributed worldwide. IBM Research Africa EPIC platform & analytics for front line HCW decision￾making 527 7/15-3/17 Yes EPIC gathers and integrates data from stove￾piped information sources (DHIS-2, IP data sources, etc.) to provide both health and context data relevant to district level health services decision-making. Successful test in one district in Sierra Leone. Modula S Modula-S Rapidly Deployable Emergency Medical Treatment Unit - Antimicrobial Copper CuVerro Materials 500 5/15- 12/15 Yes Hard-shell popup ETU building, autonomous & solar self-powered with direct contact kill antimicrobial CuVerro copper interior surfaces. Wall mounted beds, and solid floor surface pitched to drains to facilitate easy and thorough cleaning. Third generation design, now available, one third the shipping size and ISO container building modules start at $500,000. Makerere University “Epi-tent” – a next generation hospital tent with improved air flow and heat exchange 482 6/15-4/16 Yes Production in Uganda; In use in refugee settlements; Included in WHO Compendium of Innovative Health Technologies for Low Income Countries. 4 design patents filed, new housing use case design optimization. M–116 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Table M–SL7. Fighting Ebola “Grand Challenge” innovations International Personal Protection Test 3 concepts/designs to improve PPE 324 5/15-8/16 Yes Participated in expert group; tested 3 models and identified preferred. Overall cost still too high ($85/unit). Shift Labs “Drip Assist” precision infusion monitor 319 5/15-6/16 Yes In collaboration with Z/Map, Drip Assist scaled throughout West Africa. In use in 18 developing countries Dimagi Inc. Adapt CommCare platform for EVD surveillance and buid capacity 299 5/15-5/16 Yes Developed 7 templates based on CommCare, Dimagi’s open-source mobile platform that is in use in 50 countries. It allows tracking of individuals over time, so it suited the need to track symptoms of suspected EVD cases for 21 days. 3D Family Productions Africa Stop Ebola campaign in Guinea 269 5/15-8/15 Yes Implementing partner – MSF. ANNEX M. DESK REVIEWS – SIERRA LEONE n M–117 SUMMARY OF KEY FINDINGS IN FOUR DOMAINS Performance AGRICULTURAL SUPPORT AND HOUSEHOLD FOOD SECURITY Based on program data generated by various USAID Pillar II recovery activities, it is estimated that more than 67,000 households, including over 364,000 individuals, received cash transfers under USAID-funded agriculture and food security interventions. In addition, 400 predominantly female small-scale traders received conditional cash transfers, conditional upon their completion of training and demonstration of business skills. Grants to small-scale traders helped traders increase their food stocks in the market, thereby making it possible for farmers to purchase agricultural inputs. Inputs of seeds, fertilizer, and training, provided through IPs, increased productivity and diet diversity, positively impacting household income and nutrition. USAID/FFP support to EVD recovery efforts in Sierra Leone has also helped to establish or reactivate 359 Village Savings and Loans groups thereby mobilizing over USD $84,340 USD in savings; and a total of USD $29 million in loans were disbursed to 65 of 168 cluster farmers (USAID, November 27, 2017). The overall impact of USAID/FFP CT interventions in Sierra Leone has been positive. CT participants were able to purchase food, meet other household needs and invest in other critical areas such as paying school enrollment fees for their children and engaging in economic activities. Some evaluation results have shown that CT recipients were more likely to engage in economic activities and believed they were in a better position to cope in the event of another crisis (a possible indicator of resilience), while non-cash recipients believed they remained vulnerable to future shocks. The targeted cash transfers and agricultural input vouchers, combined with training and other complementary activities, have helped food insecure households to access food and increase their purchasing power, while enhancing the recovery of markets. Reports suggest that the food security situation in Sierra Leone is improving. The harvest of the main seasonal food crops has matched expected targets, and poor households are better able to meet their basic food needs and other household requirements. As of January 2018, according to FEWS NET, recent harvests and labor activities are returning to pre-EVD levels and enabling poor households across the country to meet their basic food security needs. As recovery from EVD progresses in Sierra Leone, FFP will be gradually transitioning out of emergency programming to a development focus, while maintaining its food security monitoring and reporting activities in the country (USAID, January 2018). HEALTH SERVICES AND HEALTH SYSTEMS Impressive results have been achieved in the health sector, over a relatively short period (18 months or less for most activities). 305 PHUs have been rehabilitated, and equipped with basic lifesaving medical equipment, improving not just their physical conditions, but also their functionality and the quality of services available. Over 900 health professionals, 1,500 CHWs and peer supervisors, and members of 214 FMCs have been trained. These interventions have improved health services for an estimated two million Sierra Leoneans. The health promotion activities and community engagement activities have restored some trust and encouraged people to seek care at public healthcare facilities (JSI Training and Research Institute, June 2017). LESSONS LEARNED 1. Multi-sectoral, integrated programming of CT programs enhances their effectiveness. The CT recovery interventions were multi-sectoral and integrated, to address the multifaceted needs of highly vulnerable populations. 2. Financial and social safety nets are essential for building resilience. The Pillar II interventions contributed to both financial and social safety nets through cash transfers, training, and support to seed banks, and VSLAs. 3. Building human capacity contributes to resilience and promotes sustainability. The findings from CT programs in Sierra Leone show that providing CT program beneficiaries with the right kinds of messages on health and nutrition, and with complementary skills training in business, savings, and other productive areas, and linking them to economic opportunities not only helped them to make informed decisions about how to use their cash, but better prepared them to cope in the event of another crisis. 4. When given proper guidance and support, FMCs can bridge the communication gap between local health services and local communities, and improve healthcare utilization. 5. VSLAs have been an effective complementary strategy used in Sierra Leone for building the resilience of vulnerable households receiving cash transfers through the SNAP+ and other CT programs. In addition to providing a safe place for their members to store their cash, VSLAs have promoted investments in income-generating activities, to increase the resilience of program beneficiaries. M–118 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Resilience A qualitative evaluation of the effects of the SNAP+/EFSP cash transfer intervention on household financial decision-making processes and coping strategies revealed that: i) CTP participants were better placed to cope as a result of receiving the cash than were non-CTP participants from at-risk households. During the EVD epidemic, the majority of CTP participants interviewed could not borrow from money lenders or relatives because they could not pay it back. Non-CTP participants interviewed coped by eating less, with parents skipping meals so that their children had at least one meal a day. After the EVD crisis, none of the CTP participants interviewed borrowed money from money lenders, as they were better able to use the cash received to meet household needs. In contrast, 13 percent of non-CTP participants sought to borrow from money lenders, but many of them were not eligible for a loan. CT participants who were VSLA members decided to borrow from VSLAs at a much better rate than the rates offered by money lenders. This study demonstrates that membership in a VSLA can empower vulnerable populations, particularly women. It gives members an opportunity to access credit, improve numeracy, literacy, and business skills, and build resilience and financial independence (Mbevi, November 2016). Most of the CT beneficiaries under RESSNER engaged in economic and livelihood activities to increase their income and to protect themselves from future shocks. Eighty-nine percent of the household beneficiaries reported to have invested the cash received in agricultural activities, and 59 percent engaged in savings activities. (Inclusive Development and Research Consultancy, October 2017). Sustainability The sustainability framework developed by the ET for the overall Ebola Pillar II evaluation outlines key sustainability determinants: (1) Policy, (2) Finances, (3) Accountability, (4) Institutional Support, (5) Technical Choices, (6) the Human Dimension, and (7) External Threats and Enabling Environments (see Box 1). 1. Policy—Sierra Leone contributes to an enabling environment for sustainability. Sierra Leone’s Pillar II activities supported national policies and/or included explicit or implicit strategies for building sustainability with particular emphasis on building local capacity, fostering community engagement and ownership, and developing policies and tools at the national level to institutionalize innovations and improved practices. 2. Finances—As described earlier, many activities are of short duration and funded by outside sources. In some cases, community efforts may help to build resilience against economic shocks. In AFS activities, VSLAs build community capacity to purchase agricultural inputs in lean times. However, it remains to be seen how larger level investments such as building public￾private partnerships, and government commitments to social investments, may contribute to sustainability in the future. 3. Accountability—Community engagement in health services monitoring through community dialogues, radio listening groups, and participation in health promotion activities have increased accountability at the community level. Facility Maintenance Committees are helping to facilitate communications about community needs and concerns, and to re-establish trust between the health facilities and the communities. 4. Institutional Support—Pillar II activities infused GoSL efforts with state-of-the-art information, tools and approaches. When formal guidance did not exist, activities worked closely with GoSL counterparts to make that guidance available. These activities often funded the development of manuals and job aids to roll out the policies and incorporate them into training. MSH/SIAPS improved and updated the monitoring tools available for MOH workers and HC3 helped to develop SBCC manuals for improving public health communications. All Pillar II health activities worked in collaboration with the relevant MOH teams to improve capacity and policy that supported the GoSL EVD recovery plan. 5. Technical Choices—Pillar II activities centered on approaches with proven effectiveness. Nonetheless, specific choices regarding intervention modalities have implications in terms of program effectiveness and, ultimately, sustainability. This is certainly the case when one examines different approaches undertaken with social protection strategies pursued by different Pillar II AFS activities (e.g., direct cash distribution via banks versus mobile money platforms). As another illustration, among Pillar II IPs implementing health-sector activities, strategic decisions in relation to how the activity addressed supply chain gaps have implications in terms of sustainability. Building GoSL capacity and supply chain management mechanisms may be a more sustainable choice than temporary procurement of necessary items. 6. The Human Dimension—Community engagement and ownership of HC3 project activities was critical to sustaining activity achievements. Community members were involved in planning, implementation and monitoring of project activities, and documenting progress and achievements. This process built trust between the activity team and the communities. HC3 was able to document increased community ownership of health activities, improved relationships between facilities and communities, and improved awareness and intention to use services. Participation in capacity building and economic activities: Significant portions of USAID/FFP CT beneficiaries reported using a portion of the cash they had received to engage in economic and livelihood activities in order to protect themselves ANNEX M. DESK REVIEWS – SIERRA LEONE n M–119 BOX 1. DEFINITIONS: KEY DETERMINANTS THAT CONTRIBUTE TO SUSTAINABILITY IBTCI utilizes seven potential categories of distal and proximate determinants that enhance or impede sustainability: policy, finances, institutional support, technical choices, accountability, human dimension, external threats and enabling environment. Below are the definitions for each determinant. 1. Policies, Strategies, and Plans—Policies are predicated by procedures, rules, and allocation mechanisms to provide services and/or to set priorities and guide programs. Policies may be implemented through four types of instruments: laws, regulations, assignment of responsibilities to institutions operating in the sector, and economic incentives (i.e., subsidies) for a given sector to operate properly. Hence, a determinant labeled “policy” will indicate both how it is manifested and also how it is implemented. IBTCI submits a broad definition for policy: Policy is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. Policies with sound strategies are more likely to be adopted. Likewise, strategies need comprehensive and clear plans in order to be implemented. Thus, policies, strategies, and plans are subsumed under one overarching determinant, as they should be inextricably linked in order for the effect of the policies to be realized. Therefore, the IBTCI definition for the first determinant is expanded: There is a law, regulation, directive, or guidance in place that outlines priorities and standards to guide the efforts of an institution, sector, or the country as a whole. There are also extant strategies and plans for policy implementation. Policies, strategies, and plans create a framework for the implementation of any of the activities associated with post￾EVD recovery. An example might be a health policy or an agricultural development policy that was conceived either as the result of USAID funding and/or influenced or shaped by the Pillar II interventions that USAID supports. 2. Finances—As noted, policies and strategies are operationalized through plans. The implementation of plans, however, requires budget allocations and access to funds. In regard to finances, IBTCI defines this determinant as: Allocation of financial resources for the implementation of plans is in place, with a determination of how much funding is needed, the source of those funds, and how the funds will be apportioned and used. 3. Accountability—Accountability describes the mechanisms and procedures through which decision-makers, policy makers, and implementers justify and assume responsibility for the actions, decisions, services, and products that they undertake. Accountability is best proven through written documentation. There are different types of accountability. Further defined, the salient aspects of accountability follow (UNDP Water Governance Facility/UNICEF, 2015): a. Financial accountability means that institutions and individuals must document the intended and actual use of financial resources allocated to them. A clear demonstration of how the funds were expended should be explicit and without ambiguity. b. Political accountability occurs when constituents request their representatives to justify their actions. If there is political accountability, there are no questions as to the legitimacy of the authority or position of a politician. There may be vertical channels of accountability that link citizens to their government. Horizontal accountability refers when partners of equal standing request their counterparts to account for their actions (i.e., when one local government requests another to justify and explain their actions. c. Social accountability are actions taken by civil society, the media and individuals to incite or implore decision-makers to account for decisions and actions undertaken. d. Administrative accountability is exemplified when one level of an administration asks another to account for their decisions and actions and is documented in an organized manner. Guidance and protocols need to be in situ to ensure accountability. Governmental and other institutional and organizational systems are put into place to decide who gets what, when, and how. These mechanisms should be clearly delineated, and expectations should be communicated to relevant stakeholders. These may include incentives for high performance or disincentives/sanctions for inadequate or irresponsible performance. In sum, this determinant is defined as: Systems and/or mechanisms are in place to ensure appropriate use of financial resources and rights holders’ and duty bearers’ access to information (e.g., on performance and results). These systems and/ or mechanisms safeguard the responsible and constructive use of resources and have transparent processes that are manifested by the populace’s respect for the respective institution. 4. Institutional Support—This determinant refers to organizations that are responsible for implementing actions in support of a policy, strategy, plan, or program. Institutional support examines the actions that are implemented as part of that support and how it may be coordinated among the actors involved. Further down this determinate’s pathway to sustainability, exit and transition strategies are established. Institutional support is thus defined by IBTCI as: Organizational arrangements and/or mechanisms are in place that a) operationalize agreed-upon inputs, roles and responsibilities for M–120 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES implementing policies and interventions and b) produce exit and transition strategies when feasible and appropriate. 1. Technical Choices— Assuming resources are constant, a list of technical choices can be infinite. Hence, the Team will measure a technical choice based on the following criteria: a) appropriateness and relevance to the problem at hand and whether it will solve that problem in an efficient and effective fashion at best value; b) timeliness; and c) demand on resources (human, capital, physical). Thus, the IBTCI definition of the technical choices determinant is: The best technical approach has been selected to lead to implementa￾tion decisions determining whether interventions are “fit for purpose” (effective in addressing the identified problem), feasible given time and resource constraints, and technically relevant. 2. Human Dimension—The importance of the human dimension in the sustainability of development programs is multi-faceted. The human dimension might be as straight-forward as having a person available whose job is to maintain a newly refurbished school or health facility, or a cadre of information technology experts to maintain a fiber optic cable around Monrovia. On the other hand, the human dimension might include more oblique characteristics, such as cultural or institutional norms. Norms present challenges to discussion or systematic analysis as their salient components may be difficult to define. Questions to ask might include, “Is there stigma and discrimination and if so, to what degree?” The Team is exploring the degree to which Pillar II funding ameliorated the consequences of negative norms and/or directed their activities to change them. Transparent and equitable decision-making may be a harbinger for successful implementation of an activity or project, the uptake of services, or changes in social behaviors. This particular determinant stretches across all intervention typologies. Interventions need involvement from a human being to sustain it and serve as its champion. Arguably, this is a somewhat oblique determinant that is open to interpretation and further definition. Thus, for the purposes of evaluation, IBTCI defines human dimensions as follows: Any situation whereby a human being’s input is needed to sustain the intervention. Identify the human dimension— describe it and define it—and the level at which it is available in the appropriate quantity and quality. These nuances are then examined qualitatively to identify activities poised for success and sustainability. 3. External Threats and Enabling Environments—Complex environments, such as those areas in which EVD was most palpable, are influenced by external threats and/or buoyed by enabling environments. This category of determinants is focused on external threats and factors that influence the intervention outcome. They revolve around socio-political stability, geo-politics, climatic conditions, and population migration/internal displacement. For example, roads are needed for the transportation of goods and services to and from villages (e.g., the cold chain for vaccines). Torrential rains or natural disasters may affect the lifespan of roads, thus increasing the need for frequent maintenance or replacement. Climate then is a contextual factor that affects the sustainability of road access and use. Other threats may include internal conflicts between territories, elections, or natural disasters. External threats and enabling environments as a determinant are thus defined: Those factors that have a direct or indirect influence on the ability of an intervention to be sustained, even when all other determinants are supportive of a sustainable outcome. Sustained Results Outbreak Thematic Areas of Response Innovation, Technology, & Partnerships Economic Crisis Mitigation Governance Agriculture & Food Security Basic Education Non-EVD Health Services, Health Systems Recovery Social Protection Frontline Worker Support Management, Coordination, & Partnerships Information, Communication, & Technology Institutional Enhancements Social & Behavior Change Communications Intervention Types Policies, Strategies, and Plans Finances External Threats and Enabling Environments Human Dimensions Determinants of Sustainability Accountability Institutional Support Technical Choices Immediate Results Ebola Pillar II Response Challenge: Mitigate second-order impacts of the outbreak at the individual, community, and institutional level Survivor Programs Figure M–SL2. Analytical framework to measure sustainability ANNEX M. DESK REVIEWS – SIERRA LEONE n M–121 from future shocks. For instance, most of the RESSNER cash transfer and seed voucher recipients in the two districts reported using a portion of the cash to purchase agricultural inputs (78%), hire farm labor (59%), for small trade/business (54%), savings (37%), and animal rearing (19%). (Inclusive Development and Research Consultancy (IDRC), October 2017). The evidence from the SNAP+ program also shows that the VSLAs supported through the program enabled 9,600 CT program beneficiaries to sustain their livelihoods and their economic activities (ACDI/VOCA, 2018). The RESSNER evaluation found that the beneficiaries who received business skills training under the program were two times more likely to engage in income-generating activities than those who never received training (Inclusive Development and Research Consultancy (IDRC), October 2017). Under the UNICEF activity, CHWs are being trained to use RapidPro/mHero to report on outputs, successes and challenges. This will strengthen their ability to analyze and report on field activities and health service delivery at the facility level, and in their respective communities. All these results suggest that capacity building and economic activities may be key factors for building resilience in cash transfer programs. 7. External Threats and Enabling Environments—Although all Pillar II efforts focused on recovery and enhancements to aid preparedness for future emergencies, they generally did not explicitly address external threats such as climate change or sociopolitical instability, two factors that can thwart progress and/or undo past gains. Opportunities and Gaps OPPORTUNITIES Improvements in post-harvest management practices: It is estimated that cereal farmers in Sierra Leone lose about 35% of their produce at the harvesting stage, until the time the rice is ready for consumption or marketing. Getting farmers to be aware of these losses, training them and encouraging them to adopt practices that could minimize these losses would be a sound investment policy for sustainable food security. Addressing the business needs and interests of local private companies and businesses: The role of the private commercial sector was primarily transactional and limited to the provision of cash disbursement services for the CT interventions. Other opportunities for involving local private sector institutions in capacity building and productive sectors (e.g., in agricultural production, microenterprise development) were not fully explored. For example, local private financial service providers and microfinance institutions could play a larger role in ensuring that CT program beneficiaries receive much-needed training in business development and resource management, and by ensuring that CT beneficiaries are aware of and linked to available credit and savings opportunities. This may lead to the sustainability of income-generating activities of program beneficiaries after USAID funding ends. GAPS The USAID Pillar II Ebola recovery activities in Sierra Leone were not without setbacks and challenges. Some of the challenges and limitations faced by implementers informed the lessons learned and recommendations presented in this report. The APC activity faced challenges in finding sustainable water sources that would be available year-round. In some cases, geophysical survey readings indicated a very high chance of finding a sustainable water supply, but drilling yielded no water. Over time, some boreholes and hand-dug wells collapsed. At some sites, water quality tests indicated high iron content in the water after boreholes were drilled, while the original water quality tests had shown no iron content. Management The President’s Delivery Team (PDT) oversaw Sierra Leone’s recovery priorities. It provided technical support in monitoring and embedded its members in relevant ministries, departments, and agencies (MDAs). International and national CSOs have collaborated with these MDAs to reach the recovery priorities. The level of collaboration with sector-relevant ministries varied. The more formal relationships existed with CSOs working with the MOHS, which required service level agreements before the CSOs could operate. Dalan (2016) suggested that a middle of the road approach was adopted by the Ministry of Social Welfare’s Gender and Children’s Affairs (MSWG&CA), which relied heavily on CSOs to aid with social protection while the ministry’s staff monitored the process. At the end of the spectrum was the Ministry of Agriculture, Forestry and Food Security (MAFF) where the affiliated CSOs are reported to have largely implemented interventions without much collaboration with the Ministry. Sierra Leone enacted in 2011 the National Social Protection Policy, which covers CTPs. The cash transfer program implemented with Pilar II funding was aligned with the National Standards and Policies on Social Safety Net. The National Commission for Social Action (NaCSA) is a semi-autonomous agency in charge of the coordination and implementation of social protection interventions and maintains a database of all cash transfer beneficiary households in the country. NaCSA appeared to have operated in the same manner as the MOHS, as NaCSA worked closely with USAID/FFP activities in the identification potential M–122 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES CT beneficiaries, training of field staff on the beneficiary selection process, in establishing the cash transfer system, and in monitoring activity implementation. There are different synergies that have been productive between CSOs themselves as well as between line ministries operating on the ground at the district level. Pillar II activities in the health sector made substantial efforts to leverage synergies, technical expertise, and resources to avoid duplication of effort and maximize impact of. For example, HC3 linked with MSH’s SIAPS activity to ensure PHUs received essential storage equipment and packing support for facility drug stores when these needs could not be covered by the HC3 activity. HC3 also collaborated and coordinated closely with the APC activity, and HC3 implementing partner (GOAL) was required to have monthly activity partner meetings with JSI IPs in each district. UNICEF and USAID developed an effective partnership in Sierra Leone in several areas. USAID Implementing partners also coordinated and consulted regularly with their counterparts in the MOHS at all levels and program review meetings as well as Technical Working Groups (TWGs) are used by implementing and development partners and GOSL officials to review activities and resolve implementation problems and issues. During the response to the epidemic, new IPs were not always familiar with operating procedures in Sierra Leone. The District Health Management Committee had to provide guidance and coordinate the efforts of multiple IPs operating in the same geographic areas. This happened during the recovery efforts as well. Weekly coordinating meetings were organized at the district and were crucial to align and integrate activities. RECOMMENDATIONS 1. Continued support to community engagement activities is essential for sustainability. The evaluation findings for recovery activities in both the health and food security sectors has shown that involving local community-based organizations and structures (including CSOs and community-based organizations) in the planning, implementation, and monitoring of interventions and services is an effective strategy for building community support, and ownership of the program. 2. Targeted training activities designed to build specific skills are an essential ingredient for building individual and community resilience to future emergencies and should be included in any cash-based response to a humanitarian crisis. The RESSNER evaluation results revealed that beneficiary households that had received business training under the program were more likely to use a portion of the cash received for economic activities (Inclusive Development and Research Consultancy, October 2017). 3. Protecting access of the most vulnerable households to land and other critical resources. A qualitative assessment of the SNAP+ program revealed that in some communities in Bombali district, widows are sent back home to their parents and lose access to the land they owned with their husbands when their husbands die (Mbevi, November 2016). Food security and agricultural support interventions should look into and address land rights and laws. They should educate communities about existing laws to ensure women, and other vulnerable groups, are protected from being denied access and ownership of land during disease outbreaks and other humanitarian crises. ANNEX M. DESK REVIEWS – SIERRA LEONE n M–123 ACDI/VOCA. 2018. Sierra Leone Emergency Food Security Program. Supporting Vulnerable Households Recover from the Ebola Crisis. http://www.acdivoca.org/projects/sustainable-nutrition￾and-agriculture-promotion-snap-program-2. (n.d.) ACDI/VOCA. 2016. Sustainable Nutrition and Agricultural Program Plus (SNAP+): FY16 Annual Report, October 2015- September 2016. ACDI/VOCA. 2017b. Emergency Food Security Program (EFSP): FY17 Quarterly Report, January-March 2017. ACDI/VOCA. 2017a. Sustainable Nutrition and Agricultural Program Plus (SNAP+): FY17 Quarterly Report 1, October￾December 2016. Blanchet Karl, Nam Sara L, Ramalingam Ben, and Pozo–Martin Francisco. Governance and Capacity to Manage Resilience of Health Systems: Towards a New Conceptual Framework. International Journal of Health Policy and Management, August 2017. 6(8), pgs 431–435. Accessed online on November 24, 2017. CARE Sierra Leone. FY17 Q2 Program Report. Rapid Ebola Social Safety Network and Economic Recovery. Sierra Leone. AID - FFP – G – 15-00074. Submitted 18 April 2018. CRS. 2016a. Project Overview. Sierra Leone Country Program. Cash Based Food Security Assistance for Ebola Virus (EVD)- Affected Sierra Leoneans in Kenema District. CRS. 2016b. EFSP Post Distribution Monitoring Survey. EFSP Cash Assistance to EVD-Affected Sierra Leonean 2015-2017. Dalan Development Consultants and Forcier Consulting. Government of Sierra Leone (GoSL), 6–9 Month Early Recovery Priorities: Independent Evaluation and Assessment, Sierra Leone. May 2016. Department for International Development. DFID Sierra Leone. n.d. Emrey, B. (2016). Preparing for future shocks: Building resilient health systems. In J. Charles, T. Ifafore, & K. Greene (Eds.). USAID Global Health Mini-University. De Zalduondo Barbara, Bongiovanni Annette, Sadaphal Swati, and Moore Zhuzhi. Ebola Pillar II Monitoring, Evaluation, and Learning Project: Performance Evaluation Inception Report. AID-OAA-1-15-00022/AID-OAA-TO-16-00040. September 29, 2017. FEWSNET. 2015. Guinea, Liberia, and Sierra Leone Special Report. Washington, DC. Retrieved from www.fews.net/sites/ default/files/documents/reports/Ebola_SR_2015_03_en.pdf Gandi Sagestine L. Mid-term Evaluation Final Report “Giving Voice to the Youth of Sierra Leone”‐ Project. December 2016. Retrieved from http://www.taksvarkki.fi/tv/wp-content/ uploads/2017/01/Sierra-Leone-MID-TERM-EVALUATION.pdf Geraldine Fraser-Moleketi. AfDB. Ebola: The need for a gendered approach. 05 September 2014. Retrieved from https://www. afdb.org/en/blogs/investing-in-gender-equality-for-africa’s￾transformation/post/ebola-the-need-for-a-gendered￾approach-13559. Gillespie Amaya M., Obregon Rafael, El Asawi Rania, et al. Social Mobilization and Community Engagement Central to the Ebola Response in West Africa: Lessons for Future Public Health Emergencies. Global Health Science and Practice. 2016. Volume 4, Number 4. GoSL, 2015. An EVD Response – Social Safety Net Cash Transfers Standard Operating Procedures. Freetown. GoSL. 2008. The National Commission for Social Action (Amendment). Freetown. Parliament of Sierra Leone. www. sierra-leone.org/Laws/2008-08.pdf GoSL (a). Sierra Leone National Health Information System Strategic Plan (2017-2021). n.d. GoSL (b). Quick Guide to the President’s Recovery Priorities. n.d. GoSL. National Ebola Recovery Strategy for Sierra Leone 2015– 2017. July 2015. GoSL. The Agenda for Prosperity. Road to Middle Income Status. Sierra Leone’s Third Generation Poverty Reduction Strategy Paper (2013 – 2018). n.d. Health Communication Capacity Collaborative. HC3 Sierra Leone Country Program. End of project report. August 2017 (a). Health Communication Capacity Collaborative. HC3 Sierra Leone Country Program. End of project Report.; Annex I, Indicator Reporting Sheet. Final report. August 2017 (b). International Organization on Migration, n.d. Statement of Work, International Organization on Migration Grant. AID-GH￾IO-16-00001. Amendment 1. National Ebola Training Academy – Phase II. SUB-ANNEX A. REFERENCES AND SOURCES CONSULTED M–124 n PERFORMANCE EVALUATION OF USAID EBOLA PILLAR II ACTIVITIES Inclusive Development and Research Consultancy (IDRC). Evaluation Report. Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone. October 2017. JSI Research and Training Institute, Inc. Advancing Partners and Communities. Program Overview: Sierra Leone, Liberia, and Guinea. September 2016. JSI Research and Training Institute, Inc. Advancing Partners and Communities. Post-Ebola Health Services Strengthening Baseline Report. October 2016. JSI Research and Training Institute, Inc. Advancing Partners & Communities, Sierra Leone. Improving Community Health Services in the Post-Ebola Context [Infographic Fact Sheet]. n.d. (e) JSI Training and Research Institute. Inc. Advancing Partners and Communities. Post-Ebola Recovery: Strengthening Primary and Community Health Services. Sierra Leone. June 2017) JSI Research & Training Institute. Technical Brief (a). Strengthening Access to Mental Health Services in Sierra Leone. n.d. JSI Research & Training Institute. Technical Brief (b). Building Capacity Within the Sierra Leone Association of Ebola Survivors. n.d. JSI Research & Training Institute. Advancing Partners & Communities, Sierra Leone. Ebola Transmission Prevention and Survivor Services. n.d. (d) JSI Research and Training Institute, Technical Brief (c). Rebuilding Trust in Health Services: Engaging Communities through Facility Management Committees. n.d. JSI Research and Training Institute, Technical Brief (g). Rehabilitating Peripheral Health Units to Save Lives. n.d. JSI Research and Training Institute, Technical Brief (f). Strengthening the Health Workforce in Sierra Leone. n.d. JSI Research and Training Institute, Inc. Arlington, Virginia, USA. Advancing Partners & Communities. Ebola Transmission Prevention and Survivor Services Program. September 30, 2017. Mbevi Lydia. Gender Impact Assessment of Cash Transfers Support to Vulnerable Households in Sierra Leone. ACDI/VOCA. November 2016 Minister of Health and Sanitation (MOHS), Republic of Sierra Leone. National Community Health Worker Policy, 2016 - 2020. n.d. MOHS. Freetown. 30 June 2015. Health Sector Recovery Plan (2015 – 2020). Morlai Teddy Amara. End of Project Evaluation Report: Rapid Ebola Social Safety Net and Economic Recovery (RESSNER) Project in Bombali and Tonkolili Districts in Sierra Leone – Phase 1. June 2017. Management Sciences for Health (MSH), n.d. Sierra Leone Country Page. Retrieved from: https://www.msh.org/our-work/ country/sierra-leone. Accessed February 2018. Murphy, E.; Oot, L.; Sethuraman, K. 2016. USAID Office of Food for Peace Food Security Desk Review for Sierra Leone, 2018–2022. Washington, DC: FHI 360/FANTA. Radice Welcome Holly. June 2017. Cash Transfers for Food Security in Epidemics. A Review of the USAID Food for Peace Response to the Ebola Crisis in Liberia and Sierra Leone. Sierra Leone Health Facility Survey 2014. Assessing the impact of the EVD Outbreak on Health Systems in Sierra Leone. Survey conducted 6-17 October 2014. December 3, 2014. Retrieved from https://www.unicef.org/emergencies/ebola/files/SL_ Health_Facility_Survey_2014Dec3.pdf Squire Sylvester J, Hann K, Denisiuk O, et al. The Ebola Outbreak and Staffing in Public Health Facilities in Rural Sierra Leone: Who is Left to Do the Job? Public Health Action. 2017. 7 (S1), page S47 – S54. UNICEF. Briefing Note - Update on UNICEF’s USAID Supported Work. May 2017. UNICEF. Proposal. Strengthening Post-Ebola Health Governance in Sierra Leone. Project Period: 1 April 2017-30 September 2018. n.d. UNICEF. Progress on drinking water, sanitation and hygiene 2017 update and SDG baseline. Accessed online on 2 February 2018. USAID Africa Bureau. (2015). SBU AEU Strategic Program Review 10-28-2015. USAID. Powerpoint Presentation, Ebola Pillar II Learning Conference in Sierra Leone. November 27, 2017. USAID. Powerpoint Presentation. ICS Goal 3: Health Initiatives. 4 October 2017. USAID. Powerpoint Presentation, Ebola Programs Team – Africa Bureau TDY Debrief + Strategic Program Review. October 3, 2017. USAID (a). Partnering to Improve the Daily Lives of Sierra Leoneans. Increasing Health, Food Security, Nutrition, Government Accountability, Transparency, and Responsiveness. www.usaid.gov/sierra-leone.ppt. n.d. USAID (b). Pillar II: Mitigating Second-Order Impacts of the Ebola Virus Disease (EVD) Epidemic. A Strategic Framework for Liberia, Guinea, Sierra Leone. n.d. ANNEX M. DESK REVIEWS – SIERRA LEONE n M–125 USAID (c). Food Security. n.d. Retrieved from https://www.usaid. gov/ebola/food-security. USAID. Food Assistance Fact Sheet–Sierra Leone. January 2018. Accessed at https://www.usaid.gov/sierra-leone/food-assistance on 2/25/2018. USAID Office of Inspector General. (2015). U.S. Government International Ebola Response and Preparedness Activities. Fiscal Year 2016, First Quarter. Washington, DC. The World Bank Group. The economic impact of the 2014 Ebola Epidemic: Short and Medium Term Estimates for West Africa. October 7, 2014. The World Bank. World Bank Group Ebola response Fact Sheet. April 6, 2016. Retrieved from http://www.worldbank.org/ World Food Programme, Food and Agriculture Organization, World Bank Group, European Union, and African Development Bank. State of Food Security in Sierra Leone 2015. Comprehensive Food Security and Vulnerability Analysis. 2015. https://www.wfp.org/content/sierra-leone-comprehensive￾food-security-and-vulnerability-analysis-december-2015. Accessed on February 13, 2018. World Food Programme. Protracted Relief and Recovery Operation – Sierra Leone 200938. n.d. World Food Programme. 2014. West and Central Africa Markets Update. Special Issue on the Ebola Outbreak. OMD VAM Food Security Analysis. https://www.wfp.org/content/west-and￾central-africa-markets-updates-special-issue-ebola-outbreak￾september-2014 WHO. Ebola Situation Report. 16 December, 2015. Accessed December 20, 2017. WHO. Ebola Situation Report. 11 November, 2015. http://apps. who.int/ebola/current-situation/ebola-situation-report-11- november-2015. Accessed 12 February, 2018. WHO. January 2016. Regarding the 2014-2015 Ebola outbreak in West Africa. WHO. Ebola Situation Report. Ebola Virus Disease. 10 June, 2016. http://apps.who.int/iris/bitstream/10665/208883/1/ ebolasitrep_10Jun2016_eng.pdf. Accessed February 01, 2018. World Health Organisation (WHO). “Ebola in Sierra Leone: A slow start to an outbreak that eventually outpaced all others.” n.d. http://www.who.int/csr/disease/ebola/one-year-report/ sierra-leone/en/. Accessed February 12, 2018. WHO. Strengthening the health workforce to strengthen health systems. http://www.who.int/hrh/resources/hrh_flyer.pdf.