This publication was produced at the request of the United States Agency for International Development. It was 
prepared independently by Samuel S. Lee, Evaluation Co-Team Leader, Institute for Development Impact; Dr. Bridget 
C. Lavin, Evaluation Co-Team Leader, World Vision United States; Dr. Azra Kacapor Nurkic, Institute for Development 
Impact; Jose “Pepe” Santos, Institute for Development Impact; Sibongani Kayola, Innovations for Poverty Action; 
Leslie Alex, Innovations for Poverty Action; Arthur Baker, Innovations for Poverty Action; and Arja Dayal, Innovations 
for Poverty Action 
The contents are the responsibility of the Institute for Development Impact, Innovations for Poverty Action, and World 
Vision and do not necessarily reflect the views of USAID or the United States government.
EVALUATION
Endline Performance Evaluation of the USAID Global 
Health Ebola Team (GHET) Ebola Virus Disease (EVD) 
Survivor Program - November 2018
Cover Photo: On the wall of the Bong County Ebola Treatment Unit in Liberia, Ebola virus disease 
survivors left their handprints (2015). Photo credit: USAID_Images on Flickr.com (Adam Parr, 
USAID).
i
Endline Performance Evaluation of the USAID Global 
Health Ebola Team (GHET) Ebola Virus Disease (EVD) 
Survivor Program
November 2018
ABSTRACT
Within the GHET Pillar II portfolio, this performance evaluation is an outcome-level study to determine 
the endline effectiveness of the program’s survivor-specific services. The evaluation questions are focused 
on whether program health services addressed the needs of Ebola survivors and were integrated into the 
public health system in order to mitigate the effects of further Ebola infection among the population.
In Guinea, Liberia, and Sierra Leone, this mixed-methods evaluation drew on quantitative health facility 
assessments, quantitative provider surveys, qualitative key informant interviews at endline, and use of 
baseline and endline quantitative surveys conducted among EVD survivors by implementing partners. 
Overall, the GHET program on survivor services was effective, particularly in reducing the stigma EVD 
survivors experienced from health providers in Guinea and Liberia. In Sierra Leone, provider stigma was
already the lowest among countries at baseline and remained so at endline; thus, the same reduction was 
not evident. While the GHET EVD survivor program also improved the availability of clinical services for 
EVD survivors, accessibility of care across the continuum of care (primary to tertiary facilities) was more 
difficult to improve at the primary level given systemic public health weaknesses in Guinea, Liberia, and 
Sierra Leone. 
When considering program implementation and preparation for future public health crises, the 
establishment of and capacity-building support to EVD survivor networks and government bodies 
contributed positively to the program’s efforts. However, the integration of survivor-specific services and 
processes into the general public health systems of Guinea, Liberia, and Sierra Leone remains a work in 
progress.
ii
TABLE OF CONTENTS
Acknowledgements & Disclaimer.......................................................................................................... iv
About the BalanceD-MERL consortium ................................................................................................. iv
List of Acronyms...................................................................................................................................... v
Executive Summary................................................................................................................................. 1
Introduction & Program Background...................................................................................................... 5
Program Background .......................................................................................................................... 5
Evaluation & Program Timeline .......................................................................................................... 7
Evaluation Purpose & Objectives............................................................................................................ 7
Results Framework & Lines of Inquiry ................................................................................................ 7
Intended Use & Users of Report......................................................................................................... 9
Methodology & Analytic Approach......................................................................................................... 9
Limitations ............................................................................................................................................14
Evaluation Findings by Domain.............................................................................................................15
Healthcare for Survivors ...................................................................................................................15
Accessibility of EVD Survivor Services...............................................................................................17
EVD Health Provider Stigma..............................................................................................................20
Governance & Management Effects on EVD Policies & Systems.....................................................25
Program Implementation & Barriers................................................................................................27
Conclusion.............................................................................................................................................35
Annexes.................................................................................................................................................36
Annex A: References.........................................................................................................................36
Annex B. Instruments........................................................................................................................39
Annex C. Data Quality Verification summary ...................................................................................71
Annex D. BalanceD-MERL Monitoring of EVD Survivor Surveys.......................................................74
iii
Annex E: Disclosures of Conflict of Interest....................................................................................103
Annex F: Original Statement of Work.............................................................................................111
Annex G: Research Team Biographies & Roles...............................................................................133
iv
Acknowledgements & Disclaimer
This publication was produced by the Institute for Development Impact (I4DI), Innovations for Poverty 
Action (IPA), and World Vision (WV) under the BalanceD-MERL Program, Cooperative Agreement 
Number AID-OAA-A-15-00061, funded by the U.S. Agency for International Development (USAID). This 
report is made possible by the generous support of the American people through the USAID. The 
contents are the responsibility of I4DI, IPA, and World Vision and do not necessarily reflect the views 
of USAID or the United States government. The total budget for this evaluation was $1,033,983.
The authors would like to thank all research partners, supervisors, and enumerators. Manny Sanchez
at I4DI provided institutional operational support for the project. The research team would also like 
to thank Elizabeth Hughes and Vasco Veloso for conducting key informant interviews in all three 
countries. Saikou Yaya Kollet Diallo supervised local enumerators in Guinea. 
The field staff of the lead implementing partner JSI R&T in Guinea, Liberia, and Sierra Leone were 
cooperative and collaborative throughout the entire evaluation process.
We also wish to thank USAID, particularly the Global Health Ebola Team, for their time and effort in 
providing feedback on the evaluation design, instruments, and draft versions of this document. We 
appreciate the facilitation and support of the USAID mission staff in Liberia, Sierra Leone, and Guinea, 
without whom field work would not have been possible. 
Finally, we extend deep gratitude to Ministry of Health officials, survey and interview participants, and 
the people of Liberia, Sierra Leone, and Guinea, who generously gave their time and shared their 
experiences and insights for the purposes of this evaluation. 
About the BalanceD-MERL consortium
The Balanced Design, Monitoring, Evaluation, Research, and Learning (BalanceD-MERL) consortium, 
under the U.S. Global Development Lab’s Monitoring, Evaluation, Research, and Learning Innovations 
(MERLIN) program at the United States Agency for International Development (USAID), is identifying 
solutions to commonly faced challenges in design (D) and implementation of monitoring, evaluation, 
research, and learning (MERL). D-MERL refers to the integration of program design, strategy, and 
implementation with MERL activities. The consortium provided support and technical assistance to 
USAID programs to enhance and conduct D-MERL activities. The consortium consists of five 
organizations: World Vision (prime awardee), Innovations for Poverty Action, Institute for 
Development Impact, Search for Common Ground, and William Davidson Institute at the University of 
Michigan.
U.S. Global Development Lab contact: Shannon Griswold 
v
List of Acronyms
ANSS Agence Nationale de Sécurité Sanitaire (Guinea)
APC Advancing Partners & Communities 
BalanceD-MERL Balanced Design, Monitoring, Evaluation, Research, and Learning
CI Confidence Interval
CPES Comprehensive Program for Ebola Survivors (Sierra Leone)
DQV Data Quality Verification
ETP&SS Ebola Transmission Prevention and Survivor Services
EVD Ebola Virus Disease
GHET Global Health Ebola Team
HMIS Health Management Information System
IP Implementing Partner
IPA Innovations for Poverty Action
IPC Infection Prevention and Control
IR Intended Results
IRB Institutional Review Board
JSI R&T JSI Research & Training Institute, Inc.
I4DI Institute for Development Impact
MERL Monitoring, Evaluation, Research, and Learning
MOH Ministry of Health
RENASEG Réseau National des Associations de Survivants d’Ebola de Guinée
SLAES Sierra Leone Association of Ebola Survivors
UN United Nations
USAID U.S. Agency for International Development
WHO World Health Organization
WV World Vision
1 
Executive Summary
This performance evaluation assessed the USAID Global Health Ebola Team’s (GHET) survivor-specific 
programs in Liberia, Guinea, and Sierra Leone. The evaluation explored the achievement of several 
Intermediate and sub-Intermediate Results (IR) in the GHET Portfolio Results Framework, namely IR 
3: Access to appropriate healthcare services is improved for Ebola survivors; sub-IR 3.1: Clinical 
services for Ebola survivors are strengthened and more accessible throughout the continuum of care; 
sub-IR 3.2: Stigma experienced by Ebola survivors when seeking care is reduced; and sub-IR 2.2: 
Governance and management are improved to advance national policies and systems. The evaluation 
drew upon primary data collected by the evaluation team at endline, including qualitative key 
informant interviews, quantitative health facility assessments, and quantitative health provider 
surveys. Secondary data sources included a variety of program and national reports, as well as 
implementing partner monitoring data. The evaluation also utilized data from baseline and endline 
quantitative surveys of Ebola virus disease survivors in each of the three countries, administered by 
JSI Research & Training Institute, Inc (JSI R&T), the lead program implementing partner.
Overall, the GHET program on survivor services was effective, particularly in reducing the stigma EVD 
survivors experienced from health providers when seeking care in Guinea and Liberia. In Sierra Leone, 
provider stigma was already the lowest among countries at baseline and remained the lowest at 
endline; thus, the same relative reduction was not evident. Furthermore, given the low baseline level 
and subsequent small change in observed stigma, the difference in provider stigma in Sierra Leone 
could not be determined with statistical significance. While the GHET program also improved the 
availability of clinical services for EVD survivors, accessibility of care across the continuum of care 
(primary through tertiary facilities) was more difficult to improve given systemic public health 
weaknesses in Guinea, Liberia, and Sierra Leone. While the program successfully established and 
implemented EVD-survivor specific health protocols and processes at all levels, the lower reported 
rates of adequate services for survivors available at the primary level and survivor reluctance to follow 
the referral system process were systemic deterrents to delivering accessible care, i.e. in closer 
proximity to where survivors resided. The individual findings considered for study conclusions by 
domain follow and represent sub-components that (1) explain why the overall GHET program activity 
was effective, (2) detail the systemic challenges the program faced and what approaches helped
overcome some of those challenges, and (3) share how some of these challenges could be addressed 
by future implementers in similar settings. 
Improved Healthcare Services for EVD survivors
According to EVD survivors surveyed, healthcare services for EVD sequelae improved. 74.7% of 
survivors in Guinea, Liberia, and Sierra Leone reported their access to healthcare was adequate and 
79% of surveyed survivors reported satisfaction with the care they received. Furthermore, 81.7% of 
survivors indicated that survivor services were “much better” or “a little better” since the beginning 
of program implementation.
2
Strengthened but Limited Accessibility of EVD Survivor Services
While survivor care health services overall have been strengthened, accessibility of care across the 
continuum of healthcare is skewed heavily towards tertiary facilities. While this is an expected 
weakness of the public health system in Guinea, Liberia, and Sierra Leone, this distribution of survivor 
care services still represents a gap in survivor care accessibility. As such, the program successfully 
brought survivor service care capacity to tertiary facilities. Half of the tertiary facilities sampled had 
specialist healthcare providers for EVD sequelae on-site and 57% of these tertiary facility providers 
reported being able to “often” or “always” provide adequate care for survivors. These findings confirm 
the importance of a properly functioning referral system for survivors, and the push to get survivors 
with EVD follow-up needs to tertiary facilities. However, only 46% of providers felt EVD survivors were 
satisfied with the referral process. Qualitative findings also surfaced challenges in getting survivors to 
utilize and follow the referral system. 
Strides in Health Provider Stigma Reduction for EVD Survivors
The stigma that EVD survivors experienced when seeking care was greatly reduced in Guinea and 
Liberia. Between baseline and endline, there was a 12.3% reduction (20.7% to 8.4%) of stigma in 
Guinea and 14.48% reduction (23.3% to 8.8%) in Liberia but a 2.2% increase (3.6% to 5.8%) in Sierra 
Leone (absolute reductions and increase). Results in Guinea and Liberia are statistically significant 
(p<0.001) but insignificant in Sierra Leone. The lack of statistical significance on provider stigma data 
in Sierra Leone was expected given the small change that was required for detection. In terms of 
representing this change as a percentage of baseline stigma, Guinea and Liberia respectively showed
59.50% and 62.1% relative reductions in provider stigma. While data showed an increase, the level of 
reported provider stigma remained the lowest in Sierra Leone across all three countries at both 
baseline and endline. Additionally, survivor perceptions of stigma reduction were positive across all 
three countries, with 84% of survivors who received treatment reporting an improvement. 
Furthermore, nearly three-quarters (73%) of providers reported stigma training changed their 
behavior towards survivors “very much.” This was further triangulated by analysis of interview data. 
All respondent stakeholders (implementing partners, government representatives, survivor network 
representatives, and USAID field staff) cited provider stigma trainings as highly effective and 
contributing to this reduction in provider stigma.
Governance and Management Effects on EVD Policies & Systems
The program’s key impacts on improved national policies and systems were primarily through 
contributions to each country’s national guidelines on EVD survivor care, specific clinical guidelines on 
EVD care, and capacity building and support to survivor networks and government bodies responsible 
for implementation and management of survivor services. Additionally, in Guinea the program 
supported effective operation of surveillance sentinel sites. Overall, support to and often creation of 
survivor networks and government bodies responsible for implementing survivor services and care 
was successful. These institutions had continued positive impact on the program through direct 
engagement with or feedback to the program. Simply put, the networks and bodies performed 
necessary functions that positively benefited implementation of survivor work. Survivor networks 
organized and represented survivor interests, often directing survivors to services and providing 
feedback to implementing partners on how to best operationalize services and programming like 
3
approaches to stigma reduction in communities and how to best engage survivors on referral 
processes. The government bodies engaged directly and eventually took more ownership of EVD 
programming and processes implemented throughout the course of the program. 
At project endline, nearly three-quarters (73.8%) of survivor respondents in Guinea and Liberia 
reported they agreed with a statement that their Survivor Network could look out for their interests 
and represented them well. Similarly, 70% of respondents in Sierra Leone reported being “very 
confident” in their Survivor Network. However, when JSI R&T’s baseline and endline data are 
compared, two trends emerged. Confidence in the Survivor Network increased in Guinea (+13.2%)
and decreased in Liberia (-15%). In Guinea and Liberia at endline, 69% of survivors agreed with a 
statement of confidence on the National EVD Government bodies. Similar to the trends for Survivor 
Network confidence, this represented an increase (+8.1%) in Guinea and a decrease (-19.1%) in Liberia. 
Collaborative Program Implementation & Systemic Barriers to Implementation
Overall, program implementation was characterized as collaborative, and qualitative analysis showed 
that the program implementing partners regularly sought and received feedback from program 
stakeholders. Two-thirds of qualitative interviews described their interaction with the program 
implementing partners as “high” on both level and quality of engagement. This collaboration was cited 
to inform and update program stakeholders on progress and also contributed to minimizing 
duplication of efforts. Within this positive backdrop, interviews yielded suggestions for improving 
collaboration and communication- in particular to increase engagement more locally at the district 
level (Guinea) and engagement with the survivor network (Liberia & Sierra Leone). 
Qualitative Insights from Implementation for Future Programming
Emerging from implementation experience, there are a number of important insights that may apply 
to future programming in similar post-epidemic settings. These insights were derived from qualitative 
analysis of key informant interviews and tie to program implementation barriers encountered. 
Interviews were conducted with implementing partners, government representatives, survivor 
network representatives, and USAID field staff with knowledge of implementation, engagement with 
the program, and sectoral context. 
Insight: More Local Level Community Engagement
While the APC ETP&SS program did have local components,respondents frequently noted the benefits 
of engaging the broader community affected by EVD, not just survivors or healthcare providers in 
order to ease community concerns at all phases of planning and implementation. Additionally, with 
the understanding that this particular program operated within allotted funding and limited 
timeframe, interview respondents (particularly in Guinea, which had extensive local engagement 
through sentinel surveillance sites) voiced the potential benefits of engaging more at the local district 
level, which they believed would also help ensure better monitoring and preparation for future public 
health outbreaks.
Insight: Overcoming Systemic Barriers to Treatment & Referral Actualization
Building on the theme ofsystemic public health system challenges and unanticipated costs of referrals 
for survivors, interview respondents felt that being more aware of the limits of the healthcare system 
4
(in particular lack of medication and limited ability to deliver goods and services to health facilities) 
and fully understanding and responding to the hidden costs of referral for survivors would be 
beneficial for future programming. In Sierra Leone, the issues around referrals were partly addressed 
through the use of “referral coordinators,” who helped bridge gaps in understanding that both 
healthcare providers and survivors had on the referral system and bureaucratic processes. Embodying 
the softer human side of public health administration, roles like referral coordinator should be 
considered and factored into similar future interventions.
Insight: Health Systems Overall vs Survivor Specific Activities
Given the systemic nature of weak public health systems, interviewees also noted the tension 
between focusing on survivor specific interventions as opposed to broader public health systems 
strengthening work. While this program was mandated to focus specifically on improving survivor 
services within a limited timeframe, many respondents raised the tradeoff between improving 
survivor specific services versus improving the health system overall given the existing public health 
system challenges in Guinea, Liberia, and Sierra Leone. Respondents raised the point that the program 
could have benefited from taking a broader approach to improving healthcare versus focusing 
exclusively on setting up survivor-specific services and processes. 
Insight: Engaging Early & Often with Government Entities
Interviews for this study yielded both concerns about the pace of government responsiveness and 
effective approaches to building trust and encouraging government ownership of programming. In 
post-epidemic settings, the importance of maintaining communication and relations with government 
agencies in host countries is well-known. Respondents across stakeholder types often mentioned the 
collaborative nature of program implementation and high levels of engagement with stakeholders. 
Implementing partners in all three countries worked closely and sought positive relationships with 
government partners and some illustrative best practices emerged in each country that speak to 
approaches fostering buy-in and ownership of program efforts over time, including co-design of 
program components, a focus on a systemic view towards public health outbreaks, high levels of 
engagement, and encouragement of host government ownership of programming. 
Conclusion
Overall, the GHET program for survivor services was effective, particularly in reducing the stigma EVD 
survivors experienced from health providers when seeking care in Guinea and Liberia. While the GHET 
program also improved the availability of clinical services for EVD survivors in Guinea, Liberia, and 
Sierra Leone, accessibility of care across the continuum of care (primary to tertiary facilities) was more 
difficult to improve at the primary level given systemic public health weaknesses in Guinea, Liberia, 
and Sierra Leone. While the program successfully established and implemented EVD-survivor specific 
health protocols and referral processes, lower levels of adequate services for survivors available at the 
primary level and survivor reluctance to follow the referral system process were systemic deterrents 
in delivering accessible care particularly in closer proximity to survivors at the primary facility level. 
Finally, when considering accessibility of care and preparation for future public health crises, the 
establishment of and capacity-building support to EVD survivor networks and government bodies 
contributed positively to the program’s efforts through strategic direct engagement with and 
5
feedback to the program. However, the integration of survivor-specific services and processes into the 
general public health systems of Guinea, Liberia, and Sierra Leone remains a work in progress.
Introduction & Program Background
Program Background
Between 2014 and 2016 the nations of Guinea, Liberia, and Sierra Leone experienced an 
unprecedented Ebola epidemic, which resulted in more than 28,600 cases of Ebola virus disease (EVD) 
and more than 10,000 survivors (Center for Disease Control and Prevention, 2017). In partnership with 
host governments, the global response included UN agencies, bilateral and multilateral donors, and 
non-governmental organizations. Among these institutions, the United States Agency of International 
Development (USAID) launched a four-pillar strategic response: control the epidemic (Pillar 1); recover 
from second-order impacts of Ebola (Pillar 2); build coherent leadership and operations (Pillar 3, 
crosscutting); and strengthen global health security in Sub Saharan Africa (Pillar 4) (Office of Inspector 
General US Agency for International Development, 2018). 
Since January 2015, the USAID Global Health Ebola Team, situated in the Global Health Bureau, has 
overseen a $150.5 million portfolio of health-related projects falling under Pillar II of the U.S. 
Government’s response. Activities supported critical non-Ebola health services, health systems, and 
transmission prevention of Ebola virus from the West African survivor community. Funding for 
survivor work commenced in June 2016. The GHET program’s development objective is that affected 
countries will be better equipped to sustain a “resilient zero” of Ebola cases (World Health 
Organization, 2015; USAID, 2017), meaning:
1. Effective infection prevention and control (IPC) within a wider context of patient safety
2. Strengthened integrated disease surveillance and response capacities (IDSR) as part of 
enhancing the resilience of local health systems
3. Reactivation of a basic package of essential health services in affected countries 
4. Rebuilding of health workforce capacities as a foundation for sustainable health 
workforce strengthening
5. Services addressing the specific needs of Ebola survivors are in place and integrated into 
the public health system to help mitigate further Ebola infection among the population
The 23 projects of the GHET portfolio, carried out by 12 implementing partners at national and 
regional levels across the three countries, aimed to mitigate the second-order impacts of the Ebola 
crisis, prevent further development losses, and strengthen key institutions and infrastructure. Their 
objectives were:
1. Delivery and utilization of essential primary healthcare services are restored;
6
2. Health system functions are strengthened to respond appropriately and adequately to 
future public health crises
3. Prevention and treatment services that address the needs specific to Ebola survivors are 
available and utilized.
In 2016, GHET commissioned a performance and process evaluation of its survivor-specific programs 
in Guinea, Liberia, and Sierra Leone. Implemented by JSI R&T in conjunction with local partners in each 
country, the Ebola Transmission Prevention and Survivor Services (ETP&SS) project (AID-OAA-A-12-
00047) was embedded in the larger Advancing Partners and Communities (APC) program. The APC 
program is a five-year USAID-funded project that supports the expansion and strengthening of 
community health services, especially in relation to family planning. The three ETP&SS country 
programs had a budget of $21,925,000, including a $650,000 Regional program (the latter was not 
part of this evaluation). In each country, various activities focused on the geographic areas with the 
highest numbers of EVD survivors, with select activities conducted at a national level.
In Liberia, the ETP&SS focused in four priority counties with the preponderance of registered EVD 
survivors: (1) Bong, (2) Lofa, (3) Margibi, and (4) Montserrado. In these locations, key program 
activities included, but were not limited to, health provider training on survivor clinical care guidelines, 
as well as clinical rotations and other training, coordination with the EVD survivor network, and 
renovation and equipping of health facilities. Partnering subgrants included those with the Liberia 
College of Physicians and Surgeons for clinical specialist consultants to provide EVD survivor care and 
train other pre- and in-service medical staff; Phebe School of Nursing and Rural Training Institute for
implementation of mental health clinical training in Bong County; and Eternal Love Winning Africa 
Hospital and Saint Joseph’s Catholic Hospital, both faith-based organizations that provided free clinical 
care services for registered EVD survivors.
The program in Sierra Leone focused on six USAID priority districts with the most EVD survivors: (1) 
Bombali, (2) Kailahun, (3) Kenema, (4) Port Loko, (5) Western Area Rural, and (6) Western Are Urban. 
Key program activities included, but were not limited to, health provider training and mentorship on 
specialty care for EVD survivors, including strengthening of the referral network, and capacity building 
to the EVD survivor network. Partnering subgrants included those with Kings Partnership Sierra Leone 
for specialty care capacity building; Partners in Health for survivor network capacity building; GOAL 
and Save the Children for various field implementation and/or technical assistance activities; and War 
Trauma Foundation for psychological first aid training.
In Guinea, the ETP&SS focused on three priority regions with the most EVD survivors: (1) Conakry, (2) 
Kindia, and (3) Nzérékoré. Program activities included, but were not limited to, support to survivor 
networks and stigma reduction sensitization campaigns; support for community-based survivor 
monitoring; clinical training on EVD sequelae management; and renovation and equipping health 
facilities. A partnering subgrant with International Medical Corps covered capacity building for 
survivor organizations, health provider training on EVD sequelae management, and strengthening the 
referral system. 
With regard to the latter, in all three countries, examples of basic primary care for EVD-specific 
sequelae include conditions such as arthritis or headache. Complicated sequelae were to be referred 
7
to designated secondary and tertiary health facilities, where specialists were more likely to be located 
in accordance with national health policy and planning strategies to manage scarce human resources 
for health (Ministere de la Sante, 2015; Ministry of Health and Sanitation, 2015; Republic of Liberia 
Ministry of Health and Social Welfare, 2011).
Evaluation & Program Timeline
Some implementing partners had multiple projects within and across countries, and survivor-related 
activities overall were staggered across the period of mid-2016 through 2018 following official EVD￾Free declarations by the WHO.
Evaluation Purpose & Objectives
The overall purpose of this evaluation was to determine to what extent the third intermediate and 
sub-intermediate results, as well as subIR 2.2, of the GHET portfolio were achieved during the life of 
the survivor program activities to date. In lay terms, the evaluation sought to answer two questions: 
(1) was the survivor program an effective activity and (2) why or why not?
Results Framework & Lines of Inquiry
The lines of inquiry for this evaluation were based on whether services addressing the specific needs 
of Ebola survivors were in place and integrated into the public health system to help mitigate further 
Ebola infection among the population. The lines of inquiry were also linked to the GHET results 
framework, specifically IR3 and its two sub-IRs, as well as sub-IR 2.2 (Figure 1).
8
Figure 1: GHET Results Framework
Given the complexity of GHET survivor programming, no single indicator captured all the desired 
outcomes. Thus, the evaluation lines of inquiry unpacked the IRs and subIRs into activities, outputs, 
and outcomes (Box 1). For example, what survivor specific health services were available and where, 
the degree to which they were utilized, whether they were adequate and sufficient, and whether they 
were improved. With regard to stigma, the evaluation followed a similar pattern of examining what 
activities had been carried out and whether survivors subsequently experienced less stigma from 
providers. With regard to governance and management, the evaluation assessed role and additions 
to national policies, the role of the survivor networks and survivor-related government bodies and 
survivor confidence in these groups, and the tangible improvements to governance and management.
Box 1: Evaluation Lines of Inquiry (Linked to GHET Results Framework)
Availability and Use of Quality Services
1. To what extent are quality health services focused on Ebola survivors available at 
different types of health facilities? Specific health services established for survivors? 
Regular health services? Referral to specific survivor services at higher level facilities? 
(Outcome) 
2. Capacity/Equipment Question: To what extent are survivor health services (other than 
primary healthcare services) and resources adequate and sufficient to meet the health 
needs of Ebola survivors? (Output) 
3. Survivors question: To what extent are Ebola survivors educated on services and 
accessing them? To what extent are Ebola survivors utilizing services available to 
them? (Outcome)
Stigma
9
1. To what extent do survivors experience stigma from health providers, and do 
survivors report that this stigma has decreased since the beginning of the program?
(Outcome)
2. Management of health workers: To what extent are health professionals trained to 
minimize the effects of stigma during patient-provider interaction? (Output)
Policy/Systems
1. To what extent have policies, structures or networks (eg survivor networks) 
improved the services available for Ebola survivors? (Outcome)
2. What policies, structures or networks have been strengthened to ensure adequate 
services are available for Ebola survivors? (Output)
Intended Use & Users of Report
This report is intended to provide a view into the effectiveness of program implementation as related 
to intended outcomes and goals outlined above, of particular interest to program stakeholders 
including but not limited to funding program managers, implementing partners, and government 
representatives. Additionally, the report seeks to share qualitative-insights derived from 
implementation experience relevant to future public health interventions in post-epidemic settings in 
West Africa and beyond.
Methodology & Analytic Approach
The study employed mixed methods, using both quantitative and qualitative methods. The study 
design primarily assessed cross-sectional endline data from quantitative health facility assessments, 
quantitative health provider surveys, and qualitative key informant interviews, carried out in the three 
countries in May and June 2018 (See Annex B for instruments). These data were supplemented with 
quantitative survey data collected from EVD survivors (see Annex B for relevant survey items), 
administered by JSI R&T at program baseline and endline, as well as a mix of other secondary data 
sources. 
For data collection from facilities and providers, the health facility was the primary sampling unit. 
Facilities were sampled from USAID priority areas in each country- typically those with the highest 
proportions of EVD survivors. In total, more than 80 facilities were assessed, including a mix of facility 
types across the continuum of care (primary, secondary, tertiary) that had received APC ETP&SS 
interventions (Table 1). 
In Liberia and Guinea, a near census of facilities was completed in the USAID priority areas (29 of 30 
facilities in Liberia, 14 of 14 facilities in Guinea). In Sierra Leone, 39 of 66 facilities receiving ETP&SS 
interventions were assessed in areas with high EVD survivor presence. One respondent per facility 
10
answered the facility assessment, most typically the person or administrator-in-charge. To be included 
in the facility assessment analysis, the facility respondent had to have three years or more of facility 
work experience so that pre-Ebola comparisons could be made. The facility assessment inquired about 
the provision of essential health services, presence of trained personnel, and availability of 
commodities, as related to EVD survivor needs.
Table 1: Endline Facility Assessments and Provider Survey Respondents, Number by Country and Type
Country 
(Region)
Facility Type Health facility 
assessments 
Providers
Guinea 
(Conakry, 
N’Zerekore, 
Kindia)
Primary (Health 
Posts)
1 1 
Secondary (Health 
Centers, CMC)
10 21
Tertiary
(Hospitals)
4 5 
Liberia 
(Montserrado, 
Lofa, Bong, 
Margibi)
Primary (Clinic) 9 9 
Secondary (Health 
center)
12 23
Tertiary (Hospital) 8 14
Sierra Leone 
(Western Area 
Primary (MCHP, 
CHP, CHC)
311 32
 
1 While 32 primary facility assessments were attempted, one facility did not have any staff available with 
sufficient work history at that facility, so only 31 primary facility assessments were analyzed.
11
Urban, 
Bombali, Port 
Loko, Western 
Area Rural, 
Tonkolili)
Secondary
(District Hospital)
3 6 
Tertiary National 
Hospital)
5 7 
In the second stage at each facility, health providers were sampled for participation in a quantitative 
survey measuring their participation in various training initiatives and their opinions and practices with 
regard to treatment of Ebola survivors. At smaller facilities, one provider of any type was recruited to 
participate, and two or three health providers were recruited at larger facilities, such as hospitals or
larger secondary facilities. In total, there were 27 providers surveyed in Guinea, 46 in Liberia, and 45 
in Sierra Leone. Among providers, nurses were the most common type of respondent overall, but 
there was some variation between countries.
Twenty individuals per country were selected for in-person or phone-based qualitative key informant 
interviews. Using an open-ended, semi-structured questionnaire, the interviews explored the 
efficiency, effectiveness, and sustainability of GHET-funded survivor specific programming; 
management of program activities; systemic challenges and opportunities; and other process 
evaluation topics. Informants were selected based on their knowledge of and responsibility for 
program implementation and familiarity with past and current levels of clinical care for EVD survivors. 
The final set of interview participants represented varying levels of management seniority, across a 
variety of institutional affiliations (Table 2). 
12
Table 2: Endline Qualitative Key Informants, Number by Type and by Country
Country Implementing 
Partners & 
Subgrantees
Government
Representatives
Survivor Network 
Representatives
USAID
Field
Total
Interviews
Guinea 6 4 2 3 15
Liberia 10 3 1 1 15
Sierra Leone 9 3 3 3 18
The primary data collected by the BalanceD-MERL team were supplemented with other secondary 
sources. Most notably, survivor insights were captured from two cross sectional quantitative surveys 
carried out by JSI R&T at baseline (early 2017) and endline (mid 2018) in Guinea, Liberia, and Sierra 
Leone. Their sampling frame was derived from various survivor lists in each country, such as Ministry 
of Health or survivor network records. Then random samples were drawn that considered the 
population of survivors in the target geographic areas and survivor gender. There were some 
variations in the sampling approach that JSI R&T used in each country; the detailed methodologies 
can be found in JSI R&T’s baseline and endline reports for each country. In all three countries, the 
BalanceD-MERL analysis is limited to the unweighted sample in the USAID high priority geographic 
areas and/or the areas with high presence of EVD survivors, in accordance with the evaluation Scope 
of Work (Table 3 and Annex F). 
13
Table 3: Survivor Survey Sample in USAID High Priority Areas, by Country
Country Number of Participants in JSI R&T’s
Baseline Survivor Survey
Number of Participants in JSI R&T’s 
Endline Survivor Survey
Guinea 442 531
Liberia 433 213
Sierra Leone 491 480
The survivor surveys differed somewhat between the three countries. At endline only, the BalanceD￾MERL team embedded a module of questions into each country’s survivor survey to answer questions 
specific to the GHET survivor program performance evaluation. For the purpose of this evaluation, the 
survivor surveys were used to answer questions regarding perceptions of survivor healthcare services, 
provider stigma, and survivor networks.
As available, other secondary data sources (Annex A) were used to understand program plans, 
activities completed, reach, and challenges. These included IP reports, program monitoring and 
evaluation data derived from IPs, and other program-level documents. A supplementary data quality
verification assessed the integrity, reliability, and timeliness of the IP data, to inform decisions about 
its usefulness (Annex D).
All evaluation materials were reviewed and approved by the internal Institutional Review Board (IRB) 
at Innovations for Poverty Action. Additionally, individual ethical approvals were obtained locally in 
each country through the following organizations: Ministry of Health and Sanitation, Ethics and 
Scientific Review Committee (Sierra Leone); University of Liberia Pacific Institute for Research and 
Evaluation (Liberia); and the National Committee for Ethics in Health Research (Guinea).
14
LIMITATIONS
The principal limitation of this performance evaluation was the lack of comparative data, particularly 
trustworthy baseline data for certain outcomes of interest. Key informants, facility personnel, and, in 
some instances, EVD survivors used recall to help reconstruct the situation at baseline. As such, there 
is a threat of recall bias, in which respondents do no remember prior events accurately or completely. 
Triangulation across a variety of sources allowed for conclusions to be drawn about the contribution 
of the GHET survivor program to outcomes. 
The JSI R&T survivor survey, while a good opportunity to leverage resources and reduce burden on 
survivors, presented some analytical challenges. While the instruments in Liberia and Guinea were 
very similar to one another at both baseline and endline, there were notable differences in the 
structure of the instruments used in Sierra Leone. For example, the items used to measure provider 
stigma were different between Sierra Leone and the other two countries. Where the BalanceD-MERL 
items were added to the instrument used in Sierra Leone at endline, there was a substantial shift in
the instrument to explore care seeking and the care experience by each specific sequela, in parallel to 
the general care seeking and experience questions that were maintained between baseline and 
endline instruments. This made it difficult to compare to the endline measurements in Guinea and 
Liberia, as well as the baseline situation in Sierra Leone. On several items in the Guinean and Liberian 
instruments, there were inconsistent time reference points, hampering the ability to analyze 
responses for certain subsets of respondents with validity. 
That said, the survivor surveys gave voice to the experience and perceptions of the program’s primary 
intended beneficiaries and offered a number of high quality items to evaluate the program. Moreover, 
a survey monitoring exercise conducted by the BalanceD-MERL team revealed a high degree of 
reliability in the data that JSI R&T collected from survivors (Annex D). It should be noted, however, 
that how the survivor survey data were analyzed for this report may differ from JSI R&T’s analysis and 
reporting to GHET (e.g., this performance evaluation report limited the analysisin Sierra Leone to only 
those in five contiguous districts with high prevalence of survivors; how the stigma items were 
analyzed; etc.). 
In Liberia, the saturation of viewpoints sought from Survivor Network representatives and USAID field 
staff was not reached, giving only single interviews for each category after repeated attempts to 
secure more interviews. 
While not evaluation limitations, per se, there were a variety of challenging situational factors that 
the APC survivor program faced during its implementation. The general difficulties of working in a 
post-emergency environment are acknowledged in this report, particularly as they relate to the 
systemic public health challenges. However, there were some challenges that were not picked up by 
instruments for analysis in the study like elections in Liberia and Sierra Leone and a lawsuit brought 
by the Sierra Leone Association of Ebola Survivors against the government of Sierra Leone. 
15
EVALUATION FINDINGS BY DOMAIN
Healthcare for Survivors 
Appropriate healthcare services for EVD survivors have improved. 
This was determined by analyzing EVD survivor responses to the following question which was 
included in JSI R&T’s endline surveys of survivors: “Do you feel that your access to healthcare for this 
problem was adequate?” On average across countries at endline, for EVD sequelae for which 
treatment was sought, three-quarters of EVD survivor responses (74.7%) indicated access to 
healthcare services were adequate. While 84% of EVD survivor respondents in Guinea and in Sierra 
Leone reported access to healthcare services for EVD sequelae were adequate, just 56% of 
respondents in Liberia reported the same (Figure 2). Perceptions about access to adequate care 
services did not differ statistically by respondent gender in any of the three countries. 
Figure 2: Adequate EVD Sequelae Care % by Country (Data Source: JSI R&T Endline BalanceD-MERL module)
To measure satisfaction with care received, survivors were asked the following question in JSI R&T’s
endline surveys of survivors: “Was the quality of healthcare that you received satisfactory?” On 
average across countries at endline, for EVD sequelae for which treatment was sought, over three￾quarters of EVD survivor responses (78.7%) indicated satisfaction with services received (Guinea 91%, 
Sierra Leone 81%, Liberia 64%) (Figure 3). In Liberia, satisfaction differed by gender, 72.1% of females 
reported they were satisfied with the quality of healthcare received versus 53.7% of males (p=.019). 
In Sierra Leone, where satisfaction was measured separately for each sequelae type, there were 
gender differences only for satisfaction on eye care, with 82.2% of males satisfied with treatment 
versus 62.0% of females (p=.007). In Guinea, there were no statistical differences in satisfaction by 
gender.
16
Figure 3: Satisfaction with EVD Sequelae Care% (Data Source: JSI R&T’s Endline BalanceD-MERL module)
Furthermore, survivor perceptions of EVD care improvements were measured through the following 
question inserted into the JSI R&T endline surveys of survivors: “In the past two years (since the end 
of the Ebola time), do you feel that care for this problem has- possible responses: gotten much better, 
gotten a little better, stayed the same, gotten a little worse, gotten much worse.” At endline, a little 
over four-fifths (81.7%) of surveyed EVD survivors in Guinea, Liberia, and Sierra Leone stated that 
survivor services had gotten either “much better” or a “little better” since the beginning of program 
implementation (Guinea 79%, Liberia 79%, Sierra Leone 87%) (Figure 4). In each country, a very small 
percent of respondents indicated that survivor care had gotten worse. There were no significant 
differences in perceptions of care improvement by gender.
Figure 4: Survivor Perception of Improved EVD Care% (JSI R&T Endline BalanceD-MERL module)
17
Accessibility of EVD Survivor Services 
Many health providers surveyed at facilities for this study reported being trained on EVD sequelae2
, 
but training numbers alone did not tell a complete story about the extent to which EVD survivor 
services were accessible. While specialist care providers for EVD sequelae were not intended to be 
present at primary health facilities, a closer look at the presence of specialist care providers for EVD 
sequelae at all facilities was useful in helping to explain findings on adequate care for survivors at 
varying levels of facilities. In addition to a general checkup, recommended specialty care for EVD￾related sequelae included musculoskeletal evaluation, ocular evaluation, auditory evaluation, 
abdominal evaluation, neurological evaluation, mental health evaluation, and sexual health 
evaluation (World Health Organization, 2016).
When only considering tertiary facilities, the presence of specialist care providers for EVD sequelae 
was 50.99% of facilities sampled (Figure 5). While specialist care providers were not expected to be at 
primary facilities, the lack of presence at primary facilities was a challenge for initiating the referral 
process for survivors which is designed to be initiated at the primary level.
“For many in the general population, [the notion is] that a tertiary [facility] is the only place 
that has good [specialized] care. So [survivors] bypassed the primary and secondary [facilities] 
and went straight to the tertiary care [facility] to receive primary healthcare.” Implementing 
Partner – Liberia
Figure 5: Tertiary Facilities with Specialty Care Providers % (Data Source: BalanceD-MERL Health Facility Assessment)
 
2 As research design and sampling was not a census of health providers or facilities, reporting data on 
percentages of providers who received training on EVD-sequelae were determined not to be useful given 
performance evaluation objectives. In Liberia, 79% of sampled facilities reported providers trained on EVD￾sequelae care. In Guinea, 93% of sampled facilities reported providers trained on EVD-sequelae care. In Sierra 
Leone, 87% of sampled facilities reported providers trained on EVD-sequelae care.
18
More telling on accessibility of care across the continuum of care were the perceptions of providers 
on their respective facility’s ability to provide adequate care for EVD survivors. This was determined 
through a question in the BalanceD-MERL provider survey: In your opinion, how often is this facility 
able to provide Ebola survivors with health services sufficient in quantity to achieve their therapeutic 
needs? – possible responses: Always, Often, Sometimes, Rarely, Never. On average, only 38.5% of staff 
at all sampled health facilities (primary, secondary, and tertiary) self-reported that their facilities 
“always” or “often” provided adequate care for EVD survivors. This trend differed by country, with a 
higher proportion of providers in Guinea and Sierra Leone reporting “always” or “often” compared to 
Liberia (Guinea 44.5%, Liberia 26.5%, Sierra Leone 44.4%) (Figure 6). When only considering provider 
responsesfrom tertiary facilities, the perceptions of care adequacy were better, with 57% of providers 
at tertiary facilities self-reporting that their facilities are “always” or “often” able to provide adequate 
care for EVD survivors (Guinea 60%, Liberia 50%, Sierra Leone 50%) (Figure 7). 
Figure 6: Providers at all facility levels (primary, secondary, and tertiary) reporting adequate services for EVD survivors
(Data Source: BalanceD-MERL Provider Survey)
Figure 7: Providers at tertiary facilities reporting adequate services for EVD survivors (Data Source: BalanceD-MERL 
Provider Survey)
19
Care accessibility among EVD survivors is also somewhat illuminated by an analysis of the number of 
visits and actualized referrals to tertiary and secondary facilities provided by the main implementing 
partner, JSI R&T to GHET through quarterly reports. These figures indeed showed that the referral 
system was utilized in each country (Figure 8 & 9). However, this study cannot definitively comment 
on the rise and fall in visits and actualized referrals, but given the program established the referral 
system and process for survivors, the program can certainly be linked to the usage patterns that have 
been reported. Additionally, these figures may be underreported as a quality verification of these data 
revealed inconsistent tracking of survivor visits at facilities. In some instances, survivor visits were 
purposefully not tracked in order to encourage EVD survivors to visit facilities when needed (though 
implementing partners note that this situation was adjusted when survivor care guidelines were 
updated). 
“There was initially a patient identification card for Ebola survivors and a little booklet for each 
survivor… to show to get the free medical care and medicine. Based on input from RENASEG 
[Guinea Survivor Network], which felt that this would further stigmatize patients and made 
them more identifiable and different from other patients, we did not [require use] of those 
cards” Implementing Partner - Guinea
According to qualitative key informant respondents, survivors themselves hesitated to self-identify as 
survivors. As such, referral data (not officially in HMIS but reported to GHET by implementing partners)
likely do not capture the full usage of clinical services by EVD survivors, but instead only paint a picture 
of referral system use in broad strokes. Implementing partners in Sierra Leone were able to provide 
the GHET team with data on actualized referrals, and while the numbers are lower compared to visits 
reported in Guinea and Liberia, actualized referrals represent a higher-level outcome to measure the 
referral system. The referral coordinators deployed in Sierra Leone certainly allowed for capture of 
higher quality data- in this case on actualized referrals. 
Figure 8: Visits by EVD survivors to secondary and tertiary facilities in Guinea & Liberia (Data Source: APC Quarterly 
Reports)
20
Figure 9: Actualized referrals by EVD survivors to secondary and tertiary facilities in Sierra Leone (Data Source: APC 
Quarterly Reports)
Qualitative analysis also highlighted that implementation partners only fully recognized the “hidden 
costs” associated with the referral system of care (e.g., the extent of associated costs for survivors in 
having to travel for care) as a major barrier to provision of care after implementation was well 
underway. Across countries from the view of caregivers, only 46% of providers felt that their EVD 
survivor patients were satisfied with the referral process. Qualitative key informants revealed that 
when survivors did seek care they often skipped the referral process altogether and proceeded directly 
to tertiary facilities--becoming a barrier to effective care implementation given that it strained the 
health system’s ability to both track and provide care efficiently.
“A lot of the survivors are not taking advantage of the [referral] system. They bypass the 
system and go the next level. There are a few of them [survivors] that will follow the system, 
but a lot of them do not.” Implementing Partner - Liberia
EVD Health Provider Stigma
Several different measures explored EVD survivor perception of stigma from health providers. In 
combination, the measures showed that while some provider stigma remained, there has been a 
significant reduction over time in Guinea and Liberia. In Sierra Leone, stigma remained the lowest 
among countries but a statistically significant change could not be detected.
Included in the JSI R&T endline survivor survey through a BalanceD-MERL module, the following 
question sought perceptions of survivors on their treatment by health providers: Do you think that 
treatment of Ebola survivors by doctors and nurses has gotten better, gotten worse, or stayed the 
same since just after the Ebola time? – responses: Better, Worse, Stayed the Same. Most surveyed 
EVD survivors in each country indicated that treatment of EVD survivors by physicians and nurses had 
21
improved since the program began (Figure 10). However, small proportions indicated that survivors 
were treated worse by providers over time. There were no significant differences by gender.
Figure 10: Survivor Perceptions on Treatment by Doctors & Nurses (JSI R&T Endline BalanceD-MERL module)
Across countries, EVD survivors were queried about specific examples of provider stigma at baseline 
and endline by JSI R&T.
3 Each country collected data on provider stigma differently. 
In Guinea, the JSI R&T survivor survey baseline and endline asked survivors to indicate whether they 
experienced the following 5 examples of provider stigma:
Medical personnel refused to treat you or do tests for you because of your Ebola status.
The wait time to be seen by a medical staff was longer for you than for other patients.
Medical staff took excessive precautions when examining you.
 
3 In Guinea and Liberia, perceived provider stigma was measured using five items among all survey respondents, 
without screening for whether they had recently received care at a health facility. In Sierra Leone, perceived 
provider stigma was measured using three items and limited to those who had received care at a health facility 
in approximately the year prior to the survey. The composite stigma data presented herein is an average of 
respondent answers across items, by country. This differs from the stigma index approach used by JSI R&T. In 
Liberia and Guinea, JSI R&T calculated perceived stigma as the percent of respondents who indicated two or 
more of the five stigma items. In Guinea, this equated to 29.9% at baseline and 6.8% at endline. In Liberia, this 
equated to 30.3% at baseline and 8.9% at endline In Sierra Leone, JSI R&T calculated perceived stigma as the 
percent of respondents who had received care at a health facility in the year prior to the survey and who 
indicated any of three stigma items. JSI R&T’s data in Sierra Leone were from a national sample and weighted. 
In Sierra Leone, this equated to 23.5% at baseline and 11.9% at endline. 
22
Medical staff gossiped about your Ebola status. 
You received less attention / care than the other patients.
In Liberia, the JSI R&T survivor survey baseline and endline asked survivors to indicate whether they 
experienced the following 5 examples of provider stigma:
The doctor/nurse refused to treat you or refused to do relevant tests.
You had to wait longer for the doctor/nurse to treat you than other patients.
The doctor/nurse appeared nervous or uncomfortable treating you.
The doctor/nurse gossiped about your survivor status.
You received less care/attention than other patients.
In Sierra Leone, the JSI R&T survivor survey baseline and endline asked survivors to indicate whether 
they experienced the following 3 examples of provider stigma:
You were denied health services because of your Ebola survivor status.
Healthcare workers talked badly about you because of your Ebola survivor.
A health worker disclosed your Ebola survivor status without your permission.
When survivors indicated they had experienced one of the examples above, the response was counted 
as an experience of provider stigma. Given the variance in measurement by country, the BalanceD￾MERL team created a stigma percentage score for each country by taking the average of each provider 
stigma measure for use as a composite country average of provider stigma for each country. 
The perception of provider stigma towards EVD survivors declined substantially between baseline and 
endline in Guinea and Liberia; in absolute percentage terms, there was a 12.3% reduction of stigma in 
Guinea and 14.5% reduction in Liberia (Figure 11). The results in Guinea and Liberia were statistically 
significant (p<0.001). In Sierra Leone, the level of provider stigma as reported by survivors increased 
over time (2.2% in absolute terms). However, Sierra Leone had the lowest levels of provider stigma at 
baseline and endline across all three countries. Furthermore, results in Sierra Leone are not 
statistically significant- in part due to the smaller change that needed to be detected and resulting 
larger required sample size. In terms of representing these changes as a percentage of baseline stigma, 
Guinea and Liberia showed a 59.5% and 62.1% relative reduction in provider stigma, respectively. 
23
Figure 11: Provider Stigma, Baseline and Endline % (Data Source: JSI R&T Survivor Baseline & Endline)
Perceived provider stigma rarely varied by respondent gender. In Liberia, there were no statistical 
differences by gender at baseline or endline, although the endline data trended toward more men 
reporting these problems than women. In Sierra Leone, there was no statistical difference by gender 
at baseline; at endline more males reported that “healthcare workers talked badly about you because 
of your Ebola survivor status” (8.1% of males versus 2.5% of females, p=.03). There were no other 
statistical differences in individual stigma items in Sierra Leone. In Guinea, there were no statistical 
differences by gender at baseline or endline. 
Survivor perceptions on the reduction of provider stigma were further validated by the perspectives 
of health providers who received stigma training. Providers at sampled health facilities were asked the 
following question: To what extent did your behavior when caring for Ebola survivors change as a 
result of this training? – responses: Not at all, A little, Some, Very Much. Nearly three-quarters of 
providers (73%) across countries reported that stigma training changed their behavior towards 
survivors “very much” (Guinea 63%, Liberia 68%, Sierra Leone 88%) (Figure 12). In fact, all provider 
respondents in Liberia and Sierra Leone reported that stigma training changed their behavior, and 96% 
of providers in Guinea reported the same.
24
Figure 12: Provider Behavior Change After Stigma Training (Data Source: BalanceD-MERL Provider Survey)
Qualitative analysis provided additional supporting evidence of reduced provider stigma. All 
stakeholder types interviewed (IPs, Government, USAID, and Survivor Networks) felt the program
made strong contributions to reduction of provider stigma, most effectively through provider training 
and education. However, key informants also noted that a residual amount of general stigma around 
mental health issues remained, regardless of EVD status. 
“Survivors indicated that since after the training [health workers] received [survivors] have 
been treated with respect and that confidentiality has been improved...and [providers] are no 
longer afraid to treat them[survivors]” Implementing Partner - Liberia
“Not only [has] training helped reduce stigma, but continuous engagement with [providers] 
about all the programs. So slowly [providers] came to accept that [survivors] are not infectious, 
and [survivors] came to access healthcare.” Government Representative - Sierra Leone
“Training brought a great change by explaining to providers that survivors, once they recover 
from the disease, the disease can't be transmitted- that you can treat them as you treat other 
people. This built confidence in [providers], helped them treat survivors in their facilities, and 
increased survivors’ confidence in seeking care.” Survivor Network Representative – Liberia
25
Governance & Management Effects on EVD Policies & Systems
The program strengthened national policies and systems primarily through contributions to each 
country’s national guidelines on EVD survivor care, specific clinical guidelines on EVD care, and 
capacity building and support to survivor networks and government bodies responsible for 
implementation and management of survivor services. Additionally, in Guinea the program supported 
effective and efficient operation of surveillance sentinel sites. 
Overall, the support to, and often creation of, survivor networks and government bodies responsible 
for implementing survivor services and care was successful. Moreover, these activities had 
downstream ripple effects, via the contributions that the networks and government bodies 
subsequently made to survivor program design and implementation. These institutions had continued 
positive impact on the program through direct engagement with or feedback to the program. Simply 
put, the networks and bodies performed necessary functions that positively benefited 
implementation of survivor work. Survivor networks organized and represented survivor interests, 
often directing survivors to services and providing feedback to implementing partners on how to best 
operationalize services and programming, like approaches to stigma reduction in communities and 
how to best engage survivors on referral processes. The government bodies engaged directly and 
eventually took more ownership of EVD programming and processes implemented through the 
program.
The JSI R&T survivor baseline and endline surveys in Guinea and Liberia shed light on how survivors 
perceived various EVD bodies. At endline, nearly three-quarters (73.8%) of survivor respondents in 
Guinea and Liberia reported they agreed that the Survivor Network looked out for their interests and 
represented them well. There were no significant differences in perception by gender in either 
country. Similarly, 69% of respondents in Sierra Leone reported being “very confident” in their 
Survivor Network at endline. However, in Sierra Leone, confidence in the Survivor Network differed 
by gender. Among males, 76.0% were very confident in the Survivor Network, compared to just 62.2% 
of females (p<.001). Moreover, 17.2% of females said they did not have any confidence in the Survivor 
Network in Sierra Leone, compared to 6.6% of males. 
While there were no baseline data from Sierra Leone for comparison, when comparing baseline and 
endline survivor data for Guinea and Liberia, two trends emerged. Confidence in the Survivor Network 
increased in Guinea (+13.2% in absolute numbers) and decreased in Liberia (-15% in absolute 
numbers) (Figures 13 and 14). Quantitative data collected from survivors did not allow for additional 
analysis on what led to the increase and decrease in survivor confidence in these respective 
institutions.
26
Figures 13 & 14: Confidence in Survivor Networks Guinea & Liberia (Data Source: JSI R&T Survivor Survey 
Baseline/Endline)
At endline, 69% of surveyed survivors in Guinea and Liberia agreed that they had confidence in the 
National EVD Government Bodies. In Guinea, more men had confidence in their governmental EVD 
body than women, 75.5% versus 61.4% (p=.023). There was no significant difference by gender in 
Liberia. Similar to the trends for Survivor Network confidence, over time there was an increase in 
confidence in the National EVD Government Body in Guinea (+8.1% in absolute numbers) and a 
decrease in Liberia (-19.1% in absolute numbers) (Figures 14 and 15).
Figures 14 & 15: Confidence in National EVD Bodies Guinea & Liberia (Data Source: JSI R&T Survivor Survey 
Baseline/Endline)
As noted earlier, survivor networks and national EVD government bodies were centrally located 
strategic stakeholder bodies that also contributed to survivor program implementation decisions. 
Qualitative data analysis showed that survivor networks in particular had an impact on program 
implementation, in shaping survivor community engagement and in referral system implementation. 
Survivor networks not only were the conduit for feedback from survivors on services but also held the 
power to both encourage and deter survivors from visiting health facilities (Liberia) and were 
identified as a key to better working relationships with the government (Sierra Leone). These 
qualitative insights will be further elaborated later in this report.
27
Program Implementation & Barriers
Collaborative Nature of Implementation
Overall, program implementation was characterized as collaborative. Qualitative data showed that 
the program implementing partners sought and received feedback from program stakeholders. Two￾thirds of qualitative interviews described their interaction with the program implementing partners 
as “high” on both level and quality of engagement. This collaboration was cited to not only have kept 
all program stakeholders informed but to also have contributed to minimizing duplication of efforts.
“JSI [Research & Training, Institute Inc.] wouldn't do anything without engaging us, they had 
us very fully engaged; in fact, [we] had very major input... into the processes all along the way. 
I don't even remember the number of times we have met. We met so many times, every step 
of the way...so we were involved from the very beginning up to the very end.“ Government 
Representative - Liberia
“It was a beautiful program from the beginning till the end- the way it was structured, the way 
it was designed... addressing the issue from two sides, creating two networks and making sure 
that they connected together; working with both of them to try and make them better and 
bigger all the time, and then connecting them. That was really, really good.” Implementing 
Partner - Sierra Leone
Against this positive backdrop, key informant interviews nevertheless yielded suggestions for 
improving the reach and impact of programming. In particular, interviewees suggested deepening 
engagement at the district level (Guinea) and improving engagement with the survivor network 
(Liberia & Sierra Leone). While both the overall collaborative nature of implementation and the short 
time-period for implementation were acknowledged, the following suggestions are offered for future 
public health interventions working through similar constraints. 
District Level Engagement
While the program worked with limited budget and mandate, interview respondents noted that 
overall, greater district level engagement could have increased receptiveness of survivors and the 
broader community beyond survivors- effectively a call for more “bottom-up” decision making.
“How to communicate and ensure that messages reach the targets [at the local level] should 
have been researched and analyzed before. The [EVD] Secretariat was not passing messages 
down to the district level. Messages about when specialists would be available should also 
have been put out before specialists started making visits.” Implementing Partners - Liberia
The best approach is to involve survivors, religious leaders, communities, all of them in 
decision-making. So that decision comes from the bottom-- bottom-up and not top-down.”
Implementing Partner – Guinea
Deeper insight on this point was drawn from qualitative interviews with key informants:
28
Insight: More Local Level Community Engagement
There are two dimensions to this expanded insight, and the study team has earlier referenced the 
program’s implementation contribution to local surveillance sites in Guinea. First, interview 
respondents frequently noted the benefits of engaging the broader community affected by EVD, 
not necessarily just survivors or health providers in order to ease community concerns at all phases 
of planning and implementation.
“Not only [knowledge] for survivors but for the general population... when people get 
knowledge, this influences their behavior to make change… [knowledge was] one of the 
most important things that stopped Ebola in its tracks here.” Government Representative￾Sierra Leone
“I would suggest from the onset of any Ebola crisis, the survivors, the community, the 
healthcare workers, should all form a target of behavior change communication 
intervention. Why do I say that? Survivors live in the community, and continuous 
engagement with the community will empower the community to take the actions that they 
can. People in the community have different practices, so engagement would help 
[implementers] see what they need to seek to help that community.” Survivor Network 
Representative - Liberia
Second, with the understanding that this particular program operated within allotted funding and 
limited timeframe, interview respondents (particularly in Guinea) voiced the potential benefits of 
engaging even more at the local district level, which they believed would also help ensure better 
monitoring and preparation for future public health outbreaks.
“You need the engagement of the districts to prevent another outbreak, and the districts 
were actually not so much supported by the project.” Implementing Partner - Guinea
SA-Ceint [surveillance sites] started a bit late; we should have organized that from the 
moment the outbreak ended. If we don't do the same in the DRC we risk having unpleasant 
surprises in the future, as we don't know the final outcomes of Ebola. These kinds of [local 
surveillance] structures should be in place until we have the knowledge and means to avoid 
resurgence.” Government Representative - Guinea
“Yes, the central level is there, but it [only] takes into account what the community level 
may want.” Implementing Partner – Guinea
Survivor Network Engagement
Overall, the survivor networks were determined to be helpful in organizing survivors and representing 
their views. While there were varying levels of tension between the Survivor Network and 
implementing partners (especially in Liberia), interview respondents reiterated the need for greater 
engagement and the benefits to even more coordination and collaboration with survivor networks, 
despite differences and varying levels of capacity. 
“You had a high participation rate in your last testing campaign. What do you think that was 
due to? I think it's due to the local associations, the peer interaction between the Ebola 
29
survivors, and by the survivor network.” Implementing Partner - Guinea
“Any institution that wants to work with survivors has to work with the survivor network. You 
work together, you will be able to achieve your objectives. If you don't work with the survivor 
network, you won't achieve [your objectives].” Survivor Network Representative - Liberia
“I think having that common link [with the survivor network], where there is somebody who 
understands Ebola, who has fought so hard to end this outbreak, and who is going to help 
survivors…. is an important message.” USAID Field Staff – Guinea
Persistent Barriers to Implementation
Systemic Barriers
Primarily from the implementing partner perspective, there were a number of oft-cited barriers to 
program implementation, many at a systemic level which persisted throughout implementation. 
While the program was not mandated to address systemic-level issues affecting provision of 
healthcare in all three countries, implementation was of course impacted by these underlying 
conditions. The largest barriers identified across countries were related to a lack of both general health 
infrastructure and logistics in-country required to get equipment and medication4 where it was 
needed. These conditions limited provision of and access to care and also were cited as reducing 
survivor motivation to seek care directly at primary health facilities or to follow the referral system 
(referring survivors to tertiary facilities) for survivor care.
 
4 Interviews for this study were conducted after media reports of medicine stock-outs in early May (example 
Liberian Observer, May 10, 2018.)
30
Further qualitative insights drawn from interviews with key informants 
further illustrates the challenges faced in implementation and a promising 
practice that emerged on the use of “referral coordinators” in Sierra Leone.
Insight: Systemic Barriers to Treatment & Referral Actualization
As referenced, interview respondents felt that being more aware of the limits of the healthcare 
system (in particular lack of medication and services and ability to deliver these goods and services 
to health facilities) and fully understanding the costs of referral for survivors would be beneficial 
for future programming. The lack of medication raised by the first and second quotes in this box 
were not in ETP&SS control.
“The system has problems. Sometimes there are drugs you need to buy because they are 
not there from the government. It's free healthcare from the government but the 
government is not providing them. I want to operate [on a survivor]. But after the operation, 
this patient needs... some drugs to survive. I have to ensure that these drugs are there before 
I operate. So, these patients are given a [task]... they have to bring those postoperative 
drugs.” Implementing Partner - Sierra Leone
Consultants started paying for medications themselves. But we told them to stop because it 
was not solving the problem. Medications did not arrive until the very end of the project. In 
some locations the hospital did not even have aspirin.” Implementing Partner – Liberia
“Referral everywhere costs a lot of money... for the patient because [when] you go to the 
primary level they give you a referral slip and usually it is a long distance [to the next facility]. 
You have to pay for more… laboratory tests. [The cost] is part of the referral… it costs more 
money and more resources.” Implementing Partner - Guinea
“One of the main barriers to getting mental healthcare especially for those that are located 
far away is the cost. To travel from Lofa to Montserrado is at least 10 hours, the travel costs 
[include] accommodation... so most people are very unlikely to seek this care or use the 
referral system” Implementing Partner - Liberia 
In Sierra Leone, the issues around referrals were partly addressed through the use of “referral 
coordinators,” bridging gaps for both providers and survivors on the referral system and process. 
As part of the Survivor program, referral coordinators were assigned to each District Health 
Management Team to support clinical services and referrals for EVD survivors, often assisting both 
survivors and providers with the nuances and necessary coordination of the referral process.
Embodying the softer human side of public health administration, roles like referral coordinator 
should be considered and factored into similar future interventions.
“There was a significant increase in the use of this system by survivors. And this was strongly 
helped by the referral coordinators and the teams moving between the communities, the 
district hospitals, and the tertiary care centers” Government Representative - Sierra Leone
“It is the referral coordinator that has improved care... one in terms of documentation- the 
referral form. Completing that [form], [the referral coordinator] also provides a lot of 
support... but these are people that have [also] been trained on hospital preparedness. If 
the patient is coming to Freetown for an emergency or for the laboratory, the referral 
coordinator can just call [ahead].” Government Representative - Sierra Leone
31
“Now there is a much more formalized referral pathway with a network of referral 
coordinators who collect data on service availability. Therefore, [referral coordinators] are 
able to provide that service availability data to get patients to the right place at the right 
time. Timing is quite important.” Implementing Partner - Sierra Leone
If you have a referral coordinator in your facility, you're more likely to receive [patients]” 
Implementing Partner - Sierra Leone
While again acknowledging the limited and focused scope of the program on survivor services, related 
to the systemic issues that program stakeholders noted, qualitative interviews yielded insights and 
considerations for future implementation. 
32
Insight: Health System Overall vs Survivor Specific Focus
While the program was mandated to focus specifically on improving survivor services within a 
limited timeframe, many respondents raised the tradeoff between improving survivor specific 
services versus improving the health system overall given the existing public health system 
challenges in Guinea, Liberia, and Sierra Leone.
Specifically, from the perspective of IPs, Government, and USAID Field respondents, given that EVD 
survivor care is embedded in and affected by the quality of the overall public health system, 
respondents felt that the program would have benefited from a broader approach to care for 
survivors versus focusing exclusively on setting up survivor-specific services and processes. 
“From the side of government, with the Survivors Care Support Policy, other line ministries 
besides the MOH should have offered [more], including the Ministry of Gender or Ministry 
of Justice to better enforce the other pillars of the policy. Unfortunately, the MOH has [only] 
done [its part] on three pillars- clinical care, stigma reduction & discrimination, and mental 
health” Survivor Network Representative - Liberia
“The project did not do anything with regards to the other parts of the [EVD survivor] policy 
that were for legal, education, or economic support. But the MOH is [also] not doing 
anything either for [survivors] in this regard- even though the policy says they should.” 
Implementing Partner - Liberia
Again, while the program’s mandate was limited to survivor services and programming, integration 
of Ebola specific healthcare, processes, and data into the broader public health system has not been 
completed and are likely to pose ongoing challenges in the future. 
“Are they tracking Ebola inside HMIS or the new management information system? They're 
only tracked within the [Ebola] Secretariat or the Survivor Network Secretariat. Nobody 
[else] has tracked them in this country. Nowhere. Nowhere else. Only here.” Liberia - 
Government Representative
“There have been many kinds of [data] identification but up to now, there is no official data. 
There is one form at the Ministry of Social Welfare. If you go to SLAES they have their list. If 
you go to the Ministry of Health they have theirs. So, we still have this issue.” Sierra Leone￾Implementing Partner
Program Specific Barriers
In Guinea and Sierra Leone, the slow pace and responsiveness of government bureaucracy were cited 
as having contributed to delays in programming. In Liberia, poor relations between the Implementing 
Partner and the Survivor Network were cited at times as even discouraging survivors from accessing 
care at health facilities.
“There were some delays, sometimes because of the Ministry of Health or the Ministry of Social 
Welfare. Working with the Ministry of Social Welfare, was at times a little bit difficult…” 
33
Implementing Partner - Sierra Leone
“The ANSS [representative] was... a problem for program delays… not allowing the program 
to move forward and without responding [to the IP] for over 4 months… from the period of 
October 2016 until March 2017, he did not accept activities to be implemented.” Implementing
Partner - Guinea
“In late 2017, no survivors presented themselves at the healthcare facilities. According to a 
[survivor], the Survivor Network leader was dissatisfied about some issue with JSI Research & 
Training Institute, Inc. and had instructed survivors not to attend [medical] consultations.” 
Implementing Partner - Liberia
“We found out that the Survivor Network coordinator asked survivors not to go in [to medical 
facilities] because of the transportation reimbursement issue and this was why they were not 
coming in” Implementing Partner – Liberia
Despite these difficulties, qualitative interviews with key informants shed additional insight and 
emerging best practices for future implementation with respect to host government engagement in 
similar settings.
34
Insight: Engaging Early & Often with Government Partners
Especially in post-epidemic settings, the importance of maintaining regular communication and 
good relations with government agencies in host countries is well-known. Interview respondents 
across stakeholder types often mentioned the collaborative nature of program implementation and 
high levels of engagement with stakeholders. Given implementing partners in all three countries 
worked closely and sought positive relationships with government partners, there were some 
illustrative best practices that emerged in each country that speak to approaches that work to foster 
buy-in and ownership of program efforts over time.
“We did our [design] analysis together with ANSS; they showed us the activities [for] SA￾Ceint [surveillance]... We had our activities list and analyzed them in relation with the needs 
identified and priorities set by ANSS. In agreement with ANSS, we identified the [joint] needs 
and those were the needs that JSI Research & Training Institute, Inc. [was asked to] 
address.” Implementing Partner - Guinea
“I think the big impact, the big success here is that we are working and educating the 
government to think more about a health system approach perspective rather than [just] 
response.” Implementing Partner - Sierra Leone
“The biggest difference between CPES Phase 1 and Phase 2… Phase 2 is Ministry led. This 
was a conscious decision to [have the] Ministry lead- made by the Ministry, implementing 
partners, and USAID.” Government Representative - Sierra Leone
“We had participation from the Minister of Health. We supported the MOH Communication 
Unit as well as the Liberia College of Physicians and Surgeons. We sat together to develop 
the messages.” Implementing Partner - Liberia
35
Conclusion
Overall, the GHET program for survivor services was effective, particularly in reducing the stigma EVD 
survivors experienced from health providers when seeking care in Guinea and Liberia. Comparing 
levels of stigma at baseline and endline, survivors reported a 59.5% reduction in provider stigma in 
Guinea and a 62.1% reduction in Liberia. In Sierra Leone, provider stigma was already the lowest 
among countries at baseline (3.6% of survivors reporting stigma when seeking care) and remained the 
lowest at endline (5.8%); thus, the same relative reduction was not evident. Qualitative findings 
confirmed that from the perspective of implementing partners, government representatives, survivor 
network representatives, and USAID field staff that provider trainings and education campaigns on 
stigma were effective.
While the GHET program also improved the availability of clinical services for EVD survivors, 
accessibility of care across the continuum of care (primary to tertiary facilities) was more difficult to 
improve given systemic public health weaknesses in Guinea, Liberia, and Sierra Leone. Survivors did 
report high levels of access to adequate care (74.7%), satisfaction with care received (79%), and an 
improvement in survivor services since the beginning of program implementation (81.5%). However, 
only 38.5% of healthcare providers at primary, secondary, and tertiary facilities sampled indicated 
their facilities were able to “often” or “always” provide adequate care for survivors. When only 
considering responses from tertiary facilities, the figure increased to 57%. While the program 
successfully established and implemented EVD-survivor specific health protocols and referral 
processes, lower levels of adequate services for survivors available at the primary level and survivor 
reluctance to follow the referral system process were systemic deterrents in delivering accessible care 
particularly in closer proximity to survivors at the primary facility level. 
Finally, when considering accessibility of care and preparation for future public health crises, the 
establishment of and capacity-building support to EVD survivor networks and government bodies 
contributed positively to the program’s efforts. However, the integration of survivor-specific services 
and processes into the general public health systems of Guinea, Liberia, and Sierra Leone remains a 
work in progress.
36
Annexes
Annex A: References
Cited References
Centers for Disease Control and Prevention. December 27, 2017. 2014 - 2016 Ebola Outbreak in 
West Africa. Retrieved from: https://www.cdc.gov/vhf/ebola/history/2014-2016-
outbreak/index.html
Ministere de la Sante. (2015). Republique de Guinee Plan de Relance du Systeme de Sante 2015-
2017. Retrieved from: http://invest.gov.gn/document/plan-relance-systeme-sante-guinee-2015-
2017
Ministry of Health and Sanitation. (2015). Sierra Leone Basic Package of Essential Health Services 
2015-2020. Retrieved from: https://mohs2017.files.wordpress.com/2017/06/gosl_2015_basic￾package-of-essential-health-services-2015-2020.pdf
Office of Inspector General US Agency for International Development. (January 2018). Lessons from 
USAID’s Ebola Response Highlight the Need for a Public Health Emergency Policy Framework. Audit 
Report 9-000-18-001-P. Retrieved from: https://oig.usaid.gov/sites/default/files/2018-06/9-000-18-
001-p_0.pdf
Republic of Liberia Ministry of Health and Social Welfare. (2011). National Health and Social Welfare 
Policy and Plan 2011-2021. Retrieved from: http://moh.gov.lr/wp-content/uploads/2017/03/2011-
2021-National-Health-Policy-Plan_MOH.pdf
United States Agency for International Development. (June 2017). Global Health Ebola Team Results 
Framework.
World Health Organization. April 2016. Clinical care for survivors of ebola virus disease. Retrieved 
from: 
http://apps.who.int/iris/bitstream/handle/10665/204235/WHO_EVD_OHE_PED_16.1_eng.pdf;jsessi
onid=3E7440C2B544FEF1B70A21213B2694C5?sequence=1
World Health Organization September 2015. Ebola Response Phase 3: Framework for achieving and 
sustaining a resilient zero. Retrieved from:
http://apps.who.int/iris/bitstream/10665/184693/1/ebola_resilientzero_eng.pdf?ua
Reviewed Documents & References
APC JSI Research & Training Institute, Inc. Quarterly Reports – FY2017 Q1 - FY2018 Q2
- Sierra Leone APC ETP&SS Reports
- Guinea APC ETP&SS Reports
37
- Liberia APC ETP&SS Reports
- Regional APC Survivors Reports
GHET Guinea Country Strategy, June 2016, Internal USAID Document
GHET Liberia Country Strategy (Date Unknown), Internal USAID Document
GHET Sierra Leone Country Strategy (July, 2015), Internal USAID Document
Guinea National Strategy, Government of Guinea. (Nov 2014).
Sierra Leone National Ebola Recovery Strategy, Government of Sierra Leone. (March 2015).
Investment Plan for Building a Resilient Health System in Liberia, Government of Liberia. (April 2015) 
Oxford University Press for the Infectious Diseases Society of America. (June 2016). Ebola Virus 
Disease Complications as Experienced by Survivors in Sierra Leone. Retrieved from: 
https://www.ncbi.nlm.nih.gov/pubmed/27001797
WHO. (March 2018). Emergency Risk Communication: Lessons Learned from a Rapid Review of 
Recent Gray Literature on Ebola, Zika, and Yellow Fever. Retrieved from: 
https://www.ncbi.nlm.nih.gov/pubmed/29558199
Global
Congressional Research Services. (May 2016). Status of the Ebola Outbreak in West Africa: Overview 
and Issues for Congress. Retrieved from: https://fas.org/sgp/crs/row/R44507.pdf
Kaiser Family Foundation. (November 2015). One Year Later, Where Does the U.S. Response to Ebola 
Stand. Retrieved from: https://kaiserfamilyfoundation.files.wordpress.com/2015/11/final-slide￾deck-kff-ebola-event-11-23-15.pdf
WHO. (2016). Guidelines on Core Components of Infection Prevention and Control Programmes at 
the National and Acute. Retrieved from: http://www.who.int/gpsc/core-components.pdf
WHO. (2015). Infection Prevention and Control Recovery Plans and Implementation: Guinea, Liberia, 
and Sierra Leone Inter-Country Meeting. Retrieved from: 
http://apps.who.int/iris/bitstream/handle/10665/204370/WHO_HIS_SDS_2015.23_eng.pdf;jsessioni
d=85562A5DE88083A5E52E7C4CD3930448?sequence=1
Makerere Medical School. (March 2017). Role of contact tracing in containing the 2014 Ebola 
outbreak: a review. Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5636234/
Government of the Republic of Sierra Leone: Ministry of Health and Sanitation. (2015). National 
Infection Prevention and Control Guidelines. Retrieved from: 
https://afro.who.int/sites/default/files/2017-05/ipcguide.pdf
38
Republic of Liberia Ministry of Health. (2015). WASH and Environmental Health Package in Health 
Facilities. Retrieved from: https://www.washinhcf.org/documents/Final-WASH-EH-Package-for￾Health-Facilities-003.pdf
39
Annex B. Instruments
BalanceD-MERL Facility Assessment 
Starting on next page
40
Field Question Answer
Survey Information
enumerator_name (required) Enumerator name
county_select County
district District name district district
type Facility type
facility_name_search Select the correct facility from the list below: -77 Cannot find the
name
facility_name facility_name
Survey Information > consentgrp
introduction Hello, my name is . (Enumerator name)
I am a researcher for Innovations for Poverty Action, a research and policy non-profit organisation that 
discovers and promotes effective solutions to global poverty problems.
I am visiting you today because we are a conducting a study about the health system in Liberia.
purpose The purpose of this survey is to better understand the current state of the health system, including information
about local health facilities, health providers, and community members who may use those health services.
procedures If you choose to participate, you will be asked to complete a survey. 
The survey will cover:
Information about the facility, including staffing, infrastructure, and services offered
Information about external support the facility has received since the Ebola Virus Disease outbreak 
The survey will take approximately 1 hour of your time.
risks_benefits Risks and Benefits:
We hope that this research will help us better understand the current state of the health system for Ebola 
Virus Disease survivors to improve future public health interventions.
We do not anticipate any risks to you from study participation. You are free to skip any question that makes
you feel uncomfortable.
confidentiality No names will be stored with survey responses and no names will be published from the study.
The research staff will not share your personal information with anyone outside the study, and they will do 
their best to protect your information.
voluntary_participation Participation in this study is voluntary. That means you do not have to participate if you do not want to. 
Ending participation at any time for any reason will not have any negative consequences for you. You may 
withdraw at any time, which you can do by simply telling me that you no longer want to be part of the study.
Refusal to answer any individual questions will not have any negative consequences for you.
contact For questions regarding the research or about your rights as a research participant please contact:
questions Do you have any further questions?
consent (required) If I have answered all your questions, do you agree to participate in this study? 1 Yes
0 No
signature (required) Please sign here to indicate that you agree.
Question relevant when: ${consent} = 1
All_questions
Group relevant when: ${consent} = 1
All_questions > Respondent profile
enu_note_1 I'd like to ask you a few questions about you and the facility you work in
facility_in-charge (required) Are you the facility in-charge? 1 Yes
0 No
41
Field Question Answer
position (required) What is your position
Question relevant when: ${facility_in-charge} = 0
facility_operational (required) How long has this facility been operational? 1 3-4 years
2 5-7 years
3 8-10 years
4 above 10 years
-999 Don't know
-888 Refuse to say
facility_work_exp (required) How long have you been working at this facility? 1 Less than 3 years
2 3 years or more
-999 Don't know
-888 Refuse to say
All_questions > Questions for qualified respondents
Group relevant when: ${facility_work_exp} = 2
All_questions > Questions for qualified respondents > Facility Profile
cathment_pop (required) What is the size of the catchment area population of this facility?
All_questions > Questions for qualified respondents > Facility staff
providers_physicians_full(required) How many physicians are full time at this facility?
providers_physicians_part(required) How many physicians are on part time at this facility?
providers_physicians_ass_full(required) How many physicians assistants are on full time at this facility?
providers_physicians_ass_part(required) How many physicians assistants are on part time at this facility?
providers_nurse_full (required) How many nurses are on full time at this facility?
providers_nurse_part(required) How many nurses are on part time at this facility?
providers_nurse_aide_full(required) How many nurse aides are on full time at this facility?
providers_nurse_aide_part(required) How many nurse aides are on part time at this facility?
providers_midwives_full (required) How many midwives are on full time at this facility?
providers_midwives_part(required) How many midwives are on part time at this facility?
facility_inpatients (required) Does this facility take inpatients? 1 Yes
0 No
-999 Don't know
-888 Refuse to say
facility_bed (required) Does this facility have beds? 1 Yes
0 No
-999 Don't know
-888 Refuse to say
facility_bed_number (required) How many beds does the facility have?
Question relevant when: ${facility_bed} = 1
All_questions > Questions for qualified respondents > EVD_Clinical_Services
All_questions > Questions for qualified respondents > EVD_Clinical_Services > EVD
Enu_note_2 I now want to ask you about some of the services offered to EVD survivors at this facility.
ebola_survivors_record (required) Do you keep a record of how many Ebola survivors are treated at this facility? 1 Yes
0 No
Enu_note_3 I would like to ask you some questions about how many ebola survivors are treated here.
If they do not have records or cannot get the records , ask them to answer from memory.
answer_type (required) Is the person giving answers from a written record?
Question relevant when: ${ebola_survivors_record} = 1
1 directly in a written record
2 written record and memory
3 Telling you from memory
registered_survivors_received (required) How many registered EVD survivors have received care at this facility since the end of the ebola time? (From 
June 2016 to date)
If the respondent does not know please enter -999
All_questions > Questions for qualified respondents > EVD_Clinical_Services > screen_refer
serv_offer For each of the following EVD survivor services, please could you tell me if they were delivered before the
Ebola time, during the Ebola time, and now?
screen_refer_eye (required) Basic eye screening and referral
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
42
Field Question Answer
screen_refer_auditory (required) Basic auditory screening and referral
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
screen_refer_abdominal (required) Abdominal clinical screening and referral
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
screen_refer_neuromusc (required) Neuromuscular screening and referral
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
screen_refer_mental (required) Mental health screening and support (counselling, trauma healing, stress management, psychotherapy)
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
screen_refer_sex (required) Sexual health and Family planning services
Please select all that apply
1 Offered before ebola
2 Offered during ebola
3 Offered now
4 Never offered
-999 Don't know
-888 Refuse to say
lowest_number (required) C.03. In the last seven days how many EVD survivors used this facility
Ask the respondent to estimate, If the respondent does not know please enter -999
evd_survivors_specific (required) C.04. Of the [lowest_number] of EVD survivors who used this facility, how many needed survivor specific 
services?
If the answer to any of these is yes, it is a survivor specific service:<br/>1) Is the health problem a direct result of
survivor status?<br/>2) Is the service only available to survivors?<br/>3)Does the health worker need specific training
on EVD survivors to deliver this service?
Question relevant when: ${lowest_number} >=1
All_questions > Questions for qualified respondents > EVD_Clinical_Services > specialists
intro_spec How many specialists who can treat the ailments/illnesses common to EVD survivors do you have at this
facility?
specialist_opthamologist (required) Ophthalmologist
specialist_mental_health (required) Mental health specialist
specialist_sex_health (required) Sexual health specialist
specialist_internsit (required) Internist (for EVD specific abdominal complications)
specialist_physio (required) Physical rehabilitation therapists (Physiotherapist)
specialist_otorhinolaryngologist (required) Doctor specialised in ear, nose and throat diseases (Otorhinolaryngologist)
specialist_lab_diagnostic (required) Lab diagnostic service specialist
All_questions > Questions for qualified respondents > EVD_Clinical_Services > trained_providers
providers_trained C.10. How many regular health providers at this facility have been trained to provide EVD specific services?
train_eye_screen (required) Basic eye screening and referral
train_auditory_screen (required) Basic auditory screening and referral
train_abdominal_screen (required) Abdominal clinical screening and referral
train_neuromusc_screen (required) Neuromuscular screening and referral
train_rheumatology (required) Rheumatology screening and referal
train_mental_screen (required) Mental health screening and support (counselling, trauma healing, stress management, psychotherapy)
train_sex_screen (required) Sexual health and Family planning services
43
Field Question Answer
adequate_services (required) C.11. On a scale of 1 to 5, How would you rate your facility’s capacity when it comes to providing EVD 
survivors with adequate access to health services (other than primary healthcare services) and resources to 
meet their health needs?
You may read the options to the respondent
5 5...Excellent
4 4…Good
3 3…Fair
2 2…Poor
1 1.. Very poor
-999 Don't know
-888 Refuse to say
sufficient_services (required) C.12. On a scale of 1 to 5, How would you rate your facility’s ability to provide EVD survivors with sufficient 
quality health services (other than primary healthcare services) and resources to meet their health needs? 
You may read the options to the respondent
5 5...Excellent
4 4…Good
3 3…Fair
2 2…Poor
1 1.. Very poor
-999 Don't know
-888 Refuse to say
All_questions > Questions for qualified respondents > Section D: REFERRAL SYSTEM QUESTIONS
refer_other_facility (required) D.01. Are there facilities you can refer Ebola survivors to for survivor-specific services that are not provided 
here?
1 Yes
0 No
refer_track (required) D.02. Does the facility record referrals made?
Question relevant when: ${refer_other_facility} = 1
1 Yes
0 No
refer_counter D.02b Does the facility record counter-referrals?
Counter referrals are referrals made to the facility.
1 Yes
0 No
refer_other_facility_num (required) D.03. In the last three months, how many Ebola survivors have you referred to other facilities for survivor￾specific services?
Question relevant when: ${refer_track} = 1
refer_this_facility (required) D.04. Do other facilities refer Ebola survivors to this facility for survivor specific services? 1 Yes
0 No
refer_this_facility_num (required) D.05. In the last three months, how many Ebola survivors have been referred to this facility for survivor￾specific services? Question relevant when: ${refer_this_facility} = 1
medication Section F : Medicine
All_questions > Questions for qualified respondents > observations
meds I now want to ask you some questions about medication and equipment available at the facility. When 
possible, please show me the actual medication or equipment.
Please ask to observe the following medicines. If the nurse cannot show you them, please ask if they are just
unavailable today, or never available.
paracetamol (required) Paracetamol
Used for mild to moderate pain relief
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
ibuprofen (required) Ibuprofen
Used for treating pain, fever, and inflammation.
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
prednisone (required) Prednisone (oral and IV)
Medication that treats inflammation
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
44
Field Question Answer
methotrexate (required) Methotrexate
Used to treat cancer and autoimmune diseases
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
ranitidine (required) Ranitidine
Medication that decreases stomach acid production
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
omeprazole (required) Omeprazole
Medication that decreases stomach acid production
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
amoxicillin (required) Amoxicillin
Antibiotic useful for the treatment of a number of bacterial infections
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
ivermectin (required) Ivermectin
Medication that is effective against many types of parasites
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
fluoxetine (required) Fluoxetine
Used to treat depression
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
45
Field Question Answer
diazepam (required) Diazepam
Used for the short-term relief of symptoms related to anxiety disorders; and ...
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
haloperidol (required) Haloperidol
Antipsychotic medication
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
amitriptyline (required) Amitriptyline
Medication used to treat mental/mood problems such as depression
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
prednisolone (required) Prednisolone acetate drops 1%
Steroid medicine used to treat eye swelling
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
atropine_drops (required) Atropine drops 1%
Used before eye examinations and to treat certain eye conditions
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
cyclopentolate (required) Cyclopentolate 1%
Used before eye examination
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
46
Field Question Answer
timolol (required) Timolol drops 0.5%
Used to treat increased pressure inside the eye
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
tetracycline (required) Tetracycline ointment
Antibiotic that works by stopping the growth of bacteria
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
tears (required) Artificial tears
Lubricant eye drops used to treat the dryness and irritation associated with deficient tear productin
1 Observed and in date
3 Observed but expired
4 Reported to have today,
but not observed
5 Not present today, but
normally in stock
6 Not usually stocked
-999 Don't know
-888 Refuse to say
All_questions > Questions for qualified respondents > medicine_use
generated_note_name_130 I would like to ask who these medicines are used for
paracetamol_evd (required) Paracetamol
Used for mild to moderate pain relief
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
ibuprofen_evd (required) Ibuprofen
Used for treating pain, fever, and inflammation.
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
prednisone_evd (required) Prednisone (oral and IV)
Medication that treats inflammation
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
methotrexate_evd (required) Methotrexate
Used to treat cancer and autoimmune diseases
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
ranitidine_evd (required) Ranitidine
Medication that decreases stomach acid production
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
47
Field Question Answer
omeprazole_evd (required) Omeprazole
Medication that decreases stomach acid production
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
amoxicillin_evd (required) Amoxicillin
Antibiotic useful for the treatment of a number of bacterial infections
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
ivermectin_evd (required) Ivermectin
Medication that is effective against many types of parasites
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
fluoxetine_evd (required) Fluoxetine
Used to treat depression
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
diazepam_evd (required) Diazepam
Used for the short-term relief of symptoms related to anxiety disorders; and ...
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
haloperidol_evd (required) Haloperidol
Antipsychotic medication
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
amitriptyline_evd (required) Amitriptyline
Medication used to treat mental/mood problems such as depression
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
prednisolone_evd (required) Prednisolone acetate drops 1%
Steroid medicine used to treat eye swelling
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
atropine_drops_evd (required) Atropine drops 1%
Used before eye examinations and to treat certain eye conditions
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
cyclopentolate_evd (required) Cyclopentolate 1%
Used before eye examination
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
48
Field Question Answer
timolol_evd (required) Timolol drops 0.5%
Used to treat increased pressure inside the eye
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
tetracycline_evd (required) Tetracycline ointment
Antibiotic that works by stopping the growth of bacteria
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
tears_evd (required) Artificial tears
Lubricant eye drops used to treat the dryness and irritation associated with deficient tear productin
1 Available only to EVD
survivors
2 Available only to people who
have never had ebola
3 Available to both
4 Not available
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities
bhc Section H: Basic Equipment and Commodities
intro_bhc I now want to ask you some questions about equipment available at the facility. When possible, please show
me the actual equipment.
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment > Lampes à fente
lamp Are slit lamps
A slit lamp is a lamp that emits a narrow but intense beam of light, used for examining the interior of the eye.
slit_lamps_normal_stock (required) Normally in stock 1 Yes
0 No
slit_lamps_available (required) Reported to be in stock 1 Yes
0 No
slit_lamps_observed (required) Observed 1 Yes
0 No
slit_lamps_evd (required) Available to EVD survivors 1 Yes
0 No
slit_lamps_non_evd(required) Available to people who have never had ebola 1 Yes
0 No
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment > Tonometers
tonometer Are Tonometers:
A tonometer is an instrument for measuring the pressure in a part of the body, such as the eyeball (to test for
glaucoma) or a blood vessel.
tonometers_normal_stock (required) Normally in stock 1 Yes
0 No
tonometers_available (required) Reported to be in stock 1 Yes
0 No
tonometers_observed (required) Observed 1 Yes
0 No
tonometers_evd (required) Available to EVD survivors 1 Yes
0 No
tonometers_non_evd (required) Available to people who have never had ebola 1 Yes
0 No
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment > ophthalmoscopes
opthalmoscope Are ophthalmoscopes :
An Opthalmoscope is an instrument for inspecting the retina and other parts of the eye.
ophthalmoscopes_normal_stock(required) Normally in stock 1 Yes
0 No
ophthalmoscopes_available (required) Reported to be in stock 1 Yes
0 No
ophthalmoscopes_observed (required) Observed 1 Yes
0 No
ophthalmoscopes_evd (required) Available to EVD survivors 1 Yes
0 No
49
Field Question Answer
ophthalmoscopes_non_evd (required) Available to people who have never had ebola 1 Yes
0 No
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment > audiometers
audiometer Are audiometers :
An audiometer is a device used for measuring hearing.
audiometers_normal_stock (required) Normally in stock 1 Yes
0 No
audiometers_available (required) Reported to be in stock 1 Yes
0 No
audiometers_observed (required) Observed 1 Yes
0 No
audiometers_evd (required) Available to EVD survivors 1 Yes
0 No
audiometers_non_evd (required) Available to people who have never had ebola 1 Yes
0 No
All_questions > Questions for qualified respondents > Section H: Basic Equipment and Commodities > equipment > tuning_forks
tuning_fork Are Tuning Forks:
A tuning fork is a device used for measuring hearing
tuning_forks_normal_stock (required) Normally in stock 1 Yes
0 No
tuning_forks_available (required) Reported to be in stock 1 Yes
0 No
tuning_forks_observed (required) Observed 1 Yes
0 No
tuning_forks_evd (required) Available to EVD survivors 1 Yes
0 No
tuning_forks_non_evd (required) Available to people who have never had ebola 1 Yes
0 No
end We have now come to the end of the interview, thank you for taking part in this interview.
Please inform the respondent about the small gift you have for them.
comment (required) Comment
50
BalanceD-MERL Provider Survey
Field Question Answer
A. Survey Information
enumerator_name Enumerator name
county_select (required) County
district (required) District name district district
type (required) Facility type
facility_name_search (required) Select the correct facility from the list below: -77 Cannot find the
name
facility_name facility_name
A. Survey Information > consent
introduction Hello, my name is . (Enumerator name). I am a researcher for Innovations for Poverty Action, 
a research and policy non-profit organisation that discovers and promotes effective solutions to global poverty 
problems.
I am visiting you today because we are conducting a study about the health system in Liberia.
purpose The purpose of this survey is to better understand the current state of the health system, including information about
local health facilities, health providers, and community members who may use those health services.
procedures If you choose to participate, you will be asked to complete a survey. The survey will cover Demographic information 
(e.g. level of education)
Information about your work at the health facility (e.g. what kind of services you provide, what kind of training you 
have received)
Your thoughts on your work (e.g. your beliefs about caring for patients with Ebola Virus Disease and EVD survivors)
The survey will take approximately 1 hour of your time.
risks_benefits Risks and Benefits:
We hope that this research will help us better understand the current state of the health system for Ebola Virus 
Disease survivors in order to improve future public health interventions.
We do not anticipate any risks to you from study participation. You are free to skip any question that makes you feel
uncomfortable.
confidentiality Confidentiality:
No names will be stored with survey responses and no names will be published from the study.
The research staff will not share your personal information with anyone outside the study, and they will do their best to 
protect your information.
voluntary_participation Voluntary Participation:
Participation in this study is voluntary. That means you do not have to participate if you do not want to.
Ending participation at any time for any reason will not have any negative consequences for you. You may withdraw at 
any time, which you can do by simply telling me that you no longer want to be part of the study.
Refusal to answer any individual questions will not have any negative consequences for you.
contact For questions regarding the research or about your rights as research participant please contact:
questions Questions:
Do you have any further questions?
consent_select (required) Do you agree to participate in this interview? 1 Yes
0 No
signature (required) Please sign here to indicate that you agree.
signature
Question relevant when: ${consent_select} =1
51
Field Question Answer
A. Survey Information > Consent Yes
Group relevant when: ${consent_select} =1
A. Survey Information > Consent Yes > B. Demographics
position (required) B.01. What is your position at the health facility? 1 Medical Doctor
2 Physician Assistant
3 Nurse
4 Nurse Aide
5 Medical Officer
6 Midwife
7 Community health
worker/officer
-777 Other (specify)
position_specify (required) Specify other:
Question relevant when: ${position} =-777
specialization (required) B.02. What is your specialization?
Question relevant when: ${position} =1 or ${position} = 3
1 No Specialization/general
practice
-777 Other (specify)
specialization_specify (required) Specify Other:
Question relevant when: ${specialization} =-777
hf_duration (required) B.03. How long have you worked at this health facility? 1 Less than 1 year
2 1-2 years
3 3-4 years
4 5-10 years
5 More than 10 years
work_other (required) B.04. Did you work at another facility before this one? 1 Yes
0 No
incharge (required) B.05. Are you the facility in-charge? 1 Yes
0 No
trained_EVD (required) B.06. Have you been trained and/or recruited to provide EVD-specific services?
If the answer to any of these is yes, it is a survivor specific service:<br/><br/>1) Is the health problem a direct result of survivor
status?<br/><br/>2) Is the service only available to survivors?<br/><br/>3)Does the health worker need specific training on EVD
survivors to deliver this service?
1 Yes
0 No
A. Survey Information > Consent Yes > C. EVD Services
enu_note_2 I now want to ask you some questions about the services available for Ebola survivors at the health facility
EVD_training (required) C.01. What training have you received to provide specific EVD services?
Select all that apply
Question relevant when: ${trained_EVD} =1
1 EVD service specialist type
2 Basic eye screening and
referral
3 Basic auditory screening
and referral
4 Abdominal clinical
screening and referral
5 Neuromuscular screening
and referral
6 Mental health screening 
and support (counseling, 
trauma healing, stress 
management,
psychotherapy)
7 Sexual health and Family
planning services
-777 Other (specify)
training_specify (required) Specify other:
Question relevant when: ${EVD_training} =-777
survivors_serv (required) C.02. In the In the last three months, how many Ebola survivors have you yourself provided services to?
If no exact number, please provide your best estimate
facility_serv (required) C.03. In your opinion, how often is this facility able to provide Ebola survivors with health services sufficient in quantity 
to achieve their therapeutic needs?
1 Never
2 Rarely
3 Sometimes
4 Often
5 Always
A. Survey Information > Consent Yes > D. Referral System
52
Field Question Answer
facility_no_refer (required) D.01. How often is this facility able to provide survivor services without needing to refer patients to other places?
Please read the options to the respondents
1 Never
2 Rarely
3 Sometimes
4 Often
5 Always
facility_refer (required) D.02. Are there facilities you can refer ebola survivors to for survivor specific services that are not provided here? 1 Yes
0 No
A. Survey Information > Consent Yes > D. Referral System > Survivors Referred
survivor_refer_fac D.03. In the last three months, how many ebola survivors have you referred to an EVD service specialist AT THIS 
FACILITY?
For each service how many people has the provider referred to a specialist within the facility? (AT THE SAME FACILITY)
refer1 (required) Basic eye screening and referral
refer2 (required) Basic auditory screening and referral
refer3 (required) Abdominal clinical screening and referral
refer4 (required) Mental health screening and support (counselling, trauma healing, stress management, psychotherapy
refer5 (required) Neuromuscular screening and referral
refer6 (required) Sexual heallth and family planning services
A. Survey Information > Consent Yes > D. Referral System > Survivors Referred Other
Group relevant when: ${facility_refer} =1
survivor_refer_oth In the last three months, how many ebola survivors have you referred to an EVD service specialist AT ANOTHER 
FACILITY?
For each service how many people has the provider referred to another facility?
refer_other1 (required) Basic eye screening and referral
refer_other2 (required) Basic auditory screening and referral
refer_other3 (required) Abdominal clinical screening and referral
refer_other4 (required) Mental health screening and support (counselling, trauma healing, stress management, psychotherapy
refer_other5 (required) Neuromuscular screening and referral
refer_other6 (required) Sexual heallth and family planning services
other_facility_refer (required) D. 04. Do other facilities refer Ebola survivors to this facility for survivor specific services? 1 Yes
0 No
survivor_referred (required) D.05. In the last three months, how many ebola survivors have been referred to this facility for survivor specific 
services?
Question relevant when: ${other_facility_refer} =1
A. Survey Information > Consent Yes > D. Referral System > Shared documentation
information_shared D.06. When a patient is referred to another facility, what documentation if any, is shared between providers/facilities 
about that patient's treatment history (e.g. diagnosis, lab results etc).
1 Information/ records shared
between providers
2 Information/ records shared
between facilities
3 Not shared
share1 (required) Referral sheet 1 Information/ records shared
between providers
2 Information/ records shared
between facilities
3 Not shared
A. Survey Information > Consent Yes > D. Referral System > Provider referral satisfaction
provider_satisfied D.07. How satisfied are you with the following aspects of the referral system? 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
availability (required) Availability of providers to whom you can refer your patients 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
proximity (required) Proximity and ease of access of your patients to the referral providers 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
53
Field Question Answer
quality (required) Quality of providers to whom I can refer my patients 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
documentation (required) Documentation sharing between health providers in the referral system 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
collaboration (required) Collaboration between health provideres in the referral system 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
survivor_satisfied (required) D.08. How satisfied are your EVD survivor patients with the referral system? 1 Very Satisfied
2 Satisfied
3 Neutral
4 Dissatisfied
5 Vey Dissatisfied
A. Survey Information > Consent Yes > E. EVD Stigma
enu_note_3 I now want to ask you some questions about the risk of ebola transmission associated with certain activities
A. Survey Information > Consent Yes > E. EVD Stigma > E.1. Beliefs Transmission
enu_note_4 E.1.1. We want to understand how risky you think it is to do the following activities on a scale of 0 (no risk) to 10 (high 
risk). So if you thought an activity posed no risk of transmission, you would rate it a zero. If you thought it posed a 
moderate risk of transmission you might rate it a 5 or 6. If you thought it posed an extremely high risk of transmission 
you might rate it a 9 or 10
MAKE SURE THE RESPONDENT UNDERSTANDS THE RISK SCALE
risk_handshake (required) Shake hands with someone who had Ebola, but has now recovered and been declared well 0 0 (no risk of transmission)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 (high risk of transmission)
risk_inject (required) Give an injection to someone who had Ebola, but has now recovered and been declared well 0 0 (no risk of transmission)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 (high risk of transmission)
risk_services_evd_ppe (required) Providing services to someone who CURRENTLY has Ebola, using PPE 0 0 (no risk of transmission)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 (high risk of transmission)
54
Field Question Answer
risk_services_evd_noppe (required) Providing services to someone who CURRENTLY has Ebola, WITHOUT PPE 0 0 (no risk of transmission)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 (high risk of transmission)
risk_services_baby (required) Give routine checkup to a healthy baby who has never had ebola 0 0 (no risk of transmission)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
10 10 (high risk of transmission)
precaution (required) E1.2. Do you need to take any EXTRA precautions when working with Ebola survivors who have recovered and been 
declared well? (compared to someone who never had Ebola)
1 Yes
0 No
precaution_specify E.1.3. What extra precaution do you need to take?
Question relevant when: ${precaution} =1
A. Survey Information > Consent Yes > E. EVD Stigma > E.2. Stigma- Training
enu_note_5 We now want to ask you some questions about perceptions and care and treatment of Ebola survivors
train_stigma (required) E.2.1. Have you been trained on Ebola Survivor Stigma reduction?
Training can be either formal classroom-based OR on the job mentoring/supervision.
1 Yes
0 No
number_train (required) E.2.2. How many such trainings have you received?
Question relevant when: ${train_stigma} =1
trainers (required) E.2.3a. Who delivered the trainings?
Select all that apply
Question relevant when: ${train_stigma} =1
1 County health team (CHT)
2 District health team (DHT)
3 Health facility staff
4 Civil Service Organization
(CSO) / Non Governmental 
Organization (NGO)
5 Medical / Nursing schools
-777 Other (specify):
trainer_specify (required) Specify other:
Question relevant when: ${trainers} =-777
organisations_train E.2.3b. Please name the organisation(s) who delivered the Ebola Survivor Stigma reduction training
Question relevant when: ${trainers} =-777 or ${trainers} =4 or ${trainers} =5
knowledge_change (required) E.2.4a. To what extent did your knowledge about caring for Ebola survivors change as a result of this training?
Knowledge is what we KNOW
Question relevant when: ${train_stigma} =1
1 Not at all
2 A little
3 Some
4 Very much
knowledge_specify (required) E.2.4b. Could you describe an example of this change in knowledge?
Question relevant when: ${knowledge_change} =2 or ${knowledge_change} =3 or ${knowledge_change} =4
attitude_change (required) E.2.5a. To what extent did your attitudes about caring for Ebola survivors change as a result of this training?
Attitude is how we FEEL
Question relevant when: ${train_stigma} =1
1 Not at all
2 A little
3 Some
4 Very much
attitude_specify (required) E.2.5b. Could you describe an example of this change in attitudes?
Question relevant when: ${attitude_change} =2 or ${attitude_change} =3 or ${attitude_change} =4
behavior_change (required) E.2.6a. To what extent did your behavior when caring for ebola survivors change as a result of this training?
Behaviour is how we ACT
Question relevant when: ${train_stigma} =1
1 Not at all
2 A little
3 Some
4 Very much
55
Field Question Answer
behavior_specify (required) E.2.6b. Could you describe an example of this change in behavior?
Question relevant when: ${behavior_change} =2 or ${behavior_change} =3 or ${behavior_change} =4
A. Survey Information > Consent Yes > E. EVD Stigma > E.3. Stigma - Perceptions
enu_note_6 I now want to ask you some questions about how ebola survivors should be treated
stigma1 (required) E.3.1. Ebola survivors should have separate screening and treatment areas to prevent transmission to others
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
stigma2 (required) E.3.2. Ebola survivors have more access to care and treatment compared to patients with other ilnesses, this is unfair
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
stigma3 (required) E3.3. Community stigma negatively affects the lives of Ebola survivors
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
stigma4 (required) E.3.4. You feel comfortable providing health services to Ebola survivors
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
stigma5 (required) E3.5. Ebola survivors have a responsibility to disclose their status to healthcare providers
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
stigma6 (required) E.3.6. Stigma in the community or at the health facility currently prevents Ebola survivors from seeking care
Please read out the options to the respndent
1 Strongly agree
2 Agree
3 Neither agree nor disagree
4 Disagree
5 Strongly disagree
A. Survey Information > Consent Yes > E. EVD Stigma > E.4. EVD Stigma - Enacted
stigma8 (required) E.4.1. During the Ebola epidemic, did you ever see someone with Ebola denied services at a health facility? 1 Yes
0 No
stigma8a (required) E.4.2. Why were they turned away?
Question relevant when: ${stigma8} =1
stigma9 (required) E.4.3. In the last year, have you seen an Ebola survivor denied care at any health facility? 1 Yes
0 No
-999 Don't know
-888 Refuse to say
stigma9a (required) E.4.4. Why were they turned away?
Question relevant when: ${stigma9} =1
stigma10 (required) E.4.5. Does Ebola-related stigma occur at this health facility currently? 1 Yes
0 No
-999 Don't know
-888 Refuse to say
A. Survey Information > Consent Yes > E. EVD Stigma > E.4. EVD Stigma - Enacted > E4.6a. Enacted Stigma, HF
Group relevant when: ${stigma10} =1
generated_note_name_125 I. What are some examples of Ebola-related stigma that still occur in this facility?
stigma11 1.
stigma12 2.
stigma13 3.
stigma14 4.
stigma15 (required) II. If you ever saw any of the above happening, would it be possible to report such an incident?
Question relevant when: ${stigma10} =1
1 Yes
0 No
56
Field Question Answer
stigma16 (required) III. If you ever saw any of the above happening who could you report it to?
Select all that apply
Question relevant when: ${stigma10} =1
0 No one
1 My supervisor
2 Facility Administration
3 Other health facility staff
member
4 District health team staff
member
-777 Other (specify)
-999 Don't know
stigma17 (required) Specify other:
Question relevant when: ${stigma16} =-777
stigma18 (required) IV. If a survivor believes that they are a victim of stigma and unfair treatment due to their Ebola survivor status, would 
it be possible for them to report such an incident?
1 Yes
0 No
stigma19 (required) V. Who could they report it to?
Select all that apply
Question relevant when: ${stigma18} =1
0 No one
1 Their healthcare provider
2 Other health facility staff
member
3 Facility administration
4 District health staff team
member
-777 Other (specify)
-999 Don't know
stigmaother (required) Specify other:
Question relevant when: ${stigma19} =-777
stigma20 (required) VI. Does Ebola-related stigma occur in the community? 1 Yes
0 No
A. Survey Information > Consent Yes > E. EVD Stigma > E.4. EVD Stigma - Enacted > E.4.6. Enacted Stigma, Community
Group relevant when: ${stigma20} =1
generated_note_name_139 I. What are some examples of Ebola-related stigma in the community?
stigma21 1.
stigma22 2.
stigma23 3.
stigma24 4.
generated_note_name_147 We have now come to the end of the interview, thank you for taking part in this interview.
Please remember to present the small gift to the respondent.
comment (required) Comment
generated_note_name_150 Thank you for taking the time to speak with me today.
Question relevant when: ${consent} !=1
57
BalanceD-MERL Key Informant Interview Framework
Framework of questions for KIIs with different respondent types
EVALUATION CRITERIA: EFFECTIVENESS, EFFICIENCY, RELEVANCE and ADAPTIVE MANAGEMENT
Domain Purpose Respondents Questions
Topic 1: CLINICAL HEALTHCARE 
SERVICES FOR SURVIVORS
Effectiveness: To understand the 
various perspectives of project 
stakeholders about whether EVD 
survivors had improved access to 
appropriate healthcare services. 
To provide additional context to 
quantitative data from survivor 
and provider surveys to ascertain 
to what extent improvements to 
services are attributable or not 
attributable to GHET funding
Implementing 
Partners, 
Governments 
(Relevant 
Ministries and 
Agencies), 
Survivor 
Networks
1. In the context of EVD survivors, how would you define 
“appropriate” healthcare services?
- If I was to ask other stakeholders, do you think that everyone 
would have a shared understanding of appropriate 
healthcare services for EVD survivors? 
2. In your opinion, do EVD survivors have access to appropriate 
healthcare services in [country] context today?
Follow up question: 
- Was it always like this? How has this access changed over 
time? 
- Has this change led to improvement of access to appropriate 
healthcare services for ebola survivors? 
3. Do you believe the program has contributed to the 
improvement of healthcare services for EVD survivors? If so, how 
and how much?
Follow up question: 
- Who else or what other program(s) contributed to making 
improvements to services?
- If so, how does the contribution of this program compare to 
the contributions of these other programs? Please provide 
details and/or evidence to support your response.
Adaptive management and 
efficiency: To understand how 
activities related to 
Implementing 
Partners, 
Governments 
1. What improvements to EVD survivor healthcare services did the 
program make throughout the duration of implementation?
58
strengthening clinical services 
and accessibility were 
implemented and adapted by 
implementing partners over 
time. 
(Relevant 
Ministries 
and 
Agencies), 
Survivor 
Networks
2. How did the IPs decide which services would be implemented 
and where?
Follow up questions:
- Did this change at any point in the program’s 
implementation?
- If so, how was that change managed? (Prompts: Who was 
involved? What was the process? What was the outcome?)
3. How has the intensity in the need for Ebola related clinical 
healthcare services changed throughout the implementation 
period? (please be specific) 
Follow up question: 
- Did that affect how you supported the provision of EVD 
survivor services through your targeted health facilities? 
Please explain your response and be as specific as possible.
Topic 2: STIGMA REDUCTION Effectiveness: To understand 
how different stakeholders 
perceive the effectiveness of the 
program as it relates to the 
reduction of stigma survivors 
face from health providers and 
members of their community 
(Liberia & Guinea only on 
community component)
Implementing 
Partners, 
Governments 
(relevant 
Ministries and 
Agencies), 
Survivor 
Networks
1. In your opinion, has there been a reduction in the enacted 
stigma of health providers against EVD survivors? [Int. note: 
Enacted stigma is defined as discrimination or unfair treatment by 
providers against EVD survivors] Please provide examples or data 
to explain your response. (Prompts: How was it at the beginning 
of the outbreak vs. now?)
Follow up question: 
- If so, what do you think led to this reduction? What 
contributed to it?
2. Do you believe this program has contributed to the reduction of 
survivor stigma from providers in your [country] context? 
Follow up question: 
- If so, how? 
- Who else or what other program(s) has contributed to 
59
reducing stigma?
- If so, how does the contribution of this program compare to 
the contributions of these other programs? Please provide 
details and/or evidence to support your response.
3. In your opinion, how effective were the various campaigns 
implemented in your [country] context in reducing stigma 
experienced by survivors? (Prompts: SBCC? Other campaigns?) 
Follow up questions:
- If these campaigns weren’t implemented, how would things
be different? (prompt: What difference did these campaigns 
make?)
- How would you compare the effectiveness of the various 
campaigns in your country? (prompt: Were some more 
effective than others and if so, how?) Please explain.
4. How effective were stigma trainings provided to health 
professionals and community health volunteers/workers (int. note: 
Liberia & Guinea only) in minimizing the effects of stigma in the 
community? 
Follow up questions: 
- What components of the stigma training were most and least 
effective? 
- Were there any instances where stigma trainings weren't 
effective? Please explain.
5. Are stigma reduction trainings equally effective for both health 
providers and community members (Liberia & Guinea only)? Why 
or why not?
Efficiency and Adaptive 
Management: To 7uunderstand 
what stigma reduction approaches 
were implemented by IPs and the 
Implementing 
Partners 
1. Were the stigma reduction approaches you implemented 
uniform?
Prompts: a) within country, b) across different regions/prefectures 
and c) with approaches implemented in other countries (Guinea, 
60
extent to which stakeholders have 
collaborated throughout the 
duration of implementation on any 
changes to approaches
Sierra Leone, Liberia)?
2. How were stigma reduction campaigns developed? Prompts: 
Were they adapted from existing models or custom built? 
Follow up questions: 
- Has there been collaboration on designing and coordinating 
SBCC and other campaigns with other EVD response 
implementers in your country of operation or other countries 
affected by EVD? 
- If so, what was the process? 
- How was the process documented?
3. How were stigma trainings designed? 
Follow up questions: 
- Has there been collaboration on designing and coordinating 
trainings with other GHET stakeholders in your country of 
operation or other countries affected by EVD? 
- If so, what was the process? 
- How was the process documented?
- Who determined what went into stigma trainings? 
- Were there adaptations? If so, how were they managed?
Topic 3: ACCESSIBILITY OF 
CLINICAL SERVICES 
Effectiveness: To understand 
various perspectives of program 
stakeholders about its 
effectiveness in delivering 
outputs related to strengthening 
accessibility of clinical services 
through referral systems
Implementing 
Partners, 
Governments 
(Relevant 
ministries and 
agencies), 
Survivor 
Networks
1. In your opinion, what does a successful referral system look like?
2. Do you think the referral system for EVD survivors was 
successful? Follow up questions: 
- Yes, why? No, why?
- What are the main barriers to access?
- How did the program address these barriers?
61
Efficiency: To understand how 
the program contributed to 
survivor education on access to 
services, including the referral 
system for survivors. To provide 
additional context to quantitative 
data from Facility Assessments, 
Survivor and Provider Surveys to 
ascertain to what extent success 
is attributable or not attributable 
to GHET funding
Implementing 
Partners,
Government 
(Relevant 
Ministries 
and Agencies)
Providers, 
Survivor 
Networks 
1. How does the referral system for survivors work?
Follow up questions: 
- Are staff at health facilities (e.g. physicians and nurses) clear 
on the criteria for referrals?
- In your opinion, how closely are those guidelines followed? 
- In your opinion, does the referral system work equally well for 
all survivors or are there some individuals (or groups) that 
systematically fall out of the scope of referral? If so, please 
explain in as much detail as possible.
2. What is your perception of what survivors think about the 
referral system? Please provide details on what your opinion is based 
on?
3. In your opinion, how educated are survivors in general about 
services available to them and how to access them?
4. Do you believe the program has contributed to education of 
survivors about available services? If so, how and how much?
Follow up question: 
- What training/education programs were offered and to 
whom?
- Who else or what other program(s) contributed to survivor 
education in this regard?
- How does the contribution of this program compare to the 
contributions of these other programs? Please provide details 
and/or evidence to support your response.
5. If you were to do this over, what would you do differently? 
Adaptive management and 
relevance: To understand how 
relevant the service package for 
survivors was from the 
Implementing 
partners, 
Government 
(Relevant 
1. How relevant do you think the service package available to 
survivors was and is to survivor needs? (prompt responses by 
periods):
● during the active ebola outbreak
62
perspective of implementation 
partners and survivor networks 
and to identify lessons learned 
that can inform future 
programming. To provide 
additional context for 
quantitative data collected 
through Facility Assessments, 
Survivor and Provider Surveys to 
ascertain to what extent success 
is or is not attributable to GHET 
funding
Ministries 
and 
Agencies), 
Survivor 
Networks
● post ebola outbreak [use the date of the WHO ebola-free 
declaration for each country as a point of reference- current 
period]
2. What lessons did you learn [can be drawn] from the 
implementation of this component of the program? 
Follow up questions: 
- How were these lessons integrated into implementation? 
What changes (if any) were made?
- What was the process through which these changes were 
made?
- If you were to do it over again, what would you do differently? 
- Would you remove or add anything?
Topic 4: GOVERNANCE AND 
MANAGEMENT IMPROVEMENT 
(national policies and systems)
Effectiveness: To harvest 
outcomes and understand the 
effects that the program had on 
national policies and systems and 
improving governance and 
management as a result of 
capacity building activities of the 
program. To understand how 
National Governments and 
Survivors Networks are 
perceived, including confidence 
in their work from various 
respondents 
Implementing 
Partners, 
Governments 
(relevant 
ministries and 
agencies), 
Survivor 
Networks
1. What changes to national policies and systems relating to ebola 
or IPC took place during the course of the program?
Follow up questions:
- How did these changes affect services available to survivors?
(prompts: if respondents need prompts, use words such as 
improvements, enabling, facilitated)
- Are these changes specifically tailored to survivors? If not, 
were survivor considerations integrated into broader changes 
focused on ebola and/or ipc?
- Have these changes led to improvements? If so, what 
contributed to these improvements? 
2. Do you believe the program has contributed to improvements to 
national policies and systems (governance and management) that 
occurred as a result of capacity building activities of the program? 
(prompts: which enabled/facilitated improvements in survivor 
services?) Please explain in as much detail as possible.
63
Follow up question: 
- Who else or what other program(s) contributed?
- How does the contribution of this program compare to the 
contributions of these other programs? Please provide details 
and/or evidence to support your response.
3. In your opinion, how well known are National Government 
bodies working towards EVD support for survivors and Survivor 
Networks among different stakeholder groups? (Please list) 
Follow up questions:
- Does awareness of a survivor network connect with access 
and use of services? How?
- Are they trusted to represent interests of survivors 
adequately? Alternative: What is the level of confidence 
stakeholders have in the work of the National Government 
and Survivor networks?
- What do you think attracts survivors to or deters them from 
joining the networks?
Efficiency: To understand 
challenges faced by the program 
and identify and document 
lessons learned from the 
perspective of Implementing 
Partners and government 
stakeholders
Implementing 
Partners,
Government 
(relevant 
ministries 
and agencies)
1. What lessons were learned with respect to how programs like 
this one contribute to policy change and challenges associated 
with policy change? Please be specific.
64
Relevance: To understand the 
relevance of capacity building 
components of the program 
aimed at strengthening the 
management and administration 
of survivor services
Implementing 
partners, 
governments 
(relevant 
ministries 
and agencies)
1. Do you think that the trainings offered by the program were an 
efficient way to address capacity gaps in the administration of 
survivor services? Please explain your answer.
Follow-up questions
- What are the capacities [skills and knowledge] that were 
built?
- How did your experience with these trainings affect your 
perspective on what skills are the most critical for 
management and administration of survivor services? Please 
explain and provide examples.
65
Topic 5: LESSONS LEARNED 
RELATED TO PROGRAM 
EFFECTIVENESS
Effectiveness: To understand 
how governance and 
management have improved to 
advance national policies and 
systems and based on the 
experiences of this program, 
what do managers and policy 
decision-makers consider the 
best approaches for behavior 
change
Implementing 
Partners, 
Governments 
(relevant 
ministries 
and 
agencies), 
USAID
1. Based on the experiences of this program, what do you consider 
as the best approaches for behavior change? [Behavior change 
related to survivors accessing health facilities, reducing provider 
stigma of ebola survivors, and increasing government capacity to 
administer survivor programs] Please explain. 
Follow up question:
- What worked and what did not?
- Why or why not?
- What factors influenced effectiveness? 
Topic 6: STAKEHOLDER 
ENGAGEMENT AND FEEDBACK 
LOOPS
Adaptive management and 
relevance: To understand the 
level of stakeholder engagement 
and feedback loops used by the 
program to inform decisions and 
adapt to changing circumstances 
from the perspective of IPs and 
various stakeholder groups.
Implementing 
Partners, 
Governments 
(relevant 
ministries and 
agencies), 
Survivor 
Networks, 
International 
Orgs, USAID
1. To what extent have key program stakeholders been engaged to 
inform programmatic decisions (who was engaged, how, and 
what was their role)?
Follow up questions:
- What sources of feedback were sought? (Prompts: for 
example, were/and if so how/ stakeholders involved to 
determine satisfaction by survivors and determining 
distribution of services across levels of access?) 
- How was this done? How often? 
- What communication channels were used?
- How were subsequent decisions communicated back to 
stakeholders? 
2. In your opinion, how satisfied are stakeholder groups with level 
and quality of their engagement on programmatic decisions?
Follow up questions:
- Is satisfaction level uniform across all stakeholder groups or 
are some stakeholders more satisfied than others? Please 
explain your perspective and provide details. 
66
Topic 7: BARRIERS TO SERVICE 
DELIVERY
Efficiency: To understand 
barriers to service delivery and 
how program partners addressed 
them, as well as to identify 
lessons learned that can inform 
future programming
Implementing 
Partners, 
Government 
(relevant 
Ministries 
and 
Agencies), 
Survivor 
Networks
1. What have been the barriers to appropriate service delivery for 
EVD survivors?
Follow up questions:
- How did the program address these barriers? 
- What was the process that was followed? 
- How was that process documented?
What are some lessons learned from implementation about 
effective ways of addressing barriers to service delivery in 
contexts such as this one?
Topic 8: GENERAL PROGRAM 
PERFORMANCE (PRODUCTIVITY) 
Efficiency: To understand to 
what extent the program has: 1. 
Produced outputs on time; 2. 
Met targets for outputs; and the 
process through which these 
were determined and what 
accounts for any variance
Implementing 
Partners, 
USAID
1. To what extent have program implementation plans been 
implemented as planned? 
2. To what extent have you met your program targets? 
Follow up question: 
- What accounts for the variance? 
3. Tell me about factors that influenced your team/program’s 
performance as it relates to meeting targets on time? 
Follow up questions: 
- Why was this so? What contributed to this?
- How were the program targets set?
- What factors were taken into consideration when setting 
output targets?
- What lessons did you learn from the process of setting the 
targets and managing towards achievement on this program?
67
JSI Research & Training Institute, Inc. Endline Survivor Survey Questions
Question Response 
Code
Response Option
Healthcare
Do you feel that your access to healthcare for this problem was adequate?5 1 Yes
0 No
Was the quality of healthcare that you received satisfactory? 6 1 Yes
0 No
In the past two years (since the end of the Ebola time), do you feel that care 
for this problem has: 7
1 Got much better
2 Got a little better
3 Stayed the same
4 Got a little worse
5 Got much worse
98 Don’t know
Stigma (Liberia)
The doctor/nurse refused to treat you or refused to do relevant tests
1 Yes
0 No
You had to wait longer for the doctor/nurse to treat you than other patients 1 Yes
 
5 In Sierra Leone, asked per health problem identified. In Guinea and Liberia, asked about health problems in general.
6 In Sierra Leone, asked per health problem identified. In Guinea and Liberia, asked about health problems in general.
7 In Sierra Leone, asked per health problem identified. In Guinea and Liberia, asked about health problems in general.
68
0 No
The doctor/nurse appeared nervous or uncomfortable treating you
1 Yes
0 No
The doctor/nurse gossiped about your survivor status 1 Yes
0 No
You received less care/attention than other patients 1 Yes
0 No
Stigma (Guinea)
Medical personnel refused to treat you or do tests for you because of your 
Ebola status.
1 Yes
0 No
The wait time to be seen by a medical staff was longer for you than for other 
patients.
1 Yes
0 No
Medical staff took excessive precautions when examining you. 1 Yes
0 No
Medical staff gossiped about your Ebola status. 1 Yes
0 No
You received less attention / care than the other patients. 1 Yes
0 No
Stigma (Sierra Leone)
69
You were denied health services because of my Ebola survivor status 1 Yes
0 No 
99 Don't know
Healthcare workers talked badly about you because of your Ebola survivor 
status
1 Yes
0 No 
99 Don't know
A health worker disclosed your Ebola survivor status without your permission 1 Yes
0 No 
99 Don't know
Stigma (All)
Do you think that treatment of Ebola survivors by doctors and nurses has 
gotten better, gotten worse, or stayed the same since just after the Ebola 
time?
1 Better
2 Worse
3 Stayed the same
EVD Bodies (Liberia and Guinea)
I feel very confident about the survivor Network’s ability to get better support 
for people like me.
1 Agree 
2 
Neither agree nor 
disagree
3 Disagree
99 No response
1 Agree 
70
I feel very confident about the [National EVD Government Body’s]8 ability to 
get better support for people like me. 2 
Neither agree nor 
disagree
3 Disagree
99 No response
EVD Body (Sierra Leone)
How confident are you in the survivor network’s ability to support/address 
your needs and represent you?
1 Very confident
2 
Somewhat 
confident
3 
Not at all 
confident
777 Skip
 
8 Name varied by country.
71
Annex C. Data Quality Verification summary
The Global Health Ebola Team (GHET) supplied a list of indicators related to their Pillar II portfolio for 
which they desired data quality verification (DQV), including data originating from health facilities 
and/or health management information systems (HMIS), training data, and data derived from other 
sources, such as project surveys. The purpose of the verification activity was to guide GHET on public 
use of these indicator data. The BalanceD-MERL team completed a verification of 55 indicators in total, 
among which were 14 training and ‘other’ indicators9 relevant to the GHET survivor program activities: 
seven in Guinea, four in Liberia, and three in Sierra Leone. 
For each indicator, the verification included an extraction of data from implementing partner reports, 
a sampling of up to two data reporting periods, a request for lowest level “raw” data from the 
implementing partner, and then comparison of data reported by the implementing partner against 
the lowest level “raw” data available that was re-calculated by the BalanceD-MERL consortium. Issues 
with the data and analysis were recorded, including those related to integrity, reliability, and 
timeliness – the three USAID quality standards that this Data Quality Verification activity was best 
positioned to assess. Each training or other indicator verified was given a color code based on this 
exercise to guide GHET on public use of these indicator data. 
● Green = Safe for public use. No or very low error rate overall (less than 5%) and sufficient 
ability to verify data.
● Yellow = Cautioned for public use. Low to moderate margins of error overall (5% or greater, 
but less than 8%) and/or minor challenges with verifying data. Consider potential limitations 
before posting.
● Orange = Cautioned for public use. Moderate margins of error (between 8% and 12%), notable 
data reliability or integrity issues, and/or significant challenges with verifying data.
● Red = Do not use publicly. Large margins of error (12% or greater) and/or inability to verify 
data.
As seen in Table E.1, of the 14 indicators specific to the GHET survivor program activities, six were 
given a green designation, three were given a yellow designation, and five were given an orange 
designation. Most indicators designated green were recommended for public use without caveats and 
may even be safe for scientific use. Indicators designated orange were not recommended for scientific 
purposes, and their public use would require major caveats. The data associated with these indicators 
were found to have substantial integrity and reliability issues and, as such, have very limited value to 
support a narrative around GHET’s contribution. 
 
9 The BalanceD-MERL team did not verify any HMIS-derived indicators that were specific to the GHET survivor 
program activities.
72
Table E.1: Summary of DQV Findings
Country Frequency/Type Indicator
Code
Guinea Quarterly/ Training # of healthcare workers trained or mentored to provide appropriate and 
safe clinical services needed by Ebola Survivors Green
Guinea Quarterly/ Training # of health workers trained or mentored in stigma reduction Green
Guinea Quarterly/ Training # of community members trained or mentored in stigma reduction Green
Guinea Quarterly/ Other # of visits for secondary and tertiary clinical care services at supported 
secondary and tertiary healthcare facilities
Yellow
Guinea Quarterly/ Other # of visits by Ebola survivors for secondary and tertiary clinical care 
services at supported secondary and tertiary healthcare facilities 
Yellow
Guinea Quarterly/ Other # of referrals issued at primary care sites for secondary and tertiary care 
services at secondary and tertiary healthcare facilities 
Yellow
Guinea Twice during project/ 
Other
Proportion of Ebola survivors reporting the perception of health provider 
stigma 
Green
Liberia Monthly/ Training # of healthcare workers trained or mentored to provide appropriate and 
safe clinical services needed by Ebola survivors
Orange
Liberia Quarterly/ Other # of visits for secondary and tertiary clinical care services at supported 
secondary and tertiary healthcare facilities
Orange
Liberia Quarterly/ Other # of visits by Ebola survivors for secondary and tertiary clinical care 
services at supported secondary and tertiary healthcare facilities
Orange
Liberia Life of project 
indicator/ Other
Proportion of Ebola survivors reporting the perception of health provider 
stigma
Green
Sierra Leone Quarterly/ Other # of referral visits actualized by Ebola survivors for specialized clinical care 
services at secondary and tertiary healthcare facilities
Orange
73
Sierra Leone Twice during 
project/Other
Proportion of Ebola survivors reporting stigma during their last visit to a 
healthcare facility
Green
Sierra Leone Quarterly/ Other # of visits by Ebola survivors for primary clinical care services at supported 
primary healthcare facilities
Orange
74
Annex D. BalanceD-MERL Monitoring of EVD Survivor Surveys 
1. PURPOSE & OBJECTIVES 
As part of its own program monitoring and evaluation activities, JSI R&T conducted program baseline 
and endline quantitative interviews with Ebola virus disease (EVD) survivors in Sierra Leone, Liberia, 
and Guinea. The BalanceD-MERL team leveraged the JSI R&T data from both points in time, including 
adding a module to the JSI R&T endline survey, for its performance evaluation of the GHET survivor 
programming. To this end, the BalanceD-MERL team sought to understand (1) whether the EVD 
survivor data as recorded by JSI R&T was a correct representation of the data and (2) what implication 
this had on the usefulness of the data for the performance evaluation.
The BalanceD-MERL team employed a methodology in which BalanceD-MERL enumerators 
accompanied JSI R&T enumerators and independently recorded a sample of EVD survivor responses 
separately. This established a framework in which a subsample of respondents had two 
measurements for every question in the survey, collected at the same time. The dual measurements 
theoretically were expected to be equal. This reliability assessment broadly involves analyzing the 
inconsistency or measurement error, if any, between the data recorded by the BalanceD-MERL 
enumerators versus the JSI R&T enumerators for every subject at endline.
2. BACKGROUND
Survey measurement errors occur under the assumption that the variable being measured has a true 
value that is constant (Kane, 2008). Such errors derive from two types of sources. Sampling errors 
occur when a subset of the population is measured rather than a complete census. Sampling errors 
may be minimized through the development of a comprehensive sampling frame, random sampling, 
and the drawing of larger sample sizes. Non-sampling errors are any other type of measurement error 
that is unrelated to sampling, such as those occurring from the process of data collection (Banda, 
2003).
There are many potential causes of non-sampling errors. For example, a poorly designed instrument, 
including confusing questions or vague terms, may result in a respondent providing an inaccurate 
answer (Marsden and Wright, 2010). Such instrument errors can be minimized through rigorous 
survey design. Interviewers may cause measurement error by leading or suggesting answers to a 
respondent, fabricating data, or making transcription errors (Marsden and Wright, 2010). Robust 
enumerator training may staunch such interviewer errors. However, irrespective of interviewer 
performance, respondents can create measurement error by incorrectly reporting their response to 
the interviewer (Marsden and Wright, 2010). Techniques to promote accurate reporting from 
respondents include rapport building and assurances of respondent confidentiality.
The present analysis concerns itself with variations in measurement that occur due to non-sampling 
errors, particularly those caused by the interviewer. This annex includes an overview of the 
methodology and the analytical framework, followed by detailed procedures and findings for each of 
the three countries where data were collected. Finally, a conclusion section summarizes findings and 
their implications for use of the JSI R&T data.
75
3. METHODOLOGY OVERVIEW 
Considered highly accurate for estimating measurement error, a gold-standard re-interview occurs 
when a second interview is conducted in similar conditions using questions whose answers should not 
change over time. These data allow the estimation of simple response variance and the reliability ratio 
between the two interviews, creating an estimate of measurement error (Biemer and Lyberg, 2003).
For the present study, the gold standard re-interview was deemed too taxing on the Ebola virus 
disease survivors. Hence the BalanceD-MERL team adapted the gold standard re-interview process. In 
this study, the BalanceD-MERL enumerators accompanied a random sample of JSI R&T survivor 
surveys, observing interviews, and recording survivor responses separately. In Sierra Leone and 
Liberia, Innovations for Poverty Action (IPA) monitored the JSI R&T surveys while in Guinea, the 
Institute for Development Impact (I4DI) monitored the JSI R&T surveys. 
This adapted re-interview methodology involves the following assumptions:
a) Errors come from the following sources:
1. The two measurements are made using the same instrument and at the same time. This would 
imply that any non-sampling errors that arise due to the instrument and/or respondent 
cognitive processes to arrive at the answer would be common to both measurements.
2. Hence, any differences between recorded data can be ascribed to the enumerator 
transcription process (that is, differences in the way the answers are comprehended and 
reported by the two enumerators).
b) The response(s) as recorded by the BalanceD-MERL enumerator is a stand-in for the “truth”
1. Accompaniment of JSI R&T enumerators without “blinding” could have caused the 
respondents and JSI R&T enumerators to become conscious of being observed or evaluated. 
Therefore, the performance and responses observed may not be representative of a typical 
‘unaccompanied’ interview.
2. In general, interviewers may perform better than usual in an observed interview, thus. 
underestimating the true error rate.
3. A well trained BalanceD-MERL enumerator is still capable of making data entry mistakes. 
Therefore, it is acknowledged that the exact source of discrepancies in the two measurements 
remain unknown.
c) The adapted re-interview methodology offers information on how often transcription mistakes are 
likely to occur on average. The procedure allows the measurement of how often both the enumerators 
agree to the response, rather than measuring how often the item is entered correctly. 
76
4. ANALYSIS FRAMEWORK 
Based on the characteristics of this re-interview framework and its methodological limitations, the 
BalanceD-MERL team concluded that the best estimators to answer questions about usefulness of the 
data would be:
a. Simple agreement/error rates with confidence intervals for categorical questions 
b. Reliability ratios for continuous and ordinal questions, as calculated using intra-class correlation 
coefficients
Categorical Questions
Simple agreement rate is related to the gross differences for measurements of binary data in a 
survey—the difference in response between the two data points for each question averaged over the 
element in the population. For example, a simple agreement rate of 0.75 would mean that in 75% of 
the observations there was agreement in the measurement by the two enumerators. Simple error 
rate is the inverse of the simple agreement rate: 1 – 0.75 = 0.25 is the simple error rate.
The simple agreement rate in the sample of dual observations is not the same as the distribution of 
the true error rate in the population. To obtain the true agreement rate, a survey of the entire 
population (a census) would be necessary. Hence, the simple agreement rate estimates are reported 
along with their confidence intervals reported at the 95% confidence level. Confidence intervals (CIs) 
are the range of possible values within which an estimated value of a population parameter could fall. 
For example, consider a simple agreement rate of 0.96 with a confidence interval of (0.87 - 0.99). This 
means that for the monitored set of surveys, 96% of the time the two measurements agree (an error 
rate of 4%). Confidence intervals are always reported with the confidence level - the desired ''degree 
of confidence." In error measurement literature, a 95% confidence level is used most often. Using the 
above example, if repeated measurements were made of the same variable, 95% of the time, the true 
agreement rate of the population will lie between 0.87 and 0.99.
The confidence intervals also will account for the fact that some questions were observed more than 
others. Questions with narrower CIs will be better estimates of the true values than ones with wider 
CIs. Consider the agreement rates and CIs for two hypothetical variables: 1 (0.48 - 1) for question A 
and 0.96 (0.87 - 0.99) for Question B. Even though question A has a higher agreement rate, it has a 
much wider confidence interval. This suggests that 0.96 is a better estimate of the agreement rate for 
Question B than 1 is for Question A.
Ordinal and Continuous Questions
Simple agreement/error rates treat all errors the same way. This is acceptable for categorical 
questions where one category error is similar to another. But this is not the case for ordinal and 
continuous questions where an error due to a difference of 1 is very different from an error by a 
difference of 2 or 4 or 20.
77
To account for this, the reliability ratio is used. The reliability ratio gives a quantitative measure of how 
much of the variance in the observations is true variance instead of variance from measurement error. 
For example, a reliability ratio of 0.8 implies the data are capturing 80% of the true variance (with 20% 
coming from error); the JSI R&T enumerators could be reported to be 80% reliable. As with agreement 
rates, reliability ratios are also reported with their respective confidence intervals.
Acceptable levels of agreement and reliability ratios
Since there is no definitive measure for an acceptable level of agreement, this report will utilize a 
measure of 75-90% as a benchmark for a reliable question for simple agreement rates, as suggested 
by Bajpai and Chaturvedi (2015) and their comparison of different measures. For the reliability ratio 
classification, this report will utilize a similar benchmark as based on existing literature (Bajpai & 
Chaturvedi, 2015). Below is the benchmarking used in evaluating results:
Simple Agreement Rates
Level Classification Color Code
Less than 0.75 Poor Gray
0.75-0.9 Moderate Yellow
Greater than 0.9 Excellent Green
Reliability Ratio
Level Classification Color code
Less than 0.5 Poor Gray
0.5 – 0.75 Moderate reliability Beige
0.75 – 0.9 Good reliability Yellow
78
Greater than 0.9 Excellent reliability Green
Simple agreement rates and reliability ratios are evaluated in conjunction with their respective 
confidence intervals. Consider a reliability ratio of 0.80 with a CI of (0.75 – 0.85). The width of the CI 
is calculated as the difference between the upper limit and lower limit (0.85-0.75 = 10). In this 
example, the estimate could be classified as ‘good’, per the table above, because the entire range of 
estimates fall within the Good reliability range of 0.75 to 0.9. However, a reliability ratio of 0.90 with 
a CI of (0.50- 0.95) would be difficult to classify because the range spans across multiple classification 
categories. In this example, it is impossible to state whether the value would fall in the poor, 
moderate, or good category.
Therefore, confidence interval widths are also considered in the assessment of the simple agreement
rates and reliability ratios. Estimates that exceed the maximum confidence interval width were not 
evaluated for this report. Such estimates were not color coded at all.
Confidence interval widths
Width Classification
21- 30 Moderate, with 30 being the maximum width
11- 20 Good
0- 10 Excellent
Survey questions analyzed
Across the three countries, there were differences in the overall surveys designed by JSI R&T, with the 
biggest differences occurring within the survey fielded in Sierra Leone compared to those fielded in 
Liberia and Guinea. The BalanceD-MERL team inserted questions into each of the three instruments 
to address the research objectives of its performance evaluation, and these additional questions also 
varied somewhat between countries, due to the existing structural differences in the JSI R&T
instruments.
The survey items analyzed for this reliability assessment are the core measures used to inform the 
performance evaluation of the GHET survivor program. Given the differences in survey instruments, 
as well as varying field procedures, the results for each country are presented separately in this report. 
However, the analysis framework was the same in each country regardless of survey design.
79
5. COUNTRY SPECIFIC ANALYSIS
Sierra Leone
1) Field Process
In Sierra Leone, there were four BalanceD-MERL monitors (from IPA) for this activity, all of whom with 
considerable experience both conducting and monitoring surveys. As well as standard IPA training, 
the four monitors received the full three-day enumerator training run by JSI R&T and Focus-1000, the 
organization that JSI R&T contracted to carry out its endline surveys. On the final day of training, 
several EVD survivors were invited to the training location to enable enumerators to fill out test 
entries. Given the training coordination between the JSI R&T enumerators and IPA monitors, 
subsequent discrepancies in the data are unlikely to be due to training methods.
JSI R&T informed IPA of the field plan of Focus1000. IPA then designed a monitoring plan that would 
allow observation of 75 surveys in USAID’s five priority districts. The field plan was organized such that 
IPA could observe all 12 enumerators working in these districts. IPA aimed to monitor surveys 
proportionately by district. However, in some cases this was not possible. For example, Focus-1000 
changed the field plans to reach its target number of respondents, and this did not give IPA sufficient 
time to deploy monitors to new locations. In Bombali district, Focus1000’s data collection had nearly 
concluded before IPA was made aware that data collection had started earlier than planned. Focus￾1000’s enumerators delayed the final two interviews in Bombali, allowing IPA monitors to observe 
those interviews. 
Initially, IPA had planned to monitor a sample of 10% of JSI R&T’s surveys in Sierra Leone. JSI R&T
planned to conduct 742 surveys in Sierra Leone, including 476 in USAID’s five priority districts. 
Ultimately, JSI R&T conducted 751 surveys nationwide and 480 in USAID’s priority districts. Of these, 
IPA monitored 70 JSI R&T surveys (14.5%) in the five priority districts, which equated to monitoring 
9.3% of JSI R&T’s nationwide total (Table F.1).
Table F.1: Surveys monitored by district- Sierra Leone
District # of JSI R&T surveys 
conducted
# of surveys 
monitored
% of JSI R&T surveys that were 
monitored
Bombali 40 2 5%
Port Loko 200 27 14%
80
Tonkolili 39 8 21%
Western Area 
Rural
101 24 24%
Western Area 
Urban
100 9 9%
Total 480 70 15%
Both IPA monitors and Focus-1000 collected data electronically on tablets using SurveyCTO. Before 
the launch, JSI R&T sent IPA their final coded tool to ensure there were no differences in coding that 
would cause discrepancies. To protect the anonymity of respondents, names or other identifying 
information were not collected by monitors. The surveys were linked using a unique ID assigned by JSI 
R&T. In JSI R&T’s full dataset of the 480 respondents, 95% of cases were unique by this ID but JSI R&T’s 
data contained eleven pairs of duplicates (22 surveys), 3 pairs (six surveys) of which were among the 
70 surveys monitored by IPA. At the time of this analysis, BalanceD-MERL was unable to resolve the 
issue with the IDs, which was likely due to enumerator error, and therefore the 3 surveys in question 
were excluded from the BalanceD-MERL data file. Thus, in Sierra Leone there were 67 surveys 
available for reliability comparison.
2) Reliability Findings
The BalanceD-MERL team compared answers recorded by monitors and enumerators on key 
variables. Survey respondents were first asked which health problems they had experienced in the 
past year, and for each health problem there were four follow-up questions. For three of these items 
(S.1, S.3, and S.4), Table 2 shows the percentage of complete responses for which both the Focus-1000 
and IPA enumerator recorded the same response (i.e., the simple agreement rates). The confidence 
intervals appear in parentheses. For example, on item S.2 among four surveys in which respondents 
reported abdominal problems, there was a 75% agreement. This means that the JSI R&T and IPA 
enumerators entered the same value on three of the four relevant surveys. For some health problems 
appearing in Table 2, there are very low agreement values because the sample is small. For example, 
some cells show an agreement coefficient of 0% in cases where n=1.
In Tale F.2, there are six cells for which the number of observations and the confidence interval width 
allow for the assessment of data quality. In each of these six cells, the data have been classified as 
excellent, per the benchmarks described above.
81
Table F.2: Sierra Leone Agreement Rates (Categorical Questions): Health Problems and Other
Health Problems
Respondents were 
asked these questions 
about each health 
problem they 
reported
S.1 Did you 
seek care for 
your [health 
problem]?
(yes/no)
S.3 Do you feel that 
your access to care for 
your [health problem] 
was adequate?
(yes/no)
S.4 Was the quality of 
healthcare that you 
received for your 
[health problem] 
satisfactory?
(yes/no)
Number of 
observations
Abdominal Problems 0.75
(0.19 - 0.99)
1.00
(0.40 - 1)
1.00
(0.40 - 1)
4 
Emotional problems 1.00
(0.03 - 1)
1.00
(0.03 - 1)
1.00
(0.03 - 1)
1 
Eye problems 0.75
(0.51 - 0.91)
 20
Fever 0.80
(0.44 - 0.97)
1.00
(0.69 - 1)
0.90
(0.55 - 0.99)
10
Headache 0.96
(0.79 - 0.99)
0.92
(0.73 - 0.99)
1.00
(0.86 - 1)
24
Hearing related 
problems
- - - 0 
Joint problems 0.74
(0.57 - 0.88)
0.91
(0.75 - 0.98)
0.97
(0.84 - 1)
35
82
Numbness, dizziness, 
or pins and needles
0.0
(0 - 0.98)
1.00
(0.03 -1)
1.00
(0.03 - 1)
1 
Other problems 0.60
(0.26 -0.89)
0.90
(0.55 - 0.99)
0.90
(0.55 - 1)
10
Pain 0.91
(0.71 - 0.98)
0.85
(0.62 - 0.97)
0.90
(0.68 - 0.99)
22
Problems during 
pregnancy
- - - 0 
Reproductive health 
problems
1.00
(0.03 - 1)
1.00
(0.03 -1)
1.00
(0.03 - 1)
 
1 
Sexual health 
problems
1.00
(0.03 - 1)
0.0
(0 - 0.975)
1.00
(0.03 - 1)
1 
Table F.3 below presents simple agreement rates for questions relating to stigma. Like the results in 
Table F.2, the average agreement rates for individual questions with a reasonably large number of 
observations (n ≥ 10) and acceptable confidence interval widths ranges from 90% to 97%.
Table F.3: Sierra Leone Simple Agreement Rates (Categorical Questions): Stigma
Question: In the last year, have you faced these issues 
(Yes/No)
 Number of 
observations
83
People talked badly about you because of your Ebola 
survivor status
0.90
(0.8- 0.96)
67
Has there been any improvement since then? 0.80
(0.59 - 0.93)
25
Someone else disclosed your Ebola survivor status without 
your permission
0.94
(0.8 - 0.98)
67
Has there been any improvement since then? 0.83
(0.59 - 0.96)
18
You were verbally insulted, harassed and/or threatened 
because of your Ebola survivor status
0.96
(0.87 - 0.99)
67
Has there been any improvement since then? 1.00
(0.66 - 1)
9 
You were physically assaulted because of your Ebola 
survivor status
0.96
(0.87 - 0.99)
67
Has there been any improvement since then? 0.40
(0.05 - 0.85)
5 
You felt that people did not wanted to sit next to you, for 
example on public transport, at church or mosque, or in a 
health facility because of your Ebola survivor status
0.90
(0.79-0.96)
67
84
Has there been any improvement since then? 1.00
(0.48 - 1)
5 
You were denied health services because of my Ebola 
survivor status
96
(0.87 - 0.99)
67
Has there been any improvement since then? 1.00
(0.48 -1)
5 
Healthcare workers talked badly about you because of your 
Ebola survivor status
0.96
(0.87 - 0.99)
67
Has there been any improvement since then? 0.67
(0.09 - 0.99)
3 
A health worker disclosed your Ebola survivor status 
without your permission
0.93
(0.83 - 0.98)
67
Has there been any improvement since then? 0.75
(0.19 - 0.99)
4 
Have you ever been treated badly by health staff because 
of your status as an Ebola Survivor (for example, whether 
you have been refused service, or made to wait for longer 
than other patients)?
0.97
(0.90 - 1)
67
Table F.4 presents the reliability ratios for answers recorded to question S.7—an item that was 
measured on a 5-point ordinal scale (1. Got much better, 2. Got a little better, 3. Stayed the same, 4. 
Got a little worse, 5. Got much worse). Among the four items with sufficient observations and suitable 
85
confidence interval widths, two have an ‘excellent’ reliability ratio and the other two have a ‘good’ 
reliability ratio. This could be because some judgment is necessary to code these answers. If a 
respondent gives an answer that could reasonably be interpreted as either “Got much better”, or “Got 
a little better”, then the monitor and enumerator might code different answers. In such situations, 
the enumerator ideally should have probed further to obtain a conclusive answer, but this may not 
have always happened. Almost all the observed differences between the monitor and enumerator 
coding on this item are of one degree, as in the example above. Further, the discrepancies are not 
systematic. They did not skew in one direction.
86
Table F.4: Sierra Leone Reliability Ratios (Ordinal Questions): Health Problems and Other
Health Problems
Respondents were asked these questions 
about each health problem they reported
S.7 In the past two years (since the end of 
the Ebola time), do you feel that care for 
[health problem] has got 
Number of 
observations
Abdominal Problems 1.0 4 
Emotional problems - 1 
Eye problems 0.89
(0.72 - 0.96)
20
Fever 0.95
(0.79 - 0.99)
10
Headache 0.80
(0.55 - 0.92)
24
Hearing related problems - 0 
Joint problems 0.89
(0.78 - 0.94)
35
Numbness, dizziness, or pins and needles - 1 
Other problems 0.98 10
87
(0.93 – 1)
Pain 0.79
(0.5 - 0.91)
22
Problems during pregnancy - 0 
Reproductive health problems - 1 
Sexual health problems - 
 
1 
Note: For questions with 0 or 1 observation the reliability ratio cannot be calculated
3) Other quality findings
In Sierra Leone, the IPA monitors noted issues with the administration of the survivor surveys including 
the following issues, which could affect the accuracy and precision of the data:
● Leading questions: During questions about whether services for various health problems had 
improved, some JSI R&T enumerators read out only the positive options: “Got much better, Got a little 
better”, but not the neutral or negative options “Stayed the same, got a little worse, or got much 
worse”. This might guide respondents towards selecting a positive response. This problem was 
reported in less than 10% of surveys monitored.
● Translation: The tool was coded into the mobile devices in English, with each JSI R&T enumerator 
responsible for translating the survey into Krio (Sierra Leone’s lingua franca most commonly spoken 
in the capital) or another local language spoken by the respondent, at the time of the interview. IPA 
monitors, who are all fluent in the local language of the areas they were monitoring, reported that in 
some cases the translations varied between different surveyors.
● Probing: IPA monitors reported that some enumerators did not take sufficient time to explain some 
questions, or probe when respondents gave answers that might only have been partially complete.
Finally, as with all surveys, social desirability bias can influence results. While it is difficult to determine 
whether social desirability bias affected the data provided by EVD survivors in Sierra Leone, very few 
survivors indicated that they had experienced emotional, sexual, or reproductive problems. Indeed, 
88
in JSI R&T’s full survey of 459 respondents (in 5 districts), only 3.5%, 2.6%, and 2.1% respectively 
reported experiencing these problems.
Liberia
1) Field Process
There were five BalanceD-MERL monitors (from IPA) in Liberia, all with extensive experience both 
conducting and monitoring surveys. As well as standard IPA training, the monitors received the full 
four-day enumerator training run by JSI R&T. On the fifth day of training the IPA team participated in 
a field test and debrief session with the JSI R&T team. Given the training coordination between the JSI 
R&T enumerators and IPA monitors, subsequent discrepancies in the data are unlikely to be due to 
training methods.
The number of interviews to be conducted in each Liberian county was pre-determined based on 
targets that JSI Research & Training Institute, Inc. had established in their field plan. IPA aimed to 
monitor 30% of the surveys conducted in each of the four counties of interest. IPA monitors randomly 
selected interviews to be observed by writing the names of all the respondents due to be interviewed 
on a given day on small pieces of paper which were folded and placed in an envelope. A team member 
then blindly picked names from the envelope. If a respondent did not consent to have their interview 
monitored another name would have been randomly selected; however no refusals were recorded, 
and no replacements were needed.
Due to the difficulty associated with conducting unannounced visits, the IPA teams accompanied the 
JSI R&T teams for the entire duration of field work, which lasted 20 days in part due to a survey boycott 
mounted by the survivor network in Lofa and parts of Montserrado. The IPA team was able to monitor
surveys conducted by all 11 of JSI R&T’s enumerators. JSI R&T Liberia had a sample size of 213 and IPA 
observed 69 surveys (Table F.5). This equated to 32% of the national sample. While surveys were 
monitored in each of the four counties of interest, the preponderance of monitored surveys were in 
Montserrado, in proportion to the JSI R&T survey sample.
89
Table F.5: Surveys monitored by county - Liberia
County # of JSI R&T surveys 
conducted
# of surveys 
monitored
% of JSI R&T surveys that were 
monitored
Bong 14 3 22%
Lofa 26 8 31%
Margibi 36 11 31%
Montserrado 137 47 34%
Total 213 69 32%
Both IPA monitors and JSI R&T enumerators collected data electronically on tablets. The IPA monitors 
used SurveyCTO and the JSI R&T enumerators used ONA. The difference in platforms used for data 
collection had no impact on findings as JSI Research & Training Institute, Inc.sent IPA their final coded 
tool prior to launch ensuring that both teams had the same tool. To protect the anonymity of 
respondents, no names or other identifying information were collected by monitors. The surveys were 
linked using a unique ID assigned JSI R&T. In JSI R&T’s full dataset of the 213 respondents, all cases 
were unique by this ID.
2) Reliability Findings
The BalanceD-MERL team compared answers recorded by monitors and enumerators on key 
variables. Tables F.6 and F.7 present the simple agreement rate and confidence intervals for a variety 
of categorical questions asked of survivors in Liberia. Table F.8 presents reliability ratios and 
confidence intervals for select ordinal questions. Overall, the simple agreement rates and reliability 
ratios were deemed excellent. In almost all instances the answers recorded by the IPA monitors 
matched those recorded by the JSI R&T enumerators.
90
Table F.6: Liberia Simple Agreement Rates (Categorical Questions): Health Problems and Other
Question (Yes/No) Number of 
Observations
When you had this problem, did you go to a clinic or health 
facility to get treatment?
0.98
(0.90 – 1)
52
Was the quality of healthcare that you received satisfactory? 1.00
(0.93 – 1)
52
Do you feel that your access to healthcare for this problem was 
adequate?
0.92
(0.81 – 0.98)
52
Have you heard of any group which represents survivor 
interests?
0.97
(0.90 – 1)
69
Have you heard of a group called the Ebola Survivors Network? 1.00
(0.95 – 1)
69
Are you a member of the survivor network? 0.95
(0.87 – 1)
63
Have you heard of a group within the Ministry of Health called 
the National Ebola Survivors Secretariat?
0.99
(0.92 – 1)
69
Have you ever received information about services that are 
available for Ebola survivors?
0.97 69
91
(0.90 – 1)
Table F.7: Liberia Simple Agreement Rates (Categorical Questions): Stigma
Question: In the past 12 months, has any of the following things 
happen to you at a health facility because you were an Ebola 
survivor? (Yes/No)
 Number of 
Observations
The doctor/nurse refused to treat you or refused to do relevant 
tests
0.97
(0.90 – 1)
69
You had to wait longer for the doctor/nurse to treat you than 
other patients
1.00
(0.95 – 1)
69
The doctor /nurse appeared nervous or uncomfortable treating 
you
0.97
(0.90 – 1)
69
The doctor/nurse gossiped about your survivor status 1.00
(0.95 – 1)
69
You received less care/attention than other patients 0.99
(0.92 - 1)
69
Have you ever avoided or delayed seeking healthcare treatment 
because you were afraid of the doctor/nurse’s attitudes 
towards you as an Ebola Survivor?
0.99
(0.92 - 1)
69
92
Table F.8: Liberia Reliability Ratios (Ordinal Questions): Health Problems and Other
Question (Agree, neither agree nor disagree, disagree) Number of 
observations
I feel very confident about the survivor network’s ability to get 
better support for people like me
1.00
(0.94 – 1)
63
I am happy with the work that the survivor network is doing 0.97
(0.89 - 1)
63
I feel very confident about the secretariat’s ability to get better 
support for people like me
0.97
(0.83 – 1)
32
I am happy with the work that the secretariat is doing 0.94
(0.80 – 1)
32
Question (gotten better, gotten worse, or stayed the same) 
Do you think that treatment of Ebola survivors by doctors and 
nurses has gotten better, gotten worse, or stayed the same 
since just after the Ebola time?
0.98
(0.92 – 1)
69
Question (got much better, got a little better, stayed the same
got a little worse, got much worse)
 
In the past two years (since the end of the Ebola time), do you 
feel that care for this problem has
0.96
(0.93-0.98)
52
93
3) Other quality findings
In Liberia, monitors noted issues with the administration of survivor surveys during the field testing. 
For example, it was noted that some enumerators were paraphrasing questions and not reading the 
options correctly to respondents. Monitors also observed that in some survey sections that required 
explanation the JSI R&T enumerators were not explaining as prompted. These issues were addressed 
during the re-training session conducted by JSI R&T and were not reported again during final data 
collection.
The tool was coded and administered in English, with each enumerator responsible for explaining the 
questions in Liberian English if the respondent did not understand the question. This situation was 
anticipated during the training and standard explanations in colloquial were given for all enumerators 
to adhere to. Monitors reported that JSI R&T enumerators adhered to these explanations uniformly.
Guinea
1) Methodology
There were two BalanceD-MERL monitors (from I4DI) in Guinea, with experience conducting and 
monitoring surveys. The monitors received enumerator training run by I4DI. Given that instruments, 
training materials, and the field plan were provided to I4DI only after the JSI R&T enumerator 
training had been completed, BalanceD-MERL monitors were unable to participate in JSI R&T
provided training for enumerators. Given the training coordination issue, subsequent discrepancies 
in the data are possibly due to inconsistent training methods.
The number of interviews to be conducted in Guinea was pre-determined based on targets that JSI 
R&T had established in their field plan. I4DI aimed to monitor 10% of administered survivor surveys, 
which took longer given JSI R&T’s difficulties in coordinating survivors. I4DI extended collection 
monitoring times in Conakry to accommodate for this delay on the ground. I4DI monitors paired up
JSI R&T enumerators to observe interviews. If a respondent consented to have their interview 
monitored, the I4DI monitor would join the interview and record responses separately, noting the 
shared identification code. In total, 11% of the JSI R&T surveys were monitored, all from the Conakry 
region (Table F.9).
94
Table F.9: Surveys monitored by region - Guinea
Region Surveys conducted Surveys 
Monitored
% of JSI R&T surveys monitored
N’Nzérékoré 181 0 0 
Conakry 107 58 54 
Kindia 160 0 0 
Faranah 24 0 0 
Boké 17 0 0 
Kankan 42 0 0 
Total 531 58 11
95
2) Reliability Findings
The BalanceD-MERL team compared answers recorded by monitors and enumerators on key 
variables. Tables F.10 and F.11 present the simple agreement rate and confidence intervals for a 
variety of categorical questions asked of survivors in Guinea. Table F.12 presents reliability ratios and 
confidence intervals for select ordinal questions. Overall, most of the simple agreement rates were 
deemed excellent. Due to wide confidence intervals, only one reliability ratio was assessed against the 
benchmarks. It, too, showed excellent reliability.
Table F.10: Guinea Simple Agreement Rates (Categorical Questions): Health Problems and Other
Question (Yes/No) % Correct Number of 
Observations
When you had a health problem: Did you visit a health facility 
or clinic for treatment? 
0.92
(0.73 – 0.99)
24
Was the quality of healthcare you received satisfactory? 1.00
(0.92 – 1)
46
Have you received information on the services available to 
Ebola survivors?
0.96
(0.88 - 1)
56
Do you feel that your access to healthcare for this problem was 
adequate?
0.94
(0.83 – 0.99)
50
In the last two years (since the end of the Ebola time), do you 
feel that the problem for which you received health service has 
been resolved?
0.67
(0.22 – 0.96) 
6 
Do you know one or more local groups or associations that 
represent the interests of Ebola survivors?
0.96
(0. 88 – 0.99) 
56
96
Have you heard about a group called the National Network of 
Ebola survivors in Guinea? (RENASEG)?
0.92
(0.81 – 0.98)
52
Are you a member of an Ebola survivors association? 1.00
(0.93 – 1)
52
Have you heard of an institution called the National Agency for 
Health Security (ANSS)? Formerly known as National 
coordination of Ebola Control.
0.87
(0.68 – 0.97)
24
Table F.11: Guinea Simple Agreement Rates: Stigma
Question: In the past 12 months, have any of the following 
things or actions occurred in the health facility or did you visit 
for care as a survivor or healed of Ebola? (Yes/No)
% Correct Number of 
observations
Medical personnel refused to treat you or do tests for you 
because of your Ebola status 
1.00
(0.59 – 1)
7 
The wait time to be seen by a medical staff was longer for you 
than for other patients
0.98
(0.89 – 1)
50
Medical staff took excessive precautions when examining you 0.98
(0. 56 – 0.82)
50
Medical staff gossiped about your Ebola status status 0.71
(0.56 – 0.83)
51
97
You received less attention / care than the other patients 0.86
(0.73 – 0.94)
50
Have you ever avoided or delayed seeking healthcare
treatment because you were afraid of the medical staff 
attitudes towards you as an Ebola Survivor 
0.88
(0.76 – 0.96)
51
Table F.12: Guinea Reliability Ratios (Ordinal Questions): Health Problems and Other
Question (Agree, Neither Agree nor Disagree, Disagree) Reliability Ratio Number 
Surveys
I am very confident in the ability of the Ebola survivors network 
to get the best support for people like me 
0.59
(0.39 – 0.78)
27
I am happy with the work that the Ebola survivors network 
does
0.52
(0.32 – 0.71)
27
I have confidence in ANSS's ability to provide the best support 
for people like me 
0.67
(0.35 – 0.90)
12
Do you think that the treatment of Ebola survivors by doctors 
and nurses has improved, worsened or remained the same 
since the outbreak of Ebola?
0.96
(0.88 – 0.99)
56
98
3) Other quality findings
Unlike Sierra Leone and Liberia, JSI R&T Guinea’s baseline and endline collection were conducted 
manually, using paper copies of the survey instrument. This created multiple opportunities for 
potential data entry issues both in initial recording of data on paper forms to eventual submission and 
conversion of data into digital format.
I4DI monitors noted that there were some language difficulties encountered by enumerators in 
administering the survey. There were local dialects that the survey instrument itself was not translated 
into, but enumerators made efforts to translate the questions to the best of their ability.
For monitoring data entry, the tool was coded in French, with each monitor responsible for submitting 
their entries to a database for analysis. This code was then crosswalked against JSI R&T Guinea’s 
framework and assessed. A digital collection would have simplified both the accuracy of collection and 
subsequent analysis. 
6. CONCLUSION
In summary, the three countries varied in terms of the number of survey questions available for 
analysis and the number with sufficient observations to be scored in this reliability assessment (Table 
F.13). Liberia had the highest number of survey questions scored, followed by Guinea and then Sierra 
Leone. This variation primarily results from the instrument design in Sierra Leone; a subset of 
questions were asked for each specific health problem that a respondent experienced. Because every 
respondent would not have experienced all the listed health problems, the number of observations 
related to specific health problems were low.
99
Table F.13: Number of questions analyzed and number of questions scored, by country
 Health Problems 
and Other 
(Categorical)
Stigma
(Categorical)
Health Problems 
and Other 
(Ordinal)
Total
Sierra Leone
Total No of 
Questions
39 17 13 69
Total Scored 6 9 4 19
Liberia
Total No of 
Questions
8 6 6 20
Total Scored 8 6 6 20
Guinea
Total No of 
Questions
9 6 4 19
Total Scored 8 5 1 13
Within the questions that have an adequate number of observations, there were high estimates for 
agreement rates and reliability ratios in all three countries (Table F.14). The data from Liberia had the 
best reliability, followed by Guinea and then Sierra Leone. These differences also could be due to the 
survey structure.
100
Table F.14. Reliability scores, by analysis type, by country
 Health Problems and 
Other (Agreement 
rates)
Stigma (Agreement 
Rates)
Health Problems and 
Other (Reliability 
Ratio)
Sierra Leone 
Excellent 6 (100%) 9 (100%) 2 (50%)
Good NA NA 2 (50%)
Moderate 0 0 2 (50%)
Poor 0 0 0 
Liberia 
Excellent 8 (100%) 6 (100%) 6 (100 %)
Good NA NA 0 
Moderate 0 0 0 
Poor 0 0 0 
Guinea 
101
Excellent 7 (88%) 2 (40%) 1 (100 %)
Good NA NA 0 
Moderate 1(12%) 2 (40%) 0 
Poor 0 1(20%) 0 
Due to the nature of the estimates (agreement rates and reliability ratios) and the varying number of 
observations reported, it is impossible to summarize them meaningfully into a single estimate for 
every country. Within the sample of questions that have sufficient observations, the agreement 
estimates are generally excellent, although a couple were good and one rated as poor (Guinea). These 
high reliabilities generally suggest that JSI R&T enumerators did not make transcription errors when 
entering answers reported by respondents, with the exception of the one item with poor agreement 
rates. Assuming that performance on the questions with sufficient observations is reflective of those 
questions without sufficient observations, as well as those not analyzed herein, there would likely be 
high agreement estimates across the entire survey dataset in each country.
In cases where the BalanceD-MERL monitor and JSI R&T enumerator recorded different responses, it 
is impossible to determine which of them (if any) was correct. Indeed, there is some subjective 
judgment required when entering verbal responses into electronic tools.
The findings presented herein only explore issues of enumerator reliability; they are not indicators of 
accuracy. There are several potential biases that could reduce accuracy. These include social 
desirability bias, where respondents give answers they believe are wanted or necessary, such as 
minimizing answers that might make the respondent look bad or embellishing answers that might 
result in the respondent obtaining additional assistance. The survey boycotts in Liberia may have 
affected the validity of those data uniquely. Regardless, any issues with data validity are not captured 
by this analysis.
102
7. REFERENCES 
Bajpai, R C, and H K Chaturvedi. 2015. “Evaluation of Inter-Rater Agreement and Inter-Rater 
Reliability for Observational Data: An Overview of Concepts and Methods.” Journal of the Indian 
Academy of Applied Psychology 41 (3). Indian Academy of Applied Psychology (IAAP): 20.
Banda, Jeremiah P. 2003. “Nonsampling errors in surveys.” United Nations Secretariat, Statistics 
Division. November 3, 2003.
Biemer, Paul P., and Lars E. Lyberg. 2003. Introduction to Survey Quality. Hoboken: John Wiley & 
Sons, Inc.
Campbell, Richard T., and Michael L. Berbaum. 2010. “Analysis of Data from Complex Surveys.” In 
Handbook of Survey Research, Second Edition, edited by Peter Marsden and James Wright, 221-259. 
Bingley: Emerald Group Publishing Limited.
Gwet, Kilem L. 2014. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the 
Extent of Agreement Among Multiple Raters (4th Edition). Gaithersburg: Advanced Analytics, LLC.
Kane, Michael. 2008. “Errors of Measurement.” William H. Angoff Memorial Lecture. November 19, 
2008.
103
Annex E: Disclosures of Conflict of Interest
Among the research team, there were no conflicts of interest. The following are signed statements 
verifying that there were no conflicts of interest.
104
105
106
107
108
109
110
111
Annex F: Original Statement of Work
Introduction
The BalanceD-MERL consortium is conducting the end-line evaluation for the portfolio of GHET 
Survivor programs in Sierra Leone, Liberia, and Guinea. The field data collection components of this 
evaluation will take place between February and April 2018, with the final evaluation report due at 
the end of September 2018.
This evaluation scope of work and design has been collaboratively prepared by Innovations for Poverty 
Action (IPA) and the Institute for Development Impact (I4DI), with support from World Vision US. 
USAID GHET has been engaged in prioritization of the lines of inquiry and will approve the final design 
of the evaluation by November 30, 2017.
Program description
The USAID Global Health Ebola Team (GHET) oversees a portfolio of health-related projects that fall 
under Pillar II of the U.S. Government’s (USG’s) response to the 2014 to 2016 Ebola virus outbreak in 
Sierra Leone, Guinea, and Liberia. The GHET has oversight of a $150.5 million portfolio of activities 
that support critical non-Ebola health services, health systems, and transmission prevention of Ebola 
virus from the West African survivor community. The GHET program’s development objective is that 
affected countries will be better equipped to sustain a “resilient zero” of Ebola cases10, meaning:
1. Effective infection prevention and control (IPC) within a wider context of patient safety.
2. Strengthened integrated disease surveillance and response capacities (IDSR) as part of 
enhancing the resilience of local health systems. 
3. Reactivation of a basic package of essential health services in affected countries
4. Rebuilding of health workforce capacities as a foundation for sustainable health workforce 
strengthening
5. Services addressing the specific needs of Ebola survivors are in place and integrated into the 
public health system to help mitigate further Ebola infection among the population
 
10 World Health Organization, Ebola Response Phase 3: Framework for achieving and sustaining a resilient 
zero, September 2015,
http://apps.who.int/iris/bitstream/10665/184693/1/ebola_resilientzero_eng.pdf?ua=1, and USAID Global 
Health Ebola Team Results Framework, June 7, 2017.
112
Implemented by 12 partners at national and regional levels across the three countries, the projects in 
the GHET portfolio aim to contribute to the mitigation of second-order impacts of the Ebola crisis, 
prevent further development losses and strengthen key institutions and infrastructure, with the 
following stated objectives:
1. Delivery and utilization of essential primary healthcare services are restored;
2. Health system functions are strengthened to respond appropriately and adequately to 
future public health crises
3. Prevention and treatment services that address the needs specific to Ebola survivors are 
available and utilized.
Some implementing partners have multiple projects within and across countries, and activities 
overall have been staggered across the period of 2016-2018. Other evaluation exercises are 
examining the first two sets of activities focused on the restoration of delivery and utilization of 
primary healthcare services and health systems strengthening. This is a cross-sectional study of the 
current status of GHET portfolio Ebola survivor-focused activities only.
As much as possible, baseline data will be researched from literature to provide context. Any GHET￾funded activities that will be continuing beyond the timeframe of this evaluation may consider this a 
midline survey and are welcome to use the data and findings for appropriate course correction.
Evaluation purpose 
The overall purpose of this evaluation is to determine to what extent the third intermediate and sub￾intermediate results of the GHET portfolio have been achieved during the life of the program to date.
113
Evaluation risks (limitations) & monitoring / contingency plans
Risk Mitigation Strategy
Overlapping Ebola response 
evaluation efforts
BalanceD-MERL is planning to coordinate with JSI R&T to share 
information, instrument development and even sampling 
strategies to eliminate burden on evaluation participants and 
ensure consistency where appropriate. This is particularly 
critical as it relates to conducting survivor surveys. 
Survivor survey dependencies Data collection from survivors is dependent on JSI R&T’s plans 
for respective endline survivor surveys and ability to coordinate. 
BalanceD-MERL will rely on USAID assistance in ensuring that 
our input in JSI R&T survey development is incorporated and 
this data is obtained.
Alignment of IPs’ result frameworks 
and reporting, to GHET’s theory of 
change and overall results framework
GHET has aligned IR and sub-IR indicators in the GHET results 
framework with the indicators which partners have been 
tracking and reporting over the course of their activities. This 
design is around lines of inquiry to evaluate the specific Survivor 
IR (3) and sub-IRs (3.1, 3.2, 2.2) in the overall results framework.
Access to data and documents for 
literature and secondary review, as 
well as other non-financial resources. 
We will work together to review resources required at each 
stage of engagement to ensure resources are available from 
GHET and the implementing partners. Early and persistent 
engagement with GHET and IPs is critical to ensuring that all 
documentation is acquired and/or made available for review, 
prior to the analysis and reporting on primary data collected 
through this exercise.
Availability of GHET primary points of 
contact for final decision-making on 
evaluation design
Biweekly meetings with GHET are scheduled. This ensures that 
a critical threshold of meetings enables adequate contact with 
primary stakeholders.
Seasonal liabilities - rainy season Data collection activities are planned before the onset of the 
rainy season in West Africa in 2018.
114
National Elections in Liberia (2017) 
and Sierra Leone (2018)
Data collection activities will be planned around the election 
cycles in Liberia (October 2017) and Sierra Leone (March 2018) 
to reduce potential security concerns.
IPs’ staff availability Early sampling and adequate notice will be supplied to IP key 
informants so they can be either available in the field or 
remotely for interview. If they are no longer with the activity 
being evaluated then we plan to arrange telephone or Skype 
interviews.
No baseline survey data for 
comparison
Where possible, findings will be compared with pre-outbreak 
and pre-GHET activity data. However, data availability and 
reliability is variable and this will limit comparison. Therefore, 
this evaluation should be considered a cross-sectional study and 
conclusions will be made against standards set out in country 
and UN policy.
Risks to evaluation independence As noted above, this evaluation relies on coordination with JSI 
R&T on design and input into an existing endline survivor 
survivor. As such, particular care will be given to maintain 
evaluation independence through (1) conducting independent 
training of our enumerators; (2) availability of all training 
materials for our enumerators to GHET (3) conducting of 
unannounced visits to observe JSI R&T interviews (4) use 
standardized instruments to evaluate data collection. 
115
Study design
Main Evaluation Questions
The lines of inquiry for this evaluation are based on whether services addressing the specific needs of 
Ebola survivors are in place and integrated into the public health system to help mitigate further Ebola 
infection among the population. The study will entail findings on Ebola survivor response strategy and 
activities, identification of program strengths and weaknesses. 
These evaluation questions link specifically to the GHET results framework: IR 3: Access to appropriate 
healthcare services is improved for Ebola survivors; sub-IR 3.1: Clinical services for Ebola survivors are 
strengthened and more accessible throughout the continuum of care; sub-IR 3.2: Stigma experienced 
by Ebola survivors when seeking care is reduced; sub-IR 2.2: Governance and management improved 
to advance national policies and systems.
Availability and Use of Quality Services
1. To what extent are quality health services focused on Ebola survivors available at different 
types of health facilities? Specific health services established for survivors? Regular health 
services? Referral to specific survivor services at higher level facilities? (Outcome) 
2. Capacity/Equipment Question: To what extent are survivor health services (other than primary 
healthcare services) and resources adequate and sufficient to meet the health needs of Ebola 
survivors? (Output) 
3. Survivors question: To what extent are Ebola survivors educated on services and accessing 
them? To what extent are Ebola survivors utilizing services available to them? (Outcome)
Stigma
4. To what extent do survivors experience stigma from health providers, and do survivors report 
that this stigma has decreased since the beginning of the program? (Outcome)
5. Management of health workers: To what extent are health professionals trained to minimize the 
effects of stigma during patient-provider interaction? (Output)
Policy/Systems
6. To what extent have policies, structures or networks (eg survivor networks) improved the 
services available for Ebola survivors? (Outcome)
7. What policies, structures or networks have been strengthened to ensure adequate services are 
available for Ebola survivors? (Output)
Detailed lines of inquiry questions for these main evaluation questions can be found in Appendix A.
116
Methods
The following considerations apply to this study:
● It is balanced, according to D-MERL principles- relevant, right-sized, responsible, and 
trustworthy.
● The study is a single study across the three Ebola affected countries, even though activities 
and their timing have some variation.
● Data collection will be conducted simultaneously in each country. IPA will be responsible for 
data collection in Liberia and Sierra Leone. I4DI will be responsible for data collection in 
Guinea.
● Agreement needs to be reached on the quantitative and qualitative division of labor around 
instrument development, data collection, and data analysis. 
● Health facilities are the main entry point for data collection; however, the focus will be on 
those facilities where USAID-funded activities are being conducted, in high Ebola survivor 
prevalence areas. 
● When possible, an assessment of all major health facilities that received JSI R&T 
interventions will be conducted (Liberia & Guinea). In Sierra Leone, a sample of facilities 
stratified by type and intensity of intervention will be taken.
● The evaluation design does not include a separate survivor survey but seeks only to 
coordinate with existing JSI R&T plans to survey survivors potentially through the inclusion 
of a separate module. If this proves not possible, no separate survivor data collection will 
take place through this evaluation.
The study design comprises cross-sectional data collection activities to evaluate the research 
questions, allowing conclusions on overall program efficiency, effectiveness and achievement. Data 
collection activities, described below, center around health facilities, except for the key informant 
interviews and implementing partner monitoring data. 
Health Facility Ebola Survivor Services Assessment
Survey and observation will focus on health facilities, with specific focus on Ebola survivor 
services:
● Infrastructure rehabilitation/renovation
● Availability and organization of survivor-specific services, equipment, medication, and 
commodities. 
● Expected numbers: Sierra Leone (37), Liberia (30), Guinea (17)
Health Provider Survey
Quantitative surveys will measure:
● Health provider knowledge, skills and attitudes, particularly those towards survivors and 
reduction of stigma
● Availability of EVD survivor medical equipment, medicine, and commodities
● Development and availability of survivor-specific protocols, including referral procedures
117
● Facility-based and centralized levels of human resources for health (HRH) service 
provision.
● Familiarity with specific partner training programs, focusing on the skill level of providers 
regarding Ebola survivors, determining if they attended any GHET funded provider 
trainings.
● Access to care and utilization
● Expected numbers: Sierra Leone (44), Liberia (51), Guinea (33)
Key Informant Interviews
Key regional and central level informants (and district level informants as appropriate) based on 
country-specific programming will be interviewed to discuss:
● Capacity building with central level officials and district health management teams
● Programmatic decision-making, including any stakeholder engagement
● Logistics and supply chain management
● Development of new survivor-specific protocols
● Governance support
● Survivor network integration
● Survivor network management
● Expected numbers: Sierra Leone (20), Liberia (20), Guinea (20)
Key informant interviews will include but not be limited to the Evaluation Stakeholders List & KII Pool 
(Appendix C).
Survivor Survey
As previously mentioned in this document, this evaluation design does not include a separate survivor 
survey component, but intends only to coordinate with JSI R&T on an upcoming endline survivor 
survey. Specifically, this will entail working with JSI R&T HQ and respective JSI R&T field offices in Sierra 
Leone, Liberia, and Guinea to coordinate through input into design and questions of JSI R&T’s endline 
survey instruments- proposing inclusion of an additional module (exploring lines of inquiry on survivor 
perceptions of availability of survivor-specific services, perceptions of survivor-specific policies and 
networks, etc), light complementary training of JSI R&T enumerators on the additional module as 
appropriate, and data quality/process checks on 5-10% of survivors surveyed. Data quality/process 
checks will consist of being present for surveys being conducted by JSI R&T enumerators and cross￾checking processes and data entry. Pending further engagement with JSI R&T, additional details will 
be provided. JSI R&T is not a primary source of data, but the evaluation team seeks to leverage JSI 
R&T’s endline survey with survivors to limit external surveying and taxing of survivors. Additionally, 
survivors are better interviewed by people who have already established trust and a history of 
engagement.
Per guidance from GHET, the operating assumption is that JSI R&T /APC will survey 700-800 survivors 
in each of the three countries. This would represent roughly half of the total number of registered 
survivors in Sierra Leone and Guinea.
118
Implementing Partner Monitoring Data
As part of this evaluation, a review of implementing partner monitoring data against implementing 
partner workplans will be conducted. 
Facility Sampling Strategy, logistics and data collection plan
Health facilities will be used as the entry point for:
● Health facility assessments
● Health [paid] provider surveys
Health Facilities and Health Providers
A sample of up to 100 primary, secondary, or tertiary health facilities providing survivor health services 
in geographic regions that have a high prevalence of Ebola survivors that have received JSI R&T
survivor-specific interventions, will be selected across the three countries. The sample frame is 
compiled from information supplied by implementing partners and from DHIS211. When necessary, 
facilities will be stratified by geography and type (e.g. country/district or prefecture; 
secondary/tertiary health facility). Only health facilities and associated providers in USAID-funded 
regions that received GHET funded interventions will be included in the sample. It is also expected 
there will be efficiencies to be captured between this sample and a broader data verification exercise 
proposed to GHET under separate forthcoming cover.
An enumerator will be deployed to selected health facilities, where he/she will conduct a 2-3-hour
health facility assessment to observe and note the presence of survivor-specific programs and 
interventions. After completing the health facility assessment, the enumerator will interview the 
selected provider at that facility.
When a health facility is chosen for assessment, the senior health provider associated with that facility
will be invited for interview. If they are not available, then the next most senior health provider will 
be invited. They must be a physician, nurse in charge or practitioner providing essential survivor 
services and paid to work at the selected health facility. For larger health facilities, 2-3 staff ranging 
from senior to lower level health providers (e.g. nurses) will be interviewed.
Health Provider interviews will be planned for the same day of the health facility assessment.
Instructions will be provided to the health facility so that chosen staff can be available at the health 
facility on the day of assessment. This will minimize logistical constraints and costs. 
Data will be collected using SurveyCTO on electronic devices. This facilitates real-time validity and 
completeness checks of data, as well as monitoring and analysis of descriptive statistics during the 
 
11 https://data.humdata.org/dataset/health-facilities-in-guinea-liberia-mali-and-sierra-leone
119
period of data collection. Responses in local languages and / or French (Guinea) will be translated into 
English for analysis.
Facility-based Sampling Strategy 
Sierra Leone
In Sierra Leone, the Advancing Partners & Communities (APC) Ebola Transmission Prevention & 
Survivor Services program (ETP&SS) supports the implementation of the President’s Recovery 
Priorities for Health, Key Result Area #3: the Comprehensive Program for Ebola Survivors. The 
Comprehensive Program for Ebola Survivors (CPES) is jointly led by the Ministry of Health and 
Sanitation and the Ministry of Social Welfare, Gender and Children’s Affairs.
Through its support, the ETP&SS program seeks to prevent further Ebola virus disease transmission 
and reduce the risk of resurgence from viral persistence in survivors. The program strives to improve 
access to quality health services for survivors by (1) strengthening the capacities of the health system
to provide specialized services needed by survivors, particularly through training and mentorship on 
mental health and neurology; and defining clear referral pathways. To reduce stigma and other 
barriers that keep survivors from accessing healthcare and social services, the program is (2) building 
the capacity of the Sierra Leone Association for Ebola Survivors and the CPES Program Implementation 
Unit to lead program implementation and monitoring. Through this program, Advancing Partners & 
Communities also (3) provides technical assistance to the Ministry of Health and Sanitation and the 
Ministry of Social Welfare, Gender, and Children’s Affairs on clinical and non-clinical related matters.
We propose to focus data collection in the five districts that have the highest number of ebola 
survivors, Bombali, Port Loko, Tonkolili, Western Rural, and Western Urban. JSI R&T’s CPES program 
conducted work in 66 of facilities in this district. We propose to assess 37 facilities, around 56% of 
facilities that received CPES interventions in these districts, and around 25% of all CPES facilities in 
Sierra Leone. These facilities will be randomly selected, stratified by district, facility type, and intensity 
of treatment. Facilities that received more survivor specific programming will be sampled more 
heavily. 
Health Facilities by 
district Bombali Port Loko Tonkolili
Western 
Area 
Rural
Western 
Area 
Urban
Total by 
Type
Proposed 
sample
MCHP (primary) 8 8 3 0 0 19 10
CHP (primary) 2 3 0 1 0 6 3
CHC (primary) 6 4 4 9 11 34 17
120
District Hospital 
(secondary) 0 1 1 0 0 2 2
Hospital (tertiary) 0 0 0 0 5 5 5
Total 16 16 8 10 16 66
Proposed sample 8 17 4 5 11 37
Source: JSI R&T Sierra Leone
121
Liberia
In Liberia, APC’s ETP&SS operates in four priority counties – Bong, Lofa, Margibi, and Montserrado. 
These counties account for over 90 percent of the population of registered Ebola survivors. Program 
activities in these focus counties include: (1) monthly coordination with the county-level chapters of 
the National Ebola Survivors Network of Liberia; (2) training on Ebola virus disease Survivor Clinical 
Care Guidelines led by ETP&SS; (3) clinical rotations and training provided by the Liberia College of 
Physicians and Surgeons’ Clinical Specialist Consultant Program; (4) renovations for selected hospitals 
and clinical services; (5) distribution of equipment prioritized during the November 2016 facility 
infrastructure assessment; and, (6) a one-time distribution of essential medicines for use by survivors 
in target health facilities.12
JSI R&T’s APC ETP&SS facility-based work involved 30 health facilities in the four priority counties 
(Bong, Lofa, Margibi, and Montserrado). The majority of this work was concentrated in eight 
secondary and tertiary facilities (six hospitals and two health centers), where JSI R&T provided training 
of health practitioners, provision of equipment, and infrastructure support. Work at the remaining 22 
facilities (two hospitals, nine health centers, and nine clinics) was primarily limited to provider training. 
We propose to assess all 30 facilities that received survivor-specific JSI R&T interventions. 
The table below breaks down health facilities by district and facility type:
Health Facilities by 
District13 Bong Lofa Margibi Montserrado
Total by 
type
Proposed 
sample
Primary Facilities 4 2 1 2 9 9 
Secondary Facilities 0 2 2 7 11 11
Tertiary Facilities 1 3 1 5 10 10
Total by district 5 7 4 14 30 30
 
12 The program carries out supplementary activities in the seven non-focus EVD-affected counties, specifically 
EVD survivor mapping and clinical guidelines training; these counties include Bomi, Gbarpolu, Grand Bassa, 
Grand Cape Mount, Nimba, Rivercess, and Sinoe. The training activities are carried out in collaboration with 
the Ministry of Health and the World Health Organization (WHO).
13 A note on health facility classifications in Liberia: Primary facility = PHC Clinic Level 1&2; Secondary facility = 
Health Centers, District Hospitals, County Hospitals; Tertiary facility = Regional Hospitals, National Hospital
122
Proposed sample 30
Data Source: JSI R&T Liberia)
Guinea
In Guinea, APC’s ETP&SS program focuses core activities on three priority regions—Conakry, Kindia, 
and Nzérékoré—with national coverage for selected activities. These three regions account for more 
than 80% of registered Ebola survivors.
The project assistance consists of: (1) support for supervision and coordination; (2) support for 
longitudinal care in selected high-volume health facilities in the three regions, including renovation 
and provision of small medical equipment and supplies; (3) support to community-based surveillance 
of Ebola survivors; (4) support to the national Ebola Survivors Association and stigma-reduction 
activities, including sensitization campaigns; and, (5) support for biological monitoring of viral 
persistence in semen.
Specifically, JSI R&T’s APC ETP&SS facility-based work involved 14 health facilities in three priority 
prefectures and regions. The majority of this work is concentrated in eight high-volume facilities (two 
facilities in the region of Kindia, four in Conakry, including one tertiary care facility, and two facilities 
in the region of Nzérékoré), where JSI R&T is providing infrastructure support and provision of 
equipment. Work at the remaining six facilities is primarily limited to equipment support and training. 
We propose to assess all 14 facilities that received survivor specific JSI R&T interventions. 
Beyond major health facility-based interventions, JSI R&T’s work in Guinea also includes support for 
the design and implementation of a more sustainable “sentinel site” surveillance system to streamline 
community-based surveillance. The sentinel sites are currently undergoing a planned reduction from 
700 sites at the height of the Ebola outbreak to a more sustainable 50 sites across the country. Given 
the transition underway, we propose assessing the sentinel site coordination units in each of the 
priority region and prefectures.
123
The table below gives a proposed sample size for each prefecture/region and facility type. 
Health Facilities by 
Region14
N’Zérékoré
Region
Conakry 
Region
Kindia 
Region Total by type Proposed sample
Sentinel Site Coordination 
Units 1 1 1 3 3 
Primary Facilities 1 0 0 1 1 
Secondary Facilities 3 3 4 10 10
Tertiary Facilities 2 1 0 3 3 
Total by Prefecture/Region 6 4 4 17 17
Proposed sample 17
(Data Source: JSI R&T HQ/Guinea)
Key Informants
In each country, up to 20 key informant interviews will be conducted with interviewees nominated 
from the Stakeholder & KII Pool List in Appendix C. Interviews will be conducted using an open-ended 
question guide, by a lead field researcher familiar with the GHET survivor activities that have been 
undertaken in that country.
Unless deemed culturally unacceptable, the interview will be audio recorded to facilitate note taking, 
coding and subsequent analysis. Sufficient time is built into the data collection plan so that the notes 
can consolidated following each interview. French or local language discussions (Guinea) will be 
translated into English for analysis.
 
14 A note on health facility classifications in Guinea: Primary facility = Health Posts (PS); Secondary facility = 
Health Centers (CS), CMC (Urban); Tertiary facility = Hospitals, Regional Hospitals; National Hospitals.
124
Analysis Plan
Analysis of outcomes:
Because this final evaluation is primarily a post-test, only measurement of sites with varying degrees 
of program exposure, there likely are systematic differences between the groups in observable and 
unobservable confounding factors. A rigorous impact evaluation with statistical power is not possible 
in this scenario. Thus, the analysis of outcomes generally will follow a contribution analysis approach, 
whereby the occurrence of various quantitative results (e.g., ‘outcomes’ measured via health facility 
assessments, and provider surveys) will be documented and will be considered in tandem with 
programmatic and qualitative information about the delivery of the GHET portfolio and the process 
by which change may have occurred. While the resulting report will not offer definitive proof of 
‘impact’, it will provide evidence from which we can draw a plausible conclusion that the GHET 
investment has made an important contribution to the documented results. 
For analysis of the quantitative data, the research team will use SPSS or Stata statistical software to 
calculate descriptive statistic and conduct bivariate analyses of subgroup differences. For analyses of 
qualitative data, the research team will use Nvivo transcript-coding software to organize and 
synthesize the key themes that emerge. While this will include outcomes and results, as mentioned 
above, analyses of qualitative data also will focus on insights that might explain the quantitative 
findings. An a priori coding framework will be constructed using the research lines of inquiry and the 
results of the quantitative analyses, and the framework will be expanded as new themes emerge in 
the qualitative data using a constant comparative method to inductively develop a theory around the 
variables, their linkages to one another, and their contribution to the results.
Analysis of process:
The analysis of process will provide descriptive information on how the GHET Pillar II Ebola survivor￾specific portfolio was implemented, the operational context, and barriers and facilitators to 
implementation uptake. It will rely heavily on program and policy documents, secondary data, and key 
informant interviews at central levels. The BalanceD-MERL team will gather as much documentation 
as possible on program implementation from policy documents, project work plans, quarterly reports, 
and secondary data records, and will complement these data with semi-structured key informant 
interviews in each of the three countries. As with the qualitative data collected for the outcome 
evaluation, the research team will use Nvivo software to analyze the key informant interviews data 
and will triangulate across respondents and source material to answer the relevant evaluation 
questions.
Products
The end-line evaluation report will be prepared following the period of data collection and analysis 
and will discuss in detail the study design, methodologies used, findings, conclusions and 
recommendations. The report will be co-written by members of the evaluation team and be submitted 
in draft to USAID GHET. Time will be allocated for two rounds of review and edits with USAID GHET, 
125
at which point the report will be finalised and submitted to USAID GHET, marking the completion of 
this exercise.
A plan for sharing the evaluation findings with implementing partners and Government bodies will be 
finalised in conjunction with USAID GHET.
Administrative considerations
The following list is provided as important administrative considerations which are on the critical 
path of the study and may affect the currently proposed timeline:
1. Instrument development (including incorporation of cultural norms) 
2. Translation and testing of instruments
3. IRB requirements and clearances
4. Collaboration with JSI R&T on survivor survey
5. The data-collection logistics plan (including travel and accommodation for enumerators)
6. Data access, privacy, and documentation (public use data-files, de-identifying (outlier removal)), 
and codebooks
7. Reporting/dissemination plan - expectations for USAID review/feedback/report finalization, in￾person GHET presentation, webinar for field participation
Team members
Arthur Baker (Senior Research Associate - IPA Sierra Leone)
Arja Dayal (Research Manager - IPA Liberia)
Osman Siddiqi (Country Director - IPA Sierra Leone)
Samuel Lee (Lead Researcher - I4DI)
Al Housainne Bah (Lead Field Researcher - I4DI Guinea)
Azra Nurkic (Qualitative research expert - I4DI)
Linda Gassama (Field Manager - IPA Sierra Leone)
Team advisors
World Vision United States - Bridget Lavin
I4DI - Manny Sanchez 
USAID GHET - Jeffrey Muschell
USAID GHET - Andy Rebold
Jessie Pinchoff – Population Council (previously IPA)
126
Budgets
The detailed budgets and budget narratives for this evaluation are presented under a separate 
cover. Please contact USAID GHET for detail.
GANTT chart
The following high-level Gantt chart details major activities. Timing is contingent upon approval of 
this plan, subsequent development and testing of instruments, country IRB processes, collaboration 
with JSI R&T on survivor surveys, and elections in Sierra Leone.
Key 
activities
2017 2018
Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep
Study design / 
protocol
Instrument 
development
Develop tools
Policy Review
JSI R&T
Survivor 
Survey 
Collaboration
IRB 
submission 
and clearance
Finalize 
research 
protocol
127
Pilot/test 
instruments 
in-country
Logistics 
planning
Recruit 
enumerators
Training of 
enumerators
Data 
collection
SL G/L G/L 
Data cleaning
Analysis
Report 
writing 
Report 
dissemination 
/ review & 
finalization
Project Close
128
Appendix A - Evaluation Question Lines of Inquiry
Availability and Use of Quality Services
1. To what extent are quality health services focused on Ebola survivors available at different types 
of health facilities? Specific health services established for survivors? Regular health services? 
Referral to specific survivor services at higher level facilities? (Outcome) 
- Where are the survivors?
- Where are the services (EVD specific and general) available?
- What are the standard packages of survivor services at different levels?
- How does the referral system for survivors work?
2. Capacity/Equipment Question: To what extent are survivor health services (other than primary 
healthcare services) and resources adequate and sufficient to meet the health needs of Ebola 
survivors? (Output) 
- What is the availability of survivor specific equipment, medicine, and commodities?
- What structures or networks have been put in place to ensure adequate services are 
available for Ebola survivors?
- To what extent is the program producing outputs on time and meeting targets for outputs?
- To what extent have IP project plans related to clinical services for survivors been 
implemented as planned?
- What have been the barriers to appropriate service delivery? How were they overcome?
- What adaptations, if any, have been made? What accounts for the variation?
- How did the IPs identify which services would be necessary and where to locate them? 
- To what extent have the key stakeholders been engaged to inform programmatic decisions 
(who was engaged, how, and what was their role)?
- What proportion of survivors report that adequate services are available to them?
3. Survivors question: To what extent are Ebola survivors educated on services and accessing them? 
To what extent are Ebola survivors utilizing services available to them? (Outcome)
- What proportion of survivors report receiving education on services? 
- What proportion of survivors accessed services when needed?
- What are the utilization patterns for survivor services?
- What proportion of survivors report improvement to access to appropriate health services 
as a result of the program?
Stigma
4. To what extent do survivors experience stigma from health providers, and do survivors report that 
this stigma has decreased since the beginning of the program? (Outcome)
- What are the main reasons leading to stigma from health providers?
- To what extent have SBCC and other campaigns reduced stigma reported by survivors from 
health providers and members of their community?
129
5. Management of health workers and communities: To what extent are health professionals and 
community health volunteers (volunteers in Liberia only) trained to minimize the effects of stigma in 
the community? (Output)
- What are the stigma reduction approaches implemented by IPs? 
- How did the IPs identify these approaches and where to locate them?
Policy/Systems
6. To what extent have policies, structures or networks (eg survivor networks) improved the 
services available for Ebola survivors? (Outcome)
- To what degree have national policies been implemented to enable/facilitate 
improvements in services available to survivors?
- What is the degree of awareness of and confidence levels in the work of the National 
Government Survivors Body and Ebola Survivors Networks?
- Based on the experiences of this program, what do managers and policy decision￾makers at different levels (national, regional, local, USAID-based) consider as the best 
approaches for behavior change - what worked and what did not?
7. What policies, structures or networks have been strengthened to ensure adequate services are 
available for Ebola survivors. (Output)
- What are the relevant policies, structures, and official survivor networks by country?
- How are the networks affiliated, supported by their respective MOH or equivalent?
- What is the level of organizational development of these Survivor Networks?
130
Appendix B - Summary of data collection instruments and purpose 
Data Collection 
Instrument
Duration 
per 
collection 
activity
Who will be interviewed? Purpose
Health Facility 
Assessment 
2 - 3 hours Nurse in charge at selected 
HF; observation of facility.
To visually inspect the HF for 
infrastructure improvements, 
provision of essential health 
services, availability of 
commodities, and presence of 
trained personnel
Provider Survey 1-1.5 hours Sample of paid, trained 
providers based at 
selected HF (including 
community volunteers in 
Liberia)
To assess provider knowledge re: 
Ebola stigma and assess 
participation in GHET-funded 
provider training programming.
Key Informant 
Interviews
1 hour Individuals selected with 
certain knowledge or 
experience in the health 
system 
To assess the efficiency, efficacy 
and sustainability of GHET￾funded survivor specific 
programming, management of 
activities, systemic 
challenges/opportunities, other 
process evaluation areas.
JSI R&T Survivor 
Survey Collaboration 
(TBD) 
Survivors identified and 
surveyed by JSI R&T, likely 
through individual country 
survivor networks 
To assess survivor perceptions of 
survivor services and the reach 
and effectiveness of survivor 
specific programming and 
activities
131
Appendix C - Stakeholder Groups & KII Pool
The following list are the groups (and individuals) who have a major interest in the evaluation, or will 
be substantially affected by it:
1. USAID Global Health Bureau
2. USAID Africa Bureau
3. USAID Liberia Mission
4. USAID Guinea & Sierra Leone Mission
5. FHI360 - Mobile Solutions Technical Assistance and Research (mSTAR)
6. International Organization for Migration (IOM) - International Organization for Migration 
(IOM)
7. John Snow, Inc. (JSI) - Advancing Partners and Communities Project (APC)
8. John Snow, Inc. (JSI) - Deliver Project (Deliver II) Task Order 7
9. John Snow, Inc. (JSI) - Supply Chain Management System (SCMS)
10. IBTCI
11. Johns Hopkins University-Center for Communications Programs (JHU-CCP) - Health 
Communication Capacity Collaborative (HC3)
12. Management Sciences for Health (MSH) - Systems for Improved Access to Pharmaceuticals 
and Services (SIAPS)
13. National Academy of Sciences (NAS) - Partnerships for Enhanced Engagement in Research 
(PEER) (Liberia)
14. University of North Carolina (UNC) - Monitoring and Evaluation to Assess and Use Results 
(MEASURE) Evaluation Phase IV
15. Government of Liberia
a. Ministry of Health
i. M&E Team
ii. County Health Teams
iii. District Health Teams
16. Government of Guinea
a. Ministry of Health and Public Hygiene
i. National Agency for Health Security (NAHS)
ii. District Health Teams
17. Government of Sierra Leone
a. Ministry of Health and Sanitation
i. CPES Program Implementation Unit (PIU)
ii. District Health Teams
b. Ministry of Social Welfare, Gender and Children’s Affairs (MSWGCA)
i. District Teams
18. Survivors Associations
a. Sierra Leone Association of Ebola Survivors (SLAES)
b. National Ebola Survivors Network of Liberia (NESNL)
i. County Chapters
c. Ebola Survivors Association (RENASEG) (Guinea)
19. APC Sub-grantees in Guinea
132
a. International Medical Corps (IMC)
i. Sentinel Site Coordination Units
20. APC Sub-grantees in Liberia
a. ELWA Hospital
b. Saint Joseph’s Catholic Hospital
c. The Liberia College of Physicians and Surgeons
d. Phebe School of Nursing Rural Training Institute
21. APC Sub-grantees in Sierra Leone
a. Kings Partnership Sierra Leone
b. Partners in Health
c. Save the Children
d. GOAL
22. Post Ebogui15 (Guinea)
23. National Laboratory of Hemorrhagic Fever (Guinea)
24. Liberia College of Physicians and Surgeons’ Clinical Specialist Consultant Program
 
15 Post Ebigui, a research project supported by the French government, has been conducting a longitudinal study 
in Guinea since the end of the Ebola outbreak to monitor and provide free clinical care to 801 survivors in Guinea 
(approximately two-thirds of survivors). Post Ebogui works in special care units (within the health facilities) in 
the regions of Nzérékoré and Kindia, and run their own private health care facility in the Conakry Region. The 
services provided by Post Ebogui are not following the official Guinean health system. Post Ebogui is scheduled 
to end in December 2018. Post Ebogui research areas overlap with the three priority regions and the survivors 
group supported by the ETP&SS program; therefore, it is expected that this will have an impact on the number 
of visits by Ebola survivors recorded by the program-supported health facilities.
133
Annex G: Research Team Biographies & Roles
Dr. Bridget C. Lavin has a PhD in health systems management and more than 15 years of experience 
evaluating public health and development programs. In addition to her role as Project Director of the 
BalanceD-MERL consortium, Dr. Lavin served as co-lead investigator for the endline performance 
evaluation of the GHET EVD survivor program. In this capacity, she contributed to the evaluation and 
instrument design, analyzed quantitative data, and authored portions of the final report.
Samuel S. Lee has a Master’s in Public Policy and over 15 years of experience in international 
development, social impact investing, innovation, and program evaluation. Sam is the co-lead 
investigator for the endline performance evaluation of the GHET EVD survivor program. In this 
capacity, he contributed to evaluation and instrument design, managed field data collection and 
engagement, training of enumerators, completed data analysis integrating quantitative and 
qualitative data, and authored the final report.
Dr. Azra Kacapor Nurkic is a social development expert with over 20 years of experience and expertise 
in strategic program design, program effectiveness and staff capacity development in a range of 
operational and regional contexts. Dr. Nurkic advised on and contributed to study design and led all 
qualitative work including instrument design and analysis.
Jose "Pepe" Santos has a Master’s in Global Affairs & Management with relevant experience in 
multinational business operations, strategic development, and research. As part of the consortium, 
Pepe conducted qualitative analysis on key informant interview data. 
Arthur Baker has a Bachelors in Econ Politics and a Graduate Certificate in Economics. Arthur managed 
BalanceD-MERL activities in Sierra Leone including quantitative surveys and survivor survey 
monitoring. He was involved in the development of quantitative survey instruments, training of field 
staff, securing local ethics review board approval, coordinating between various organizations, 
quantitative data analysis and co-authoring the reliability assessment (Annex D). 
Sibongani Kayola has an M.S postgraduate degree in Evidence-based social intervention and policy 
evaluation. Sibongani managed BalanceD-MERL activities in Liberia including quantitative surveys and
survivor survey monitoring. She was involved in the development on the quantitative survey 
instruments, training of field staff, securing local ethics review board approval, coordinating between 
various organizations, quantitative data analysis and co-authoring the reliability assessment (Annex
D). 
Leslie Alex has a Bachelors in Industrial Engineering and micromasters credential in Data,Economics 
and Public policy. Leslie worked on designing, coding, and testing quantitative instruments, 
monitoring data collection and quality in Sierra Leone, development of evaluation framework for 
reliability assessment monitoring piece and co-authoring the reliability assessment (Annex D). 
Arja Dayal has a Masters in Public Administration in Development Practice and has multi-sectoral 
knowledge, working on projects encompassing education, health, nutrition and WASH, both at 
grassroots as well as policy level. Arja managed all IPA activities in the BalanceD-MERL project across 
134
Sierra Leone and Liberia. She oversaw project development efforts, stakeholder management and 
project finance management. Arja also co-authored the reliability assessment (Annex D) in the report. 
135
U.S. Agency for International Development
1300 Pennsylvania Avenue, NW
Washington, DC 20523