SSDI-SERVICES PERFORMANCE
EVALUATION
October 2016 
This publication was produced at the request of the United States Agency for International Development. It was 
prepared independently by Dr. Marcel Sagbohan, Dr. Spy Munthali, Dr. Nicholas Kanlisi, Mr. Willie Kachaka and Mr. 
Dyton B. Mukhuna through DevTech Systems Inc.
SSDI-SERVICES PERFORMANCE 
EVALUATION 
October 14, 2016
AID 612-TO-16-00003
DISCLAIMER 
The author’s views expressed in this publication do not necessarily reflect the views of the United States 
Agency for International Development or the United States Government. 
SSDI-Services Evaluation i 
ACKNOWLEDGEMENTS 
The SSDI-Services evaluation team would like to thank the individuals who generously took time out of 
their busy schedules to provide candid information and perceptions on the quality, access, capacity 
building and HMIS activities implemented under the Support for Service Delivery-Integration (SSDI￾Services) Program; all health workers who patiently and kindly answered our questions and filled out a 
lengthy survey, despite long lines of patients waiting for their attention; the community members and 
community volunteers (CAG and Care groups) who accepted to speak to us so enthusiastically about 
their role in improving quality and access in their communities, and participated in focus group 
discussions in the thirteen districts research sites often with little advance notice. Throughout this 
evaluation, the team benefited from the support and assistance of Ministry of Health (MOH) directors, 
managers, district health officers, program coordinators and health care providers, of the Essential 
Health Package (EHP) in Malawi. The team would like to acknowledge the dedicated USAID Malawi staff 
who provided on-going feedback and support throughout the evaluation, especially Chimwemwe 
Chitsulo, USAID/Malawi’s Monitoring, Evaluation and Learning Specialist and Contracting Officer’s 
Representative (COR). Lastly, the team is grateful to SSDI-Services, JHPIEGO, Save the Children, Plan 
International, CARE, Nkhotakota AIDS Support Organization for making their staff and project 
information available to the team. These professionals are working tirelessly to improve quality and 
access of EHP services to the Malawian people and build capacity of MOH staff in EHP services and 
utilization of health data for decision making. 
SSDI-Services Evaluation ii
This page is intentionally left blank
SSDI-Services Evaluation iii
CONTENTS 
ACRONYMS iv
EXECUTIVE SUMMARY vii
I. PROJECT OVERVIEW 1
II. EVALUATION PURPOSE AND QUESTIONS 3
Evaluation Purpose 3
Evaluation Questions 3
III. EVALUATION DESIGN, METHODS AND LIMITATIONS 4
Evaluation Design 4
Methods 4
Limitations of Data 7
IV. FINDINGS AND RECOMMENDATIONS 8
Evaluation Question 1 8
Evaluation Question 2 11
Evaluation Question 3 15
Evaluation Question 4 18
Evaluation Question 5 21
Gender Analysis 26
Annex 1. Evaluation Statement of Work 28
Annex 2. Summary Evaluation Methodology 37
Annex 3. Data Collection Instruments 38
Annex 4. People Consulted 57
Annex 5. Documents And Sources Reviewed 61
TABLES 
Table 1: Changes in quality of care indicators SSDI-Services-facility data 8
Table 2: Community level capacity development for CBDAs 17
Table 3: Data collection and reporting indicators 20
Table 4: Performance achievements on the 10 core indicators 22
FIGURES 
Figure 1: SSDI-Services districts 1
Figure 2: Percentage of clients reporting on satisfaction with services 11
Figure 3: Trend of number of HF with established CMAM 12
Figure 4: Trend of HIV-exposed infants tested for HIV within 12 month 12
Figure 5: Trend of number of functional village clinics providing iCCM 13
Figure 6: Trend of CYP through CBD 14
Figure 7: Access to quality services for men, women and people with disability 15
Figure 8: Trend in supervision from district to HF 17
Figure 9: Best practices reported by stakeholders (%) 24
Figure 10: Difference in best practices reported by stakeholders 25
SSDI-Services Evaluation iv
This page is intentionally left blank
SSDI-Services Evaluation v 
ACRONYMS 
AIDS Acquired Immune Deficiency Syndrome 
BEmONC Basic Emergency Obstetric and Newborn Care 
CBDA Community Based Distribution Agent 
CBS Community Based Services 
CDCS Country Development Cooperation Strategy 
CMAM Community Management of Acute Malnutrition 
CMED Central Monitoring and Evaluation Division 
CSO Community Service Organization 
DCS Department of Clinical Services 
DHIS2 District Health Information System Version 2 
DHO District Health Office 
DHMT District Health Management Team 
DP Development Partner 
DPPD Department of Policy Planning and Development 
EHP Essential Health Package 
FGD Focus Group Discussion 
FP Family Planning 
HAC Health Advisory Committee 
HF Health Facility 
HSSP Malawi Health Sector Strategic Plan 
GI Group Interview 
GIS Geographic Information System 
iCCM Integrated Community Case Management 
IE Impact Evaluation 
iHRIS Human Resource Information System 
IP Infection Prevention 
IPM Internal Planning Meeting 
IPT2 Intermittent Preventive Treatment Second Dose 
IT Information Technology 
HIS Health Information Systems 
HIV Human Immunodeficiency Virus 
HQ Headquarters 
HMIS Health Management Information System 
KMC Kangaroo Mother Care 
SSDI-Services Evaluation vi
KII Key Informant Interview 
MoH Ministry of Health 
NAC National Aids Commission 
NASO Nkhotakota AIDS Support Organization 
NGO Non-governmental Organization 
NHA National Health Accounts 
NMCP National Malaria Control Project 
PBI Performance Based Incentive 
PDU Policy Development Unit 
PE Performance Evaluation 
PMP Performance Monitoring Plan 
PMS Performance Management System 
PQI Performance Quality Improvement 
QA Quality Assurance 
RH Reproductive Health 
SBMR Standard Based Management and Reward 
SOW Statement of Work 
SSDI Support for Service Delivery Integration 
SSI Semi Structured Interview 
STA Senior Technical Advisor 
SWAp Sector Wide Approach 
TA Technical Assistance 
ToT Training of Trainers 
TL Team Leader 
USAID United States Agency for International Development 
ZHO Zonal Health Office 
SSDI-Services Evaluation vii 
EXECUTIVE SUMMARY 
The Support for Service Delivery Integration (SSDI-Services) project was a five-year project worth an 
estimated US $85,000,000 funding support from United States Agency for International Development 
(USAID) to the Government of Malawi (GoM) toward the implementation of the Health Sector 
Strategic Plan (HSSP) 2011-2016. Implemented by a consortium of partners (Jhpiego, Save the Children, 
CARE International, Plan International) and led by Jhpiego as the Prime, the SSDI-Services project set 
out to improve the health and well-being of the population of Malawi by implementing an integrated 
service delivery program that sought to significantly expand and improve quality of priority Essential 
Health Package (EHP) services at the community and referral (health centers and hospitals) levels in 
order to achieve the following results: 
1. Increased access and utilization of EHP services for women and children that could make a 
difference in their health and engage men in care. 
2. Improved quality of health services at community and facility level in 15 districts. 
3. Improved health-seeking behavior by individuals, families and communities. 
4. Strengthened health care delivery system through development, testing, and scale up of 
innovative and sustainable community-based service delivery approaches. 
PURPOSE OF THE PERFORMANCE EVALUATION 
The purpose of this SSDI-Services end-of-project performance evaluation is to measure the 
effectiveness, relevance and efficiencies of project activities. Primarily the evaluation aimed to determine 
the effect of SSDI-Services’ interventions on improved service delivery and quality of care at supported 
community clinics and health facilities; expanded coverage of quality EHP services; and increased uptake 
of quality integrated EHP. The main objectives of the evaluation included to: 1) determine the extent to 
which SSDI-Services interventions influenced quality of and access to care; 2) provide in-depth insights 
into the facilitating and limiting factors of increased service utilization at each level of service delivery; 
and 3) document progress made towards building MoH capacity to deliver quality EHP services. The 
evaluation also documented key approaches that contributed to achievement of project outcomes and 
to provide evidence-based recommendations on key actions required of USAID/Malawi and MoH to 
further improve integrated health service delivery in Malawi. USAID/Malawi and the MoH are the 
primary beneficiaries of this report, however, a much wider group of interested stakeholders are 
expected to utilize the findings and recommendations in their efforts toward more efficient and effective 
health service delivery in Malawi. 
EVALUATION QUESTIONS 
The performance evaluation aimed to answer the following key questions: 
1. To what extent did SSDI-Services improve quality of care for priority EHP interventions at 
targeted health facilities and community service delivery points? Were there any adverse or 
unexpected results on quality of care? 
2. To what extent did SSDI-Services improve access to priority high impact EHP interventions in 
the target health facilities and community service delivery points? 
3. To what extent did the direct involvement of MoH staff to facilitate the in-service training of 
health practitioners, on-the-job mentoring and quality assurance strengthen the MOH’s capacity 
to institutionalize these components as part of routine delivery of high impact priority EHP 
services? 
SSDI-Services Evaluation viii 
4. To what extent did SSDI’s investments in performance monitoring and Health Information 
Systems improve the MoH’s (national, district and facility) capacity to collect; access and utilize 
routine service delivery data for decision-making? 
5. What are the most significant accomplishments, best practices, and lessons learned from the 
SSDI-Services activity? Explicitly identify and document the facilitating and inhibiting factors to 
positive performance for each of the above questions. 
EVALUATION METHODOLOGY 
The evaluation utilized a mixed-method approach that included both qualitative and quantitative 
methods. In addition to reviewing data from secondary sources, the evaluation team gathered primary 
data from two health facilities in 13 out of 15 project districts, for a total of 26 health facilities. A 
number of Key Informants were interviewed at national, zone and district levels. The main criteria for 
selecting the districts to be included in the evaluation were 1) representation on Country Development 
Cooperation Strategy (CDCS), and 2) Comprehensive HIV test and treat program delivery and the 
Performance Based Incentives (PBI) program. 
In summary 34 key informant interviews at National, Zonal representatives, and District level 
informants, at least 3-4 interviewees per facility including self-assessments, and community-level focused 
discussions and score-card assessments as well as 15 to 20 client exit interviews per facility were 
conducted in these sites. To gauge the performance on various indicators and incorporate views of 
respondents, the team analyzed the collected primary qualitative data in terms of themes and narratives 
using NVIVO 11 software, while the quantitative primary and secondary data were all analyzed in 
STATA 14.0 software and excel. Qualitative methods included thematic analysis of primary data 
collected through KIIs, client exit interviews and focus group discussions. All qualitative data was 
analyzed using NVIVO 11 software. The team conducted a complementary comprehensive desk review 
of the project documents provided by USAID and JHPIEGO. The literature review provided insights into 
the design of the project, the background situation of the service delivery prior to the interventions, and 
an understanding of a selected list of indicators routinely tracked by the M&E system over the life of the 
project for triangulation with data collected from primary sources. The 10 core indicators at the center 
of the services component of the project were reviewed as a requirement of this evaluation. 
The evaluation team also completed 26 group interviews (GI) with on average 3-4 health care workers 
per group at 26 SSDI-Services supported health facilities to gain an in-depth understanding of their 
experiences with SSDI-Services program interventions. 
A total of 112 self-assessment questionnaires were completed. The evaluation team also administered 
490 consumer questionnaires to clients upon exit from facility utilization on the day of the evaluation 
team visit to each facility. 
KEY FINDINGS 
Q1: On Quality of Care, the performance evaluation found that: 
 Quality of Care (QoC) has improved significantly at target HFs and community service delivery 
points through Quality Assurance (QA) approaches, service integration and community 
empowerment 
 Performance Quality Improvement (PQI) approaches have contributed to improved Quality of 
Care 
 Community mobilization and empowerment has improved quality of care 
 More women are satisfied with the capacity of facilities to meet clients’ needs than men
Q2: On Access to Care, the performance evaluation found that: 
SSDI-Services Evaluation ix
 SSDI-Services expanded coverage of EHP services and provided opportunities for greater access 
to clients 
 Trained HSAs have improved communities access to CBMNH services through village clinics 
 SSDI services increased access in hard to reach facilities and communities but distances and 
transport are still major barriers to access in many places 
 Performance Based Incentives (PBI) improved access to services 
 Community members agreed that SSDI-Services has increased their access to EHP services 
Q3: On Capacity Building, the performance evaluation found that: 
 The project built capacity for utilization of TOT, mentorship and supervision 
 Trainers were being utilized in mentorship, supervision and follow up activities 
 Supervision of health facilities in SSDI districts considerably increased 
 Community Score Cards and Community-Based Volunteers facilitated Demand Side 
Contribution to Improved Quality of service delivery 
 Capacity Institutionalization is faced with funding Challenges 
Q4: On Use of HMIS and Use of Data for Decision Making, the performance evaluation found that: 
 SSDI supported the migration from paper based reporting to DHIS 2, this included provision of 
equipment 
 HMIS has been strengthened at National level 
 The project built the capacity at District and HF levels for data collection, reporting, entry in 
DHIS2 and utilization 
 Regular Data collection and reporting are being institutionalized at District level 
 Generation and access to data has been increased, and data review and analysis meetings are 
conducted every quarter 
 Some challenges tend to undermine SSDI Services effort to strengthen HMIS 
Q5: On Achievements, Lessons Learned, and Best Practices, the performance evaluation found that: 
The following were cited as key best practices: 
 Establishing 15 HFs that met specific quality standards as centers of excellence 
 Using mentorship as a capacity building approach 
 Using Standard Based Management and Rewards for improving quality of care 
 Regularly conducting facility data review meetings 
 Camping 4-5 days in hard to reach areas with delivery of integrated health services.
 Use of Data Reporting Booklets 
 Use of Mobile Phone Application for reporting 
 Use of Community Score Cards, Community Action Groups (CAGs) and Care Groups 
 Employing participatory approaches at designing activities and selected implementation stages 
The following were the lessons learned: 
Central Level: 
 Investment in integrated service delivery in the National Health System has potential for 
reaching tremendous outcomes 
SSDI-Services Evaluation x 
 Inclusive partnerships from initial design stages is conducive to successful implementation and 
outcomes 
 Coordination and harmonization of implementation for many partners is very difficult to achieve 
 The Malawi Health System is not yet ready to independently sustain implementation of activities 
hitherto supported by SSDI after project phase-out 
 Service Integration and PBI are effective approaches to improve quality. 
District Level: 
 By focusing on data quality HFs are enabled to improve other technical domains too 
 PBI improves provider’s confidence and increases participants program ownership and financial 
capacity to acquire and maintain equipment, infrastructure and supplies 
KEY CONCLUSIONS ON THE FINDINGS 
 SSDI interventions had significant positive effects on quality of and access to care 
 PBI has great potential for improving Quality of Care
 Clear progress was made towards building MoH capacity 
 Inconsistent financing by the MoH threatens the continuity of gains achieved through SSDI 
KEY RECOMMENDATIONS 
1. MOH and Partners should develop a strategy to scale up the PBI program, IP, Mentorship and 
Supportive supervision 
2. MOH and Partners should progressively extend outreach clinics and CBS to more hard to reach 
areas 
3. Partners and DHMT should integrate their activities and implementation schedules for more 
efficient use of transportation. 
4. For routine utilization of the built capacity strategies, MOH and partners should ensure 
availability of supportive resources 
5. MOH & Partners to improve integration of parallel health related data systems with DHIS2 for 
synergies 
6. MOH and Partners should ensure there are no transitional gaps during program/project phasing 
to avoid erosion of health gains already attained. 
SSDI-Services Evaluation 1 
I. PROJECT OVERVIEW 
The Support for Service Delivery Integration–Services (SSD-I Services) is an $85,000,000 five-year 
USAID-funded flagship project that aims to support the Government of Malawi in the implementation of 
the Health Sector Strategic Plan (HSSP) 2011-2016. The SSD-I Services project is implemented to assure 
significant expansion of coverage and improved quality of priority Essential Health Package (EHP) 
services at the community and referral (health centers and District hospitals) levels with the aim of 
improving the health and well-being of the population of Malawi. 
The project was launched in 2011 with JHPIEGO, PLAN, CARE, and Save the Children as principal 
implementing partners and will close-out in 2017. At the time of inception of the project, Malawi 
suffered from poor health indicators, inefficient and understaffed service delivery systems, gender 
inequalities and widespread use of harmful traditional practices. USAID/Malawi sought to address these 
issues through the inter-related SSD-I programs: 1) SSD-I Services, 2) SSD-I Communication, and 3) 
SSD-I Systems. The 3 sub-programs were implemented in a coordinated manner and in strong 
collaboration with Malawi Ministry of Health (MOH) to improve the health of Malawian families. The 
focus of this performance evaluation is primarily on the SSD-I 
Services component of the project. 
SSDI-Services works at the national level, and in 15 districts in 
all the five health zones of Malawi. At the national level the 
project assists the development and review of national 
policies, strategies, protocols and guidelines. At the zonal, 
district and health facility level, the project provides direct 
technical support to implementing high impact EHP 
interventions. 
At the community level, SSD-I Services supports 
improvements in the coverage and provision of quality high 
impact EHP interventions through Health Surveillance 
Assistants (HSAs), outreach clinics and community volunteers 
(community Action Groups, Community Care Groups and 
Community Based Distribution Agents). These interventions 
are supported by both formal and on-the-job-training, 
mentoring, and supervision using an integrated approach. The 
project holds regular update and joint review meetings for 
health workers to encourage greater peer-to-peer support 
and utilization of data for decision-making. An important 
aspect of this work is support to community volunteers and promotion of linkages between the 
community and the health facilities using the community score card approach. 
The project has a critical focus on improving the quality and access of EHP services, improving the 
capacity of health workers and community volunteers to facilitate community access to quality care, and 
improving MoH’s capacity to collect, manage and use data for decision making. SSDI-Services key 
technical areas of focus included the following: 
 Main EHP Areas: Maternal Health, Newborn and Child Health, Family Planning and Reproductive 
Health, HIV/AIDS and TB, Nutrition and Malaria 
 Cross-cutting Approaches: Key Technical Areas: Community Mobilization and Community￾based Services, Performance Quality Improvement (PQI), Mentoring, Performance Based 
Incentives (PBI), Monitoring and Evaluation (M&E).
Figure 1: SSDI-Services districts
SSDI-Services Evaluation 2 
Gender and Disability 
Gender and disability issues contribute directly and indirectly to health outcomes. In its activities, SSDI￾Services addressed: barriers to use of EHP services and behavior change such as gender biases and 
inequalities through the use of community based distribution agents (CBDAs), community scorecards, 
community action groups (CAGs) and care groups. The project facilitated increased involvement of men 
in safeguarding family health; and incorporated messages of gender mainstreaming and inclusion of 
people with disabilities into training, mentoring, supervision and health promotion activities. 
SSDI-SERVICES PROGRAM VISION, GOAL AND OBJECTIVES 
Development Hypothesis 
Theory of change/development hypothesis 
SSDI-Services postulates that by providing training and mentoring, commodity facilitation, equipment and 
supplies, effective data management, performance-based incentives, community mobilization and 
expanded community-based services, the activity will improve the technical capacity of health workers, 
increase commodity security, improve facility client flow, improve health facility functioning, improve 
governance and increase the use of data-informed decision-making. These short term outcomes, in turn, 
are assumed to lead to increased client access to and utilization of health services, improved quality of 
care, a strengthened health system, improved health-seeking behavior and empowered communities. In 
the long-term, these outcomes are assumed to result in reduced maternal, newborn and child morbidity 
and mortality and lowered risk of HIV, ultimately contributing to the goal of “Healthier Malawian 
Families” according to the SSDI program document.
SSDI-Services Evaluation 3 
II. EVALUATION PURPOSE AND QUESTIONS 
EVALUATION PURPOSE 
The primary objective of the evaluation is to measure and determine the extent to which SSDI Services 
interventions effected quality of and access to care; provide in-depth insights into the facilitating and 
limiting factors of increased service utilization at each level of service delivery, and document progress 
made towards building MoH capacity to deliver quality EHP services. 
The findings of this evaluation will inform MOH, USAID, and other development partners in the design 
and prioritization of future investments in strengthening Malawi’s Health System.
To achieve that purpose the SSDI-Services performance evaluation examined the intervention logic; 
analyzed the completion of planned activities and achievement of performance targets; and assessed how 
the availability and quality of the EHP services has changed in intervention communities over the life of 
the project. The evaluation also documents the extent to which SSDI-Services’ theory of change proved 
to be correct and the determining factors for the achievement of project outcomes. 
EVALUATION QUESTIONS 
The performance evaluation aimed to answer the following key questions: 
1. To what extent did SSD-I improve quality of care for priority EHP interventions at targeted 
health facilities and community service delivery points? Were there any adverse or unexpected 
results on quality of care? 
2. To what extent did SSDI-Services improve access to priority high impact EHP interventions in 
the target health facilities and community service delivery points? 
3. To what extent did the direct involvement of MoH staff to facilitate the in-service training of 
health practitioners, on-the-job mentoring and quality assurance strengthen the MoH’s capacity 
to institutionalize these components as part of routine delivery of high impact priority EHP 
services? 
4. To what extent did SSDI’s investments in performance monitoring and Health Information 
Systems improve the MoH’s (national, district and facility) capacity to collect; access and utilize 
routine service delivery data for decision-making? 
5. What are the most significant accomplishments, best practices, and lessons learned from the 
SSDI-Services activity? Explicitly identify and document the facilitating and inhibiting factors to 
positive performance for each of the above questions. 
SSDI-Services Evaluation 4 
III. EVALUATION DESIGN, METHODS AND 
LIMITATIONS 
The evaluation was conducted between 23rd August and September 23, 2016 and reviewed the period 
from SSDI-Services inception in April 2012 until June 2016. The core evaluation team consisted of five 
consultants: Dr. Marcel Sagbohan (Team leader), Dr. Nick Kanlisi (Public Health Specialist- Community 
Based Services), Dr. Spy Munthali (Public Health Specialist – Facility Based Services), Mr. Willie Kachaka 
(Research Analyst) and Mr. Dyton B. Mukhuna (Evaluation Specialist). In addition data collection was 
supported by 12 research assistants from a local research firm, Evidence for Change (E4C) Solutions. 
Logistics support was provided by DevTech through Ellen Kumwenda a Lilongwe based logistical 
consultant. 
EVALUATION DESIGN
The evaluation methodology used various approaches including preparatory meetings to strategize, a 
mix of quantitative and qualitative techniques including document review; key informant interviews 
(KIIs), facility staff interviews and self-administered questionnaire; focus group discussions (FGDs) with 
community members (male, female, CAG and care groups) and community score cards to collect 
relevant information, in a non-experimental performance evaluation design focusing on the key 
evaluation questions. Application of Qualitative and Quantitative approaches was utilized to analyze the 
secondary and primary data to inform the evaluation and the findings were disseminated to stakeholders 
in a presentation. The feedback from the workshop and inputs from USAID, JHPIEGO, Social Impact and 
DEVTECH were incorporated in the final report. 
METHODS 
Sampling 
A multi-stage purposive sampling procedure was used to select the 13 out of 15 SSDI project districts, 
facilities and individual respondents. The criteria included district representation on core functions of 1) 
Country Development Cooperation Strategy (CDCS), and 2) Comprehensive HIV test and treat 
program delivery and 3) the Performance Based Incentives (PBI) pilot program. These three criteria led 
to the sample of 5 core districts namely Chitipa, Mangochi, Balaka, Nsanje and Nkhotakota which also 
ensures that all Ministry of Health Zones are represented. An additional 8 districts were selected from 
across the remainder of SSDI districts to get statistically sufficient numbers of respondents to consumer 
questionnaires and health facility staff interviews. 
The next stage involved strategic sampling of 2 facilities in each of the 13 districts using two approaches; 
first was the Ministry of Health (MOH) classification of Hard to Reach (HTR) versus non-HTR and 
secondly a JHPIEGO classification of Best Performing versus Poor Performing facilities on the core 
indicators of the SSDI project. The 26 facilities visited by the evaluation teams sought equal 
representation on these two criteria across the 13 districts. At each facility 3 to 4 members of the 
health workers were included in the sample for staff interviews, the numbers varied with availability of 
staff at each facility. Within proximity to each facility, one community within 3 Km radius and another 
community within 3 to 5 Km distance from the facility were sampled with the help of the HSAs to reach 
two groups of 10 women and 10 men each, and one group of 10 randomly selected women community 
members respectively. These three groups were then complemented by one Community Action Group 
(CAG), a designated community leadership (fixed-membership) group introduced by SSDI project as 
another target for interviews and discussions. 
SSDI-Services Evaluation 5 
Obtaining Approval and Prior Consent 
The evaluation, through USAID, obtained approval from the MoH to conduct the assessment and a 
written letter was distributed to all targeted data collection points prior to the evaluation team’s visit. 
Similarly, verbal consent was sought from every individual that was requested to participate in the 
interviews and provision of evaluation information or recording of the discussions by reading a written 
statement of consent on the first page of each data collection instrument regarding the voluntary nature 
of their participation. 
Data Collection, Analysis and Dissemination 
The process of planning and implementing the evaluation was a collaborative effort involving USAID, 
Dev Tech and SSDI-Services. Several planning meetings were held with key stakeholders; a training 
workshop was organized for adapting, translating and piloting the questionnaire, and for outlining a 
report combining the qualitative and quantitative results. 
Pretesting of instruments and data collection took place during the period 23rd August and September 
23, 2016 in the 13 districts, 5 MOH zones and at national level where MOH, USAID, JHPIEGO and 
representatives of Implementing Partners’ namely Plan, Care International and Save the Children 
participated in the interviews. The data collection tools are attached as Annex 5. 
Data collection was done by a team of 17 members comprising 5 key evaluation staff and 12 
enumerators that were split into two sub-teams during field visits. Dr. Marcel Sagbohan and Mr. Dyton 
Mukhuna led a team of six enumerators who did data collection in Chitipa, Nkhotakota, Salima, Dowa 
and Kasungu including Mzuzu Zone office. Dr. Nicholas Kanlisi and Dr. Spy Munthali accompanied by six 
other enumerators collected data in Nsanje, Chikwawa, Phalombe, Machinga, Balaka and Mangochi 
including Blantyre and Zomba zone offices for South West and South East respectively. In the final week 
of data collection the two teams converged in Lilongwe to complete Lilongwe district data collection, 
Zone offices for Central West and East, and Lilongwe national level interviews. During the data 
collection period Mr. Willie Kachaka was closely working with the field teams while focusing on 
producing data entry templates, pretesting data entry and interview transcription guides. 
Document Review 
The team conducted a formal document review prior to the start of the field work as well as during the 
analysis of the data. This was done in order to better understand the SSDI-Services program, its 
implementation context, and its intended outcomes and goals. All documents reviewed are listed in 
Annex 2. 
Qualitative Data 
The qualitative data was collected through key informant interviews at national, zonal and district levels, 
specifically program managers, program supervisors and technical leads from USAID, JHPIEGO, PLAN, 
CARE, MoH and Save the Children representing the management and oversight of the project activities. 
Group interviews with health care providers at facilities, and Focus Group Discussions (FGD) conducted 
with representatives of the Community Action Groups (CAG) and members of the communities who 
utilize the health services and the primary target of the program provided important insights into 
perceptions of the project’s approach and achievements.
Key informant and group interviews 
The evaluation conducted 34 KIIs with staff of the following institutions: MOH (national, zonal, and 
district level officers), USAID/Malawi, SSD-I Services, JHPIEGO, PLAN, CARE, and Save the Children 
staff. KIIs were also conducted with officials from several local organizations involved in SSD-I Services. 
A full list of officials interviewed is included in Annex 3. Key Informant Interviews (KII) provided 
qualitative information on how SSDI- Services’ contributions to improvements in the quality of and 
SSDI-Services Evaluation 6 
access to EHP services, capacity building of health care providers and how the MoH’s capacity to collect, 
access and utilize data for decision making has been built. The KIIs were also useful in exploring attitudes 
on stakeholder relations and challenges to SSD-I Services results achievements. 
The evaluation team conducted 26 group interviews (GI) with average 3-4 health care workers per 
group at 26 SSDI-Services supported health facilities to gain an in-depth understanding of their 
experiences with SSDI-Services program interventions. 
Focus group discussions 
Three sets of FGDs were conducted per facility in the five core districts in the sample. The FGDs 
included the following sets of participants: women resident within 3 km of target facility and those 
within 3-5 km of target facility; men resident within 3 km of target facility; and members of CAGs or 
community care groups. Respondents for the FGDs were selected from areas surrounding 10 health 
facilities in the 5 core districts. Four local experienced data note takers and four facilitators conducted 
all the focus groups. In total, 38 FGDs (13 Men FGD’s and 26 women FGD’s) and documented the views 
of 385 (148 males and 237 females) beneficiaries of SSDI-Services’ interventions. In addition, 38 
Community Score Cards were completed with FGDs. 
Quantitative Data 
The evaluation team also collected primary quantitative data through the use of score cards and a survey 
administered self-assessments questionnaire which was administered to gauge the quality of and access 
to care as well as capacity building to use knowledge and data for decision making. A total of 112 self￾assessment questionnaires were completed. The evaluation team also administered 490 consumer 
questionnaires to clients upon exit from facility utilization on the day of visit to each facility. Additionally, 
the evaluation adopted the scorecard used by SSDI- Services in the communities to gain additional data 
on the communities’ perspective on the quality and access to care. 
Secondary data was obtained from SSDI-services program monitoring performance database which 
provided data on 17 core PMP indicators from 2012 to the end of the project in June 2016. 
Quality Assurance 
To ensure the collection of the highest quality data, the team collaborated with USAID and SSDI￾Services program staff as well as the DEV TECH program manager in the development of the tools and 
selection of sites. The data capturing was conducted concurrently with data collection to ensure data 
quality checks during fieldwork. In addition, a USAID staff member observed evaluation site visits in 
Nkhotakota District and Nankumba HC in Mangochi on 8th and 15 September respectively to observe 
some of the data collection processes. SSDI-Services program in Lilongwe provided data and detailed 
clarification on various aspects of program information on request. 
Data Analysis 
Quantitative data analysis 
Quantitative data was entered into CSpro 6.3 software. The quantitative data was analyzed using STATA 
14.0 statistical software and Microsoft Excel in order to generate graphs. Secondary data analysis 
focused on the performance of the 10 core indicators of the SSDI project outlined in Table 5 of the 
report. 
Qualitative data analysis 
All Qualitative data (focus group discussion and KII) were recorded. The qualitative data was then 
transcribed and translated by the team enumerators and checked for quality by key staff. 
A preliminary thematic analysis was conducted on the transcribed notes based on the primary research 
questions and a review of a subset of transcripts. NVIVO 11 software was used for analysis of qualitative 
SSDI-Services Evaluation 7 
data. The transcripts also included direct quotes by topic that can be used for illustrating trends. The 
qualitative analysis used two primary approaches that included; theme analysis and narrative analysis. 
Theme analysis involved organizing data into categories by identifying recurring themes in the data and 
creating labels under different categories. Narrative analysis was used to examine the relationships 
between codes. 
LIMITATIONS OF DATA 
Limitations in the data collection included: the possibility of recall bias among key informants, group 
interview and FGD participants; the subjectivity of self-reported data; within the limited time in which 
the interviews had to be done some key informants targeted by the evaluation were missed—
particularly with MoH national and zonal officials. 
SSDI-Services Evaluation 8 
IV. FINDINGS AND RECOMMENDATIONS 
EVALUATION QUESTION 1 
To what extent did SSDI improve quality of care for priority EHP interventions at targeted 
health facilities and community service delivery points? Were there any adverse or 
unexpected results on the quality of care? 
1.1 Findings: 
Quality of Care (QoC) has improved significantly at target HFs and community service delivery 
points through Quality Assurance (QA) approaches, service integration and community 
empowerment. 
SSDI-services aims at implementing an integrated service delivery program to expand and improve 
quality of selected essential health package (EHP) services at the facility and community levels. The 
project supported 304 facilities in 15 districts. Over the five years of implementation, the project has 
positively influenced quality of care in the facilities and community service delivery points. 
Table 2 below show changes in some indicators of service delivery over the five year period of the 
project that supports the assertion that quality of care at target health facilities and community health 
delivery points have improved. 
Area of Quality Before April 2012 After March 2016
Maternal deaths 120.1/100,000 births 52/100,000 births
Neonatal deaths 15.1/1000 births 9/1000 births
Asphyxiated newborns resuscitated 52% 90% (Feb 16)
IPT2 uptake 12% 64%
HIV exposed infants tested within 12 months 7% 94%
Number of health facilities with established capacity to 
manage acute malnutrition
98 304
Facilities eligible for quality bonus 1/17 12/17
Table 1: Changes in quality of care indicators SSDI-Services-facility data
Source: JHPIEGO end line report, August 2016
Routine SSDI Services M&E reports using data from facility registers from 15 districts and MOH DHIS2 
database have documented a substantial reduction in maternal deaths from 120.1/100,000 live births at 
the inception of the project in 2012 to 52 /100,000 live births in 2016. Neonatal deaths have been 
reduced from 15.1/1000 live births in 2012 to 9/1000 live births in 2016. No specific targets were set for 
these indicators at start of the project. 
Table 2 also shows that IPT21
 uptake increased significantly from 12% at baseline up to 64% by 2016 
though falling short of the 2016 target of 75%. Ninety percent (90 %) of asphyxiated newborns were 
successfully resuscitated compared to 52% at baseline. HIV exposed infants tested within 12 months 
increased from 7% to 94%, (target 55%) and the number of health facilities with established capacity to 
manage acute malnutrition rose from 98 at baseline to 304 in 2016 above the target of 275. Additionally, 
the number of facilities eligible for quality bonus—an indicator for reaching certain quality improvements 
indicators—increased from 1 to 12 within the life of the project. By the end of the project 18 facilities 
 
11 IPT2- Intermittent preventive treatment second dose. 
SSDI-Services Evaluation 9 
had achieved recognition status for Performance Quality Improvement (PQI) for Infection prevention 
and Reproductive health. [Source JHPIEGO (SSDI-Services FY2014 PMP Final Nov.82013.doc] 
The community management of acute malnutrition (CMAM) has been successful in maintaining high cure 
rates. CMAM OTP– cure rate above SPHERE standards (>75% cure rate, <10% death rate and <15% 
default rate) increase from 80.2 % in April 2012 to 93% in March 2016 (Source: JHPIEGO, End line 
report 2016). 
Observations on substantial improvements in quality of care were also corroborated by perspectives of 
focus group discussions and consumer surveys administered to community members/ clients. About 80% 
of respondents to consumer surveys reported that they were satisfied with quality of health services 
provided to them, and even when disaggregated of gender both men and women were satisfied with the 
quality of care provided by the health facilities. 81% agreed that quality of care has improved compared 
to before. (77% hard to reach and 84 % non-hard to reach) and 90% of consumers felt that their health 
status improved after visiting the health facility. (90% of women and 87% of men) (85% hard to reach 
and 93% in non-hard to reach areas). Eighty eight (88%) of respondents said the quality of services was 
better than before. About 92% of women as against 85% of men said that the services were better than 
before. 
It is clear that there is an all-round impression of vast improvements in the quality of care as compared 
to the pre-project period when drugs stock outs and poor worker attitudes were common place. 
Performance Quality Improvement (PQI) approaches have contributed to improved Quality of 
Care. 
Health facilities and community service delivery points require the availability of equipment, supplies and 
commodities, improved skills of service providers and supervision to be able to meet the quality of care 
needs of their clients. 
SSDI-Services created an enabling environment for the provision of quality care through capacity building 
e.g. training MOH staff, ToTs, procurement of equipment, Infrastructures improvements (33 sites), 
mentorship and supportive supervision and community mobilization and empowerment. 
The project trained service providers in the 6 technical areas including maternal and new born health, 
child health, Family Planning and Reproductive Health, HIV/AIDS and TB, Nutrition and Malaria. MOH 
staffs were also trained in cross cutting areas such as mentorship and supervision. Basic competency 
training, mentorship and supervision provided knowledge and skills for service providers to deliver 
quality services. Mentoring and coaching reinforced skills acquisition and use leading to better quality 
service. 
SSDI-Services also successfully integrated service delivery at the facility level (training of a single service 
provider in multiple skills) such that more services were provide on clients during one visit reducing the 
need for clients to make multiple visits to the clinic. Similarly, the delivery of outreach services was 
integrated paving the way for many services to be provided once. 
SSDI-Services provided service delivery equipment including blood pressure machines, cord clamps, 
autoclaves; Delivery sets; Manual vacuum aspirator (MVA) kits, neonatal resuscitator, drug boxes, 
examination lamps etc. The project also provided infrastructure improvement to created space for 
quality client interactions. These improvements in 33 facilities included partitioning of spaces to improve 
privacy for counseling, making extensions to maternity or delivery rooms to provide space to conduct 
deliveries, building incinerators and placental pits to ensure proper medical waste disposal. 
The project also used a number of quality approaches to address improved quality of service including 
PBI, PQI/SBMR and community mobilization and empowerment. 
SSDI-Services Evaluation 10
PBI is a health financing program that incentivizes quality of care 
and coverage. It encourages the use of standards in delivering 
services. PBI used a number of approaches to influence quality of 
care including provision of equipment, development of skills and a 
bonus for meeting predetermined service quality and utilizations 
standards. The performance incentives encouraged health facilities 
to develop innovative approaches to achieving the quality and 
utilization target indicators (see Annex 7). By the end of the 
project 12 facilities out of 17 targeted facilities qualified for the performance bonus based on the 
achievement of the target indicators. Also when PBI was linked up with the use of the PQI approach, 
many more facilities were able to get recognition status for Infection Prevention and Reproductive 
Health (IP & RH) due to PBI. 
The collaborative approach, where a group of facilities meet to share ideas and experience, has 
engendered competition among facilities to reach recognition status. This has contributed positively to 
improvements in quality of care. However inadequate staff and staff turnover have negatively affected 
the implementation of the PBI as ToTs, trained providers and mentors move out of their positions. 
Community mobilization and empowerment has improved quality of care. 
The SSDI- Services and the SSDI- Communications projects worked together and developed a 
community mobilization strategy to mobilize and empower communities to improve health seeking 
behavior and ownership by planning with the Health Facilities and communities to address community 
health issues. These approaches included the use of the Community Score Card, Community Action 
Groups (CAG), Care groups, Community-based distribution agents (CBDA). The community score card 
provides input into the quality process at the facility while the care groups screen and identify 
malnourished children for treatment. The work of care groups 
expanded from a focus on nutrition messages to cover the 
entire spectrum of EHP. 
Qualitative data substantiated the increase in quality of care. 
Respondents to the KIIs, FGDs, and Group interviews said 
that quality of services had improved significantly pointing to 
some key quality indicators such as survival of asphyxiated 
babies, capacity to manage malnutrition and Early Infant 
Diagnosis (EID) reduction in maternal deaths in the last five years of the project. 
More women are satisfied with the capacity of facilities to meet clients’ needs of than men. 
The Consumer survey showed that about 81% of women were more satisfied with the quality of care 
they received from the facility than men (about 79%). Also about 90% of women felt their health status 
had improved as a result of coming to a facility. No adverse or unexpected results on the quality of care 
were reported. 
“A lesson I have learnt is that “not 
only what goes into my pocket can 
motivate me to change behaviour, I 
can as well change behaviour 
because I have been given working 
conditions that are favourable to 
work.”—Key informant
“Supervision of medical staff by the health 
committee from different villages which 
was formed to take care of the facility and 
the score cards have led medical staff to 
improve their working performance”—
CAG member
SSDI-Services Evaluation 11
Figure 2: Percentage of clients reporting on satisfaction with services
1.2 Conclusions: 
The above findings suggested the following conclusions with regards to the project’s effect on the quality 
of care: 
 Integration of health services reinforced by QA approaches and community empowerment are 
effective ways of improving quality of care at health facilities and community service delivery 
point 
 Use of standards ensures quality service delivery 
 Rewards for achieving standards is highly motivational to service providers 
 Community mobilization and empowerment ensures quality services at facility level 
 Client satisfaction has confirmed that quality of care has improved 
1.3 Recommendations: 
 DHMT should incorporate QA and community empowerment into the routine DIPs 
 MOH and Partners should support the scale up of PQI/SBMR linked to PBI to more facilities 
EVALUATION QUESTION 2 
To what extent did SSDI-Services improve access to priority high impact EHP 
interventions in the target health facilities and community service delivery points? 
2.1 Findings 
SSDI-Services expanded coverage of EHP services and provided opportunities for greater access 
to clients. 
The SSDI-Services Rapid Situational Analysis Report of 2011 determined that the health system faced 
several constraints. Nationally health centers were inadequately staffed with qualified service providers 
and lacked basic equipment to provide primary health care services. The facilities were overcrowded. 
To address these, SSDI-Services decided to expand the services delivered at the community by shifting 
some of the tasks to community based workers. HSAs were trained to provide DMPA injection while 
community based distribution agents (CBDA) provided condoms, pills and cycle beads. 
SSDI-Services Evaluation 12
Training, mentoring and supervision, provision of commodities, supplies and equipment improved the 
quality and availability of services at Health Facilities. SSDI-Services worked with staff at health facilities 
to re-organize the services, integrating them where applicable enabling clients to access as many services 
as possible in a single visit to the facility. This increased the range of services (a pregnant woman could 
access ANC services and also HIV testing services) provided at the facility level. For instance men were 
encouraged to accompany their wives to the antenatal clinic (ANC) reinforcing male involvement. 
Similarly expansion of access to services occurred through the conduct of Open Days and Youth Clubs 
for Adolescents where Orientations to and uptake of sexual and reproductive health services enabled 
adolescents and the youth to take decisions to protect their health. The involvement of Care Groups 
and CAGs promoted by the SSDI-Services project ensured that the project reached out to multiple 
beneficiaries. The project supported the SUN activities in the communities through the Care Groups 
who worked in their cluster households to promote optimal nutritional practices, Family planning, Early 
ANC, malaria prevention, HIV/AIDS messages, cooking demonstration and principles of hand washing. 
As shown in figure 3 below the number of health facilities with established capacity to manage acute 
under-nutrition (CMAM) increased from 98 at baseline to 304 at end line. Care Groups contributed to 
CMAM objectives of focusing on maintaining the case outcomes above the sphere standard ->75% cure 
rate, <10% death rate and <15% default rate. Scale Up of Nutrition (SUN) intervention was scaled up 
from zero districts at inception to 11 districts. In addition the number of service delivery sites that offer 
Long acting and Permanent Methods (LAPM) has doubled from 129 sites at inception to 251 while 
access to EID Services measured by percent of HIV-exposed infants tested within 12 months increased 
from 7% in 2012 to 94% in 2016 as portrayed in figure 4, and the number of HF that provide BEmONC 
increased from 50 to 87. 
Trained HSAs have improved communities access to CBMNH services through village clinics. 
HSAs were trained in Community Based Maternal Neonatal Health (CBMNH) at the community level 
and man the village clinics. These have had a palpable impact on services at the community level. As 
shown in Figure 5 the number of functional village clinics providing iCCM increased significantly from 
145 at baseline to 1721 at the end of the project providing greater access to health quarter during 
services in the communities. 
However limited number of HSAs coupled with the lack of accommodation in the communities and 
adequate supervision remain challenges. These HSAs are expected to live in the communities where 
they work but often have no accommodation. This hampers access to services when they are away from 
the community. [Source: JHPIEGO Endline Report] 
Figure 3: Trend of HIV-exposed infants 
tested for HIV within 12 months
Figure 4: Trend of number of HF with 
established CMAM
SSDI-Services Evaluation 13
Besides, KII and GI respondents said that when there are staff shortages, some HSAs are often taken 
from the communities to work in the health facility further reducing access to services in the 
communities. 
KII respondents also expressed concern about the challenges 
with supervising HSAs. Supervisors often had no means of 
transport to go on supervisory visits and mentors would rather 
call the HSAs to the Health center for the mentoring sessions 
which also leaves the communities without access to services. 
SSDI services increased access in hard to reach facilities 
and communities but distances and transport are still 
major barriers to access in many places. 
Integrated Family Health Outreach Clinics (IFHOC) were conducted in hard to reach areas. An average 
of 100 IFHOC was supported monthly. This significantly improved access to services in these hard to 
reach areas. However long distances for some communities and lack of transport to reach these areas 
limited access to referral and outreach services. 
For instance 203 out of 304 have no ambulance of their own or have no access to ambulance in their 
immediate proximity. (Source JHPIEGO database on facilities, utilities and interventions). KIIs, GI participants 
said lack of transport has limited referral and outreach activities. 
Family Planning Campaigns- this approach 
has improved access to FP services 
significantly especially for areas where 
there are no FP services e.g. in the 
Christian Health Association of Malawi 
(CHAM) catchment areas where health 
facilities do not provide FP service for 
religious reasons. KII respondents said this 
approach was used in Balaka district 
successfully. 
LAPM at Health Posts: As part of getting 
services closer to clients SSDI-Services 
expanded delivery of Long Acting and 
Permanent family planning methods at 
community health post–the lowest level of 
the health. Couple Years of Protection for 
LAPM increased from 1,716 in the third 
quarter of 2012 to 70,969 at end of project 
March 2016. 
Figure 5: Trend of number of functional village clinics 
providing iCCM
“Increase health worker providers because 
they are few and are attending a huge 
number of people which is tiresome and it 
delays people from receiving help in time of 
which others thinks they are not well 
received at the facility, Government should 
train more HSAs to work in the 
communities”—Woman client
SSDI-Services Evaluation 14
Performance Based Incentives (PBI) improved access to services. 
PBI Scheme aims to increase access to and utilization of selected Essential Health Package (EHP) services 
by providing performance incentives to encourage health facilities to develop innovative approaches to 
achieving predetermined quality and utilization targets. Health facilities develop business plans that they 
implement over a defined period of time. As they implement they also assess their achievements against 
the set targets. An external team is also involved in the assessment of achievements and a bonus is 
awarded for meeting the predetermined service utilizations standards. This has had the motivational 
effect of making service providers work harder for their facility to earn the bonus. To ensure that they 
are on course to earning a bonus facilities have reached out to the communities in their catchment areas 
to provide services and encourage them to access services in their facilities showing the improvements 
in their services. Facilities participating in the PBI initiative also meet to share, ideas and experience and 
learn from each other what works and what does not work. This has generated a healthy competition 
to increase access to services and reach recognition status or earn bonus. When PBI was linked up with 
the Standards Based Management and Recognition (SBMR) program, many more facilities were able to 
attain the recognition status due to the motivation and accountability factors facilitated through the PBI 
process 
However inadequate staff and staff turnover have negatively affected the implementation of the PBI 
scheme as trained staff move on. By the end of the project 12 facilities have so far achieved bonus status. 
Community members agreed that SSD-I Services has increased their access to EHP services. 
A review of the qualitative data shows that about 93% of respondents on the client score card agreed 
that access to a skilled provider for pregnancy and birthing services had improved (95.6% women and 
90.9% men agreed). Similarly, about 95% of respondents agreed that people are able to access health 
services for their babies, infants and children better than before. Of these 96% of women and 93 % of 
men also agreed. 
Figure 6: Trend of CYP through CBD
SSDI-Services Evaluation 15
Ninety three percent of respondents agreed 
that people are able to access contraception, 
other family planning methods and abortion 
care services. Of these 96% of women agreed 
as opposed to 90% men. Most respondents 
94% agreed that people are able to access 
HIV/AIDS and TB services in their community 
(women 94% and men 94%). 
Health providers and key informants said men, 
women and people with disability had equal 
access to quality services. However the data 
from respondents on the score card shows 
that people with disability had less access to 
quality health care services than men and 
women 
2.2 Conclusions: 
Based on the findings above following conclusions are made: 
 The approach of integration of services delivery, outreach clinics and CBS made it easier for 
communities to access high impact EHP interventions 
 HSAs that were trained to provide CBMNH have made an impact on access to services in 
communities 
 Irregular transport (vehicles and fuel) adversely affected Outreach clinics and referrals from 
health facility to District Hospital 
2.3 Recommendations: 
 Integrate outreach clinics and CBS into each district implementation plan, progressively extend 
outreach clinics and CBS to more hard to reach areas 
 DHMTs should explore the possibility of Local government and / or communities supporting the 
construction of accommodation for HSAs 
 MOH and Partners should ensure that more HSA’s are trained in CBMNH for greater impact 
 Partners and DHMT should integrate their activities and implementation schedules for more 
efficient use of transportation 
EVALUATION QUESTION 3 
To what extent did the direct involvement of MoH staff to facilitate the in-service training 
of health practitioners, on-the-job mentoring and quality assurance strengthen the MoH’s 
capacity to institutionalize these components as part of routine delivery of high impact 
priority EHP services? 
3.1 Findings:
The project built capacity for utilization of TOT, mentorship and supervision. 
The integrated capacity building approach targeted change in skills, attitudes, and motivation of providers 
in order to improve service delivery to clients. The MOH and the DHMTs played a strategic role of 
identifying training gaps during planning and facilitating implementation of the capacity building process. 
The SSD-I Services training program effectively supported the MOH in establishing a critical mass of 
trainers and personnel qualified in mentorship and supervision of health services. A total of 1130 health 
Figure 7: Access to quality services for men, 
women and people with disability
SSDI-Services Evaluation 16
care providers were trained as Trainer of Trainers, mentors and supervisors. SSD-I Services also 
directly trained health providers in specific technical areas, including FP, CBDA, BEmONC, KMC, iCCM, 
CMAM and iPT2. The contribution of SSDI-Services to the capacity building also included development 
of curricula, for example, for trainings given to the MOH management level staff, and a series of 
provider level trainings that emphasized the importance of mentorship and supervision roles. Currently 
the positions of ToTs and mentors are well recognized and appreciated by the entire MOH system 
according to the KIIs. 
Trainers were being utilized in mentorship, supervision and follow-up activities. 
SSD-I Services positively affected the utilization of the ToT, mentorship and supervision initiatives by: 1) 
placing a strategic focus on addressing specific training needs 
at each tier of the health system; 2) aligning the capacity 
building to MoH Policies; and 3) revising and updating existing 
MOH curricula for service delivery as opposed to creating 
project-specific materials. Trainers were particularly effective 
at imparting knowledge and skills for keeping track of 
program indicators to program coordinators in various areas, 
such as Safe Motherhood, while at District and Facility levels 
mentorship and supervision benefitted service providers in 
EHP services. 
GI interviews revealed that for those who received training, 
mentorship and supervision were highlighted as more 
effective instruments of change on quality of care as these approaches have more lasting impacts on the 
service providers’ abilities. Although theoretical in-class lessons were appreciated, it was the long-term 
reinforcing and incentivized processes to go along with mentorship and supervision that had long lasting 
effects on the health workers. SSD-I Services provided the resources for mentorship and supervision to 
be conducted on an ongoing basis. Ironically this could also be the undoing of the recently created 
capacity as the MOH faces massive challenges in resourcing mentorship and supervision on a routine 
basis in the absence of partner funding. Staff movements and competing priorities of staff time due to 
multiple externally supported activities within facilities and community service delivery points are also 
perceived as major barriers to sustainability of these initiatives according to MOH KIIs. 
Supervision of health facilities in SSD-I districts considerably increased. 
As shown in Figure 8, supervision activities increased dramatically during implementation of SSD-I 
Services. The project sought to influence trainings, mentoring and introduction of specific performance 
management standards and checklists variably applicable to different programs and levels of service 
delivery. Zone and District level managers had a structured program for supervising facility and 
community level service delivery. These were reinforced by supportive resources including finances and 
transportation which consequently improved the number of supervision events across the districts. 
However, frequency and quality of supervision depends on a number of factors including district specific 
factors such as leadership and availability of vehicles. The HF was supervised by district team using 
integrated supervision checklist at least once per quarter through this capacity increased from 48 in 
2011 to 225 in 2016. Concurrently, the number of districts supervised by zonal officers using integrated 
supervision checklist at least once in a quarter also increased from 10 to 15 over the life of project. This 
focus on supportive supervision suggests SSD-I Services made considerable investments to 
institutionalize routine quality assurance and quality improvement processes. 
“SSDI services introduced a number of 
capacity building activities like: mentorship 
on HBB, KMC, and antenatal, trainings for 
HMB and KMC and orientations on 
integrated management of maternal and 
neonatal care. Trainings in IP, five S 
training, BEmONC and CEmONC training, 
essential care of every newborn trainings,
training HSAs in providing quality 
community based maternal and neonatal 
care.”—Health worker
SSDI-Services Evaluation 17
Major concerns voiced during interviews with 
MOH staff suggest that the capacity of MOH to 
institutionalize supervision at the current level 
remains very weak. All levels involved in 
supervision activities continue to face funding 
bottlenecks in the absence of support from 
partners. MOH staff at zonal and district levels may 
also benefit from prioritization and integration of 
their supervision activities. 
Community score cards and community-based 
volunteers facilitated demand side contribution 
to improved quality of service delivery. 
SSDI-Services used community score cards (CSC) 
to engage communities in situation analysis of their 
respective facilities and trigger corrective measures 
or improvements on service delivery. The 
scorecard specifically targeted demand-side 
influence on the facility. At the end of the project 
42.6% of facilities are implementing CSC in the five Core Districts and 17.1% in other Districts. Some 
critical CSC achievements include the following: 
Improved DHMT and CHAM commitment which lead to the deployment of staff in almost all facilities 
where CSC rolled out, the provision of ambulance where needed, greater collaboration and partnership 
with other sectors which contributed to the provision of water, electricity in some districts. 
 Community involvement at facility has also induced infrastructure improvement like facility 
roofing, sanitation and contributed to providing electricity to some facilities 
 Health worker attitude has greatly improved. 
 New HACs were formed and trained and old ones revamped 
 Ninety one percent of consumers reported that score cards have enabled health workers to 
better meet community needs in hard-to-reach areas 
Community capacity comprising Community Action Groups (CAGs), Community-Based Distribution 
Agents (CBDA) volunteers and care groups working alongside the HSAs were empowered by the 
capacity building interventions to effect change in quality of service delivery. Care groups were 
particularly active in under nutrition assessments of under-5 children and greatly contributed to the
effectiveness of the nutrition program interventions. These interventions enabled SSD-I Services to 
reach 498,759 children with nutrition-related activities. 
Table 2: Community level capacity development for CBDAs 
It is not easy to judge the potential for sustainability of the community voluntary processes but the 
immediate results of SSDI intervention in this dimension are encouraging. Stakeholder comments in both 
KIIs and FGDs, as well as secondary quantitative data gathered on specific indicators associated with 
community approaches such as FP, Maternal Mortality and CCM, indicate that the scorecards and 
Figure 8: Trend in supervision from district to HF
CBDAs trained in FP Year 1 Year 2 Year 3 Year 4 Year 5
female 73 23 10 2 na
male 125 33 11 5 na
Grand Total 198 56 21 7 na
SSDI-Services Evaluation 18
voluntary contributions have made a significant positive contribution to delivery of high quality services. 
For example training was given to the 108 female and 174 male CBDAs in the first four years of the 
intervention. 
Capacity institutionalization is faced with funding challenges. 
During the implementation phase of capacity building activities change in policy on allowances, change in 
operational strategy and a reduction in resources allocated to activities were the notable factors to limit 
capacity building. The policy on allowances had negative consequences such as low attendance or 
complete cancellation of some of the planned training sessions. Discussions with key MOH staff showed 
that frequent staff movements in the form of departures and transfers and inadequate staffs in key 
positions owing to government freeze on filling vacant positions has had both short term and long term 
effects on the project. The districts and facilities are also faced with perennial transport challenges in 
terms of fuel and vehicles required for supervision activities. The weak government funding environment 
and staffing shortages have long term effects on the momentum for institutionalization of the capacity 
created by SSDI. 
3.2 Conclusions: 
SSDI-Services had the following areas of influence on building capacity of MOH; 
 Capacity was successfully created through training, mentorship and supervision 
 The MOH currently has ample numbers of ToTs and mentors 
 There was satisfactory utilization of the capacity in specific program areas. Phase out of SSDI 
project 
 A poor government funding environment is the biggest threat to institutionalization of ToTs, 
Mentorship and supervision 
 Supervision activities increased considerably over the life of the project. 
 On the demand side, the scorecards and voluntary contributions have made a significant positive 
contribution to delivery of high quality services 
3.3 Recommendations: 
Recommendations to strengthen capacities are the following: 
 MoH should decide whether mentoring approach to strengthen health personnel knowledge and 
skills should replace the classical training approach considering that mentorship appears to be 
more effective in improving staff skills 
 MOH should ensure to create and update a database of the available TOTs and Mentors 
 For routine utilization of the built capacity strategies, MoH and partners should ensure 
availability of supportive resources 
EVALUATION QUESTION 4 
To what extent did SSDI’s investments in performance monitoring and Health Information 
Systems (HIS) improve the MoH’s (national, district and facility) capacity to collect, access 
and utilize routine service delivery data for decision making? 
4.1 Findings: 
SSDI supported the migration from paper based reporting to DHIS 2; this included provision of 
equipment. 
SSDI-Services Evaluation 19
Performance monitoring and Health Information Systems strengthening is a core component of SSDI￾Services interventions. Strategically SSDI- Services placed routine service delivery data collection and 
utilization for decision making as a priority issue for building the HMIS capacity of all the MoH, Zone, 
District, Health Facility levels. This included the introduction of new technologies such as DHIS2 and 
mobile phone reporting systems which induced the migration of paper based data reporting to an 
electronic and standardized one, the reinforcement of capacity of staff through training, the provision of 
adequate equipment and materials and on-going performance monitoring at all levels. 
Staffs at different levels are more skilled, equipped and motivated to report routine data and use them 
to monitor performance and make decisions. This investment significantly improved the data processing 
chain from HF to District and up to the MoH. For example as shown in Table 3 below, data reporting 
from HF to District increased from 7 percent at the beginning to 83 percent at the end of the project. 
HMIS has been strengthened at the national level. 
At the national level, SSDI-Services invested heavily in the DHIS2 (e.g. training at Central, Zonal and 
District levels, equipment procurement and database management). DHIS2 which was originally housed 
by the Malawi College of Medicine became nationally accessible after the system was migrated to and 
rolled out under CMED in the MOH Headquarters. SSDI-Services’ strengthening of the CMED (through 
staff training, technical and financial support to develop HIS Policy and Strategy documents, technical 
support for setting National Data Standards, Technical working groups and National data collaborative 
system, Technical support to review and print Registers and forms as well as provision of computers 
and routers) has enabled the MOH to conduct quarterly supportive supervision and quarterly data 
quality assessments at zonal level. 
Some key informant interviews with CMED, Zone managers and some HMIS Officers suggested that the 
evolution of the DHIS2 had hitherto focused on data reporting for performance monitoring at HF, 
district and zonal levels and less so on than data quality and accuracy issues and strengthening the 
practice of making decisions based on data. There is no evidence that the system of routine DQA for 
the 15 supported districts established by SSDI- Services has yet improved the quality of the data and a 
study on this issue should be conducted. 
The project built the capacity at district and HF levels for data collection, reporting, entry in 
DHIS2 and utilization. 
Following a comprehensive needs assessment undertaken at various levels of the health system, SSD-I 
Services supported trainings of MOH staffs, including the DHMTs, HMIS personnel, program 
coordinators and health care providers on data collection, reporting and utilization. These trainings 
were integrated with broader elements of ToTs, mentorship, supervision, data entry into DHIS2 and 
application of tables and graphic analyses that would guide performance and decision making at all levels. 
Apart from reinforcing knowledge and skills of health 
personnel, SSD-I Services also invested in equipment and 
materials to foster data utilization in the MOH. Resources 
were allocated to printing and distribution of registers, double 
reporting forms and booklets, acquisition of and setting up the 
DHIS2 system, as well as provision of airtime, computers, 
routers, stationery to district HMIS staff and mobile phones 
for rapid reporting by HSAs in hard to reach health facilities. 
This has contributed to change providers and DHMT attitude 
and practices towards data. At HF level, trained providers are 
able to use the registers and forms for reporting and also 
monitor their own performance. Health workers in hard to 
“MOH capacity to collect and utilize data 
has been addressed, the issue of registers, 
tapes, graphs, hardboard, health passport 
books, and monthly reports maternity 
register has also been solved but 
submission to central server still a 
challenge. There is still some issues with 
quality of data which requires us to conduct 
quality assessments.”—District HMIS 
Officer
SSDI-Services Evaluation 20
reach areas have been equipped with, and trained to use mobile phones to transmit their monthly 
reports. At District level DHOs, programs coordinators and other DHMT members have been granted 
access to the DHIS2 system data and be able to generate performance monitoring of interest. 
Regular data collection and reporting are being institutionalized at district level. 
In order to improve and sustain data reporting, DHMTs and SSDI-Services implementing partners at the 
districts supported monthly M&E Mentorship, reporting, filing, data completeness and analysis, data and 
charts displays by making vehicles, fuel and allowances available for these activities. HMIS staff and 
program coordinators were always central to the conduct of data handling and reporting. The district 
management teams also provided support that ensured continued functionality of the mobile health 
reporting system. In addition Data Quality Assessments taking place at district level were also done at 
zonal level albeit on a quarterly basis for the latter. 
Generation of and access to data has been increased, and data review and analysis meetings are 
conducted every quarter. 
In the SSDI-Services target districts, DHIS2 data reporting and completeness from health facilities to the 
district level has significantly improved from 7 percent in project year 1 to 83 percent in project year 3 
and has been maintained at this level till the end of the project. Reporting from district to zone levels 
has also improved from 40 percent to 80 percent (see Table 3 below). There was clear evidence in the 
form of numbers and data charts on display in most HFs that data reviews were being conducted 
although the intervals at which these were being updated varied from one facility to another (observation 
from HF visited during evaluation Aug – Sept 2016). 
Table 3: Data collection and reporting indicators 
Area of data collection, reporting and use BEFORE
April 2012
AFTER
March 2016
HFs meeting DHIS2 improved reporting requirements 7% 83%
DHIS2 establishment (DHIS2 mobile) 2 districts 10 districts
Improved reporting from district to zone level 40% 80%
Source: JHPIEGO, endline report August 2016
In terms of use, Group Interviews with facility staffs as well as KIIs with DHMT members indicated that 
HMIS staffs are mainly responsible for entry of the data in DHIS2 on a monthly basis while DHOs and 
program coordinators are the key staff that access DHIS2 for use. 
Access to data has helped improvement of activities and identifying areas of poor performance of service 
delivery indicators that require corrective action. In addition to DHIS2 quarterly data review meetings, 
the evaluation found that the District Implementation Planning was another important process that used 
this data. During biannual DIP review DHMT used data to update, adjust or change activities being 
implemented. 
Some challenges tend to undermine SSD-I Services effort to strengthen HMIS. 
There is a shortage of statistical clerks at HFs and in most cases Senior HSAs are carrying out data 
collection roles and the setback is that they perceive data collection and reporting as something that 
only draws them away from their core activities. Lack of electricity and poor network connectivity are 
other barriers to access to DHIS2 on the one hand, and as well as preventing health providers in the 
hard to reach HF from sending data via mobile phones in time. In some cases staff have to use public 
SSDI-Services Evaluation 21
transportation to send data to the district level when feasible. Several districts reported of shortage of 
vehicles and/or fuel that resulted in HMIS Officer and program coordinators being unable to complete 
regular supportive supervision, mentorship and data quality assessments. 
4.2 Conclusions: 
 Investment in training and mentorship, appropriate equipment and technology as well as 
supportive supervision has contributed to create capacity to collect, access and utilize routine 
service delivery data and HF staffs are collecting routine data using registers. 
 In most HF senior HSAs are carrying out data collection due to lack of statistical clerks. 
 Data is being used at district and zone levels as data is being collected, processed and accessed 
by DHMT and also available on line through DHIS2 country wide. 
 Data quality and utilization at national, district and HF levels are being challenged by lack of staff, 
and there is need for a lot more work to be done on addressing quality of data. 
 Owing to the limited quality of the data being passed up the system, many potential program 
level users have been reluctant to use DHIS2. Consequently, data generation and management 
processes being supported by various partners running in partial or completely unintegrated 
mode with the DHIS2 are viewed to produce more reliable data outputs. The MOH and 
partners are aware of the effects of running various systems on the success of DHIS2 and efforts 
toward harmonization of the multiple systems are being slowly tackled and there is hope that 
the national DHIS2 will be transformed into a more useful HMIS. 
4.3 Recommendations: 
 DHMT should focus more on supervising data generating and utilization, and on data quality 
assessment at the facility level. 
 MoH should provide each HF with a statistical clerks. 
 MoH should integrate data quality assessment into supervision and mentoring activities. 
 DHMT to increase providers and clerks’ knowledge and skills on data quality through 
mentorship. 
 MoH and Partners to improve integration of parallel health related data systems with DHIS2 for 
synergies e.g. Baobab system. 
EVALUATION QUESTION 5 
What are the most significant accomplishments, best practices, and lessons learned from 
the SSDI-services activity? Explicitly identify and document facilitating and inhibiting 
factors to positive performance for each of the above questions. 
5.1 Accomplishments 
Quality of Care: 
 Quality of Care (QoC) has improved significantly at target HFs and community service delivery. 
 Performance Quality Improvement approaches (e.g. PQI, AQ and PBI) have contributed to 
improve QoC. 
 Community mobilization and empowerment through Score Card, CAGs and Care Groups have 
contributed to improve QoC. 
Access to Care: 
 More health services were offered as a result of SSDI which in turn provided opportunities for 
greater access to clients. 
SSDI-Services Evaluation 22
 Trained HSAs have improved communities access to CBMNC Services through Village Clinics, 
FP injectable method and CBD. 
 SSDI services increased access in hard to reach facilities and communities but distances and 
transport are still major barriers to access in many places. 
 The PBI scheme has increased access to and utilization of selected EHP services, improved the 
quality of health services by improving working conditions and recognition. 
Capacity Building: 
 The project built capacity for utilization of TOT, mentorship and supervision. 
 Trainers are being utilized in mentorship, supervision and follow up activities. 
 There is a substantial increase in number of health facilities in SSDI districts receiving one or 
more supervision visits per quarter. 
 Community capacity through utilization of score cards and community based volunteers has 
facilitated demand side contribution to quality service delivery. 
HMIS Reinforcement: 
 SSDI supported the migration from paper based reporting to DHIS 2, this included provision of 
equipment. 
 HMIS has been strengthened at the national level. 
 The project built the capacity at district and HF levels for data collection, reporting, entry in 
DHIS2 and utilization. 
 Regular data collection and reporting are being institutionalized at the district level. 
 Generation and access to data has been increased, and data review and analysis meetings are 
being conducted every quarter. 
These accomplishments have significantly improved SSDI-Services performance indicators in the 304 
supported facilities within the 15 SSDI districts as presented and discussed in earlier sections. Table 4 
shows how the interventions have impacted the 10 core performance indicators. 
Table 4: Performance achievements on the 10 core indicators 
Core Indicators (from 304 SSDI supported facilities) Before
April 2012
After
March 2016
Number of service delivery sites offering LAPM 129 251 
Number of sites/district providing BEmONC 50 87
Couple-years of protection
 Through CBFP
 Increasing CYP since inception (cumulative) 
8,298 
41,383
31,975
1,851,669
Percent of pregnant women attending ANC that were tested for HIV 66 92
Number of facilities with capacity to manage acute under-nutrition* 145 304
Malaria: Percent of pregnant women at ANC who received 2nd IPT** 12.6 64
HIV/AIDS: Percent of HIV positive pregnant women initiated on ART 89 95
Mentoring and PQI: percent of facilities receiving at least one supervisory 
visit per quarter. 
--
(Not available)
225
Source: JHPIEGO: Endline Report, August 2016; * figures from USAID, 2016; ** source is the facility registers in 
SSDI-supported facilities 
SSDI-Services Evaluation 23
5.2 Contributing Factors 
Several factors have facilitated these accomplishments. Among these factors are: 
 The complementary and interactive design of the program which targeted the main pillars of the 
Health System and the presence of the communication for behavior change initiative. 
 The focus of services delivery at lowest levels where the target population is present and the 
contribution of performance monitoring and evaluation activities involving multiple partners 
including those from other sectors. 
 SSDI-Services was implemented within the MOH –HSSP. This let the MOH take leadership. 
Furthermore, SSDI-Services facilitated the operations with resources and reinforcing the DHMT 
leadership capacity at facility and community level, through provision of supplies, the 
empowerment of the community structures such as use of the score cards, outreach clinics, 
village health clinics. 
 Integration of services and Outreach clinics: Providing services in an integrated manner ensure 
that clients receive services at one go. Outreach clinics bring services to the hard-to- reach 
areas that contributes to increases in access to services. 
 Use of HSAs and Volunteers, Score Card Approach, Involvement of CAGs and Care Group: 
The use of community approaches like HSAs and care groups, CAGs provided an opportunity 
for leveraging community resources and also get input from the community about quality of 
services. 
 CBDAs and other voluntary roles have contributed outstanding results in improving and 
extending access to quality care. For example an outstanding achievement was attained by Care 
Groups who were able to reach 4, 498,759 children with nutrition related activities. 
 Incentive Schemes (Recognition/PBI): Rewards and recognition of facilities motivated services 
providers to achieve better outcomes. 
 Provision of supportive resources by SSDI such as reporting booklets, Computers, Routers, 
Flash Disks, Air Time, fuel for transport and Phones with priority to hard-to-reach areas. 
 Good awareness and adherence of Staff achieved through mentorship and supportive 
supervision have improved performance. 
 DHMTs routinely financed internet connectivity. 
5.3 Inhibiting Factors 
Some factors have inhibited the performance of SSDI-Services or at least hampered the expected 
outcomes. Most of them are environmental factors that are embedded in the Malawi health system, such 
as: 
 The weak government funding environment and staffing shortages have long term effects on the 
momentum for institutionalization of the capacity created by SSDI. 
 High staff turnover, competing tasks and parallel implementing plans are highly affecting 
continuity of project activities. 
 Change in policy on allowances, change in operational strategy and a reduction in resources 
allocated to activities were the notable factors during the implementation phase of capacity 
building activities limit capacity building. 
 Lack of electricity hampered the quality of service delivery. 
 Transportation for referrals: Transport for service delivery (outreach clinics) and referrals was 
inadequate due to lack of ambulance for referrals in some health facilities and road conditions all 
had negative influence on the quality of service delivery. 
SSDI-Services Evaluation 24
 Shortage of technical and clerical staff to operate DHIS2, frequent power outages, inadequate 
computers and unreliable internet connectivity undermined the potential of SSDI-Services 
Investment on data collecting, reporting and utilization. 
Some of the inhibiting factors are inherent in the design of the SSDI-S strategy; in particular respondents 
to KII interviews pointed to SSDI being thinly spread to too many districts with a wide range of 
interventions (starting with 10 and extended to 15) as one of the main inhibiting factors. They observe 
that it is difficult to conclusively register immense impacts at population levels with such a spread, a 
model focusing on fewer districts with full coverage of those districts would be preferred. Respondents 
felt there was a missed opportunity to achieve the highest outcomes level in the project’s theory of 
change due to the reduced funding in latter stages of implementation without a matching recourse on 
scale of operations. 
5.4 Best Practices 
The following were cited as best practices: 
 Establishing 15 HFs that met specific quality standards as centers of excellence 
 Using mentorship as a capacity building approach 
 Using Standard Based Management and Recognition for improving quality of care 
 Conducting facility data review meetings 
 Camping 4-5 days in hard to reach areas for service delivery 
 Use of Data Reporting Booklets 
 Use of Phone Application 
 Use of Community Score Cards, Community Action Groups (CAGs) and Care Groups 
 Employing participatory approaches at designing activities and selected implementation 
stages. 
As shown by Figure 9; “Capacity building (Training and Mentorship) in provision of quality care” was 
recognized as a best practice by 19% of all respondents to the self-assessment questionnaire , followed 
by “Improvements in all EHP services, (MNCH, FP, Male involvement in ANC, Malaria testing etc.)” 18%, 
“Improved data management and completeness of reporting” 12% (District level staff: 21,4%), “Follow 
ups and Quality supportive Supervision” 11% and “Planning and implementing together” 11% whereas 
successful and popular approaches as “Awards of excellence and PBI” scored 5%, “Comprehensiveness 
of Community based services and Community involvement” 3% and “Nutrition Assessments” 1%. 
Figure 9: Best practices reported by stakeholders (%) 
SSDI-Services Evaluation 25
Figure 10 below shows best practices in different areas as attained by various HFs and how they were 
perceived by different categories of stakeholders. 
Figure 10: Difference in best practices reported by stakeholders 
For example, Nutrition Assessment, Review meetings and facility monthly meetings, Comprehensiveness 
of Community based services and Community involvement, Follow ups and Quality supportive 
Supervision as well as improvement in all EHP services were fully acknowledged by Health Facility staff 
as Best Practices. District level staff appreciated integrated planning and implementation between 
partnerships, improved data management and completeness of reporting as best practices whereas 
Zonal staff recognized Award of excellence and PBI, integrated Services and collaboration among staff 
members as well as infection prevention standards adherence and capacity building as best practices. 
These results provide insights into the level at which each category of best practices should be 
coordinated, sustained and institutionalized. 
5.5 Lessons Learned 
Central level: 
 Investment in integrated service delivery in the National Health System has potential for 
reaching tremendous outcomes. 
 Inclusive partnerships from initial design stages is conducive to successful implementation and 
outcomes. 
 Coordination and harmonization of implementation for many partners is very difficult to 
achieve. 
 The Malawi health system is not yet ready to independently sustain implementation of activities 
hitherto supported by SSDI after project phase-out. 
 Service integration and PBI are effective approaches to improve quality. 
District level: 
 By focusing on data quality HFs are enabled to improve other technical domains too. 
 PBI improves provider’s confidence and increases participants program ownership and financial 
capacity to acquire and maintain equipment, infrastructure and supplies.
 
SSDI-Services Evaluation 26
GENDER ANALYSIS 
The evaluation critically looked at the potential positive or negative impacts of SSDI-Services activity on 
gender. Providers reported that there was no gender imbalance in the EHP interventions delivery at 
district hospital, health centers and village clinics. Health services are available and accessible to 
everyone regardless of gender even though health services have been designed and organized for 
women and children as the priority groups. 
In addition the percentage of women is highest at service delivery level except for HSA cadre where 
men are predominant. Most of the RH officers were female while most DHOs and program 
coordinators were male, these are inherent systemic biases not necessarily a consequence of the SSDI￾Service delivery interventions. 
At community level more females have participated in community mobilization from score cards, CAGs 
and Care Groups during the first two years of the program implementation. Men came in later as a 
deliberate effort to lower female influence. Communication activities implemented by SSDI-Services 
targeted both men and women and increasing number of men involved in care provision, mainly in FP at 
HFs and village clinics. SSDI-Services had no impact nor did it intend to impact on issues related to 
disability. 
SSDI-Services Evaluation 27
ANNEXES 
Annex 1: Evaluation Statement of Work 
Annex 2: Summary Evaluation Methodology 
Annex 3: Data Collection Instruments 
Annex 4: People Consulted 
Annex 5: Documents and Sources Reviewed 
SSDI-Services Evaluation 28
ANNEX 1. EVALUATION STATEMENT OF WORK 
SECTION C DESCRIPTION/SPECIFICATION/STATEMENT OF WORK 
C.1 SSDI ACTIVITIES EVALUATION-SERVICES
EXECUTIVE SUMMARY 
The Support for Service Delivery Integration (SSDI) is USAID/Malawi’s flagship health activity 
to support the increased availability and quality of the Essential Health Package (EHP) services, 
reinforce health promotion and disease prevention among households, and strengthen elements 
of the health system to sustain effective EHP delivery throughout the 28 districts of Malawi. 
SSDI consists of three interrelated sector activities: Services, Communications, and Systems. 
The Services activity provides an integrated service delivery program by improving the quality 
of priority EHP services at the community- and referral-levels. Communications is a social and 
behavior change communication (SBCC) activity to promote normative and individual behavior 
change in the priority areas of HIV and AIDS, maternal and child health (MCH), malaria, 
nutrition, water and sanitation, and family planning (FP). The mission of the Systems activity is 
to assist the Ministry of Health (MoH) to improve policies, management and leadership, and 
fiscal responsibility to advance Malawi’s health system and the sustainable impact of the EHP. 
The integration of all three activities within one broad health initiative aims to achieve the 
USAID/Malawi Country Development Cooperation Strategy (CDCS) strategic goal to improve 
the quality of life for Malawians through the improvement of social development, the increase in 
sustainable livelihoods, and the exercising of citizen rights and responsibilities. 
The following sections discuss in detail DevTech Systems’ approach to conduct the final 
performance evaluation of the SSDI activity. DevTech’s formula in developing effective 
approaches to obtain valid and reliable findings has been proven in more than 150 successful 
evaluations completed over 30 years. For the SSDI evaluation, DevTech’s approach will be built 
from the following core foundations: 
Extensive Experience in Health Systems. The DevTech Team consists of proven 
implementation and evaluation specialists in the areas of health systems strengthening and 
service delivery from the national- to community-levels in Malawi and other countries. 
Combining the strengths of each member brings a strong foundation of health systems 
qualifications in key functional areas including, but not limited to, policy formulation, program 
design, innovative service delivery, implementation and management, behavior change, and 
performance monitoring and evaluations. The DevTech Team also brings experiences in key 
health sectors such as HIV and AIDS, MCH, malaria, nutrition, water and sanitation, and FP. 
This rich set of qualifications will bring the needed depth and rigor to evaluate an expansive 
initiative such as SSDI.
SSDI-Services Evaluation 29
Integrated Evaluation Approach for Valid and Reliable Findings. The SSDI consists of a 
broad range of interventions that are interrelated across the three components and are 
implemented from the zonal- to the district-levels. Thus, the overall evaluation approach will 
integrate one activity evaluation design with the other two where appropriate to have more 
accurate assessments of SSDI outcomes and results. Below represents our Thematic Areas of 
Inquiry.
One Team Approach for Technical and Management Efficiencies. Along with an integrated 
design, the DevTech evaluation team is structured along a “one team approach” that leverages 
complementarities among the members to form a cohesive evaluation unit. The DevTech Team 
for each activity will coordinate tasks closely and will be supported by research and logistics 
personnel who will work across the three evaluations. Additionally, all teams will be supported 
by DevTech HQ Senior Evaluation Advisor (STA), who will provide technical input throughout 
the evaluation and Program Manager who will provide operational support. 
Local Understanding. The Malawi health sector is unique in its own circumstances and 
challenges. The country’s need to expand coverage in all areas is a result of increasing demands 
for health services. At the same time, the quality of service delivery needs to be maintained, if 
not improved, to meet the stated development objectives. While the macro-level conditions are 
fairly evident, it is the knowledge of specific conditions at the district- and community-levels 
where evaluation findings of SSDI activities will be more useful and relevant. The DevTech 
evaluation team is heavily represented by proven Malawian health specialists and evaluators with 
extensive experience in conducting field work, even in remote (rural) locations of the country. 
Having previously worked with Invest in Knowledge a locally-based data collection firm, the 
DevTech Team will look to engage Invest in Knowledge to support all three evaluation 
activities.
Unequaled Management. DevTech Team is, a single integrated unit encompassing two very 
strong USAID partners with complementary skills, technical management expertise, logistical 
capability, geographic reach, and extended networks of local researchers. The Team offers an 
unequaled bench of Monitoring and Evaluation (M&E) expertise to successfully execute from 
day one. 
SSDI-Services Evaluation 30
Focus on Gender. Reinforcing health promotion and disease prevention among households 
cannot be effectively achieved without an awareness of relevant gender issues. Gender 
differences impact health practices such as the exposure to risk factors, access to services, 
attitudes towards personal health and maintenance, and perceptions of quality care. In all 
relevant aspects of the evaluation, the team will assess SSDI outcomes along gender lines not 
only to respond to the Mission’s CDCS and the USAID Gender policy but also to determine if 
health practices are being applied throughout the project areas. 
TECHNICAL APPROACH 
Background on SSDI-Services 
USAID/Malawi’s flagship health activity, SSDI aims to strengthen health systems in Malawi 
through the implementation of three primary health sector activities: SSDI-Services, SSDI￾Systems and SSDI-Communication. SSDI-Services specifically targets the provision of high 
quality EHP services to reduce fertility and population growth, lower the risk of HIV, and lower 
maternal, newborn, infant and under-five mortality and morbidity rates. The Mission and its 
partners focus on improving both access to and quality of care by building capacity at the 
ministry-, district-, and facility-level through 30 technical interventions including professional 
training, clinical mentorship, facility improvement, and community mobilization. Services aims 
to develop innovative community approaches to delivery such as care groups, while working in 
close collaboration with the Communications and Systems activities. 
Objectives of the Evaluation: 
The primary objective of the Services performance evaluation (PE) is, “to measure and 
determine the extent to which SSDI Services interventions had on quality of and access to care; 
provide in-depth insights into the facilitating and limiting factors of increased service utilization 
at each level of service delivery; and document progress made towards building MoH capacity 
to deliver quality EHP services.” The sections below detail our overall approach. 
a. Overall Technical Approach
a.1 Evaluation Questions and Design 
In order to assess the effect of SSDI Services activities on the quality of and access to services, 
the evaluation scope of work (SOW) outlined six evaluation questions. The DevTech Team 
proposes a mixed methods design that draws from existing data, project documents, and Health 
Information Systems (HIS) data on service provision and health outcomes, and calls for the 
collection of new qualitative and quantitative data through targeted surveys, key informant 
interviews (KIIs) and focus group discussions (FGDs). Table 1 summarizes the evaluation 
questions and describes the indicators and data sources used to answer the evaluation questions. 
Table 1: Evaluation Design Summary Table 
SSDI-Services Evaluation 31
EQ
#
Evaluation Question Theme Indicator Data Source
Q1 To what extent did 
SSDI improve quality 
of care for priority EHP 
interventions at targeted 
health facilities and 
community service 
delivery points? Were 
there any adverse or 
unexpected results on 
the quality of care?
Quality 
of 
Services
Changes in output 
and outcome 
indicators
Perceptions of care
Existing program output and 
outcome data 
Baseline and endline data gathered 
by SSDI
FGDs with healthcare providers and 
community members 
Qualitative Score Card (during
FGDs)
Healthcare provider and consumer 
surveys
KII with MoH officials
Document review
Other secondary data
Q2 To what extent did 
SSDI-Services improve 
access to priority high 
impact EHP 
interventions in the 
target health facilities 
and community service 
delivery points?
Access to 
Services
Change in uptake 
of services
Coverage increase 
(mapping)
Increase in 
community based 
provision of care
Perceptions of 
access
Existing program output and 
outcome data 
GIS data 
Baseline and endline data gathered 
by SSDI
FGDs with healthcare providers and 
community members 
Qualitative Score Card (during 
FGDs)
Healthcare provider and consumer 
surveys
KIIs with MoH officials
Other secondary data
Q3 To what extent did the 
direct involvement of 
MoH staff to facilitate 
the in-service training 
of health practitioners, 
on-the-job mentoring 
and quality assurance 
strengthen the MoH’s 
capacity to 
institutionalize these 
components as part of 
routine delivery of high 
impact priority EHP 
services?
Capacity 
for 
Service 
Delivery 
Number of 
practitioners 
trained
Changes in MoH 
capacity described 
by MoH staff and 
trainers 
(beneficiaries of 
Trainer-of-Trainer 
programs)
Perceptions of 
training received
Existing program data on number 
trained
KIIs with MoH officials
FGDs with trainers
Qualitative Score Card (during 
FGDs)
Q4 To what extent did 
SSDI’s investments in 
performance monitoring 
and Health Information 
Systems (HIS) improve 
the MoH’s (national, 
Capacity 
for Data￾Driven 
Decision￾Making
Number of 
individuals trained 
in HIS usage
Usage of HIS data
Perceptions of data 
usage
Existing program data on number 
trained
Baseline and endline HIS data on 
usage (number of times accessed, 
reports generated, etc)
KIIs with MoH officials
SSDI-Services Evaluation 32
EQ
#
Evaluation Question Theme Indicator Data Source
district and facility) 
capacity to collect, 
access and utilize 
routine service delivery 
data for decision 
making?
Document review
Q5 What are the most 
significant 
accomplishments, best 
practices, and lessons 
learned from the SSDI￾services activity? 
Explicitly identify and 
document facilitating 
and inhibiting factors to 
positive performance 
for each of the above 
questions.
Successe
s/Lessons 
Learned/
Enabling 
and 
Inhibitin
g Factors
Change in targeted 
health outcomes
Perceptions of 
SSDI-Services 
activities
Self-assessment questionnaire
Baseline and endline data gathered 
by SSDI 
KIIs with MoH and IP 
FGDs with community members and 
healthcare workers
Qualitative Score Cards (during 
FGDs)
Healthcare provider and consumer 
surveys
a.2 Evaluation Activities 
In order to meet the evaluation objectives and ensure timely submission of key
deliverables, the following key activities are proposed: 
Post-Award Meeting and Internal Planning Meeting (IPM). Following award, the DevTech 
Team will meet with Mission stakeholders via phone to clarify roles and responsibilities and to 
finalize the workplan, timeline, and relevant SSDI documents and secondary data to review and 
analyze. 
Document Review and Secondary Data Analysis. The DevTech Team will review program 
reports, performance monitoring plan (PMP) data, baseline and endline data collected by SSDI, 
and other relevant secondary data identified together with the Mission, to assess changes in 
access, quality, and effectiveness of SSDI-Services interventions. The analysis will also be used 
to identify best- and least-improved healthcare sites that will be the focus of qualitative data 
collection. 
Field Data Collection. A team of data collectors will be retained to conduct data collection at a 
sample of health facilities across the 15 districts where SSDI activities are implemented. The 
Team Leader (TL) and Evaluation Specialist (ES) will provide training and oversight of data 
collectors. 
Ongoing Data Analysis. The Research Analyst (RA) will analyze quantitative and qualitative 
data collected on an ongoing basis to facilitate discovery of preliminary results and report 
writing. The DevTech Team will meet weekly via phone to discuss progress, data collection 
issues, and discuss preliminary findings. 
SSDI-Services Evaluation 33
Bi-weekly Debriefing Meetings with USAID. The TL and HQ STA, will provide regular 
updates to USAID during the bi-weekly meetings that will also include the Communications and 
Systems meetings for more efficient management. 
USAID/Malawi Presentation. The TL and team members will deliver an oral presentation to 
USAID on preliminary findings, conclusions, and recommendations. 
Findings Workshop. In consultation with USAID, the DevTech Team will organize findings 
workshops in Lilongwe to present the main findings, conclusions, and recommendations to key 
stakeholders across the three SSDI evaluations. 
A.3 Integration of Gender and Other Disadvantged Groups 
In the process of developing its current CDCS, the Mission undertook a gender assessment to 
identify approaches to improve health outcomes of some of Malawi’s most vulnerable citizens 
— women and children under the age of five. Several of these approaches are central to the 
design of SSDI Services, including building capacity at the local level in order to implement 
community-based services and delivering integrated services that take gender and social norms 
into consideration. Therefore, as part of its analysis of the effectiveness of SSDI-Services in 
improving the level of access to, and quality of, services, the DevTech Team will consider how 
quality and services may differ by gender and other social and economic factors. Additionally, 
the sampling strategy will take into account gender and age differences. 
b. Evaluation Plan
This section describes a detailed plan the team will take to evaluate the SSDI Services 
evaluation. 
b.1 Data Sources
Existing Data. The Devtech Team will leverage a number of existing data including: baseline 
and endline data gathered at the household level by SSDI Services and SSDI Communication, 
respectively; PMP data; GIS data gathered by the Mission; HIS data; program quarterly reports; 
and potentially data gathered through the 2010 DHS and other relevant national surveys. 
Primary Qualitative and Quantitative Data. While the team will capitalize on opportunities to 
draw from existing data, the evaluation design requires that the DevTech Team also collects new 
information from a variety of stakeholders to obtain direct perspectives on the outcomes of 
SSDI-Services’ activities. Devtech proposes five distinct data collection activities as part of the 
SSDI-Services evaluation: 1) self-assessment questionnaire of Services implementers, 2) survey 
of healthcare providers; 3) survey of individuals who have received services from an SSDI￾supported facility (consumers); 40 KIIs with MoH officials, district health officers, 
representatives from IPs; and 5) focus group discussions (FGDs), including qualitative score 
cards, with community residents, healthcare providers, and trainers benefiting from MoH 
Trainer-of-Trainer (ToT) programs. The sections below provide more details about each of the 
data collection activities. 
SSDI-Services Evaluation 34
b.2 Data Collection Approach 
Quantitative Data Collection
Healthcare Provider and Consumer Surveys. The DevTech Team will carry out two brief 
surveys — one with representatives from healthcare facilities and a second with healthcare 
consumers — at 49 healthcare facilities, chosen through a stratified random sampling approach 
in all 15 districts. Each survey will take approximately 15-20 minutes to complete. In addition to 
demographic questions, the surveys will consist of items to gauge both healthcare workers' and 
consumers’ perceptions of the quality of services (Q1) and access to health care (Q2). The survey 
will also include items that specifically target female respondents, especially those of 
childbearing age and those with children under five. 
Self-Assessment Questionnaire. At the conclusion of KIIs with implementers and other 
partners, the team will administer a short self-assessment questionnaire to objectively assess 
stakeholder performance in implementing SSDI-Services, and to identify successes, challenges, 
and enabling factors of their performance. 
Qualitative Data Collection 
While the survey data will give us a broad understanding of providers’ and consumers’ 
perceptions on access to care and the quality of care, KIIs and FGDs provide depth of 
understanding of the factors that determine quality of, and access to, care, as well as the role 
SSDI-Services has played, especially in terms of increasing capacity and the use of data to 
improve outcomes. 
Key Informant Interviews. In addition to supplementing information gathered through provider 
and consumer surveys to address Q1 and Q2, KIIs will be used to understand changes in the 
MOH's capacity to gather and analyze HIS data and increase the quality and reach of services 
(Q3 and Q4). Furthermore, the DevTech Team will ask interviewees about significant 
accomplishments, best practices and lessons learned, personal experiences, and other factors that 
may have inhibited or promoted success (Q5). 
Focus Group Discussions and Score Cards. The DevTech Team will adapt a qualitative data 
collection technique used in the baseline study of the USAID/Malawi CDCS (see box 1) — a 
Rural Score Card — to examine healthcare providers' and community members’ perceptions of 
care with respect to both quality (Q1) and access (Q2). FGD participants will be asked to vote on 
whether or not there were positive or negative changes in their community related to various 
issues around access and quality of healthcare, and to discuss their responses. The DevTech 
Team will use a similar technique with individuals who participated in ToT programs to examine 
the quality of the training (Q3) and help flush out successes, best practices, and lessons learned 
(Q5). 
Instrument Development and Training. Prior to fieldwork, survey instruments and semi￾structured FGD and KII guides will be vetted with USAID staff, translated into local languages, 
SSDI-Services Evaluation 35
and piloted. All data will be collected in the respondent's preferred language. Each data 
collection team will include local language speakers as either interviewers or translators. The 
data collection team will be trained under the supervision of the TL with the assistance of the ES, 
particularly on using interviewing and facilitation techniques that encourage robust responses 
from respondents. 
Survey Sample 
The DevTech Team will follow a stratified random sampling approach at the district level to 
select a random sample of healthcare facilities in the 15 SSDI districts. Using this approach, the 
Team will conduct quantitative data collection in 49 healthcare facilities, with the goal of 
collecting two healthcare provider surveys per site for a total of 98 provider surveys, and a total 
of 385 healthcare consumer surveys across sites. Collecting two healthcare provider surveys per 
site from representatives with different roles, tenure, level, or gender minimizes the potential for 
response bias due to confounding factors. This sampling approach allows the DevTech Team to 
generalize conclusions, at a 95% confidence level, about perceptions of care quality and access 
to all healthcare facilities and catchment population within the 15 districts. This approach 
assumes that service delivery is more similar between healthcare facilities within, than between 
districts. 
The DevTech Team will consult with USAID/Malawi to finalize the sample sizes and sample 
facilities selected, with the goal of targeting healthcare facilities that offer all three SSDI sector 
activities and that are representative of urban and rural facilities and length of SSDI support, 
among other criteria. 
Qualitative Sample: The team’s qualitative sampling approach is purposive in nature. The team 
anticipates collecting qualitative data in eight districts, including Balaka, Machinga, and 
Lilongwe Rural to target the three districts in which most activities have focused. The remaining 
five districts will be determined in collaboration with the teams for the Communication and 
Systems evaluations. The team will aim to collect data in an equal number of urban and rural 
sites and in locations where all three SSDI activities overlap. Two facilities will be visited in 
each district, for a total of 16 facilities. A total of 38 KIIs and FGDs with 64 people will be 
conducted, targeting a range of MoH district/zonal and central representatives, implementers, 
USAID representatives and male and female community members. 
b. 3 Data Analysis Plan 
Survey and secondary data will be cleaned, processed, and analyzed on an ongoing basis 
throughout the evaluation. Results and trends will be discussed during weekly team check-ins. 
The Team will analyze survey data to assess changes in both quality of and access to healthcare 
over time at the level of healthcare providers and consumers, and to extract patterns in the data 
that point to key enabling and/or inhibiting factors to the objectives of SSDI-Services. The 
DevTech Team will systematically investigate differences attributable to gender and/or other 
disadvantaged group status particularly in terms of perceptions of healthcare access and quality, 
and to understand if and how SSDI-Services are effectively and equally reaching women, 
children, and men. The Team will draw upon existing quantitative data and use GIS tools to 
SSDI-Services Evaluation 36
construct maps that illustrates SSDI-Services’ coverage, including where this coverage overlaps 
with SSDI-Systems and SSDI-Communications. Qualitative data collected will be digitally 
recorded whenever possible and transcribed and translated from local languages to English as 
needed. The team will then code the data using qualitative data analysis software to identify 
themes and build a narrative that responds to the primary evaluation questions. Through 
quantitative data, the evaluation offers a discrete assessment of the level of care quality, access, 
and effectiveness of SSDI-Services, enriched and validated by narrative themes from the 
qualitative data collected. 
Challenges and Limitations. Based on the DevTech Team's past experience in Malawi, we 
anticipate some potential challenges and present solutions in Table 2. 
Table 2. Anticipated Challenges and Potential Solutions. 
Challenges Mitigating Solutions
Geographic distance. The time it takes to 
travel between sites, especially during the 
rainy season (December-March). 
Identify a reputable local data collection 
firm to oversee logistics, and conduct data 
collection, transcription, and data entry. 
Language differences. The variety of local 
languages spoken, e.g., Chechewa, Chisena, 
Tumbuka, Yao.
The local data collection firm will include
team members fluent in the wide diversity 
of local languages spoken by respondents. 
Respondent fatigue. Malawi is heavily 
saturated by programming, and as a result, 
evaluations. Many Malawians are 
experiencing respondent fatigue.
The DevTech Team aims to mitigate this 
burden by coordinating data collection 
activities across the three PEs. 
Condensed evaluation timeline. The RFP 
requests a high-level of rigor in technical 
approach, which the current proposal seeks 
to fulfill. However, the condensed timeline 
may not provide sufficient time to 
extensively codify and analyze data to form 
robust findings and recommendations.
The Team will conduct data analysis on an 
ongoing basis, utilizing an efficient data 
analysis plan that focuses on answering the 
specified evaluation questions. While in￾depth results may not be possible within the 
evaluation timeframe, the dearth and quality 
of the qualitative and quantitative data the 
Team will provide as a product of the 
evaluation will allow the Mission to revisit 
the data or request additional analysis 
through subsequent task orders. 
[END OF SECTION C] 
SSDI-Services Evaluation 37
ANNEX 2. SUMMARY EVALUATION METHODOLOGY 
Data Collection Method Stakeholder Number
KIIs USAID, IP, MOH 34
Self-Assessment Questionnaire USAID, IP, MOH, health 
providers
112
Group Interviews Health Providers 26
FGDs/Community Scorecards Community members 38 (13 men only; 26 women 
only)
Consumer Questionnaires Health Consumers 490
13 out of 15 districts sampled
26 health facilities visited (2 per district)
SSDI-Services Evaluation 38
 
ANNEX 3. DATA COLLECTION INSTRUMENTS 
A-Instrument: Key Informant Instrument Interview Guide 
Stakeholder: MoH managers (central, zonal, DHO) 
Hello! My name is________________________, I am an interviewer for DEV TECH which is doing a study for 
the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All 
answers will remain confidential. It should take about an hour. Please be advised that you are not in any way 
obliged to participate in this interview, and you can discontinue the interview at any time without any penalty. 
You can also refuse to answer any question and move on to the next one. 
At the end of this interview we would like to ask you to fill out a short self-assessment questionnaire about SSDI’s 
performance. Please answer « yes » if you agree to participate. 
For the evaluation team to fill out:
Site: Respondent Sex (M/F): Respondent ID/Number:
Stakeholder Group: Management Level 
Title/Stakeholder Group: 
National: Zone: District:
Evaluation Question 1: To what extent did SSDI improve quality of care for priority EHP 
interventions at targeted health facilities and community service delivery points? Were there any 
adverse or unexpected results on quality of care? 
1. Kindly explain what you know about how the SSDI Services program is organized and how you fit 
in? (Note: It may be known by another name). 
2. To your understanding, what are the major health issues that SSDI services has addressed? 
3. How have SSDI services activities influenced the quality of care for these health issues at health 
delivery sites? 
4. Which of these activities/approaches were successful/unsuccessful? Please explain and give 
examples. 
5. Do men and women equally have good quality of care, or is the quality of care different for men 
versus women? Please explain. What about people with disabilities? Please explain. 
6. Were there any unexpected results (good or bad) on the quality of care as a result of SSDI 
services activities? Any surprises in general? Please explain. 
7. What are your suggestions on how SSD-I Services could have been implemented differently to 
have a stronger effect on the quality of health services? Please clarify 
SSDI-Services Evaluation 39
Evaluation Question 2: To what extent did SSDI-Services improve access to priority high impact 
EHP interventions in the target health facilities and community service delivery points? 
1. What are the challenges that have affected patient and community access to EHP services in your 
unit and/or Malawi? 
2. How has SSDI Services addressed these access challenges through its support and activities? Kindly 
explain and provide examples. 
3. Which of these activities/approach were successful/unsuccessful? Please explain and give examples. 
4. In your opinion, how has the equality in access to care changed for 1) women, 2) disabled and 3) 
marginalized groups as a result of SSD-I Services 
5. In your opinion, how has SSDI services affected the access to care for men versus women? Do they 
have equal access? Why, why not? What about people with disabilities? 
6. What are your suggestions on how SSDI Services could have been implemented differently to have a 
stronger effect on access to health services at the community level? Please clarify. 
Evaluation Question 3: To what extent did the direct involvement of MoH staff to facilitate the 
in-service training of health practitioners, on-the-job mentoring and quality assurance strengthen 
the MOH’s capacity to institutionalize these components as part of routine delivery of high 
impact priority EHP services? 
1. Please describe the SSD-I Services capacity building activities of which you are aware. 
2. Have these activities addressed the capacity needs existing before SSDI services began at MOH? 
Please explain. 
3. Have TOTs/Mentors/PBI-Q been used within the system or elsewhere besides SSDI? 
Have there been any bottlenecks to full utilization of TOTs/mentors/supervision? 
4. What have been the most successful/unsuccessful components? Please give examples 
5. In your opinion, how could the institutionalization and sustainability of the 
TOTs/mentors/supervision be improved? 
Evaluation Question 4: To what extent did SSDI’s investments in performance monitoring and 
Health Information Systems improve the MoH’s (national, district and facility) capacity to 
collect, access and utilize routine service delivery data for decision-making? 
1. Please describe the HMIS capacity gaps being targeted by SSDI Services. 
2. In what manner have the performance monitoring activities affected the capacity of MOH to 
collect and utilize data? Can you provide a concrete example? 
3. Is the HMIS and its data being used on a routine basis? Who is using them, and how? If not, why 
not? 
4. Please describe the successes or improvements you have seen from this investment and the main 
factors contributing to its success? 
5. What have been the main bottlenecks in establishing this system and in its actual utilization? 
6. What will you recommend to improve the usefulness of HMIS and performance management in 
your area of responsibility? 
SSDI-Services Evaluation 40
Question 5: What are the most significant accomplishments, best practices, and lessons learned 
from the SSDI-Services activity? Explicitly identify and document the facilitating and inhibiting 
factors to positive performance for each of the above questions. 
1. Overall did the SSDI project activities meet your expectations? Why or why not? 
2. Would you please describe the SSDI Services accomplishments/lessons learned/ best practices? 
What are the main factors that contributed to these outcomes? 
3. Do you have any final remarks or questions? 
Thank you very for your participation. 
B-Instrument: Key Informant Instrument Interview Guide 
Stakeholder: Program coordinators, HMIS, and facility in-charge/health provider 
Hello! My name is________________________, I am an interviewer for DEV TECH which is doing a study for 
the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All 
answers will remain confidential. It should take about an hour. Please be advised that you are not in any way 
obliged to participate in this interview, and you can discontinue the interview at any time without any penalty. 
You can also refuse to answer any question and move on to the next one. 
At the end of this interview we would like to ask you to fill out a short self-assessment questionnaire about SSDI’s 
performance. Please answer « yes » if you agree to participate. 
For the evaluation team to fill out:
Site: Respondent Sex (M/F): Respondent ID/Number:
Stakeholder Group: Management Level 
Title/Stakeholder Group: 
National: Zone: District:
 
Evaluation Question 1: To what extent did SSDI improve quality of care for priority EHP 
interventions at targeted health facilities and community service delivery points? Were there any 
adverse or unexpected results on quality of care? 
1. Kindly explain what you know about how the SSDI Services program is organized and how you fit 
in? (Note: It may be known by another name). 
2. To your understanding, what are the major health issues that SSDI services has addressed? 
3. How have SSDI services activities influenced the quality of care for these health issues at health 
delivery sites? 
4. Can you kindly specify the activities/ approaches used to improve the quality of care within specific 
health services or categories of services? 
SSDI-Services Evaluation 41
5. Which of these activities/approaches were successful/unsuccessful? Please explain and give examples. 
6. Do men and women equally have good quality of care, or is the quality of care different for men 
versus women? Please explain. What about people with disabilities? Please explain. 
7. Were there any unexpected results (good or bad) on the quality of care as a result of SSDI services 
activities? Any surprises in general? Please explain 
8. What is your opinion on the overall effect of SSD-I Services on the quality of care in your area of 
responsibility? Please explain, give examples 
9. What are your suggestions on how SSDI Services could have been implemented differently to have a 
stronger effect on the quality of health services? Please clarify. 
Evaluation Question 2: To what extent did SSDI-Services improve access to priority high impact 
EHP interventions in the target health facilities and community service delivery points? 
1. What are the challenges that have affected patient and community access to EHP services in your 
unit? 
2. Can you kindly specify the activities/approaches used by SSDI Services to help improve access to 
health care within specific health services or categories of services? 
3. Which of these activities/approach were successful/unsuccessful? Please explain and give examples. 
4. In your opinion, how has SSDI services affected the access to care for men versus women? Do they 
have equal access? Why, why not? What about people with disabilities? 
5. In your opinion, what are the key factors that facilitated the positive achievements that you have 
observed? 
6. What have been the main bottlenecks for SSDI Services in improving access to service? Please 
explain. 
7. What are your suggestions on how SSDI Services could have been implemented differently to have a 
stronger effect on access to health services at the community level? Please clarify. 
Evaluation Question 3: To what extent did the direct involvement of MoH staff to facilitate the 
in-service training of health practitioners, on-the-job mentoring and quality assurance strengthen 
the MOH’s capacity to institutionalize these components as part of routine delivery of high 
impact priority EHP services? 
1. Please describe the SSD-I Services capacity building activities of which you are aware. 
2. In your opinion, have these activities addressed the capacity needs existing before SSD-I began in 
your/District/Facility/unit? Please explain. 
3. What changes has TOT program brought in addressing capacity building needs? How has this 
improve health access and quality? Please explain 
4. Kindly describe how TOT were utilized within your area of responsibility. What specific health areas 
or responsibilities were they used for? Please give examples. 
5. Do you continue to use the TOTs? If so, how often and for what purposes? If not why? 
6. Have the TOTs in your area of responsibilities trained other individuals or groups outside your 
area? 
SSDI-Services Evaluation 42
7. What have been the greatest bottlenecks to routine utilization of TOTs? 
8. Are you aware of or been involved in the SSDI Services mentorship and supervision activities? In 
your opinion, what has been the outcome of this process? 
9. What are the contributing factors of the successful/unsuccessful components? 
10. Have there been improvements in the quality of or access to health service indicators because of 
SSD-I capacity building activities? 
11. In your opinion, how could the institutionalization of the TOTs/mentorship/supervision be 
improved? 
Evaluation Question 4: To what extent did SSDI’s investments in performance monitoring and 
Health Information Systems improve the MoH’s (national, district and facility) capacity to 
collect, access and utilize routine service delivery data for decision-making? 
1. Please describe the HMIS capacity gaps being targeted by SSD-I Services within your area of 
responsibility. 
2. What performance monitoring systems did the SSD-I services activity introduce into your area of 
responsibility? Please describe. 
3. In what manner have the performance monitoring activities affected the capacity of your 
district/facility/unit to collect and utilize data? 
4. Is the utilization of these systems and activities routine? Who is using them, and how? If not, why 
not? 
5. Please describe the successes or improvements you have seen from this investment? 
What are the main factors contributing to these successes? 
6. What have been the main bottlenecks in 1) establishing this system and 2) in its actual 
utilization? 
7. What would you recommend to improve the usefulness of HMIS and performance management in 
your area of responsibility? 
Evaluation Question 5: What are the most significant accomplishments, best practices, and 
lessons learned from the SSDI-Services activity? Explicitly identify and document the facilitating 
and inhibiting factors to positive performance for each of the above questions. 
1. Overall did the SSDI services project activities meet your expectations? Why or why not? 
2. Would you please describe any best practices or lessons learned from SSDI Services? 
3. Do you have any final remarks or questions? 
Thank you very for your participation, etc. 
C-Instrument: Key Informant Instrument Interview Guide 
Stakeholder: Implementing partners, USAID 
Hello! My name is________________________, I am an interviewer for DEV TECH which is doing a study for 
the Ministry of Health and its partners. We are here to ask questions about health and hope you can help us. All 
answers will remain confidential. It should take about an hour. Please be advised that you are not in any way 
SSDI-Services Evaluation 43
obliged to participate in this interview, and you can discontinue the interview at any time without any penalty. 
You can also refuse to answer any question and move on to the next one. 
At the end of this interview we would like to ask you to fill out a short self-assessment questionnaire about SSDI’s 
performance. Please answer « yes » if you agree to participate. 
For the evaluation team to fill out:
Site: Respondent Sex (M/F): Respondent ID/Number:
Stakeholder Group: Management Level 
Title/Stakeholder Group: 
National: Zone: District:
 
Evaluation Question 1: To what extent did SSDI improve quality of care for priority EHP 
interventions at targeted health facilities and community service delivery points? Were there any 
adverse or unexpected results on quality of care? 
10. Kindly explain what you know about how the SSDI Services program is organized and how you fit 
in? (Note: It may be known by another name). 
11. To your understanding, what are the major health issues that SSDI services has addressed? 
12. How have SSDI activities influenced the quality of care for these health issues at health delivery 
sites? 
13. Can you kindly specify the approaches used to improve the quality of care within specific health 
services or categories of services? 
14. Which of these activities/approaches were successful/unsuccessful? Please explain and give examples. 
15. Do men and women equally have good quality of care, or is the quality of care different for men 
versus women? Please explain. What about people with disabilities? Please explain. 
16. Were there any unexpected results (good or bad) on the quality of care as a result of SSDI services 
activities? Any surprises in general? Please explain. 
17. What is your opinion on the overall effect of SSD-I Services on the quality of care in your area of 
responsibility and in Malawi? Please explain, give examples 
18. What are your suggestions on how SSDI Services could have been implemented differently to have a 
stronger effect on the quality of health services? Please clarify 
Evaluation Question 2: To what extent did SSDI-Services improve access to priority high impact 
EHP interventions in the target health facilities and community service delivery points? 
SSDI-Services Evaluation 44
8. What are the challenges that have affected patient and community access to EHP services in your 
unit and/or Malawi? 
9. Can you kindly specify the approaches used by SSD-I Services to help improve access to health care 
within specific health services or categories of services? 
10. In your opinion, how did these approaches actually influence access to care at heath delivery sites? 
11. Which of these activities/approach were successful/unsuccessful? Please explain and give examples. 
12. In your opinion, how has SSDI services affected the access to care for men versus women? (Please 
explain). Do they have equal access? Why, why not? What about people with disabilities? 
13. In your opinion, what are the key factors that facilitated the positive achievements that you have 
observed? 
14. What have been the main bottlenecks for SSDI Services in improving access to service? Please 
explain. 
15. What are your suggestions on how SSDI Services could have been implemented differently to have a 
stronger effect on access to health services at the community level? Please clarify. 
Evaluation Question 3: To what extent did the direct involvement of MoH staff to facilitate the 
in-service training of health practitioners, on-the-job mentoring and quality assurance strengthen 
the MOH’s capacity to institutionalize these components as part of routine delivery of high 
impact priority EHP services? 
1. Please describe the SSD-I Services capacity building activities of which you are aware? 
2. In your opinion, have these activities addressed the capacity needs existing before SSD-I services 
began? Please explain. 
3. Which of these activities were successful/unsuccessful? Why? 
4. To your understanding, has the MOH continued to utilize TOTs, mentoring, or supervision? How 
so? 
5. What have been the greatest bottlenecks to routine utilization of TOTs? Please explain 
6. How have SSDI services capacity building activities affected the quality of and access to health 
services, if at all? 
7. In your opinion, how could the institutionalization and sustainability of the 
TOTs/mentorship/supervision be improved? 
Evaluation Question 4: To what extent did SSDI’s investments in performance monitoring and 
Health Information Systems improve the MoH’s (national, district and facility) capacity to 
collect, access and utilize routine service delivery data for decision-making? 
1. Please describe the HMIS capacity gaps being targeted by SSD-I Services for the entire system or 
within your area of responsibility? 
2. What performance monitoring systems did the SSD-I services activity introduce? 
3. In what manner have the performance monitoring activities affected the capacity of MOH to collect 
and utilize data? 
SSDI-Services Evaluation 45
4. Please provide example of concrete products/successful results of investments in performance 
management and HMIS? 
Please provide examples of concrete unsuccessful results of investments in performance 
management and HMIS? 
5. Is the utilization of these systems and activities routine? Who is using them, and how? If not, why 
not? 
6. What would you recommend to improve the usefulness of HMIS and performance management in 
your area of responsibility? 
Evaluation Question 5: What are the most significant accomplishments, best practices, and 
lessons learned from the SSDI-Services activity? Explicitly identify and document the facilitating 
and inhibiting factors to positive performance for each of the above questions. 
1. Overall did the SSDI service project activities meet your expectations? Why (explain)? or why not 
(explain)? 
2. Would you please describe any best practices or lessons learned from SSDI Services? 
3. Do you have any final remarks or questions? 
Thank you very much for your participation, etc. 
D-Instrument: Self-Assessment Questionnaire (Likert) 
Stakeholder: SSDI implementers and partners 
For the evaluation team to fill out:
Site: Respondent Sex (M/F): Respondent ID/Number:
*************** 
PART A: Please circle the number of the stakeholder group that you belong to: 
SSDI staff (JHPIEGO, CARE, PLAN, Save the Children) 1
Health Care Provider 2
Trainer of Trainer, Trainer, Mentor, or Supervisor 3
HMIS Staff 4
Donor (USAID) 5
Ministry of Health Official/Administrator 6
Other (Please write) : 7
PART B: For each question, please circle the number that best matches your answer (1, 2, 
3, 4 or “I don’t know”):
SSDI-Services Evaluation 46
Topic: Quality of Care (Evaluation Question 1) 
Within the last 5 years, to what extent has SSDI services improved the quality of care for family 
planning services at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
1. Within the last 5 years, to what extent has SSDI services improved the quality of care for 
maternal and neonatal health at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
2. Within the last 5 years, to what extent has SSDI services improved the quality of care child 
health services at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
4. Within the last 5 years, to what extent has SSDI services improved the quality of care for 
nutrition services at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
5. Within the last 5 years, to what extent has SSDI services improved the quality of care for 
malaria services at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
6. Within the last 5 years, to what extent has SSDI services improved the quality of care for 
HIV/AIDS services at targeted health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
Topic: Access to Care (Evaluation Question 2) 
7. Within the last 5 years, to what extent has SSDI services improved access to family planning in 
the target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
8. Within the last 5 years, to what extent has SSDI services improved access to maternal and 
neonatal health in the target health facilities? 
SSDI-Services Evaluation 47
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
9. Within the last 5 years, to what extent has SSDI services improved access to child health in the 
target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
10. Within the last 5 years, to what extent has SSDI services improved access to child health 
services in the target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
11. Within the last 5 years, to what extent has SSDI services improved access to nutrition services 
in the target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
12. Within the last 5 years, to what extent has SSDI services improved access to malaria services in 
the target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
13. Within the last 5 years, to what extent has SSDI services improved access to HIV/AIDS 
services in the target health facilities? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
Topic: Capacity for Service Delivery (Evaluation Question 3) 
14. As a result of SSDI-Services, to what extent does the MOH routinely use the Training-of￾Trainers as part of ensuring quality service delivery for EHP (Essential Health Package)? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
15. As a result of SSDI-Services, to what extent does the MOH routinely use Mentoring as part of 
ensuring quality service delivery for EHP (Essential Health Package)? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
SSDI-Services Evaluation 48
16. As a result of SSDI-Services, to what extent does the MOH routinely use the Quality Assurance 
Supervision as part of ensuring quality service delivery for EHP (Essential Health Package)? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
Topic: Capacity for data-driven decision making (Evaluation Question 4) 
17. As a result of SSDI-Services, to what extent does the MOH now collect and utilize routine 
service delivery data to make decisions? 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
5
“I Don’t Know”
PART C: Please answer in your own words the following questions: 
18. What were the biggest achievements and the best practices of SSDI-Services? 
Achievements:_______________________________________________________________ 
__________________________________________________________________________ 
__________________________________________________________________________ 
Best practices________________________________________________________________ 
__________________________________________________________________________ 
19. What were the biggest challenges of designing, implementing, and managing SSDI-Services 
activities? 
__________________________________________________________________________ 
__________________________________________________________________________ 
__________________________________________________________________________ 
__________________________________________________________________________ 
E-Instrument: Perceptions of Care Survey 
Stakeholder: Consumers 
SCRIPT: We would like your personal opinion on changes in various health services in Malawi since the SSDI 
Services program began. There are three parts to this survey and it should take about 30 minutes to complete. 
Your personal observations and perceptions are anonymous and you can never be associated with them because 
they will be entered into a large database with many other opinions. We will only ask your sex and age. 
Participating in this survey is voluntary. You will not receive any compensation for participating in the survey. You 
will also not be punished if you decline to participate. At any time you can refuse to answer a question or stop 
participating altogether. 
Your personal opinion is extremely valuable for evaluating SSDI’s services performance over time. Kindly be as 
frank as you can so that health services for all Malawians can be its very best. Please say “yes” if you would like 
to take part in this survey. 
Zikomo kwambiri! 
SSDI-Services Evaluation 49
For the evaluation team to fill out:
Site
District:
Respondent Sex (M/F): Age: Respondent 
ID/Number:
PART A (Service Usage): 
1. Please indicate which of the following health services you or your family have ever received from 
this facility: 
=1 Malaria 
=2 Maternal, Neonatal and Child Health 
=3 Family Planning 
=4 HIV/AIDS 
=5 Nutrition 
=6 Water, hygiene and sanitation 
=7 Infant vaccinations 
=8 Bed nets for malaria 
=9 Malaria treatment 
=10 Birth outside the home with medical assistance 
=11 Prenatal care 
=12 Postnatal care 
=13 Water and sanitation: soap/protection of water and food/latrines 
=14 Nutrition education 
=15 Vitamin A 
=16 Other (Please name) 
2. In the last 5 visits, were you able to receive the service you came for each time? Say “yes” or “no.”
Yes No
PART B (Likert Scale): SCRIPT: Think about the services that you receive from this facility, or from the 
community based workers from this facility. Based on your last visit, please indicate how much you agree with the 
statements.
Quality of Care 
3. The people I went to for services spent enough time with me. 
1 2 3 4
SSDI-Services Evaluation 50
“not at all” “a little” “moderately” “a lot”
4. I was given information about different services that were available to me. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
 
5. I was given enough information to effectively handle my problems. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
6. In general, I was satisfied with the quality of care that I received from this facility.
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
7. My health status has improved as a result of coming to this facility. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
8. In general, this health facility is able to meet the needs of women in my community. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
9. In general, this health facility is able to meet the needs of men in my community. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
10. In general, this health facility is able to meet the needs of people with disabilities in my 
community. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
Access to Care 
11. It is easy for me to get to this facility to receive quality care. 
SSDI-Services Evaluation 51
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
12. It is easy for me to access a health worker in my community. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
13. The staff at this facility explained things to me in a way that was easy for me to understand. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
14. It is easy for men from my community to access quality care at this facility. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
15. It is easy for women from my community to access quality care at this facility. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
16. It is easy for people from my community with disabilities to access quality care at this facility. 
1
“not at all”
2
“a little”
3
“moderately”
4
“a lot”
PART C (Scorecard) 
SCRIPT: In this part, we would like to know about how things used to be, compared to how things are now. 
Please think about when you used to go to a health facility 5 years ago. Now think about going to this health 
facility now. I am going to say some statements. For each status, please say whether you think things have 
become better, stayed the same, or become worse than five years ago. You can also say that you don’t know.
No. Question BETTER SAME WORSE Don’t 
know
1 People are aware of when they should go to a 
health center.
2 People are able to get to the hospital or health 
center when needed.
How is the quality of this care compared to 
SSDI-Services Evaluation 52
before?
3 This health facility is able to meet the needs of the 
community.
4 Community health workers are meeting the needs 
of the community, especially for women of 
childbearing age, or women with children less than 
5 years of age
How is the quality of this care compared to 
before?
5 People are able to access pregnancy and birthing 
support (e.g. midwife attendant when needed).
How is the quality of this care compared to 
before?
6 People are able to access health services for their 
babies, infants and young children when needed.
How is the quality of this care compared to 
before?
7 People are able to access contraception, other 
family planning methods and post-abortion 
services. 
How is the quality of this care compared to 
before?
8 People are able to access HIV/AIDS and TB 
services in their community. 
How is the quality of this care compared to 
before?
9 Women and men have equal access to good 
quality health services.
How is the quality of care for women compared 
to before?
How is the quality of care for men compared to 
before?
SSDI-Services Evaluation 53
10 People with disabilities have access to care.
How is the quality of this care compared to 
before?
F-Instrument: Focus Group Discussion using Participatory Scorecard 
Stakeholder: Consumers, Community Action Groups (CAGs) 
Consent Script 
Thank you very much for coming today. My name is _[fill in]__ and I am working with an American company 
named DevTech. We are conducting a study to assess the impact of the SSDI program [mention SSDI, 
Jhpiego, or other possible names by which participants may know the name]. We are gathering information 
about how the program has affected your community in order to make improvements to the program. 
You have been invited to participate in this group discussion because you may be able to provide information 
about changes in health in this community in the past 5 years. This interview will take about 1.5 hours of 
discussion. 
Your participation is completely voluntary. You can choose not to participate now, or at any time. There is no 
penalty or problem if you choose not to participate. There are no known risks of participating in this activity, 
other than losing the time it will take. Should you feel uncomfortable with any question, you may refuse to 
answer it. You will not be paid to participate, and there are no direct benefits to you other than knowing your 
information may help the Malawian Government to improve its services in Malawi and to your community. 
I encourage everyone to keep the conversation confidential out of respect for your neighbors here. We will not 
ask your name and so your answers will be anonymous to outsiders. We will only ask for your age because we 
need to reflect at the different age group needs. I also request your permission to record our conversation so that 
I can remember what was said. 
Do you have any questions? 
Do you agree to participate in the study? (everyone must give a verbal answer) Yes No 
(if any say no, allow them to leave before proceeding) 
Do you agree to let me record our conversation? (everyone must give a verbal answer) Yes 
No 
(if anyone says no, allow them to leave, or do not record if large consensus to not record) 
For the evaluation team to fill out:
Community Name: Name of Nearest Health Facility:
FGD Participants Log
# Sex (M/F) Age Verbal Consent (Y/N)
1
SSDI-Services Evaluation 54
2
3
4
5
6
7
8
9
10
11
12
Introductions: 
SCRIPT: First, let’s get to know each other… [facilitate group greetings, including name, number of years living in 
community, and any leadership positions held] 
Instructions: 
SCRIPT: I am going to ask about certain topics, and for each one, please tell me whether you feel the situation 
has gotten worse, gotten better, or stayed almost the same as in the last 5 years. I expect that some of you 
may not fully agree on each thing, and that is OK. These questions ask for your own personal perceptions, and 
there are no right or wrong answers. Each person’s perspective is valuable to us, so please feel free to speak your 
mind. 
PART 1: GENERAL DISCUSSION (45 minutes) SCRIPT: I’d like us to discuss the quality of health 
services and health information in your community in the last 5 years. Note to facilitator: lead a discussion 
that hits some of the following topics. Probe only if necessary. Do not ask too many questions, rather 
allow the participants to discuss among themselves. 
1. What health services do you receive from your health facility and/or the community health worker? 
2. Do you know of any changes in recent years to community practices of visiting health centers when 
one is feeling unwell? Please explain the changes 
a. Probe: Would you say that the positive/negative changes are peculiar to specific health 
services only? What would be the explanation for that? 
b. Have you experienced any changes in how the facilities attend to community members when 
you visit them? Please explain 
3. How did the changes positive/negative come about? 
a. Probe: Are there any elements at the community level responsible for the positive/negative 
changes you have mentioned? 
SSDI-Services Evaluation 55
4. Do you think that there are specific groups of people that have been mainly affected by the 
positive/negative services? Would you explain? 
5. What improvements would you still make going forward to reach a satisfactory level of receiving 
care from the health facilities? 
PART B: SCORECARD EXERCISE (45 minutes) 
The below questions are asked and voted upon using the flipchart. Draw a happy face to mean “better,” a 
neutral face to mean “same,” and a sad face to mean “worse.”
No. Question BETTER SAME WORSE
1 People are aware of when they should go to a health center.
2 People are able to get to the hospital or health center when 
needed.
How is the quality of this care compared to before?
3 This health facility is able to meet the needs of the community.
4 Community health workers are meeting the needs of the 
community, especially for women of childbearing age, or women 
with children less than 5 years of age
How is the quality of this care compared to before?
5 People are able to access pregnancy and birthing support (e.g. 
midwife attendant when needed).
How is the quality of this care compared to before?
6 People are able to access health services for their babies, infants 
and young children when needed.
How is the quality of this care compared to before?
7 People are able to access contraception, other family planning 
methods and post-abortion services. 
How is the quality of this care compared to before?
8 People are able to access HIV/AIDS and TB services in their 
community. 
How is the quality of this care compared to before?
9 Women and men have equal access to good quality health 
services.
How is the quality of care for women compared to before?
How is the quality of care for men compared to before?
10 People with disabilities have access to care.
How is the quality of this care compared to before?
SSDI-Services Evaluation 56
TIPS FOR FACILITATOR: Probes to promote general discussion: 
 Do you agree with this view? 
 Why do you think so? 
 Can you give an example of why you think this? 
 Is there any part that has improved? Has become worse? 
 Ending question for each item ask: Is there consensus on which category I should use? Mark the 
number of votes for each topic on the flipchart. 
Thank and end here. 
SSDI-Services Evaluation 57
ANNEX 4. PEOPLE CONSULTED 
Key Informant Interviews at Zonal and District levels
North Zone 
Dr. Owen Musopole, Zonal Supervisor Northern Zone
Chitipa DHO-HMIS
Chitipa DHMIS Officer
Chitipa FP Coordinator
Chitipa Plan Representative
Chitipa District Family Planning Programme Officer
Mr. Selemani, Chitipa District Nursing Officer
Karonga: Health care provider and HMIS staff
Samuel Longwe North Zonal M&E Officer Ministry of Health Mzuzu
Central East Zone
Dr. Sosten Lankhulani, DHO Nkhotakota
Tifa Ngoma, NASO Executive Director, Nkhotakota
NASO Programme Officer, Nkhotakota 
Nkhotakota: District HMIS Officer 
Nkhotakota District Malaria Programme Officer
Happy Manda, Nkhotakota District Quality Assurance/ PBI Programme Officer
Salima: Lab Technician
Mezuwa Banda, Nutrition Coordinator Nkhotakota
Maclean Mphenzi, District Malaria Coordinator, Nkhotakota
Central East Zonal HMIS officer MoH
Central West Zone
Stella Misomali, HMIS Officer Lilongwe DHO
Dr. Evelyn Chitsa Banda Central West Zonal Supervisor, Lilongwe
Bwaila Hospital Nursing Officer 
Dowa: DHO
District HMIS /M&E Officer for Lilongwe
Kasungu HMIS Officer
South East Zone
Dr. Dickson Mabulu, DMO-Balaka
Alinafe Mangulenje Zonal Nursing Officer–Zomba
Andrew Nkhoma, Programme Coordinator Safe Motherhood 
Sabola. HMIS-Balaka
SE Zonal supervisor-Zomba 
Christabel Namondwe SSDI-S team leader for Balaka
Thoko Bema, Zonal manager for Save the Children
SSDI-Services Evaluation 58
South West Zone
Dr. Alexander Chijuwa DHO Nsanje
Dr. Malangizo Mbewe, Zonal Officer-Blantyre
Stony Makunganya District Community Coordinator–Nsanje
District HMIS Officer–Nsanje
Coordinator–Safe motherhood Nsanje
District Nursing officer Nsanje
Felix Monia Health service Administrator Nsanje
Safe-Motherhood Coordinator Nsanje 
South West Zone Office
Dr. A Majidu, DHO-Chikwawa
Group Interviews with HF providers, Consumers Survey, FBGs and Community Score 
Cards
North Zone
Chitipa District
Kapenda health centre staff
Nthalire health centre staff
Karongo District
Kapenda health centre staff
Fulira health centre staff
Wiliro health centre staff
Central East Zone
Nkhota Kota District
Dwambadzi health centre staff
Ngala health centre staff
Salima District
Thavite health centre staff
Maganga health centre staff
Dowa District
Bowe health centre staff
Kasese health centre staff
Central West Zone
Kasungu District
Bua health centre staff
Simulemba health centre staff
Lilongwe District
Mitundu rural hospital staff
Khongoni health centre staff
South East Zone
SSDI-Services Evaluation 59
Phalombe District
Nambazo health centre staff
Gogo Nazombe health centre staff
Mangochi District
Nankumba
Katula
Machinga District
Nyambi
Mkwepele
Balaka
Mbela health centre staff
Kalembo health centre staff
South West Zone
Nsanje District
Nsanje DH staff
Masenjere health centre staff
Chikwawa District
Chapananga health centre staff
Ndakwera health centre staff
Key Informant Interview–Stakeholders and Implementing Partners at Central Level
Ministry of Health
Maganizo Monawe, CMED, Technical Advisor MoH
Felix Pensulo, Director of Nutrition, Department of Nutrition, HIV and AIDS
Dr. Henry Ndindi, Deputy Director of Clinical Services-Curative
Mrs. Rose Nyirenda, Director of HIV/ AIDS Department
Rhino Mchenga, CMED, Acting Head (MoH)
Egley Chirwa, Central East Deputy Zonal Supervisor MoH
Senior Nursing Officer, MoH, Lilongwe
Safe Motherhood Coordinator, MoH, Lilongwe
Dr. Jane Namasasu, Central Zonal Supervisor, MoH, Lilongwe
USAID
Peter Halpert, Health Office Director
Chimwemwe Chitsulo, M&E Specialist USAID Malawi
Ms. Evelyne Zimba, AOR SSDI-Services
Violet Orchardson, Nutrition Advisor, USAID
JHPIEGO
Dan Wendo, Chief of Party
Deliwe Malema, Deputy Chief of Party
SSDI-Services Evaluation 60
Lolade OSENI, M&E M&E Director 
William Twahirwa, PBI advisor 
SAVE THE CHILDREN
Catherine Mkangama, Snr. Technical Nutrition Advisor 
PLAN
Chitipa, Plan Representative
CARE
Rose Tchwenko, Assistant Country Director CARE.
DEVTECH
Carol Shepard, TL Communication Component Evaluation
Iain McLellan, Community Mobilization Specialist, Communication Component Evaluation
Franck, SSDI System Component Evaluation
 
SSDI-Services Evaluation 61
ANNEX 5. DOCUMENTS AND SOURCES REVIEWED 
1. Government of Malawi, SSDE Quarterly Progress Report FY 2012: Quarter 1: October 1, 
2011–December 31, 2011: USAID and Ministry of Health, Lilongwe.
2. Government of Malawi, SSDI- PBI Program Updates USAID/Ministry of Health. March 31, 
2015. Lilongwe.
3. Government of Malawi. SSDE-Quarterly Progress Report FY 2013 Year 2 Quarter 3 Progress 
Report (April 1, 2012–June 30, 2012). USAID and Ministry of Health, Lilongwe.
4. Government of Malawi. SSDE-Quarterly Progress Report, SSD-E Quarterly Report, FY 2013 
Year 2 Quarter 3 Progress Report (April- June 2013), USAID and Ministry of Health, 
Lilongwe.
5. Government of Malawi. Support for Service Delivery Integration-Services (SSDI-Services), 
Quarterly Progress Report, FY 2012-Quarter 2: January1, 2012–March 31, 2012, USAID and 
Ministry of Health, Lilongwe.
6. Jhpiego Database for Selected Best and Poor performing facilities in 13 SSDI Districts.
7. Jhpiego, SSDI Services Evaluation, List of Facilities with utilities and interventions.
8. Jhpiego, SSDI Services Evaluation, List of Key Stakeholders National.
9. Jhpiego, SSDI Services Evaluation, List of Key Stakeholders, Central East Zone.
10. Jhpiego, SSDI Services Evaluation, List of Key Stakeholders, Northern Zone.
11. Jhpiego, SSDI Services Evaluation, List of Key Stakeholders, South East Zone.
12. Jhpiego, SSDI Services Evaluation, List of Key Stakeholders, South West Zone_Rev.
13. Jhpiego, SSDI Services Evaluation, List of Other Trainers of Trainers.
14. Jhpiego, SSDI Services Evaluation, List of Trainers of Trainers, Central West Zone.
15. SSDI End of Project Final Report 2014.
16. SSDI Performance Monitoring and Evaluation Plan for support for Service Delivery 
Integration-(SSDI-Services) Project; Malawi USG Partner: Jhpiego, Time Period Covered: 
November 2011–September 2016.
17. SSDI-Performance Monitoring and Evaluation Plan For Support for Service Delivery 
Integration-Services (SSDI-Services) Project, Malawi USG Partner: Jhpiego, Time Period 
Covered: November 2011 to September 2016.
18. SSDI-Services FY 2013 PMP Indicators and Targets.
19. SSDI-Services FY2013 Annual Progress Report (October 2012-September 2013).
20. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2012 Issue No.3, USAID–
SSDI, April 2012–September 2012).
21. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013: Issue No.4, USAID–
SSDI, October 2012).
22. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2015 Issue No.5, USAID–
SSDI, October 2012–November 2012).
23. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.6, USAID–
SSDI, October 2012–December 2012).
24. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.7, USAID–
SSDI, October 2012–January 2013).
25. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.10, 
USAID–SSDI, October 2012–April 2013).
SSDI-Services Evaluation 62
26. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.12, 
USAID–SSDI, October 2012–June 2013).
27. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.13, 
USAID–SSDI, October 2012–July 2013).
28. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.14, 
USAID–SSDI, October 2012–August 2013).
29. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin (FY 2013 Issue No.15, 
USAID–SSDI, October 2012–September 2013).
30. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin, (FY 2012 Issue No. 2, 
USAID–SSDI, April 2012–August 2012).
31. SSDI-Services Results on Key Indicators; M&E Monthly Bulletin, FY 2012 Issue No. 1, USAID–
SSDI, July 2012.
32. SSDI-Services Self-Assessment and Rapid Situation Analysis Report, MOH, USAID, SSDI, June 
2012.
33. SSDI-Support for Service Delivery Excellence (SSD-E) Program Description-ATTACHMENT 
B.
34. USAID Inception Report AID 612-TO-16-00003 SSDI Activities (Services) Evaluation, 
Submitted by: DevTech Systems, Inc., July 19, 2016.
35. USAID, Request for Applications (RFA) Number 674-10-0051 Support for Integrated Service 
Delivery: Southern Africa, August 2010.
36. USAID. Support for Service Delivery Integration Services Program Year Three Work Plan 
October 2013-September 2014 October 2013-September 2014.
37. USAID. Support for Service Delivery Integration SSDI-Services, Progress Report FY2013 Year 
2: Quarter 2 (January–March 2013).
38. USAID/Malawi, Support for Service Delivery Integration Services Program, Year Two Work 
Plan October 2012-September 2013, (Submitted August 31, 2012-Resubmitted November 15, 
2012).
39. USAID/Malawi, Support for Service Delivery Integration Services Program, Year Three Work 
Plan October 2013-September 2014 (Submitted October 23, 2013).
40. USAID/Malawi, Support for Service Delivery–Excellence (SSD-E) Program, Year One Work 
Plan, October 2011-September 2012 (Submitted February 8, 2012).