i
CLAimHealth
COLLABORATING, LEARNING, AND 
ADAPTING FOR IMPROVED HEALTH ACTIVITY
Final Report
Remote and Hybrid Monitoring and Evaluation 
Activities of the USAID Health Project in the 
Philippines
February 28, 2022
DISCLAIMER
The views expressed in this report do not necessarily reflect the views of the U.S. Agency for 
International Development or of the U.S Government.
ii
This report was produced for review by the United States Agency for International Development (USAID). It was 
prepared by Panagora Group for the USAID/Philippines Collaborating, Learning and Adapting for Improved Health 
(CLAimHealth) Activity, IDIQ No. AID-OAA-I-15-00025, Task Order No. 72049218F00001 as a technical 
deliverable per Task Order Section F.6 (b) and therefore uses USAID branding as outlined in the CLAimHealth 
Branding and Marking Plan. 
Recommended citation: USAID/Philippines Collaborating Learning and Adapting for Improved Health 
(CLAimHealth) Activity. Remote and Hybrid Monitoring and Evaluation Activities of the USAID Health Project in the 
Philippines. December 2021.
Panagora Group Contact: 
Mary Ann Lansang, MD
Chief of Party
11/F, Ramon Magsaysay Center – Tower 
Manila1004, Philippines
iii
ACKNOWLEDGMENTS 
The Alliance for Improving Health Outcomes (AIHO) research team would like to thank USAID’s 
implementing partners – Bangsamoro Autonomous Region in Muslim Mindanao for Health
(BARMMHealth); Medicines, Technologies, and Pharmaceutical Services (MTaPS) Program;
ProtectHealth; ReachHealth; RenewHealth; TB Innovations and Health Systems Strengthening (TB IHSS);
and TB Platforms for Sustainable Detection, Care and Treatment (TB Platforms), as well as the 
CLAimHealth research team, for their support in the implementation of this research. We would also 
like to thank the Department of Health and local government units for their support and participation.
AIHO RESEARCH TEAM 
Michael Caampued, MD, MPM, MPH, MZP - Principal Investigator
Ezra Valido, MD, MPM, MPH, PhD (cand) - Implementation Research Consultant 
Miguel Camacho, MPH - Monitoring and Evaluation Consultant
Chiqui de Veyra, RMT - Project Manager
Kristine Alvina - Research Assistant
Joseph Aaron Joe - Research Assistant
Lynnel Alexie Ong, RMT - Research Assistant
iv
TABLE OF CONTENTS
ACRONYMS vi
DEFINITION OF TERMS ix
EXECUTIVE SUMMARY x
1. INTRODUCTION 1
1.1. Background and Rationale 1
1.2. Statement of the Problem 2
1.3. Objectives 2
2. BACKGROUND 3
2.1. USAID Philippines HP Background 3
2.2. USAID Approach to MEL 5
2.3. Remote and Hybrid M&E during COVID-19 7
2.4. The Implementation Research Approach 8
3. METHODOLOGY 10
3.1. Initial Desk Review 10
3.2. Modeling the Approach to Data Collection and Analysis 10
3.3. Development of Data Collection Tools 11
3.4. Evaluation Framework 13
3.5. Recruitment and Selection of Respondents 15
3.6. Data Collection 15
3.7. Data Analysis and Validation 16
3.8. Study Limitations 16
4. FINDINGS 18
4.1. Scoping Results 18
4.2. Summary of Data Collection Engagement 19
4.3. Overview of Monitoring and Evaluation of Selected USAID IPs 20
4.4. Effectiveness of Remote and Hybrid M&E Practices in Achieving M&E and Adaptive Management 
Goals 32
5. DISCUSSION 40
5.1. Implementation Contexts, Opportunities, and Challenges 40
v
5.2. Adaptiveness of Practices 41
5.3. Feasibility for Broader Use of the Practices and Their Sustainability 43
6. SUMMARY AND RECOMMENDATIONS 46
6.1. Summary 46
6.2. Recommendations for Strengthening the Remote and Hybrid M&E Systems (For USAID and IPs)
46
6.3. Recommendations for Strengthening Implementation of Remote and Hybrid M&E Practices 48
6.4. Specific Recommendations per Identified Remote and Hybrid M&E Practice 49
8. REFERENCES 59
ANNEX 1: CFIR DOMAINS AND CONSTRUCTS, 62
ANNEX 2: IDENTIFIED M&E DOMAINS 63
ANNEX 3: SUSTAINABILITY DOMAINS OF THE PSAT 64
ANNEX 4: DATA COLLECTION TOOLS 65
ANNEX 5: DATA COLLECTION TABLES 65
ANNEX 6: SUMMARY OF IDENTIFIED M&E PRACTICES BY CATEGORY 66
ANNEX 7: IMPLEMENTATION FIDELITY OF EACH IDENTIFIED PRACTICE IN THE SELECTED 
USAID IPs 70
ANNEX 8: CONSOLIDATED EVALUATION FRAMEWORK OF THE IDENTIFIED REMOTE 
AND/OR HYBRID M&E PRACTICES 72
vi 
ACRONYMS
ADAO Anti-Drug Abuse Office
AMELP activity, monitoring, evaluation, and learning plan
BARMM Bangsamoro Autonomous Region in Muslim Mindanao
BARMMHealth Bangsamoro Autonomous Region in Muslim Mindanao for Health
CBDR community-based drug rehabilitation
CFIR Consolidated Framework for Implementation Research
CiTEC Cebu City-Wide TB Elimination Campaign
CLA collaborating, learning, and adapting
CLAimHealth Collaborating, Learning, and Adapting for Improved Health
COVID-19 Coronavirus disease 2019
DDB Dangerous Drugs Board
DHIS 2 District Health Information System 2
DMO Development Management Officer
DOH Department of Health
DOH-PMIS Department of Health - Pharmaceutical Management Information System
DOTS directly observed treatment, short-course
DQA data quality assessment
DQC data quality check
FAST Finding cases Actively, Separating safely, and ensuring effective Treatment
FB Facebook
FGD focus group discussion
FHSIS Field Health Service Information System
FIO Field Implementation Officer
FP family planning
FTF face-to-face
FY fiscal year
GPPI good practices and promising interventions
HCW health care worker
HCWM health care waste management
HP health project
HIS health information systems
HSS health systems strengthening
vii
IMS information management system
IP implementing partner
IPC infection, prevention, and control
IR implementation research
ITIS Integrated TB Information System
KII key informant interview
KMITS Knowledge Management and Information Technology Service, DOH
LGU local government unit
LMICs low- and middle-income countries
M&E monitoring & evaluation
MDR-TB multidrug-resistant TB
MEL monitoring, evaluation, and learning
MOH Ministry of Health
MOV means of verification
MTaPS Medicines, Technologies, and Pharmaceutical Services Program
NAP NTP Adaptive Plan
NCR National Capital Region
NTP National TB Control Program
P&R pause and reflect
PBSP Philippine Business for Social Progress
PD Pharmaceutical Division, DOH
PHO provincial health office
PIRS performance indicator reference sheet
PMDT Programmatic Management of Drug-Resistant TB
PSAT Program Sustainability Assessment Tool
RHU rural health unit
SCM supply chain management
SI strategic information
SMS short messaging service
SOP standard operating procedure
STC satellite treatment center
TA technical assistance
TB tuberculosis
TBCC TB Contact Center
viii
TB IHSS TB Innovations and Health Systems Strengthening
TB LON TB Local Organizations Network
TB Platforms TB Platforms for Sustainable Detection, Care, and Treatment
TOC theory of change
TPM third-party monitoring
TPT TB preventive treatment
UNODC United Nations Office of Drugs and Crime
USAID United States Agency for International Development
WHO World Health Organization
ix
DEFINITION OF TERMS
Health Project USAID/Philippines Health Project implemented from 2019 to 2024
Activities These refer to the HP’s nine implementing partners. Specifically of 
interest to the study are the following seven activities: ReachHealth, 
BARMMHealth, TB Platforms, TB IHSS, ProtectHealth, MTaPS, and 
RenewHealth
Remote Online/virtual; non-face-to-face
Hybrid Combination of in-person and remote
M&E activities Involves planning, data collection, analysis, validation, consolidation, 
interpretation, use, and dissemination
Practice Interventions relating to the application of technology to support an 
M&E activity or enhance its systems even before the COVID-19 
pandemic; any process or activity meant to build systems for M&E. 
“Practice” may include the following activities: data collection, 
management, analysis, validation, consolidation, interpretation, 
reporting or dissemination, and utilization.
Effectiveness of an M&E activity An M&E activity is considered effective when it has facilitated the use of 
quality M&E data to inform decisions or actions, such as program 
adaptations or adjustments to more effectively achieve Health Project 
and implementing partners’ outcomes. Other factors such as the 
practice design and implementation, as well as its integration within the 
general M&E system, are also considered. 
Fidelity Degree to which a practice was implemented as it was prescribed in 
the original protocol or as it was intended by the program developers1
Adaptations Deviations that are corrective actions and produce results toward 
overall objective2,3,4
Mandate An official order or commissioning for the practice, whether it was 
formally written or informally declared by authority.
Feasibility Extent to which a newly implemented practice can be successfully used 
or carried out within a given agency or setting5
Sustainability Extent to which a newly implemented practice is maintained or 
institutionalized within a service setting’s ongoing, stable operations6
Penetration Integration of a practice within a service setting and its subsystems7
1 Proctor, Enola et al., 2011. 
2 Allen, Jennifer et al., 2012. 
3 Pérez, Dennis et al, 2016.
4 Brownson, Ross et al., 2017. 
5 Proctor, Enola et al., 2011.
6 Proctor, Enola et al., 2011.
7 ibid.
x 
EXECUTIVE SUMMARY
Monitoring and evaluation (M&E) activities have been a core component of the U.S. Agency for 
International Development (USAID) Philippines Health Project (HP) (2019-2024) in implementing its 
activities and assessing progress toward its goals. With the persistence of the Coronavirus disease 2019 
(COVID-19) pandemic, the HP had to adjust its usual M&E activities considerably, shifting to largely 
remote and hybrid approaches. The impact of this shift on the performance of M&E and attainment of 
goals is largely unknown. 
This assessment, implemented by the Alliance for Improving Health Outcomes (AIHO) with support 
from the USAID Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) project, 
aims to document and assess the effectiveness of these remote and hybrid M&E practices and, based on 
findings, propose recommendations for the HP and the country. The practices were evaluated based on 
the implementation outcomes of fidelity, feasibility, and sustainability. The study developed a weighted
evaluation framework (with scoring from 0-3) which allowed for the identification of “good practices” 
that can be sustained within the project’s contexts. Only practices classified as Level 3 were considered 
as a good practice and promising intervention (GPPI).
The study explored the general M&E system of seven USAID HP implementing partners (IPs), 
ReachHealth, BARMMHealth, MTaPS, ProtectHealth, RenewHealth, TB Platforms, and TB Innovations 
and Health Systems Strengthening (TB IHSS), including their adaptations during the COVID-19 
pandemic. All IPs adjusted their M&E activities to allow continuation despite lockdowns and other 
community safety measures during the pandemic. A total of 17 remote and hybrid M&E practices were 
identified across the seven IPs. The M&E practices can be classified into six categories: phone-based, 
web-based, videoconferencing, information management system (IMS), remote access software, and 
hybrid practices. A weighted evaluation of this assessment’s findings placed two practices at Level 0, 
three at Level 1, five at Level 2a, three at Level 2b, and three at Level 3. The study only evaluated 16 out 
of the 17 identified practices due to misconstructions in the definition of one practice.
The following themes characterize the implementation of remote and hybrid monitoring across the IPs: 
1) remote and hybrid monitoring generally use digital technologies, 2) implementation is designed based 
on the need of the IPs, 3) the adapted M&E approaches heavily rely on external partners, and 4) 
implementation of remote and hybrid monitoring during the pandemic is dynamic. 
xi 
Weighted Evaluation of Each Identified Practice in Selected USAID IPs 
IP Remote and Hybrid M&E Practice Weighted Evaluation
ReachHealth
District Health Information System 2 (DHIS 2) Level 2a
Remote phone monitoring Level 1
Hybrid rapid feedback mechanism Level 2a
BARMMHealth
 
Hybrid monitoring of family planning (FP) program 
through site visits, phone calls, and Facebook (FB) 
messenger
Level 2a
Blended pause and reflect (P&R) sessions Level 2a
KoboToolBox Level 2b
MTaPS
Remote access to information systems 
(Department of Health - Pharmaceutical 
Management Information System [DOH-PMIS] and 
Integrated TB Information System [ITIS])
Level 1
 COVID-19 Special Response Project- hybrid 
monitoring of facilities (onsite and virtual) Level 2b
ProtectHealth Cebu City-Wide TB Elimination Campaign (CiTEC)
project regular online conference calls Level 3
RenewHealth Community-based drug rehabilitation (CBDR) IMS Level 0
TB Platforms
 
Hybrid monitoring of TB Contact Center using 
regularly scheduled phone calls and onsite 
monitoring/mentoring visits
Level 2b
Hybrid monitoring/mentoring for Finding cases 
Actively, Separating safely, and ensuring effective 
Treatment (FAST) implementing facilities
Level 1
Assessment of infection, prevention, and control 
(IPC) using Google Survey Level 0
Remote monitoring and mentoring of 
Programmatic Management of Drug-Resistant TB 
(PMDT)
Level 2a
TB IHSS
 
 
Remote P&R sessions Level 3
Remote rapid assessment survey Level 3
In general, the remote and hybrid monitoring across the IPs have maintained fidelity to M&E plans and 
achievement of project goals. Remote and hybrid monitoring substituted and/or complemented face-to-
xii
face (FTF) M&E activities and were viewed as opportunities to expand and modify original plans. Digital 
technologies used in remote and hybrid monitoring also improved the efficacy of integrating data and 
generating knowledge.8 
Most of the remote and hybrid monitoring practices in the IPs were feasible to scale up due to existing
perceptions or beliefs about the relative advantages of the practices when FTF is not viable. The 
technologies adopted were low cost, widely used, and available. The presence of leadership support and 
policies also clarified roles and expectations. However, the IPs faced challenges when adopted digital 
technologies had limited uptake or were inappropriate for the particular situation. The study further 
assessed the potential sustainability of each practice within and beyond the life of the IPs. Practices with 
limited sustainability had persisting implementation issues in funding stability, organizational capacity, and 
environmental support. 
This assessment of remote and hybrid M&E practices within the HP found that digital technologies play a 
vital part in responding to the need for remote M&E solutions. Hybrid practices consistently showed an 
optimal level of M&E effectiveness and acceptability. It is then expected that post-pandemic ways of 
working in M&E should employ more hybrid practices where remote methods will complement FTF 
activities, rather than fully replacing them.
Strengthening M&E systems for increased remote and hybrid application would require digital training, 
commensurate investments, use of multiple tools and methods to improve data validity, understanding of 
context, and end-user perspective studies. Implementation, in particular, can be strengthened by 
reducing complexity and generating user confidence in the collected data. Additional studies are 
recommended to objectively and extensively assess the data quality from remote practices compared to 
FTF methods, particularly in low- and middle-income countries. 
8 For instance, information management systems, such as KoboToolbox, improved knowledge generation by allowing for 
deeper or more efficient analysis and visualization of data.
1
1. INTRODUCTION
1.1. Background and Rationale
The USAID Philippines Health Project (HP) (2019-2024) has a nationwide outreach through nine 
activities9 to achieve its goals of strengthening healthy behaviors, fortifying quality services, and 
bolstering health systems. M&E activities have been a core component of the HP to generate, use, and 
assess the integrity and quality of family planning (FP), tuberculosis (TB), and community-based drug 
rehabilitation (CBDR) activities. 
Efforts to assess the validity, reliability, timeliness, precision, and relevance of HP performance data have 
taken different forms including data quality assessments (DQAs), data validation activities, and third￾party monitoring (TPM). These assessments have generated critical insights that have informed program 
management, service delivery, and policy and decision making. 
For example, a recent data validation of FP performance data helped the HP identify issues on FP stock 
management, FP data reporting and recording, and lack of training on standard information systems 
procedures, such as the 2018 Field Health Services Information System (FHSIS) Manual of Procedures. 
Additionally, TPM and continuous evaluation methods have supported TB and FP program 
implementation in the complex post-conflict environment of Marawi City and other areas of the 
Bangsamoro Autonomous Region in Muslim Mindanao (BARMM).
With the persistence of the COVID-19 pandemic, the HP has had to make considerable changes to its 
usual M&E activities, shifting to largely remote approaches (e.g., by telephone, e-mail, online 
collaborative workspaces, and/or Internet communications) and sometimes hybrid approaches (blended 
use of remote communications and in-person/site visits). 
The effectiveness of these various innovations in enhancing the performance of M&E and attainment of 
goals is still largely unestablished. Culling important and scalable processes, as well as identifying factors 
that affect the way M&E systems are able to function, are worthwhile measures to learn what is and is 
not working well in this new environment. Because implementation will vary significantly based on 
context, factors such as monitoring perspectives, stage of implementation, and local settings must also 
be understood.
This assessment, implemented by the Alliance for Improving Health Outcomes (AIHO) with support 
from the USAID Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) project, is 
focusing on the efficacy in the shift from the standard, in-person, to remote (and hybrid) approaches for 
M&E in the Philippines. This assessment’s findings, together with current understanding around the 
broader developing science and practice on M&E and IR, can help define strategic recommendations for 
the HP and the country.
9 The nine activities: ReachHealth, BARMMHealth, TB Innovations, TB Platforms, TB LON, ProtectHealth, MTaPS, 
RenewHealth, and CLAimHealth
2
1.2. Statement of the Problem
1. In the context of COVID-19 community quarantine rules and limited mobility, are remote or 
hybrid M&E activities effective in achieving the M&E and adaptive management goals of the HP in 
the Philippines?
2. As remote approaches are expected to increasingly become part of the standard mode of 
conducting M&E, how can remote and hybrid M&E activities be enhanced in the HP?
1.3. Objectives
1. Document the processes involved, facilitators, and hindrances in remote and hybrid M&E 
activities, including quantitative and qualitative approaches used to monitor and assess the 
quality of interventions.
2. Assess the effectiveness of remote and hybrid M&E activities in terms of ability to accurately 
answer HP performance questions and achieve adaptive management goals, taking into 
consideration data validity, reliability, timeliness, precision, and integrity. 
3. Assess the potential feasibility and sustainability (including during and post-COVID-19) of 
remote and hybrid M&E activities in terms of time and resources spent and ease of operations. 
4. Provide recommendations for quality improvement of M&E activities moving forward, including 
during and post-COVID-19. 
3 
2. BACKGROUND
2.1. USAID Philippines HP Background
USAID/Philippines HP (2019-2024) seeks “to improve the health of underserved Filipinos.” This goal is 
consistent with objectives set in the Sustainable Development Goals, Philippine Development Plan, and 
National Objectives for Health. To achieve this, HP's strategy is to institutionalize and strengthen the 
Philippine health system to enable: 1) healthy behaviors, 2) quality health services, and 3) agile health 
system functions. The goal and sub-purposes of the HP are outlined in the results framework (Figure 1). 
Figure 1. USAID HP Results Framework
HP's activities focus on TB, FP (including maternal, neonatal, and child health and nutrition), health 
systems strengthening (HSS), and drug demand reduction (see Table 1). The project sites for these 
activities are selected based on greatest need, prioritizing underserved populations. Specifically, the main 
geographic focus areas are where the TB burden, unmet need for modern FP methods, and teenage 
pregnancy rates are the highest.10 This assessment focuses on the M&E systems of seven of these 
activities. 
Table 1. USAID/Philippines HP activities, 2019-202411
10 USAID/Philippines, 2019. 11 USAID/Philippines, 2021. 
4 
Thematic 
Areas/Portfolio
USAID HP Activity* Estimated 
Timeline
Status in 
Fiscal 
Year (FY) 
2020
TB and multidrug￾resistant TB (MDR￾TB)
Treat TB: Supporting MDR-TB Activities in 
the Philippines
September 2016 –
March 2019
Ended
TB Innovations and Health Systems 
Strengthening (TB IHSS)
February 2018 – 
February 2023
Current
TB Platforms for Sustainable Detection, 
Care and Treatment (TB Platforms)
April 2018 –
April 2023
Current
TB Local Organizations Network (TB LON) October 2020 –
September 2023
Current
FP/Maternal, 
Neonatal, Child 
Health and 
Nutrition/Adolescent 
Reproductive Health
Community Maternal, Neonatal, Child Health 
& Nutrition Scale-up 
August 2016 –
December 2019
Ended
(Q1 FY 
2020) 
FP/Maternal and Neonatal Health 
Innovations and Capacity Building 
Platforms (ReachHealth)
December 2018 – 
November 2023
Current
Bangsamoro Autonomous Region in 
Muslim Mindanao Health Capacity 
Building (BARMMHealth)
February 2019 –
February 2024
Current
HSS Institutionalization of the Health Leadership 
and Governance Program
July 2017 –
September 2020
Ended
(Q4 FY 
2020) 
Health Equity and Financial Protection 
Platform (ProtectHealth)
March 2019 –
March 2024
Current
Medicines, Technologies, and 
Pharmaceutical Services Program 
(MTaPS)
September 2018 –
September 2023
Current
Human Resources for Health 2030/Philippines October 2017 –
June 2020
Ended 
(Q3 FY 
2020) 
Drug Demand 
Reduction
Expanding Access to CBDR Program in 
the Philippines (RenewHealth)
May 2019 – May 
2024 
Current
Monitoring, 
Evaluation, and 
Collaborating, Learning and Adapting for 
Improved Health (CLAimHealth) 
March 2018 –
March 2022
Current
5 
Learning (MEL)
*The study will only focus on M&E systems of the seven activities in bold font.
Robust MEL underpins the broad range of all of HP’s activities. Each implementing partner (IP) uses an 
Activity MEL Plan to help manage the process of assessing and reporting progress toward achieving 
project outputs and outcomes and to identify what questions will be addressed during evaluation.12
Results from the implementation of these activities are measured through a set of indicators at outcome 
and output levels for HP’s three sub-purposes. Each activity’s specific theory of change (TOC) is aligned 
with the HP goal and results framework. These TOCs organize the indicators into a frame to show how 
objectives will be met at both the activity and project level.
The activities also have performance indicators for cross-cutting areas (i.e., sustainability, gender equality 
and women’s empowerment, supply chain management [SCM], informed choice and voluntarism, private 
sector and civil society organization engagement, and climate risk management) important to USAID 
policies and strategies.13 Evaluation activities are conducted to provide information on implementation 
challenges and successes, as well as an overall understanding of performance. USAID activities conduct 
internal evaluation routinely but may also collaborate with external evaluators commissioned by the 
agency. Learning is collaborative throughout the program cycle to strengthen existing practices and 
processes. HP has a specific activity, CLAimHealth, that provides technical assistance (TA), advisory 
services, and relevant logistical support for MEL of the other activities.14
 
One of the major challenges for USAID is assessing these different project frames within a singular M&E 
frame. Tracking of HP’s progress overall needs to account for the activities’ being at varying stages of 
implementation, which means that data available to monitor their performance at any given time will 
vary. Monitoring is affected their performance frameworks that evolve over time, as new indicators and 
highly contextual issues emerge in the implementation of the individual activities and the HP as a 
whole.15 
2.2. USAID Approach to MEL
An M&E system is a set of interconnected processes, tools, and policies for systematic collection, 
analysis, and use of information. A strong M&E system facilitates effective project implementation by 
using the collected M&E data to support evidence-based decision making, promote accountability, and 
generate learnings to inform adaptations and improvements of programs and interventions.16
Monitoring
Monitoring plays a critical role throughout the program cycle of USAID. Monitoring data are used to: 1) 
assess whether programming is achieving expected results, 2) adapt existing activities, projects, and 
12 USAID, n.d. 13 USAID/Philippines, 2019. 14 USAID/Philippines, 2017. 15 USAID/Philippines, 2019. 16 USAID, n.d.
6 
strategies as necessary, and 3) apply USAID learning to the design of future strategies and programming. 
There are three approaches to monitoring: performance, context, and complementary monitoring. Only 
the first approach is required by USAID’s Automated Directives System. However, a well-rounded 
monitoring plan may employ all three approaches, provided that they fit into the programming needs 
and culture.17 
Performance monitoring is the ongoing and systematic collection of performance indicator data and 
other quantitative and qualitative information to reveal whether implementation is on track and 
expected results are being achieved. This includes monitoring the quantity, quality, and timeliness of 
activity outputs within the control of USAID and its implementers, as well as the monitoring of project 
and strategic outcomes that are expected to result from the combination of these outputs and other 
factors.18 USAID has identified five pathways in performance monitoring: 1) establishing performance 
indicators, 2) performance baselines and targets, 3) monitoring assumptions, 4) collecting, reporting, and 
sharing performance data, and 5) DQAs.19
USAID has identified five data quality standards that all performance monitoring indicators must meet:20
1. Validity- Data clearly and adequately represent the intended result
2. Integrity- Data have safeguards to minimize the risk of transcription error or data manipulation
3. Precision- Data have sufficient level of detail to permit management decision making
4. Reliability- Data reflect consistent collection processes and analysis methods over time
5. Timeliness- Data are available at useful frequency, are current, and timely enough to influence 
management decision making
Evaluation 
Evaluation is the systematic collection and analysis of data and information about the characteristics and 
outcomes of the project to understand and improve effectiveness and efficiency as well as to inform 
decisions about current and future programming.21 Along with monitoring, evaluation contributes 
evidence to improve strategic planning, project design, and resource design and is part of a greater body 
of knowledge and learning.22 USAID has identified five pathways in evaluation: 1) planning for evaluation 
during design, 2) managing the evaluation process, 3) high quality performance evaluations, 4) rigorous 
impact evaluations, and 5) following up on evaluations.
Collaborating, Learning, and Adapting
Collaborating, learning, and adapting (CLA) is a set of practices that helps USAID improve its 
development effectiveness. Integrating CLA into USAID’s work helps ensure that programs are 
coordinated with others, grounded on a strong evidence base, and iteratively adapted to remain relevant 
17 USAID, 2016. 18 ibid.
19 USAID, 2019. 20 USAID, 2016. 21 USAID Learning Lab, 2015. 22 USAID, 2021. 
7 
throughout implementation. The systematic application of CLA approaches largely springboards from 
the effective conduct of an M&E activities. CLA is led by people who have the knowledge and resources
to carry this out and enables the agency to be an effective learning and development organization.23
2.3. Remote and Hybrid M&E during COVID-19 
The COVID-19 pandemic has dramatically changed the general ways of working in the development and 
humanitarian sector. The limited fact-to-face (FTF) contact brought by social distancing, community 
lockdowns, and travel restrictions to control the spread of the virus has disrupted the “normal” way of 
doing programming and M&E globally. Specifically, this has had an impact on M&E activities from 
planning, data collection, data analysis, data validation, reporting, and providing feedback, up to 
dissemination, which all had to be done via phones, tablets, or other virtual platforms. Remote data 
collection activities include phone-based or online interviews and focus group discussions (FGDs), audio￾diary methods, photovoice, video documenting, social media analysis, and auto-ethnography for 
qualitative data; and phone-based surveys using short messaging service (SMS), interactive voice 
response, computer-assisted telephone interviews, self-administered postal questionnaires, and web￾based survey questionnaires for quantitative data.24,25 A case study in Zimbabwe found an increased use 
of remote data collection methods and secondary data, data collection reduction only to crucial 
information, simplification of data collection methods and tools, and rethinking of sampling designs in 
organizations.26 In non-permissive environments, USAID activities have relied on TPM systems to 
supplement monitoring data and/or verify IP reports. TPM refers to the systematic and intentional 
collection of performance monitoring and/or contextual data by a partner that is not USAID or an IP 
directly involved in the work.27 These trends can be observed for the M&E practice globally.28
Although some M&E activities have been conducted remotely in the past, especially in conflict/crisis￾affected settings, the COVID-19 pandemic magnified the need for such approaches in an unprecedented 
way worldwide. Remote M&E activities ensure the safety of staff and project participants, have wider 
coverage, reduce costs and time in data collection, and encourage more participation and honesty from 
anonymous respondents. However, switching to remote also poses some challenges including significant 
time needed to set up trainings, investments, and roll out; more easily compromised data quality and 
potential breaching of data security; and the need to invest in data collection devices.29 Moreover, not all 
M&E practitioners have experience or skills in conducting online activities and using virtual platforms; 
and the impact of the shift to online, technology-based activities on the income and gender digital divide 
and the social isolation of people with disabilities is unknown.30,31 More studies are needed to establish 
the quality, reliability, and accuracy of remote data collection methods in low- and middle-income 
23 Jessica Z. 2017. 24 Hensen, Bernadette et al., 2021. 25 Greenleaf, Abigail et al., 2017. 26 Groupe URD & CartONG, n.d. 27 USAID, 2021. 28 Dube, Thulane et al., 2021. 29 Women’s Refugee Commission, 2020. 30 Dube, Thulane et al., 2021. 31 Ukaid, n.d
8 
countries (LMICs).32
Some international organizations like the USAID and War Child Canada and Women’s Refugee 
Commission developed guidance documents for their staff and IPs on how to conduct remote M&E 
activities in their programming during the pandemic. The guidance documents provided key points to 
consider before implementing remote monitoring methods, which include understanding the information 
to collect, conducting an internal and external environmental analysis, selection of potential methods, 
identification of tools (phones, Internet, survey tools) to use, availability of geospatial remote monitoring 
tools, integration of community-based approaches, and finding additional public resources.33,34 USAID’s
guidance highlights the fact that the appropriate use of remote M&E methods varies considerably in the 
context where these methods will be implemented.
In this current operating environment, USAID and its IPs face new challenges in implementing activities, 
monitoring progress, collecting data, and tracking indicators.35 The current context moves development 
actors to consider adopting remote monitoring and management strategies not only for security or 
disease outbreak reasons, but also as a permanent measure with the broader aim of building local 
capacities and strengthening accountability among its partners.36
Ensuring data quality in remote monitoring
Ensuring the quality of data collected through remote and digital channels may also pose a 
methodological challenge. Thorough yet remote supervision of the fidelity and quality of the data 
collected may be ensured through the following strategies: 1) using automated systems for quantitative 
data collection to ensure that human error is kept to a minimum, 2) using a smartphone application to 
record calls as quality assurance method, 3) duplicating spot-checking of interviews by making three-way 
phone call with a supervisor, and 4) avoiding questions with long lists of answer choices and instead 
using a modular questionnaire design to lower the chance of high non-response rates.37 Prior to 
investing in equipment and human resources, the sustainability of remote data collection over the 
course of a project should be evaluated. Factors such as capacity, culture and acceptance, and resources 
within restrictive and remote environments may be taken into consideration.38 
2.4. The Implementation Research Approach 
Implementation research (IR) is the scientific inquiry into questions concerning any aspect of 
implementation. IR seeks to understand how an intervention works within real world conditions, taking 
into great consideration the context in which implementation occurs and factors exerting influence on 
32 Greenleaf, Abigail et al., 2017. 33 Women’s Refugee Commission, 2020.
34 USAID, 2020.
35 ibid. 36 Women’s Refugee Commission, 2020. 37 JSI, 2020. 38 Women’s Refugee Commission, 2020.
9 
it.39, 40 In the case of the HP, an analysis of desired implementation outcomes may help assess the M&E 
practices with respect to how they have adapted to changes in the operational environment, whether as 
a result of COVID-19 or other contextual needs (such as insurgency). Implementation outcome 
variables (e.g., fidelity, feasibility, sustainability) describe the intentional actions to deliver services and 
serve as indicators of the success implementation.41 
 
Fidelity is the degree to which programs are implemented as intended. Implementation fidelity is typically 
measured by comparing the original evidence-based intervention and the implemented (or adapted) 
intervention in terms of adherence to the program plan, dose or amount of program delivered, and 
quality of program delivery.42 Feasibility is defined as the extent to which an innovation can be 
successfully carried out within a given agency or setting.43 Typically used in retrospect, the concept is 
used to arrive at a potential explanation for an initiative’s success or failure. Sustainability refers to the 
extent to which an intervention is maintained or institutionalized within a service setting’s ongoing, 
stable operations.44 
Health service researchers make use of an IR approach to perform formative evaluations to assess the 
extent to which implementation is effective45 in a specific context, prolongs sustainability of an 
intervention in that setting, and promotes dissemination into other contexts.46 In formative evaluations, 
a progress-focused evaluation is appropriate for assessing concurrent implementation progress (as in the 
case of the HP). This type of evaluation is intended to optimize interventions and/or reinforce progress 
via positive feedback to key players through an analysis of impacts and indicators of progress toward 
goals.47 Formative evaluations may be guided by the Consolidated Framework for Implementation 
Research (CFIR), which is a framework for assessing context in terms of existing or potential barriers 
and facilitators to successful implementation.48 The CFIR provides a comprehensive taxonomy of 
operationally defined constructs that may exert influence on the implementation of complex programs.49
The CFIR organizes these constructs into five domains (i.e., intervention characteristics, outer setting, 
inner setting, characteristics of individuals, and process) and provides a repository of standardized 
factors that influence implementation effectiveness.50,51 The list and brief description of the domains may 
be found in Annex 1. 
39 Peters, David et al., 2013. 40 USAID, n.d. 41 Proctor, Enola et al., 2011. 42 ibid.
43 Karsh, B., 2004. 44 Proctor Enola et al., 2011. 45 “Effective” in the case of IR looks into the success of achieving implementation outcomes. This differs from the definition of 
an “effective M&E” which pertains more to success in guiding performance or goal attainment. 46 Damschroder, Laura et al., 2009. 47 Stetler, Cheryl et al., 2006. 48 CFIR Research Team, 2021. 49 Damschroder, Laura et al., 2009.
50 B Karsh, 2004. 
51 Means, Arianna Rubin et al., 2020.
10
3. METHODOLOGY 
The assessment adopted a progress-focused implementation evaluation under the assumption that 
remote and hybrid M&E practices are concurrently being implemented across the various activities. The 
assessment analyzed the effectiveness of these adapted practices in achieving the M&E and adaptive 
management goals of HP and its activities. In particular, it closely investigated their fidelity to standard 
operations, M&E plans, and adaptive management goals, as well as their ability to achieve data quality 
standards (validity, reliability, timeliness, precision, and integrity). The assessment also evaluated the 
M&E practices based on the implementation outcomes of fidelity, feasibility, and sustainability. This was 
carried out through the following systematic steps: 
3.1. Initial Desk Review
We conducted an initial desk review of selected activity MEL plans, work or implementation plans, 
progress reports (annual and quarterly), and performance indicator references sheets (PIRS) to gain a 
fundamental understanding of HP’s current overall M&E system and how this relates to the variable 
activity-specific M&E systems as well as to aid in framing the questions for the data collection.
We also reviewed relevant literature to underpin the study’s methods and frameworks for analysis. This 
included a scoping of global evidence on remote and hybrid M&E practices. The online publications 
reviewed were systematically selected through an expert-guided literature search, which focused on 
suggested key subject headings and research-related inquiries. The terms “monitoring and evaluation” 
and “implementation research,” appended with the subject headings outlined in Table 2, were searched 
in the following databases: USAID, Google Scholar, and Google. 
Table 2. Subject headings used for the literature search
Monitoring and Evaluation Implementation Research
• Remote monitoring
• Effectiveness of
• USAID (+ names of various activities)
• Performance report
• Philippines
• COVID-19 
• Data quality
• Data integrity
• M&E system
• Online
• Progress-focused
• Fidelity evaluation framework
• Feasibility evaluation framework
• Sustainability evaluation framework
3.2. Modeling the Approach to Data Collection and Analysis
We developed a consolidated assessment protocol combining separate assessments on M&E 
11
effectiveness and analysis of desired implementation outcomes to respond to the main research inquiry: 
Are remote or hybrid M&E activities effective in achieving the M&E and adaptive management goals of the 
USAID/Philippines HP?
We collected qualitative information that allowed comparison and selection of good practices for 
remote and hybrid M&E, looking into process flow, data validity, reliability, timeliness, precision, integrity 
of data, and achievement of the M&E and adaptive management goals. Specifically, we organized the M&E 
questions to inform four general activities namely: 1) monitoring of inputs and activities, 2) monitoring of 
outputs and outcomes, 3) monitoring of risks and assumptions, and 4) evaluation. 
We used an IR approach for the research question to assess the implementation outcomes of fidelity, 
feasibility, and sustainability. We assessed remote and hybrid M&E practices based on their adherence 
toward the standard operating procedures (SOPs), plans and objectives, TOCs/logical frameworks, or 
Mission and Vision in relation to their M&E and their feasibility for scale-up or application in other 
project settings. 
We also built a consolidated tool by combining the approaches or questions delineated by the M&E and 
IR consultants. For analysis, information that answered the M&E inquiry was culled first, followed by the 
extraction of information to answer the IR inquiry. Results of these analyses informed 
recommendations. We then conducted a triangulated analysis and cross-tabulation summarizing analysis 
of pertinent domains on M&E effectiveness and implementation evaluation to come up with an answer 
to the main inquiry and identify good practices.
3.3. Development of Data Collection Tools 
We developed a tool for documenting the processes of various M&E activities, both as prescribed in the 
initial activity MEL plan and how they were actually conducted. We also reviewed analytical tools for 
M&E and IR which allowed the generation of an exhaustive list of questions to be streamlined and used 
for data collection. 
M&E analytical tool
We reviewed the Monitoring and Evaluation Capacity Assessment Toolkit,52 specifically the individual 
and group assessments, to frame questions to suit the needs of the study. We added questions on 
remote M&E activities as well as the M&E and adaptive management goals based on an identified set of 
domains (Annex 2), breaking down or specifying items as needed, modifying some item scorings to 
either a 5-point Likert scale or a binary response (Yes or No) to allow relative comparison of the pre￾practice (using recall) and during practice scores. A “practice,” as referred to in the developed tool, 
pertains to the introduction or strengthening of the remote and/or hybrid M&E activities as a revision 
from the common practice or an adaptation due to the COVID-19 pandemic. 
IR analytical tool
52 MEASURE Evaluation PIMA, 2017. 
12
We used the CFIR53 as a general framework for assessing remote and hybrid M&E activities in progress 
across the various activities. The assessment considered the following for the domains: the 
“intervention” as being the remote and hybrid M&E practices across the various activities; the “outer 
settings” as the USAID HP and local partners within the current context of COVID-19; the “inner 
settings” as the USAID IPs; and the “individuals involved” as the implementers of the M&E systems. The 
various constructs of the CFIR provided guidance in the formulation of the questions. 
Implementation Fidelity: Fidelity is conceptualized as adherence of implementation to the content, 
frequency, duration, and coverage intended in a practice’s design. Figure 2 shows the process this 
assessment devised to evaluate fidelity of the practices to the HP’s M&E plans. This first involved 
identifying core components or key action points in the practice’s design/plan from selected source 
documents. We checked for presence or absence of actions that deviated from the plans, and deviations 
that contribute to the overall goal were labeled as “adaptations.” We assessed the key advantages and 
issues in their adaptations and deviations using the CFIR framework. The strength of this measure is 
dependent on the presence of a plan or an SOP. In the absence of plans or SOPs, assessments were 
based on the practice’s designs and plans as stated in written documents or interviews with project staff 
and by referencing against activity-level goals. Then, we characterized practices as on track, delayed, 
stalled (temporarily stopped), or aborted. On track and delayed practices with adaptations were 
considered as having good fidelity. Practices that were stalled and aborted were considered as having 
poor fidelity.
Figure 2. Process of Evaluating Fidelity for Each Practice 
Feasibility: Using the CFIR, we identified enablers and challenges in the implementation of each practice 
and categorized them at the individual (i.e., central office, field project, or field counterpart staff) and 
organizational level (Figure 3). All practices with good fidelity and with no major issues at the level of 
operational individuals are feasible to scale. Unresolved issues pertaining to practice mandates, budget, 
and training are major issues. Practices with major issues at the level of operational individuals are not 
feasible to scale at the moment. All practices with only major issues at the organizational level can still 
be feasible to scale. 
53 CFIR Research Team, 2021.
13
Figure 3. Enablers and Challenges Faced by Operational Individuals and the Organization 
in the Implementation of Each Practice 
Sustainability: The Program Sustainability Assessment Tool (PSAT) is a new and reliable instrument that 
allows programs to accurately assess their capacity for sustainability across eight domains: environmental 
support, funding stability, partnerships with stakeholders, organizational capacity, program evaluation, 
program adaptation, communication with stakeholders, and strategic planning.54,55 A description of each 
sustainability domain is found in Annex 3.56 We converted the sustainability domains into a checklist and 
identified domains present in each practice along with their corresponding source (documents, key 
informant interview [KII], FGD). All practices with good fidelity and feasibility with no issues on core 
sustainability domains (environmental support, funding stability, and organizational capacity) had a high 
chance of being sustainable. 
After performing a review of existing analytical tools for M&E and IR, we developed the tools for data 
collection (online survey, KII, and FGD). Questions in these tools looked closely into the complexity of 
each practice, and findings fed into the series of evaluative frames. We sought input and feedback on the 
data collection tools from the CLAimHealth M&E team and pre-tested the tools with other members of 
the said team. We then revised and finalized the tools accordingly (Annex 4). 
3.4. Evaluation Framework
We assessed each identified practice using the evaluation frames for M&E and the three implementation 
outcomes (fidelity, feasibility, and sustainability) for IR. The evaluation frame for M&E tabulated 
54 Luke, Douglas et al., 2014.
55 PSAT, 2019. 56 Center for Public Health Systems Science, 2012.
14
descriptions and identified gaps and challenges against five identified domains. The IR fidelity frame 
tabulated identified practices against implementation status, adaptations and deviations, and perceptions 
culled from the KIIs/FGDs. The IR feasibility frame tabulated the enablers and challenges at the personal 
and organizational level, while the IR sustainability frame tabulated verifications and their sources against 
the eight PSAT domains. The M&E and IR frames allowed identification of “good practices” that can be 
sustained within the project’s contexts. The results of the M&E and IR assessment of each practice fed 
into the consolidated evaluation frame in which each identified practice underwent a weighted and 
narrative evaluation for a richer depiction. Each practice was classified according to the five levels 
described in Table 3. These levels were derived from a consensus of the experts within this study. Only 
practices classified as level 3 by the evaluation will be considered as sufficient for good practice and 
promising intervention (GPPI) documentation. 
Table 3. Remote and/or hybrid M&E practices levels based on the weighted evaluation
Levels Description
Level 0 M&E: Practices at the planning stage without allocated resources; several gaps and 
challenges identified per each domain observed; very little evidence on data use for 
decision making
IR: testing feasibility (practice is halted or aborted or not feasible at the moment)
Level 1 M&E: Practices already being implemented within an activity but with very limited 
resources; collection, analysis, and reporting of data not well integrated within the general 
M&E system; evidence of poor data quality; very little evidence on data use for decision
making
IR: poor fidelity, no adaptation; considerable gaps and challenges identified on the domains 
observed 
Level 2a M&E: Practices being implemented within an activity with adequate resources; processes in 
place for analysis and reporting but aligned with the general M&E system; good quality 
evidence has been generated and used for decision making, but not consistently
IR: good fidelity, with adaptation and feasible to scale (acceptable); some gaps and 
challenges identified on the domains observed but with no major issues with operational 
individuals
Level 2b M&E: Practices being implemented within an activity with adequate resources; processes in 
place for analysis and reporting but aligned with the general M&E system; good quality 
evidence has been generated and used for decision making, but not consistently
IR: high fidelity, with adaptation and feasible to scale (acceptable and appropriate); some 
gaps and challenges identified on the domains observed but with no major issues with 
15
operational individuals
Level 3 Identified practice can be considered as a good practice
M&E: Collection of data is well resourced and integrated well within the general M&E 
system; processes in place for analysis and reporting; and high quality evidence has been 
generated and consistently used for the team’s decision making and improvement of
project performance
IR: already showing high penetration and is sustainable; fewer gaps and challenges identified 
on the domains observed; able to improve M&E performance
3.5. Recruitment and Selection of Respondents
We selected participants through scoping discussions with point persons from the activities’ M&E teams 
to learn more about the contexts in which promising adaptations of remote and hybrid M&E practices 
were realized. We sought participation of the respondents through the USAID/Philippines 
CLAimHealth, concerned local government units (LGUs), and other activity point persons. 
3.6. Data Collection
1. Documents gathering: We gathered relevant documents such as activity MEL plans, work or 
implementation plans, progress reports (annual and quarterly), PIRS, and data collection tools at the HP 
and activity levels. We extracted the following data from the project documents: Activity objectives, 
TOCs, logical framework, strategic implementation; M&E and adaptive management goals, principles 
being followed (results-based, activity-based, or both), information flows, M&E activities before and after 
the practice, and changes in the M&E plan/activities due to the pandemic.
2. Documentation of good practices: A good practice is defined as a technique or methodology that 
through experience and research has proven reliably to lead to the desired result. Domains of the 
World Health Organization (WHO) guide to identifying and documenting best practices in FP 
programs57 closely aligns with the domains assessed by the M&E and IR tools as well as the criteria for 
GPPI used by USAID/Philippines.58 The weighted assessment is then presumed to sufficiently identify a 
“good practice,” and the information gathered may be used for a full GPPI case report.
3. Online survey: We did an online survey for a preliminary analysis of the HP and IPs as well as their
remote and hybrid M&E practices. Survey respondents were relevant M&E staff at various levels of the 
HP and activities. Some survey respondents also participated in the KIIs or FGDs. The survey sought to 
57 World Health Organization Regional Office for Africa, A Guide to Identifying and Documenting Best Practices in Family Planning 
Programmes (2017). 58 USAID/Philippines Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) Activity, Good Practices and 
Promising Interventions, Technical Series No. 6: Engaging Local Chief Executives to Build Local Capacity and Strengthen Health Systems
(2021).
16
identify practices being implemented by each activity, the quality of the M&E data collected via these 
practices, and staff perceptions on their individual and organizational capacity to implement the 
practices. 
4. KIIs/FGDs: We conducted KIIs/FGDs to identify contextual factors that facilitated and hindered the 
implementation of remote and hybrid M&E and to clarify responses from the online survey. The key 
informants were M&E focal point persons with an in-depth understanding of the processes of the M&E 
system and its role in achieving M&E and adaptive management goals of the HP and IPs. FGD participants 
included M&E coordinators, field coordinators, and other project and counterpart facility staff involved 
in M&E activities from each of the IP study sites. The goal of both KIIs and FGDs was to gather data on 
the characteristics and processes related to the practice as well as the inner and outer settings in which 
the practice is implemented. We also explored pre-practice M&E activities and changes in the M&E plans 
due to the pandemic during the discussion. 
3.7. Data Analysis and Validation 
Framework analysis: We reviewed recorded responses from FGDs and KIIs to form abridged transcripts 
and conducted framework analysis for KIIs and each focus group. We extracted relevant content from 
transcripts based on domains/themes identified within the developed M&E and IR analytical frames (see 
modeling approach to analysis). We then generated the narrative responding to the joint M&E and IR 
inquiry. 
Triangulation: We reviewed and analyzed information from FGDs and KIIs to generate the process 
analyses of the M&E systems from the various IPs. In addition, we did convergence and corroboration of 
desk review, KII/FGD, and online survey analyses to inform a more in-depth evaluation of identified M&E 
systems. We also developed an evaluation criteria for remote and hybrid M&E implementation based on 
expert consensus and assigned weights to identify the GPPIs.
Expert validation: After preliminary analyses, the Project Lead and consultants further reviewed and 
validated the evaluation matrices and raw data. The CLAimHealth technical team, various IPs, and 
USAID Office of Health will also be consulted regarding the final assessment. The results of the 
assessment will be used to generate evidence-based recommendations and identification of GPPI.
3.8. Study Limitations
The study was not able to carry out the quantitative analyses indicated in the inception report. There 
were limitations in the online survey due to considerable variability of respondents and responses across 
practices and IPs. Therefore, descriptive analysis of survey results was not done. Results of the survey 
were included in the qualitative assessments done for the practices.
The study was not able to perform a cross-sectional comparison of selected performance data 
indicators collected from the baseline (pre-practice) and endline (during practice) because it was not 
clear which indicators all the M&E practices were related to, and significant changes in the availability of 
17
data collected for tracking the performance indicators were largely contingent on the impact of COVID￾19 on the provision of services at the facility level.
The study was also unable to do process documentation during field observations due to difficulties in 
managing expansion of planned data collection. Moreover, the study opted to collect data remotely to 
assure the health and safety of staff during the pandemic. The methods generated for the collection of 
data for weighted analysis were substituted for the documentation of GPPI. Hence, the specific GPPI 
protocol was not pursued.
18
4. FINDINGS
4.1. Scoping Results
We conducted scoping discussions with the M&E point persons from various IPs via email and/or zoom 
calls from June 30 to July 6; and September 7 to 10, 2021. We asked questions on the M&E and adaptive 
management goals of the USAID HP and Activity as well as their definition of an effective M&E practice 
to align conceptual understanding. Preliminary interviews, combined with the review of HP documents 
provided by the USAID IPs, revealed that the goal of M&E activities is mostly centered on tracking the 
progress of their performance toward achieving a set objectives at both the project and activity level. 
Other goals include supporting project management and partners as well as developing and finalizing 
metrics for assessing implementation of programs. The adaptive management goal is an explicit target or 
outcome, but an overarching discipline for all USAID activities to respond and adjust to changing 
contexts to continue progress in their work. Definition of an effective M&E practice was different across 
the respondents. 
We also asked the M&E point persons to identify any remote and/or hybrid M&E practices in their 
activity based on the following criteria/definition: 
● Related to the application of technology to support an M&E activity from a distance 
● Any process or activity meant to build systems for M&E 
● M&E activities supported can be in data collection, management, analysis, validation, 
consolidation, interpretation, reporting/dissemination, and utilization 
● Includes practices even before the COVID-19 pandemic 
● Considers practices applied locally or nationally 
M&E point persons also recommended potential survey and KII/FGD respondents for each intervention. 
We narrowed down the initial list of remote and/or hybrid M&E practices based on the 
criteria/definition and presented it to the CLAimHealth team for its validation. Table 4 summarizes the 
17 identified remote and/or hybrid M&E practices across the seven IPs that were subjected to further 
analysis.
Table 4. Seventeen remote and/or hybrid M&E practices subjected to further analysis
IP Remote and/or Hybrid M&E Practices
ReachHealth District Health Information System 2 (DHIS 2)
Remote phone monitoring
Hybrid rapid feedback mechanism
BARMMHealth Hybrid monitoring of FP program through site visits, phone calls, and Facebook 
(FB) messenger
19
Blended pause and reflect (P&R) sessions
KoboToolbox
MTaPS Remote access to information systems (Department of Health [DOH]-
Pharmaceutical Management Information System [PMIS] and Integrated TB 
Information System [ITIS])
COVID-19 Special Response Project - hybrid monitoring of facilities (onsite and 
virtual)
ProtectHealth Cebu City-Wide TB Elimination Campaign (CiTEC project regular online 
conference calls
RenewHealth CBDR information management system (IMS)
TB Platforms Hybrid monitoring of TB Contact Center (TBCC) using regular scheduled phone 
calls and onsite monitoring/mentoring visits
Hybrid monitoring/mentoring for Finding cases, Actively, Separating safely, and 
ensuring effective Treatment (FAST) implementing facilities
Assessment of Infection, Prevention, and Control (IPC) using Google Survey
Remote monitoring and mentoring for Programmatic Management of Drug 
Resistant TB (PMDT) 
TB IHSS Remote P&R sessions
Remote rapid assessment survey
Remote monthly data quality checks (DQCs) using information technology (IT) 
systems
4.2. Summary of Data Collection Engagement
We conducted online surveys and KIIs/FGDs, which ran for a period of six weeks from September 27 to 
November 9, 2021. Table 5 summarizes the data collection engagement for this study. We sent 
invitations to participate in the online surveys and KIIs/FGDs via email and followed up target 
respondents through email and text. We conducted online surveys first to allow clarifications of some of 
the responses during the KIIs/FGDs. Some identified respondents were not able to participate due to 
conflicts and sudden changes in schedule and Internet connectivity issues. 
20
Table 5. Data collection engagement summary
IP Online Survey KIIs/FGDs
Number of 
Target 
Respondents
Number of 
Actual 
Respondents
Number of 
Target
Attendees
Number of 
Actual 
Attendees
ReachHealth 7 4 (57.1%) 7 5 (71.4%)
BARMMHealth 13 5 (38.5%) 13 10 (76.9%)
MTaPS 4* 4 (100%) 5 5 (100%)
ProtectHealth 5 4 (80%) 5 4 (80%)
RenewHealth 3 2 (66.7%) 3 2 (66.7%)
TB Platforms 18 10 (55.6%) 18 12 (66.7%)
TB IHSS 7 7 (100%) 7 7 (100%)
Total** 57 36 (63.2%) 58 45 (77.6%)
*One of the respondents was not requested to complete the survey.
**Two respondents served as a representative for two different IPs/practices; thus the actual number of individual respondents 
in the study is 56.
4.3. Overview of Monitoring and Evaluation of Selected USAID IPs
The USAID HP in the Philippines has nine IPs to help achieve the goals of strengthening healthy 
behaviors, fortifying quality services, and bolstering health systems in the areas of FP, TB, and CBDR. 
The study explored the general M&E system of seven IPs, including the adaptations they made during the 
COVID-19 pandemic (Table 6). 
Pre-COVID/Pre-Adaptation
All IPs except RenewHealth had an established results-based M&E system before the pandemic. They 
have fully a developed TOC, M&E/Results Framework (indicators and targets), and M&E plan for their
activities. They all reported adequacy of resources, especially financial and material. Almost all 
respondents agreed that there was enough budget and equipment (e.g., laptops) to carry out their M&E 
activities even before the pandemic. In terms of human resources, all IPs have a dedicated team for M&E 
across all levels (local, regional, and national) depending on the scale of the project. However, a few of 
them, like BARMMHealth and MTaPS, lack additional M&E professionals whose responsibilities were 
being covered by only one M&E staff. For smaller IPs like ProtectHealth, the M&E team receives support 
from other teams (Table 6). 
21
Compared to other IPs, RenewHealth was still in the process of establishing its M&E system before the 
pandemic. The lack of international CBDR benchmarks and standards presented a challenge in drafting 
their indicators that are needed before initiating any M&E activities. Additionally, the LGUs lack a 
harmonized CBDR M&E system, making it difficult to integrate RenewHealth's metrics and data 
collection mechanisms. Respondents from the two IPs working on TB (TB Platforms and TB IHSS) 
raised issues on the clarity and delineation between their roles and responsibilities, which has led to 
siloed work and duplication of M&E activities, such as having similar indicators with the same target 
population and sites. They noted that significant resources could have been saved if collaborations were 
facilitated early on and if the original roles were followed.
Most IPs had been implementing traditional, paper-based, and FTF M&E practices prior to the pandemic. 
These included field/facility visits, FTF data collection methods (observation, survey, interview, FGD), 
secondary data review (either FTF or remote), FTF data validation, FTF feedback activities, and FTF 
quality assurance activities. Only TB IHSS had used hybrid approaches before COVID-19, conducting 
half of their M&E activities FTF and half remotely because they could not visit all sites. Their remote 
practices included the use of phones (call or SMS), emails, and messaging apps (FB Messenger). Regular 
submission of reports and supporting documents were also done conventionally either through FTF 
visits, mail, or email. 
All IPs had planned to conduct baseline and evaluation activities, including review and learning sessions, 
in person to ensure high engagement and in-depth collection of information. They also conducted M&E 
training and capacity building activities, as well as regular stakeholder meetings, in person. All IPs 
reported that their M&E activities enhance the capacity of their government counterparts in collecting 
and using relevant data. 
In terms of quality assurance activities, the majority of the IPs conducted regular FTF DQAs or DQCs, 
engaging either internal or external teams. Some IPs with electronic information management systems
have been especially careful to ensure that data protection and security measures are in place within 
their information system. They also have used these systems for validation rules and checks on the data 
they collected.
Post-COVID/Post-Adaptation
All IPs made adjustments in their M&E system to continue their activities with the lockdowns and social 
distancing measures during the pandemic. Project and M&E activities were postponed or canceled; M&E 
staff had to work from home; field visits were halted or limited; and most FTF data collection was
moved online. As a response to the changes brought by COVID-19 in their programming, some of them 
modified their TOCs, implementation assumptions, and M&E/Results Framework by including additional 
project components, and adding/removing indicators and updating their targets, respectively. Most of 
them also changed their administrative guidelines to comply with the government health protocols when 
doing their M&E activities. 
Even during the pandemic, most of the IPs’ M&E resources remained adequate. In fact, ReachHealth 
recruited more M&E staff to work on the additional COVID-19 component in their project. However, 
TB Platforms had limited human resources at the field level who were needed for additional data 
22
collection and encoding for their service delivery work. In terms of costs, a few of the IPs reported that 
their budget increased due to canceled FTF M&E activities during lockdown. The use of online 
collaboration platforms such as Zoom and WebEx incurred no expenses. They, however, used this 
extra budget to cover pandemic-related expenses like testing kits, phone allowance, and additional 
equipment (for work from home set-up). TB Platforms reported a limited budget at the start of the 
pandemic because they did not receive additional resources and they were not allowed to use their 
money for COVID-19, which was later addressed when they were permitted by USAID to do 
realignment (Table 6).
Most of the traditional FTF M&E practices of the IPs were replaced by remote methods, especially 
during the first phase of the pandemic. There has been a notable increase in the use of technology 
(phones), mobile and web-based tools (Google Forms, KoboToolbox,59 SurveyMonkey), video 
conferencing tools (Zoom, WebEx), and messaging applications (FB Messenger, WhatsApp, Viber, 
emails). In lieu of on-site visits, the M&E teams were communicated with their partners remotely with
phones/smartphones to collect and/or validate both primary and secondary data. Paper-based 
monitoring forms were converted into an online version (in the form of an online survey) using mobile￾based or web-based data collection tools that were then shared to partners for completion. The use of 
secondary data has been maximized, and MTaPS obtained permission to remotely access government￾run databases. All IPs have moved their meetings, training/orientations, and workshops online using 
videoconferencing platforms, which have reduced logistical issues and expenses. Remote DQAs/DQCs 
and provision of feedback to partners were conducted usually via phone calls or video conferencing, 
where partners were asked to scan or send photos of documents (as in the case of ReachHealth and 
BARMMHealth). 
IPs used hybrid methods (combination of FTF and remote) in cases where FTF practice (usually facility 
visits) was necessary, even during the pandemic, due to the type of data being collected, limited 
connectivity of the facilities, or travel restrictions in the target sites. TB IHSS has generally adapted well 
to COVID-19 mainly because of their prior use of hybrid practices, which they increased and enhanced 
during the pandemic. RenewHealth, which had no established M&E system before the pandemic, decided 
to directly implement remote or hybrid M&E activities during the pandemic.
There were not many changes in the planned FTF evaluation activities of the IPs because most of them 
are still planned at midterm and at endline. For example, TB Platforms managed to conduct an FTF 
internal evaluation due to the easing up of restrictions in 2021. On the other hand, MTaPS completed 
their midterm evaluation remotely. 
IPs used similar information management systems to manage their data before and during COVID-19. A 
few IPs such as RenewHealth tried to hasten the development of a more sophisticated information 
management system during the pandemic, where partners and/or field workers could directly encode, 
store, and process a large amount of indicator data. There were no issues reported on data protection 
and security, as measures were already in place pre-pandemic. 
59 KoboToolBox: An open-source toolkit for data collection and management.
23
Table 6 provides a summary of the general M&E system and adaptations by IPs before and after COVID￾19.
24
Table 6. General M&E system and adaptations made by IPs before and after COVID-19
IP Time M&E System 
Principle
M&E resources Routine Monitoring 
Activities
Feedbacking 
activities
Evaluation 
Activities
Quality 
Assurance 
Activities
M&E Database
ReachHealth Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate project 
staff, money, and 
equipment 
FTF Facility visits Quarterly FTF P&R 
sessions
FTF baseline, FTF 
midterm, FTF 
endline
Quarterly FTF 
DQA, system 
validation rules 
and checks
Performance 
Indicator 
Tracking Table 
(Microsoft Excel)
Post-COVID/ 
Post-adaptation
(No changes) More field 
counterpart staff 
recruited
FTF facility visits, 
phone interviews, 
online meetings
FTF and online P&R 
sessions, hybrid rapid 
feedback mechanism 
(No changes) Scanning of means 
of verification 
(MOVs) 
(No changes)
BARMMHealth Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate money 
but limited staff
Secondary data 
(FHSIS), integrated 
monitoring and 
supervisory checklist, 
FP/maternal and child 
health exit interview
Monthly feedback 
sessions
FTF midterm 
(internal), FTF 
endline (internal, 
external)
Regular FTF 
DQC, phone￾based DQA
Microsoft 
OneDrive
Post-COVID/ 
Post-adaptation
(No changes) (No changes) Phone/Zoom calls, 
mobile/web data 
collection 
(KoboToolbox)
Blended P&R sessions (No changes) Photos of MOVs, 
Online DQC
KoboToolbox
MTaPS Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate staff, 
money, and 
equipment 
Secondary data (DOH￾PMIS, ITIS, FHSIS, 
IQVIA, IOR and FP 
hotline), FTF interviews
FTF P&R, email 
feedback
FTF baseline, FTF 
midterm, FTF 
endline
FTF DQA (as 
necessary), data 
protection 
measures in place
NDrive, 
Performance 
Indicator 
Tracking Table 
(Microsoft Excel)
Post-COVID/ 
Post-adaptation
(No changes) (No changes) Remote access to 
secondary data, online 
interviews
Learning Thursday Remote midterm Intensified data 
protection 
measures
Microsoft 
OneDrive
ProtectHealth Pre-COVID/ Pre￾adaptation
Results-based Adequate staff, 
money, and 
Secondary data 
(PhilHealth, DOH)
Client feedback system FTF IR FTF visits to 
PhilHealth
Product 
Accomplishment 
25
equipment Tracking Table 
(Microsoft Excel) 
Microsoft 
OneDrive
Post-COVID/ 
Post-adaptation
(No changes) (No changes) (No changes) (No changes) (No changes) No FTF visits (No changes)
RenewHealth Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate project 
staff, money, and 
equipment
Secondary data (Anti￾Drug Abuse 
Council/Office 
[ACAC/ADAO], social 
welfare, Dangerous 
Drugs Board [DDB], 
DOH, United Nations 
Office of Drugs and 
Crime [UNODC]), 
FTF exit interviews, 
FTF observations, FTF 
FGD, desk reviews
Partner feedbacking FTF process 
(internal), FTF 
midterm (internal), 
FTF endline 
(internal)
Annual FTF DQA Microsoft Excel
Post-COVID/ 
Post-adaptation
(No changes) Adequate money, 
but limited field 
counterpart staff 
and equipment
Secondary data 
(ACAC/ADAO, social 
welfare, DDB, DOH, 
UNODC)
(No changes) (No changes) (No changes) CBDR IMS
TB Platforms Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate staff, 
money, and 
equipment
Secondary data (ITIS), 
FTF field visits
Weekly updates 
(national), during and 
end-of-project 
implementation 
(partners) 
Annual FTF 
evaluation (internal), 
endline (external)
Periodic FTF 
DQA (internal 
done at least once 
a year; external 
done every 3 
years)
Project 
Management 
Information 
System
Post-COVID/ 
Post-adaptation
(No changes) Limited staff, 
money, and 
equipment (field 
level)
Mobile/web data 
collection (Google 
Forms, SurveyMonkey), 
Online meetings, TB 
Care App, FTF field 
visits 
(No changes) (No changes) Reliance on ITIS 
and systems in 
place
Microsoft 
SharePoint, 
Microsoft Excel
TB IHSS Pre-COVID/ Pre￾adaptation
Activity-based
Results-based
Adequate staff, 
money, and 
equipment
Secondary data (ITIS, 
National TB Control 
Program [NTP], 
FTF P&R sessions, FTF 
meetings and 
workshops 
Midterm, impact 
(external)
FTF DQA, 
quarterly FTF 
DQC, data 
Microsoft 
SharePoint, 
Performance 
26
WHO), FTF Field visits, 
FTF observations, 
hybrid surveys, hybrid 
interviews, hybrid 
FGD, desk reviews
privacy and 
security policies in 
place
Monitoring 
Information 
System
Post-COVID/ 
Post-adaptation
(No changes) (No changes) Secondary data (ITIS, 
NTP, WHO), hybrid 
surveys, virtual 
meetings
Online P&R sessions (No changes) Quarterly remote 
DQC (Zoom), 
data privacy and 
security policies in 
place
(No changes)
27
Table 7 summarizes the 17 remote and hybrid M&E practices identified across the seven IPs. See Annex 
5 for detail about data collected on the general M&E systems and the specific M&E practices of the 
USAID IPs, summarized into the developed M&E and IR analytical frames. 
Table 7. Individual characterization of remote & hybrid M&E practices
Remote & 
Hybrid M&E 
Practices 
Description
I. ReachHealth
DHIS 2 DHIS 2 is web-based data management software that can store, manage, analyze, 
and visualize monitoring data routinely generated by the project through a 
dashboard. DHIS 2 only serves as ReachHealth’s information system/performance 
monitoring database and will not substitute FHSIS in any way. Implementation of 
the system was temporarily halted due to COVID-19, but in June 2021, 
ReachHealth began partial implementation of DHIS 2 as its project information 
system, starting with FHSIS data. Full implementation covering all project databases 
for routinely collected data was completed in the 4th quarter of FY 2021. 
Prior to implementation and use of DHIS 2, data collected from FHSIS reports and 
quarterly monitoring tools were re-encoded and analyzed on Microsoft Excel. 
Remote Phone 
Monitoring
Due to mobility restrictions as well as health and safety concerns during the 
pandemic, collection of quarterly monitoring data shifted from onsite facility visits 
to remote phone calls during the first two quarters after the emergence of 
COVID-19. During the call, data collectors from ReachHealth staff administer the 
standard quarterly monitoring tool to partner facilities at the regional, provincial, 
and health facility levels. After this period of purely phone-based monitoring, 
ReachHealth switched to using three modes for quarterly monitoring: phone 
interviews, facility visits (to a limited extent), and a combination of both.
Hybrid Rapid 
Feedback 
Mechanism
The hybrid rapid feedback mechanism is a subset of the regular quarterly M&E 
activity. It was developed in response to USAID’s recommendation after 
conducting a DQA in Dec. 2020 to “close the loop” between the identification of 
problems at the health facilities during quarterly monitoring and actions to address 
those problems. This feedback mechanism essentially has two phases that can 
involve hybrid approaches: 1) rapid data collection and 2) rapid feedback 
dissemination. Data are collected and managed through the rapid assessment tool 
in the DHIS 2.
II. BARMMHealth
28
Hybrid monitoring 
of FP program 
through site visits, 
phone calls, and FB 
messenger
This is a special case because the monitoring practice is primarily performed by 
provincial health office (PHO) staff from the BARMM provinces. Prior to the 
pandemic, they were already employing hybrid monitoring approaches for FP 
programs through site visits, phone calls/texts, and FB messenger. During the 
pandemic, site visits were conducted more frequently than phone calls in areas 
with few COVID-19 cases or poor mobile and Internet connectivity. In areas with 
higher COVID-19 cases and decent signal, phone calls were used. FB Messenger 
group chats were used for updates, clarification, and document sharing.
While BARMMHealth utilizes data collected by the BARMM PHOs, they also 
dovetail their monitoring for project-specific indicators. They also facilitate 
BARMM PHO monitoring by providing additional transportation for PHO staff as 
well as representation and transportation allowances to participants when they 
conduct joint monitoring activities. 
Blended P&R 
Sessions
The project aims to conduct monthly feedback sessions to discuss BARMMHealth 
activities and how they can be adapted to improve performance. P&R sessions are 
used in M&E for planning, data collection, data validation, data consolidation, and 
data reporting/dissemination.
Pre-pandemic, P&R sessions were conducted FTF each month with stakeholders 
from the municipal/city and provincial level in one venue. Due to pandemic 
restrictions, P&R sessions have shifted into a blended format. Online P&R sessions 
operate in a blended manner in the sense that technical staff from the region 
attend individually while provincial staff gather in areas with good Internet
connectivity. Zoom is used for videoconferencing. 
KoboToolbox The BARMMHealth provincial team uses KoboToolBox, an open-source platform
suitable for mobile phones and tablets, to upload data that will help health 
managers visualize and analyze their data. KoboToolBox was adopted mid-2020 
because of the challenges of submitting physical forms due to quarantine 
restrictions during the COVID-19 pandemic. The toolbox is used for data 
collection, analysis, interpretation, and reporting/dissemination. Each performance 
indicator has a corresponding form in KoboToolBox, which is based on the original 
paper-based forms.
During the third-party monitoring activity in selected BARMM areas carried out in 
2021, CLAimHealth noted that this system is only used for activity-level recording 
of client satisfaction, institutional capacity assessment, informed consent and 
voluntarism, and data on Muslim religious leaders. Moreover, in BARMM areas that 
have poor internet access, it may be more practical for BARMMHealth to simply 
provide pre-programmed Excel sheets for data recording to barangay health 
stations. BARMMHealth is also exploring the use of DHIS2, another open-source 
29
platform already being used by ReachHealth.
III. MTaPS
Remote Access to 
information systems 
(DOH-PMIS and 
ITIS)
MTaPS was given access to the recently upgraded DOH-PMIS in late 2020/early 
2021 to support the DOH Pharmaceutical Division (PD) in its monitoring and 
analysis of supply and consumption data for TB and FP medical commodities. 
MTaPS was also given “viewer’s privilege” by DOH NTP and Knowledge 
Management and Information Technology Service (KMITS) in early to mid-2021 to 
access ITIS and assess the inventory data reported. 
Prior to this remote access, by MTaPS requested needed data or reports from 
DOH PD and KMITS, and these were sent via email.
COVID-19 Special 
Response Project - 
hybrid monitoring 
of facilities (onsite 
and virtual)
The Special COVID-19 Response Project ran from March to September 2020. In 
response to the pandemic, USAID tasked MTaPS to support DOH to strengthen 
IPC, health care waste management (HCWM), and SCM practices in facilities. 
MTaPS delivered a series of virtual training sessions in the three areas. 
After conducting the remote trainings, MTaPS conducted onsite facility and virtual 
visits to 42 hospitals between July and Sept 2020 to: 1) determine the status of the 
hospitals’ IPC, HCWM, and emergency SCM practices during COVID-19, 2) 
discuss key pointers pertinent to these practice areas with the hospital staff in their 
daily operations, and 3) to provide onsite support to supplement the virtual 
training received by health care workers (HCWs). 
This COVID-19 response project developed its own M&E system which is distinct 
from that used more broadly for MTaPS. 
IV. ProtectHealth
CiTEC Project 
Regular Online 
Conference Calls
Regular conference calls with TB IHSS, the Cebu City LGU, and other stakeholders 
(e.g., Central Visayas Center for Health Development and barangay officials) started 
in April 2020 during the planning stage for the CiTEC demonstration study in Cebu 
City.
Initially, there was no regular schedule for the meetings, but eventually a regular 
time was set, Wednesdays from 2-4PM. In the current implementation phase, the 
scheduled meetings are less frequent but still held regularly. Because the project is 
jointly implemented with TB IHSS, both central office and field staff members from 
ProtectHealth and TB IHSS are involved in the meetings. 
V. RenewHealth
30
CBDR IMS CBDR IMS is one of the subdomains of the CBDR portal (cbdr.org.ph), a one-stop 
website to aid LGUs in their CBDR implementation. This will serve as the 
backbone for LGU Anti-Drug Abuse Council operations as it helps LGUs monitor 
CBDR services and better manage information on people who use drugs. The 
system gathers data from multiple sources, analyzes the information, and reports 
data to aid in organizational decision making. 
The primary purposes of the IMS are to support data gathering and retention of 
involved offices in the LGUs, and to be used to safely keep data that is easily 
accessible for reporting to Philippine government partner offices. This will not 
replace reporting systems established by the Philippine government but rather will
aid in the submission of reports.
VI. TB Platforms
Hybrid monitoring 
of TBCC using 
regular scheduled 
phone calls and 
onsite monitoring/ 
mentoring visits
From its introduction in Marawi in mid-2019, TB Platforms expanded the 
implementation of the TBCC in facilities in Region 4-A in May 2020 (later to 
Region 3 and National Capital Region [NCR]) following the NTP TB Adaptive 
Management Plan’s recommendation to explore telephone based means to follow 
up patients during COVID-19. TBCC is a remote (phone call-based) means for TB 
screening and contact investigation, monitoring of treatment, and addressing 
patient concerns related to treatment (e.g., adverse drug reactions).
TB Platforms monitors implementation of the TBCC using regularly scheduled 
phone calls and onsite monitoring and mentoring visits to TBCC. During phone 
calls, call log sheets are validated, and any issues with the implementation of TBCC 
are raised. During facility visits, the team checks records of incoming and outgoing 
phone calls, as well as whether all patients' inquiries and concerns were adequately 
addressed. Stocks of TB preventive treatment (TPT) drugs, TPT implementation, 
TPT records, and percentage of patients screened are also monitored.
Hybrid monitoring/
mentoring for FAST 
implementing 
facilities
In FY 2019, TB Platforms introduced FAST as an infection control strategy for 
nosocomial TB, especially among HCWs. FAST prioritizes rapidly diagnosing 
patients and starting treatment to prevent further transmission. FAST also doubled 
as an intensified case finding strategy within health care settings.
Prior to the pandemic, FAST implementing hospitals and health facilities were 
monitored every month once the system was in place. This included process 
checks, random patient record checks, and accomplishment of a data collection 
tool with the FAST point person and other partners and facility representatives. At 
the end of each monitoring visit, mentoring was conducted by providing feedback 
on the status of FAST implementation including issues, concerns, and suggestions 
to improve implementation. 
31
Onsite monitoring was halted at the onset of the pandemic due to mobility 
restrictions with the lockdown and strict protocols set by hospitals in the following 
months. During these times, monitoring of FAST implementing facilities continued 
through remote approaches such as Google Sheets, FB Messenger, phone calls, and 
email. Onsite monitoring resumed in late 2020/early 2021. Remote monitoring 
approaches may still be used but are not employed as often anymore.
Assessment of IPC 
using Google Survey
In Q2 2020, TB Platforms provided TA to improve IPC by conducting eMentoring 
(webinars) to facilities focused on reiteration of IPC protocols as well as integrating 
systematic screening for both TB and COVID-19 using the FAST strategy. 
Following these webinars, TB Platforms conducted a baseline assessment among 
Level 1 and 2 hospitals from Region 3, 4A, and NCR in July 2020 using Google 
Surveys to assess implementation of IPC in facilities and identify technical support 
areas for strengthening.
PMDT remote 
monitoring and 
mentoring
TB Platforms conducted remote monitoring and mentoring sessions in PMDT 
treatment facilities and its catchment rapid TB diagnostic laboratories. Google 
Meet, Zoom, and Microsoft Teams are the commonly used meeting platforms. 
Before the scheduled monitoring session, TB Platforms forwards the assessment 
tool and remote PMDT mentoring tool for completion of the facilities. The remote 
PMDT monitoring tool was created to simplify data collection and was based on 
the original PMDT monitoring tool.
Representatives from TB Platforms/FHI 360, Philippine Business for Social Progress 
(PBSP) program officers, satellite treatment center (STC) nurses, and physicians 
attend the monitoring session. FGDs with facility staff are also conducted where 
accomplishments of the facilities are discussed. Facilities apply the 
recommendations that they receive from TB Platforms during the mentoring 
session to improve PMDT implementation.
VII. TB IHSS
Remote P&R 
sessions
TB IHSS utilizes P&R sessions to determine adaptive management actions 
undertaken with key stakeholders which drew on evidence-based insights or 
lessons learned. At the national level, monthly project-level P&R sessions are held 
to reflect on key learnings and discuss progress, challenges, and hold strategic 
conversations to inform the overall direction of the project. Internally, this is held 
annually for HP performance evaluation (includes other IPs). Due to the pandemic, 
P&R sessions shifted from hybrid to fully remote, with investments to procure 
licenses for Zoom or online polling tools.
Key staff including from the Strategic Information (SI) Unit, top management, and 
32
sometimes gender focal person and technical advisors attend P&R sessions. 
Technical advisors and the SI Unit helps facilitate. Needs assessment and planning 
for the next quarter is discussed. The collected data from P&R sessions provide a
“proxy” assessment on the operating environment for TB IHSS activities. Strategies 
are adjusted based on these findings. Results can inform decision making at the 
management level as well as advocacy material with relevant government officials.
Remote rapid 
assessment survey
DOH NTP, in coordination with USAID TBIHSS, conducted a rapid assessment of 
TB services using online data collection (Google Surveys in April 2020. These 
surveys intended to determine the quantitative and qualitative impact of COVID-19 
on ongoing TB elimination efforts in Level 3 hospitals in Region 3, Region 4, and 
NCR implementing ENHANCE (Enhancing Hospital Networks and Communities 
to End TB), with findings to inform mitigating measures, advisories, and approaches 
for front-liners involved in TB care. Follow up assessments were conducted a 
month and six months later. Information and insights gathered have even informed 
the latest NTP Adaptive Plan (NAP).
Remote monthly 
DQCs using 
information 
technology systems
Further analysis of this practice was dropped. Suggested respondents to be 
interviewed weren't aware of this practice. Upon clarification with the TB IHSS 
M&E Specialist, TB IHSS was not conducting remote monthly DQCs using 
information technology systems (e.g. web-based ITIS, ITIS Lite mobile tools, data 
warehouse, and dashboards). Rather, DOH NTP requested TA from TB IHSS to 
streamline DQCs for the mandatory TB notification with their normal DQC 
process. The TA will not be limited to FASTPlus facilities but will be adaptable 
across all regions. So far, DOH NTP has an established process for DQC 
conducted by the regional, provincial, and city-level coordinators for ITIS, but none 
for the DQCs of the recently introduced mandatory TB notification in ITIS Lite. 
This TA is not yet formalized, and only a concept note has been drafted since 
DOH NTP changed their priority for TA in mid-July 2021. TB IHSS is also awaiting 
feedback from TB Platforms, who is spearheading DQCs of rural health units 
(RHUs) for FASTPlus.
4.4. Effectiveness of Remote and Hybrid M&E Practices in Achieving M&E and Adaptive 
Management Goals
An M&E practice is considered effective mainly when it has facilitated the use of quality60 M&E data to 
make informed decisions or actions, such as solutions which address gaps, adaptations, or adjustments 
to program implementation and activities toward the achievement of the HP and the IPs’ outcomes. This 
assessment also considered factors such as the practice design and implementation, as well as its 
integration within the general M&E system. 
60 Data quality criteria: Timeliness, Reliability, Validity, Precision, and Integrity
33
We assessed each remote practice identified per IP based on the following descriptors: implementation 
status, resources, process of data collection, analysis and reporting, how the data collected are being 
used, and quality of the collected data. Level 3 practices were considered as a candidate for “good 
practice” (Table 3). 
The identified remote M&E practices were classified into the following six categories: 
● Phone-based practices - one-on-one interview surveys or open-ended interviews conducted by 
phone, WhatsApp calls, or other online calling platforms
● Web-based practices - online surveys conducted using web-based data collection tools such as 
Google Forms and KoboToolbox
● Videoconferencing practices - activities with a group using videoconferencing tools such as 
Zoom, Microsoft Teams, WebEx
● Information management system practices - activities focused on the development or 
strengthening of the IP’s data management system
● Remote access software practices - activities using remote access software
● Hybrid practices - activities that combine FTF and remote methods
Phone-based Practices
ReachHealth implemented remote phone monitoring as an alternative and, later on, as a complement to 
in-person facility visits. The practice is well resourced, more cost-effective than FTF visits, and is 
integrated well within ReachHealth’s general M&E system following the same data flow, using the same 
standard facility monitoring tool, and being stored and managed in the same database (Annex 6). 
However, some challenges were identified, particularly on staff acceptability and data quality. Field staff 
felt “embarrassed” when making long phone calls with high-level officials. This could be addressed by 
shortening and simplifying the tool, which is currently 22 pages. Also, the reliability of data collected has 
been compromised in a few indicators, specifically the FP stockout rate, for which actual inventory 
inspection was the only acceptable practice for monitoring prior to COVID-19. There were low 
responses and delayed/incomplete submission of secondary data from field staff, especially during the 
peak of COVID-19, as they were involved in the response. Project and field staff conveyed their 
preference for FTF data collection methods rather than phone interviews once COVID-19 restrictions 
are loosened. Overall, remote phone monitoring can be considered as an efficient complement in cases 
where in-person facility visits are not possible. It is recommended that data collection be adapted as a 
hybrid practice to maximize the effectiveness of the appropriate method (whether FTF or phone-based) 
and ensure data quality. 
Web-Based Practices
34
TB IHSS implemented a web-based practice using remote rapid assessment surveys, mainly through 
Google Forms. Paper and soft copies of the questionnaires were also provided to address the workload 
and Internet connectivity issue of the respondents (HCWs). The short yet comprehensive online survey 
tools developed by TB IHSS were validated (content and face validity) and pre-tested before they were 
rolled out to partners. The team also adjusted the tools during the implementation as necessary. Field 
staff received orientations, informal training, materials, concept note (with methodology), coding 
manuals, and operational definitions before the pre-testing and actual roll out. TB IHSS have provided a 
communications allowance to the target respondents to answer the survey and to follow up with 
patients during the community quarantine (Annex 6). 
This practice helps TB IHSS see the external factors that hindered efficiency and effectiveness of their 
activities being implemented in different sites. The DOH NTP team, regional NTP teams, and hospitals
are using these findings to identify mitigation measures and recommendations to improve TB care in the 
facilities. They were also used as one of the points for the NAP during the pandemic. For example, TB 
IHSS supported the launch of the ITIS Lite mobile app61 to make it easier for doctors to notify their TB 
cases using their smartphone while ensuring patient confidentiality and data security.
TB IHSS encountered minor challenges with the remote rapid assessment surveys, including the low 
sample size, difficulty ascertaining reliability of responses, and the tight window in which they had to 
develop the questionnaire, orient enumerators, pre-test the forms, and collect and analyze data. The 
HCWs were busy directly responding to needs pertaining to COVID-19, which contributed to the 
difficulty of obtaining responses from them. This was addressed by having enumerators constantly follow
up with the facilities. This challenge could also be mitigated by calling them when they are likely to be 
free (before or after their shift or on off days), or making additional calls when previous attempts were 
unsuccessful.62 Though the central team conducts regular data checks, data reliability was a concern 
because not all data could be validated and data collection was rushed. Respondents should be given an 
adequate amount of time to complete the surveys, with consideration for their busy schedules. 
TB Platforms also used Google Forms to do a baseline assessment of IPC implementation. The survey 
questionnaire was based on draft questions developed prior to the pandemic, and no information is 
available on whether the new tool was validated and/or pilot tested. Field staff were given orientations 
on the questionnaire, references used in developing the tool, technical design (components of IPC), and 
IPC process (Annex 6).
In contrast to TB IHSS’ successful use of Google Forms in their remote rapid assessment survey, TB 
Platforms experienced significant challenges in implementing this intervention. Though it was integrated 
well within the general M&E system, the following major challenges were encountered: poor data quality 
characterized by numerous duplicates, incomplete/missing data, low response rates, aversion of 
respondents to open-ended questions, and limited probing of responses (which is expected from online 
surveys).63 This could be due to the design of the questionnaire which warrants reviewing to be sure 
that it is short (not more than 30 minutes) and easy (predefined choices) for the respondents to answer. 
61 ITIS Lite: The official application of the DOH Philippines for the TB mandatory notifications by all public and private 
providers.
62 Hensen, Bernadette et al., 2021. 63 Hensen, Bernadette et al., 2021. 
35
Similar to the remote rapid assessment survey of TB IHSS, pre-testing of new tools is recommended and 
multiple modes (paper and soft copies) of the tool could be provided to the target sites to address 
Internet connectivity issues.
Another limitation was the discomfort of the team in using Google Sheets and the need for them to use 
the information to analyze infrastructural aspects of IPC. TB Platforms tried to address this by hiring an 
external engineer consultant, but they have not yet found a consultant with the necessary expertise. 
There was also confusion among the facility HCWs when the results were presented because they did 
not realize that there were multiple respondents per facility. Intensive training is necessary to build 
technical capacity of staff not just for data collection/analysis but also in building rapport with the 
respondents. 
TB Platforms’ baseline assessment achieved its intention to assess the implementation of IPC in the 
facilities and to identify technical support areas for strengthening; however, provision of feedback 
regarding these results was limited. Survey results were only fed back to the regions and DOH NTP, but 
not to most of the facilities because of significant hesitancy among field staff. This was due to their lack 
of technical capacity in giving feedback coupled with their negative experiences in previous feedback 
processes when facilities became defensive after scoring low. There was no official SOP or policy in the 
use of Google Forms, limiting the use of the findings to strengthen the IPC program at the facility level. 
Nonetheless, regions used the data to strengthen their health worker surveillance programs and 
supportive supervision systems, and TB Platforms used the data to develop a comprehensive TA 
package for IPC.
BARMMHealth used KoboToolBox to help provincial staff and data managers accelerate data collection 
(no encoding needed), reporting (automatic upload), and data generation/analysis during the pandemic. 
Provincial staff were oriented on how to use the online survey forms, which were based on the original 
paper-based forms. BARMMHealth intends to transition from KoboToolBox to its own information 
management system in the future and is in discussions with ReachHealth to consider use of DHIS2, 
another open-source platform that ReachHealth already uses (Annex 6).
Results generated from this practice have informed BARMMHealth on certain project decisions such as 
service delivery as well as identifying areas for improvement. One respondent said that the practice 
made it easier to see how implementation can be provided and to provide feedback on client satisfaction 
with services. The generated information can be used for FP service targeting and coverage and 
identifying areas for improvement. However, BARMMHealth should address the non-reporting of some 
RHUs to ensure completeness of data.
Video Conferencing Practices
Various practices in the HP used video conferencing tools such as Zoom, Microsoft Teams, or WebEx 
for their M&E activities. These include the P&R sessions of BARMMHealth and TB IHSS, the regular 
conference calls of ProtectHealth, and the remote monitoring and mentoring of TB Platforms for its 
PMDT program (Annex 6).
36
IPs use P&R sessions to reflect on key learnings and achievements, as well as to provide feedback to 
LGU counterparts and beneficiaries for the overall direction of the project. During the pandemic, both 
BARMMHealth and TB IHSS moved these sessions online (through Zoom), and they were well 
resourced and integrated within their general M&E systems. BARMMHealth and TB IHSS’ remote and 
blended P&R sessions have provided an avenue for the teams to strategize and learn from well￾performing areas. However, BARMMHealth still has limited reach with the blended approach because 
some areas, especially island provinces, cannot participate in online sessions. It was not guaranteed that 
BARMMHealth staff could conduct in-person P&R due to travel restrictions. The blended approach is 
not sufficient to inform project activities by utilizing inputs from all involved partners, thus limiting its 
effectiveness. 
TB IHSS, on the other hand, had been implementing hybrid P&R sessions prior to the pandemic, which
eased their transition to being fully remote during the pandemic. It was still challenging, however, for the 
participants to suddenly shift from hybrid to remote, but they were able to adapt over time. The team 
uses the information in the P&R to provide a “proxy” assessment on the operating environment for TB 
IHSS activities. Strategies were changed in accordance to these findings. Results can inform the decision
making at the management level as well as advocacy materials with relevant government officials.
The regular online conference calls (through Microsoft Teams) for the CiTEC joint project of 
ProtectHealth and TB IHSS is well resourced and well integrated within the M&E system of the joint 
project, which is aligned with the general M&E system of ProtectHealth. The online calls were done 
weekly at the start of the project but have become less frequent. ProtectHealth uses the information 
from these calls to track progress of implementation and decisions on operational issues, and to discuss 
draft protocols and guidelines. For example, one of the meetings discussed revisions in the forms to be 
used by field implementers, which has led to significant changes in implementation in the field. No major 
challenges with data reporting, use and quality related to the conference calls were reported. But other 
aspects of the implementation could further be improved, such as having a consensus agenda on urgent 
needs to address, downloading app extensions for added features, and creation of a protocol. Another 
challenge is that added security measures in Microsoft Teams limit the free entry of internal and external 
participants during the call. Overall, though, the practice is effective in ensuring coordination between 
the two IPs, which have two different sets of project participants across geographically separated units 
(central and field level), with ProtectHealth staff receiving financial support for communications. Perhaps 
most importantly, the online calls enabled the project team members to continue collaborative 
teamwork to make timely informed decisions and take actions on activities toward the achievement of 
the outcomes of both ProtectHealth and TB IHSS (Annex 6).
TB Platforms’ remote monitoring and mentoring for PMDT has adequate resources in place, and it is 
aligned with their general M&E system. Through this, facilities were made aware of the directives to 
resume TB services during the pandemic. However, the quality of the data collected was questioned. 
Central office staff participants may have been biased because the questions were sent to them 
beforehand. Central office and field staff agreed that the limited data validation undermines data quality. 
Because of these factors, effectiveness of the remote monitoring and mentoring is difficult to assess, 
although respondents have indicated that the practice was helpful in improving PMDT implementation. 
Respondents have recommended transforming this remote practice to a hybrid practice to allow FTF 
facility visits and data validation (Annex 6).
37
Information Management System Practices
ReachHealth moved from Microsoft Excel to DHIS 2, a web-based software, to accommodate the 
massive amount of data that they have to manage, especially during COVID-19 times. DHIS 2 has 
adequate financial and human resources in place, and it is aligned with the general M&E system. DHIS 2 
has improved data management functionality and hastened ReachHealth’s reporting and visualization, as 
it has automatic features and is less prone to human errors compared to Microsoft Excel. However, 
ReachHealth's use of DHIS 2 was still in the pilot-testing phase at the time of this assessment, and more 
time will be needed to assess its performance (Annex 6).
The CBDR IMS of RenewHealth is currently under development. It has adequate resources to maintain 
and further revise the system, wherein the data collection, analysis, and reporting processes are aligned 
with RenewHealth’s general M&E system. Once operational, the intervention is expected to accelerate 
RenewHealth’s data collection and analysis by facilitating remote data collection and solving the burden 
of data encoding as well as automating the process of summarizing and generating disaggregated data for 
the project’s performance monitoring (Annex 6).
The current planning stages face some theoretical bottlenecks; for example, CBDR indicators are still 
being developed. Given its current stage, the effectiveness of the IMS in aiding RenewHealth’s M&E and 
attending to the partners’ needs cannot be fully assessed at this point. The system has not yet generated 
high quality evidence for the teamʼs decision making and improvement of project performance, but it
does have the potential to accelerate data processing and improve data quality of CBDR information of 
RenewHealth, LGUs, and partner government agencies.
Remote Access Software Practices
The provision of remote access to DOH-PMIS and ITIS was free, and it was well-integrated with the 
general M&E system of MTaPS. However, the lack of sustainable funding support from DOH PD, NTP, 
and KMITS for the day-to-day operations of the information systems presents a challenge. Moreover, 
issues with human resources, equipment, and Internet connectivity posed challenges to encoding data 
into DOH-PMIS and ITIS at the facility level (Annex 6). 
MTaPS was analyzing secondary data from DOH-PMIS and It is pre-pandemic, but the full remote access 
during the pandemic has improved data analysis and use due to ease of and fuller access to the data. This 
has facilitated MTaPS’ deeper analysis of information such as stock and consumption of TB medicines, FP 
consumption and stock-on-hand reporting, etc. However, there were some data quality concerns in 
DOH-PMIS and ITIS databases including the constant changing of data and the uncertainty about veracity 
and completeness of data entered. MTaPS is currently working with relevant partners so both 
information systems can improve the reporting and data challenges for TB medical products.
Hybrid Practices
38
A hybrid practice is a combination of FTF and remote methods. This has been the most common type of 
practice across the IPs, usually involving on-site facility visits coupled with phone calls (Annex 6). 
ReachHealth’s hybrid rapid feedback mechanism is a well-resourced practice that is aligned with the 
general M&E system. Implementation of the mechanism has allowed for immediate feedback to partners 
in some facilities but it is challenged by some perceived data quality issues stemming from remote data 
collection, and the lack of human resources and time (during the pandemic) at the field counterpart 
level. However, because ReachHealth's use of the rapid feedback mechanism was still in the pilot-testing 
phase at the time of this assessment, a performance assessment would be premature. Major aspects of 
the mechanism are still currently being finalized. For instance, low compliance to targets for data 
encoding is currently being mitigated through a “pahiyaan” (humiliation/shaming or guilting) system 
wherein status of encoding per area is tracked and made transparent to all encoders. Developing more 
sustainable and reliable accountability systems to facilitate data encoding is recommended. Additionally, a 
remote qualitative measure should be incorporated in the triangulation of quantitative information 
collected from the remote rapid data collection.
Performance monitoring by BARMM PHOs, supported by BARMMHealth, had used hybrid approaches 
(FTF site visits plus phone calls and FB Messenger) even before the pandemic. Although additional 
resources were needed for hybrid performance monitoring at the RHU level, there were adequate 
resources reported at the provincial level. With several data validation measures in place, data 
generated are of good quality. Despite specific challenges identified, hybrid monitoring was seen as more 
effective than pure FTF or pure remote monitoring, as it allows adaptation to the geographical context, 
access to decent Internet and mobile connectivity, and can flex with the constantly changing situation of 
the pandemic. Implementation of this hybrid approach to monitoring is envisioned to continue during 
and even beyond the pandemic. BARMMHealth should develop formal guidelines for conducting hybrid 
monitoring of the local health offices.
The COVID-19 project of MTaPS aims to expand the absorptive capacity of facilities, improve IPCs, 
HCWM, and SCM practices, and mitigate COVID-19 impacts. MTaPS used hybrid activities to monitor 
facilities and, though it has adequate resources to implement the practice, this monitoring does not align 
with the general M&E system. The indicators of the COVID-19 project were somewhat related to the 
overall HP indicators. Therefore, it is difficult to assess whether it was effective in terms of the project’s 
exclusive intentions. Some challenges encountered relate to human resources such as resignation of 
team members, COVID-19 infection with the team, and the dual responsibilities of staff between 
COVID-19 and regular MTaPS work. A few concerns were raised about data quality issues though these 
were mostly addressed. Data generated will be used by MTaPS and DOH to develop capacity building 
materials for training of trainers and guidelines that can address gaps at the facility level. 
TB Platforms has implemented hybrid monitoring activities (FTF facility visits and phone calls/FB 
Messenger/email/Google Sheets) for both its TBCC and FAST implementing facilities. Both practices 
were well integrated within the M&E system. Resources for monitoring TBCC were adequate at all 
levels, while the hybrid monitoring and mentoring of FAST facilities had limited human resources at the 
field and facility levels. 
39
TB Platforms conducts regular phone calls and FTF visits with the facilities for TBCC. In cases where 
HCWs are unavailable to submit reports or do phone monitoring, facilities are visited in person. 
Monitoring tools are available for facility visits but not for the phone calls, although facility focal points 
received instructions for filling out the patient contact forms and summary call log sheets. Results from 
the hybrid monitoring are used to improve TBCC implementation and for advocacy to adopt TBCC to 
other facilities/LGUs. Any issues identified in the call log sheets or facility visits are discussed with the 
HCWs, and feedback is provided by the end of the calls/visits. It is recommended that TBCC 
incorporate any FB Messenger communications in the summary call log sheet to ensure inclusion of all 
data and maximize accuracy of reports. 
For FAST monitoring, the facility focal point answers the form through Google Forms or submits a soft 
copy of records via FB Messenger or email, which are then discussed through a virtual meeting with TB 
Platforms staff. Analysis and use of the data from the FAST monitoring showed data quality issues in the 
TB cascade, prompting TB Platforms to work with the facilities to optimize patient flows, data recording 
and reporting, and to improve sputum collection and transportation mechanisms as well as patient 
follow up mechanisms. Challenges identified included the limited availability of facility staff for monitoring 
(during the pandemic) and some issues with the forms. This practice relies on the availability of human 
resources at the facility to document data; this should be addressed to ensure the availability and 
completeness of monitoring data. 
Respondents prefer the current hybrid set up of TBCC monitoring due to the current pandemic. 
However, added COVID-19 responsibilities of the HCWs resulted in challenges in monitoring FAST 
facilities in collecting and validating information remotely, scheduling FTF visits, and facilitating the 
monitoring. 
For all hybrid practices, guidelines are needed on how to conduct M&E activities. Recommendations for 
future monitoring efforts by DOH should include building capacity of HCWs to conduct hybrid 
monitoring activities, and to use technology, monitoring processes, and data to be reported in both FTF 
and virtual methods.
40
5. DISCUSSION
5.1. Implementation Contexts, Opportunities, and Challenges
The pandemic has changed our ways of working in conducting M&E. It has been particularly challenging 
for those working in public health and managing public health programs, given the need for FTF 
interaction in providing a health service, supervision, and engagement of multiple local community 
stakeholders. The current situation forces us to innovate, either by using technologies to create a new 
product/process, or by repurposing old technologies. Digital technologies are being used more 
frequently now in M&E to allow remote and hybrid monitoring, augmenting FTF M&E activities to 
minimize physical contact. To improve the implementation of these new ways of working, we need to 
investigate how they are implemented and what enables or hinders the implementation, and then as 
needed recalibrate, adapt, and further innovate to optimize their effectiveness. 
Implementation is always contextual and heavily affected by the environment, the characteristics of 
those who are implementing the practice, and the properties of the practice itself. The CFIR captures 
the key domains in investigating implementation (Figure 4). In our case, we slightly modified the 
framework to depict the relationships of these components to reflect the practice in question, which is 
remote and hybrid monitoring.
Figure 4. The CFIR (Left) and the Modified (Right) Framework to Capture the 
Relationships of the Key Domains in the Implementation of Remote and Hybrid 
Monitoring
We identified the following key themes in the implementation of remote and hybrid monitoring across 
the IPs in this assessment: 
1. Remote and hybrid monitoring generally use digital technologies. Most practices implemented by 
the IPs include one or more of the following: use or development of information systems such 
as the DHIS 2, video conferencing such as Zoom for weekly meetings, phone calls for data 
collection, social media applications such as FB Messenger for communication, and web-based 
tools such as Google Forms for reporting. Aside from DHIS 2, these technologies are widely 
available, have been used even prior to the pandemic, and the IPs have the necessary equipment 
41
to use them. Therefore, investment needs are low and implementation is fast. However, the 
level of success in implementing the same digital technology in the IPs differs. Some practices, 
even when proven to be efficacious, widely available and low cost, can fail when user 
acceptability is low or users lack needed capacity to implement them.
2. Implementation is designed based on the needs of the IPs. M&E assesses performance of 
programs or projects, involves field data collectors and managers, and is inherently dependent 
on the movement of data and information and their eventual use. Remote and hybrid monitoring 
is mostly substituting for an activity in the M&E process. The practice consists of two integral 
core components: 1) the digital technology that allows remote activity and (2) the individuals 
that implement the technology. The complexity of the digital technology used and the data 
collected depends on the IPs’ needs. Reducing the practice complexity, simplifying processes, 
and ensuring the matching of capacities of the implementers and target partners are among the 
main challenges. 
3. Reliance on an external partner. M&E of nongovernmental organizations heavily relies on data 
from government health program partners. These may be LGUs or regional or national offices. 
They are considered within the “outer” setting because the IPs do not directly control and 
supervise them; rather they can only be influenced and persuaded.
4. The implementation of the remote and hybrid monitoring during the pandemic is dynamic. The 
country’s general lockdown has led to field movement restrictions. Once the restrictions are 
lifted, pre-pandemic activities may resume. However, global and local uncertainty remain, and 
the practices may still evolve with the situation.
5.2. Adaptiveness of Practices
In general, the remote and hybrid monitoring across the IPs have been adaptive. Most such activities 
were based on M&E plans, had good fidelity, and were on track with achieving project goals (Annex 7). 
Common adaptive management practices implemented by the IPs are described further below. 
1. Remote and hybrid monitoring used digital technologies to substitute or complement FTF M&E 
activities such as data collection, capacity building activities, team meetings, and engagements 
with health partners in their pre-pandemic plans. Video conferencing tools (WebEx) were used 
for virtual visits. Counterpart staff on the field clearly illustrated the continuous presence of TB 
Platforms despite the pandemic through their remote phone monitoring, 
"They’re really here. If they cannot come to the hospital, they will still call. They really maintained it even 
during the pandemic." (FGD, Field Counterpart Staff 2, TB Platforms)
2. Remote and hybrid monitoring are viewed as opportunities to expand and modify original plans. 
An example of this is the remote rapid assessment survey of TB IHSS, where they modified their 
survey tool to evaluate their practice on the field.
“Originally, we didn’t plan to have the follow-up survey after six months…it’s good to have one to 
evaluate what has happened or what is the effect of the intervention done if ever, coming from the 
results of the baseline and one month assessment.” (KII, Central Office Staff 4, TB IHSS)
42
3. Use of digital technologies in remote and hybrid monitoring improved the efficacy of integrating 
data and knowledge generation. For instance, the use of KoboToolBox in BARMMHealth has 
facilitated data consolidation and its communication. 
“It is easier to generate summary reports (tables, charts) with Kobo, and deeper analysis can be done 
using customized charts… " (FGD, Central Office Staff 2, BARMMHealth)
Practices with lower adaptability were observed in those with delays in implementation. Key issues 
across the IPs that hindered adaptations are summarized in Table 8. Of note, practices that are more 
complex and larger in scale for planned implementation, such as DHIS 2, are expected to have delays 
due to multiple components within the practice: conduct of training virtually is difficult; the data the 
information system collects can be complicated; the security and access to the system are given to 
everyone; equipment distribution can be delayed; and there may be complex nuances in the actual use of 
the system.
Table 8. Key Issues That Hindered Adaptability of Implementation of Remote and 
Hybrid Monitoring
Issues Illustrative quotes
Complexity of the practice "It’s difficult to know if participants can understand and grasp the 
concept, if they are listening or are still on the call or not." (KII, 
Central Office Staff 3, ReachHealth)
Difficulty in verifying data 
collected
“Difficulty obtaining MOVs during phone monitoring and preference 
towards facility visits.” (KII, Central Office Staff 2, ReachHealth)
Low participation of target 
participants 
“It is common practice for nurses to answer [the surveys] even if they 
were sent to the doctors. Compared to doctors, the nurses usually have 
more knowledge on the operations side.” (KII, Central Office Staff 1, 
TB Platforms)
Development of digital tools 
takes time
"It took a while to finalize the online training materials…[partners]
were requesting if we could perhaps revise some of the materials to 
reflect this and that, so we adjusted.” (KII, Central Office Staff 2, 
MTaPS)
Difficulty in retrofitting tools for 
the pandemic
“It was difficult to come up with a tool that approximates pre-COVID 
levels of data collection.” (KII, Central Office Staff 2, TB Platforms)
43
External partners are decision 
makers
“We have no hold, no access inside [the technical working group] so 
we are just waiting.” (KII, Central Office Staff 2, RenewHealth)
“Those were delayed a bit due to the development of the materials. 
Number one, we needed to consult a lot of other development 
partners, not just DOH…” (KII, Central Office 2, MTaPS)
5.3. Feasibility for Broader Use of the Practices and Their Sustainability
Successful practices need to be replicated, but implementing them could be difficult. Therefore, 
replication needs to focus on the core components of practices, as practices cannot work without them. 
Remote and hybrid monitoring as a practice has two core components: the digital technology used and 
the implementing individuals. 
Because practices are contextualized, an understanding is first needed as to whether they are feasible to 
scale. The interaction between core components provides information on what enables or hinders these 
components. This assessment targeted end-users of digital technology as among the respondents, 
evaluating individual acceptability of the technology and the appropriateness of the technology to their 
work. Acceptability encompasses the user’s cognitive understanding, feelings and beliefs, and their 
perceived burden and benefits of the practice.64 Appropriateness is the matching of the expected task to 
the level of the implementer’s capacities.65 
Enablers and challenges encountered by the participating individuals involved in implementation were 
identified. These end users were not acting independently but rather for the organizations. Therefore, 
some of their responses pertained to what affects them directly and indirectly on an individual and on an 
organizational level. These enablers and barriers are classified and summarized in Figure 5.
64 Sekhon, Mandeep et al, 2017. 65 Proctor, Enola et al., 2021.
44
Figure 5. Enablers and Barriers at the Individual and Organizational Level with Remote 
and Hybrid Monitoring
When starting implementation of the practices, IPs emphasized 1) training of implementers to ensure 
the practice is rolled out correctly, 2) defined roles and expectations for each of the implementers, and 
3) a budget to accommodate the costs of implementation. When feasibility to scale was assessed, these 
three components were checked particularly at the individual level because issues on the field level with 
these components will most probably lead to failure in implementation. In these cases, the practice 
should not be scaled up. 
In general, most of the remote and hybrid monitoring practices in the IPs were feasible to scale due to 
perceptions or beliefs on the relative advantages of the practices, which were associated with the 
project staff’s confidence to implement the M&E practices correctly. The technologies were low cost, 
widely used, and available. There was leadership support and policies that clarified the roles and 
expectations. However, there were challenges when the digital technology being used failed or was 
inappropriate for the particular situation.
We then assessed the potential sustainability of these practices within and beyond the life of the IPs. For 
this, we focused on the organizational enablers and barriers in Figure 5 and were guided by the Program 
Sustainability Framework66 (Annex 3). Funding stability, organizational capacity, and environmental 
support reflects the three components in the assessment of feasibility to scale. Issues in scaling will 
66 Luke, Douglas et al., 2014.
45
persist when not addressed. Likewise, issues in adaptation to shifting and changing contexts as noted and 
listed in Table 8 affect sustainability as well. Partnerships are also important because M&E in the IPs is 
heavily dependent on collecting data from external health partners in the government. The general 
capacity, resources, level of authority, and status of working relationships can determine the ease or 
difficulty of collecting data. Therefore, buy-in of the external partners to the practices is critical. The 
practices have limited sustainability when the current implementation issues in the above-mentioned 
domains persist.
Annex 8 consolidates the results of the assessment of each practice using the M&E and IR analytical 
frameworks. Notably, the overall level of assessment of the practices, which considers implementation 
outcomes, largely differs from the earlier assessment which used only M&E domains. 
46
6. SUMMARY AND RECOMMENDATIONS
6.1. Summary
M&E Practice Types and Levels of Effectiveness
The M&E sections of this report discussed the general M&E systems of seven USAID Philippines HP IPs, 
along with their adaptations to the pandemic. The 16 identified specific remote and hybrid M&E 
practices were classified into six categories: phone-based, web-based, videoconferencing, information 
management system, remote access software, and hybrid practices. Various factors that contributed to 
the effectiveness level of these practices, particularly in the use of quality M&E data for decision making 
toward the achievement of their project results, were qualitatively analyzed. In summary, there was one 
practice (6%) at Level 0, three (19%) at Level 1, nine (56%) at Level 2, and three (19%) at Level 3, the 
last of which are considered candidate good practices. Almost all hybrid practices fall under Level 2, 
which indicated a good level of effectiveness as compared to other practices. This summative assessment 
of the practices is in terms of M&E effectiveness and does not yet consider the fidelity, feasibility, and 
sustainability of its implementation. 
Weighted Evaluation 
Considering the results of both M&E and IR framework analyses, the weighted evaluation placed two 
(12%) practices at Level 0, three (19%) at Level 1, five (31%) at Level 2a, three (19%) at Level 2b, and 
three (19%) at Level 3, which are considered candidate good practices.
6.2. Recommendations for Strengthening the Remote and Hybrid M&E Systems (For 
USAID and IPs)
Perform continuous digital training to ensure that staff are prepared to use the new M&E methods.67
The respondents generally felt that the remote methods were good alternatives to continue routine 
monitoring when in-person contact are not possible. Remote approaches ensured the continuity of 
routine monitoring despite mobility restrictions and safeguarded the health and safety of not only the 
project staff but also their partners and respondents. Relevant orientations to new methods and tools 
would be necessary. We recommend a more formal mandate from USAID on remote and hybrid M&E 
practicesto ensure consistency in implementation and institutionalization/full integration of the 
approaches within project M&E systems.68,69 
Keep abreast on the various emerging methodologies for data security and protection. Some IPs have 
developed or intensified their data protection and security measures by limiting people who can access 
the data. Data protection could be further enhanced by limiting the collection of sensitive and/or 
67 Dube, Thulane et al. 2021.
68 Hensen, Bernadette et al., 2021. 69 Raftree, L, n.d.
47
personal information, conducting risk-benefits assessments, establishing a data sharing agreement with 
relevant partners, and ensuring data security, from data collection (e.g., end-to-end encryption) to data 
disposal.70,71
Support additional workload and demands with commensurate investments in resources, capacity, 
training, and support such as personal protective equipment and mental health support. While almost all 
of the practices were well resourced, a few IPs were initially under-budgeted and had limited staff,
particularly at the field level. Most remote methods require an initial upfront investment - but are more 
time-efficient compared to FTF methods.72 One of the facilitating factors identified in the practices was 
the provision of communications allowance to respondents. 
Carefully consider the nature of project indicators when determining the type of remote methods to be 
used.73 Though remote practices were widely accepted, some respondents felt that these could not 
always replace the FTF practices, especially when collecting data for indicators such as FP stock out rate, 
and when conducting data validation or DQAs/DQCs, where checking the physical documents on-site is 
vital. ReachHealth and BARMMHealth have addressed the latter by asking the health facilities to scan or 
send photos of needed documents; with this adaptation, extra vigilance is needed to ensure 
confidentiality of these documents. 
Account for intrinsic methodological limitations and practicality at the level of implementers when 
considering remote/hybrid M&E approaches to adopt. The remote approaches were frequently reported 
to have lesser reliability and integrity due to challenges in physically validating the data. These and other
limitations of remote methods should be anticipated and acknowledged as part of the risk assumptions 
in M&E.74 The participants have also expressed the relative advantage of an FTF approach when it comes 
to ensuring data quality for process checks, data collection, and checking of physical means of 
verifications. Looking ahead, however, the possible impact on data quality when collecting data remotely 
instead of in-person needs further assessment. USAID should also consider balancing/reducing data for 
accountability requirements, easing compliance and/or anti-fraud requirements, and avoiding an 
extractive process, especially at times of emergencies like the pandemic. 
Digital tools or solutions must undergo end-user validation tests and refitting to enhance participation.
Participants prefer online surveys that are short (no longer than 30 minutes) and highly structured 
(predefined answer choices, no open-ended questions), particularly when they use smartphones with 
small screens as it is hard to enter large quantities of text.75,76 Incorporating these features will thus help 
improve response rates. This assessment found that doing this was one of the reasons that remote rapid 
assessment surveys of TB IHSS were more effective than the remote phone monitoring of ReachHealth 
and remote IPC assessment of TB Platforms. 
70 Hensen, Bernadette et al., 2021. 71 Raftree, L, n.d
72 Hensen, Bernadette et al., 2021. 73 USAID, 2020.
74 Groupe URD & CartONG, n.d. 75 Dabalen, Andrew et al., n.d. 76 Dette, Rahel et al., 2016
48
Understanding context is vital in bridging the digital divide.77,78 Careful examination of internal and 
external implementation environments would help in making M&E practices more inclusive. According 
to 2020 World Bank data,79 the Philippines has high mobile phone access (155 mobile cellular 
subscriptions per 100 people, compared to the global average of 107). However, only 114,000 per 1 
million people have secure Internet servers and 5 per 100 people have fixed broadband subscriptions. 
The IPs should take this into consideration by exploring more phone-based practices (SMS, voice 
messaging, calls) that require zero or minimal Internet use in their routine monitoring. In addition, IPs 
could explore existing monitoring and communications systems outside USAID with whom they can tap, 
leverage, and share data. This could help both to obtain data and build shared understanding between 
development actors.80,81
Finally, USAID could explore innovative, community-based monitoring schemes.82,83,84 USAID could 
consider engaging and training barangay health workers, community volunteers, key informants, local 
community leaders, and community-based organizations in USAID routine monitoring and validation 
activities. This could both strengthen the relationship of the IPs with the local community while at the 
same time addressing the gaps resulting from limited FTF visits, as these community members can access 
the target sites more easily. 
6.3. Recommendations for Strengthening Implementation of Remote and Hybrid M&E 
Practices 
The practices that this assessment investigated are largely distinguished and shaped by the needs of 
specific IPs. It particularly explored the domains of implementation, arriving on the following themes of 
action to strengthen implementation:
Reduce complexity of the practice by improving the acceptability and appropriateness of the digital 
technologies. This is best done by testing and validating the tools with end users and health partners 
before implementation. Complexity arises from the data that are collected and competency and 
capability of the external stakeholder expected to use the technology. With respect to the latter, the 
competency of the external health partner in using the technology was found to be a major barrier, 
especially considering that video conferencing apps and digital technology could fail in remote areas.
Modification of paper-based data collection tools which are converted for digital use (either by web￾based forms or by phone calls) should be considered. The more successful practices have considered 
redesigning their tools and focusing data collection on their most critical indicators. Flexibility and use of 
77 USAID, 2020. 78 Raftree, L., n.d. 
79 World Bank, n.d. 80 Groupe URD & CartONG, n.d. , 81 USAID, 2020.
82 Hensen, Bernadette et al., 2021 83 USAID, 2020. 
84 Raftree L., n.d. 
49
both physical and digital data collection forms should be considered based on what is available in each 
health partner's context. 
Generate confidence in the data collected via remote monitoring. This involves the development of a 
manual or verification guide for data collected remotely and virtually. Preference for field visits is mostly 
rooted in mistrust of remotely collected data, with participants citing the lack of data verification 
activities and questionable data quality. However, the quality of data collected remotely has not been 
formally tested in this study.
Engage external partners and stakeholders more inclusively and ensure their early buy-in. Ensure their 
active participation in developing tools and decision making from the start, such as by including them in 
stakeholder consultation meetings during the planning stage. Their participation and buy-in must be 
sustained throughout implementation, as external partners play a vital role in sustainability.
6.4. Specific Recommendations per Identified Remote and Hybrid M&E Practice
Table 9. Specific Recommendations per Identified Practice
Remote and 
Hybrid M&E 
Practice 
Recommendations
I. ReachHealth
DHIS 2
Weight: Level 2a
Resolve implementer-level issues (training, technology uptake, and resource).
Remote Phone 
Monitoring
Weight: Level 1
While this may be an appropriate M&E practice during the pandemic, it is difficult 
to scale given the implementation hurdles at the field level (both project and 
counterpart staff). Mitigating these challenges may also be difficult as some are 
beyond ReachHealth's control (i.e., limitations of not being on-site, poor mobile 
signal, limited availability of field counterpart staff). A hybrid monitoring approach 
may be needed, giving data collectors discretion in choosing which method is best 
their specific contexts. 
Although FTF approaches are also being used when remote data collection is not 
viable, hybrid monitoring may need to be integrated into official mandates to 
move toward process excellence and quality data. Additional qualitative remote 
measures of corroboration can help ensure data quality when using remote 
approaches. More broadly, sustained programmatic evaluation of the effectiveness 
of approaches used for M&E—especially new remote approaches—is needed. 
50
Hybrid Rapid 
Feedback 
Mechanism 
Weight: Level 2a
There is a need to finalize major phases (rapid data collection and dissemination) 
and to fully adopt refinements being made along its development. At this point, 
risks and assumptions must be adjusted to mitigate external barriers to 
implementation (i.e., lack of field partners' time and human resources during the 
pandemic). 
More is needed to ensure quality of data and accountability in monitoring 
practices. Strengthening quality of data obtained from remote methods may 
require additional qualitative remote measures for corroboration. There may also 
be more sustainable and reliable methods of increasing compliance for data 
encoding other than a pahiyaan (shaming) system. 
In general, sustained programmatic evaluations of M&E approaches used by 
USAID are urgently needed. 
II. BARMMHealth
Hybrid 
monitoring of FP 
program through 
site visits, phone 
calls, and FB 
messenger
Weight: Level 2a
The assessment for this practice is a special case because identified respondents 
were all from (non-USAID) regional and provincial counterpart health offices, 
which was then regarded as the inner setting. Although USAID BARMMHealth 
utilizes data collected by the BARMM Ministry of Health (MOH) and PHOs, they 
also conduct independent monitoring adjacent to the monitoring of the local 
health offices. 
This assessment considered USAID BARMMhealth as part of the outer setting 
that supports local monitoring activities. Given their current critical role in 
providing TA and other external support to the monitoring activities of the local 
health offices, key recommendations will focus on actions that USAID 
BARMMHealth can take to further strengthen and sustain the capacity of the 
local health offices to implement their monitoring activities, especially beyond the 
life of the project. 
In general, there is a need to address the persistent challenges inherent in 
BARMM's unique geographical and digital environment. For example, limitations 
in securing MOVs or conducting inspection through remote approaches may be 
addressed by alternative qualitative remote measures for corroboration. Given 
their technical capacity, BARMMHealth can help develop guidelines for 
conducting hybrid monitoring activities of the local health offices. This will 
translate current iterations for hybrid monitoring into official mandates that will 
help improve both the monitoring process and quality of data. 
Given the unique set up of BARMM Region’s health system, there is an urgent 
need to strengthen communication and collaboration with MOH-BARMM. The 
51
regional health office (RHO) will need to assume current support that 
BARMMHealth provides to the PHOs’ monitoring activities, as they will sustain
these activities beyond the life of the BARMMHealth project; and the RHO will 
need capacity-building support and TA for auditing and monitoring the provincial 
level. Human resources and communication allowances will also be needed to 
facilitate monitoring activities at the RHU level.
Blended P&R 
Sessions
Weight: Level 2a
There is urgency in addressing poor Internet connectivity in partner areas.
Adopting blended P&R will be difficult because FTF is still preferred. Some 
exploration on how to strengthen blended P&R is needed, focusing on refining 
and defining the protocol, building incentives, and assessing the quality of 
information generated.
KoboToolbox
Weight: Level 2b
BARMMHealth needs to determine and address causes of non-reporting among 
some RHUs to ensure completeness of data. Both paper-based and Kobo forms 
need updating to preserve data precision. Also, counterpart staff need training to 
navigate this new technology for data collection.
The interoperability of KoboToolBox with DOH data entry forms should also be 
explored to identify any duplicated processes.
There is still limited understanding about the advantages of KoboToolBox versus
the project management information system of BARMMHealth. Because the 
project intends to transition to the latter, the former needs to be 
designed/developed in a way to ease transition such that it is acceptable to the 
end users. Good practices to be adapted and maintained during the transfer must 
be identified. 
III. MTaPS
Remote Access to 
Information 
Systems (DOH￾PMIS and ITIS)
Weight: Level 1
DOH needs to resolve a number of issues around funding, equipment, server 
problems, human resources, data quality, and sustainability.
USAID, as a more meaningful partner and co-developer, should continue to 
actively participate in planning for HIS strengthening of DOH-PMIS and ITIS. 
Collaborative agreements for system development, performance analytics, TA, 
resource support, as well as TA on DQA activities for DOH should be sustained 
to ensure that DOH is able to pursue their plans for developing the HIS 
platforms. Costing of the development and sustainability of DOH-PMIS and ITIS 
should be done, and data should be used to inform how to run and manage these 
information systems.
COVID-19 Special Recommendations for future monitoring efforts by DOH would include 
52
Response Project￾hybrid monitoring 
of facilities (onsite 
and virtual)
Weight: Level 2b
capacitating HCWs to conduct hybrid monitoring activities (since they are used 
to being mobilized in the field) as well as educating the people you are monitoring 
on the use of technology, monitoring processes, and data to be reported in both 
FTF and virtual. Issues in Internet connectivity should be resolved as well. 
If partners such as DOH prefer WebEx as the online video conferencing 
platform, USAID MTaPS should adapt accordingly to facilitate its use.
IV. ProtectHealth
CiTEC Project 
Regular Online 
Conference Calls
with TB IHSS, 
Cebu City LGU, 
and other 
stakeholders (e.g.,
Central Visayas 
Center for Health 
Development and 
Barangay officials)
Weight: Level 3
A few minor issues need to be addressed to improve the effectiveness and quality 
of these regular conference calls. As recommended by the participants, the 
agenda for the calls should focus on concerns that require all-party consensus and 
issues that need to be urgently addressed to avoid repetitive updates. Software 
extensions may be downloaded to further improve participant experience in the 
meetings, such as noise-canceling applications. Additionally, participation in these 
meetings may be more effectively sustained by provision of incentives (e.g., 
representation allowance) rather than disciplinary measures (e.g., set quotas on 
the number of meetings to be attended). 
Specific guidance with legal or contractual implications (e.g., recording without 
consent of participants) can help improve the quality and security of the 
conference calls. 
While there is not plan to sustain the practice after the CiTEC project, it is
worth considering in future work with a similar set-up, particularly if the 
organization is ready to implement the practice and there are minimal challenges 
to continue implementation.
The development of a protocol may further enhance the implementation of 
regularly scheduled video conference calls across diverse and distinct 
units/participants. A set of guidelines covering the essential dimensions of an 
online conference call can improve the procedures and serve as a baseline to 
gauge quality improvements. Guidelines could cover provisions on the use of the 
online conference platform, ethical considerations, agenda-setting, and key roles 
and responsibilities of regular meeting participants (e.g., implementation leader). 
A manual may also benefit other IPs or organizations who intend to employ the 
same practice. 
V. RenewHealth
CBDR IMS The IMS needs to be finalized based on LGU and partner government agencies' 
needs, and to fully adopt the refinements being made along its development. 
53
Weight: Level 0 Despite having planned measures to mitigate risks in implementation, support to 
LGUs may need to be adjusted to unforeseen challenges once the system is 
finalized and implemented on the ground.
Ensuring the system is sustained beyond the life of the project requires intensified 
engagement with partner agencies (DOH, DDB, Department of Interior and 
Local Government, Department of Social Welfare and Development). 
RenewHealth must also secure the buy-in of an agency that may continue the 
legacy of the IMS before the project ends. Establishing plantilla (permanent, full￾time) positions for system implementers in the LGU may be needed to mitigate 
constant staff turnover. 
Assessments on the effectiveness of the system are also needed, both to support 
USAID RenewHealth M&E and its ability to improve data management and 
sharing between the LGU and partner government agencies. 
Guidelines to be developed for CBDR IMS should be both detailed to be useful 
to field-level implementers and inclusive to consider the various contexts of 
LGUs (i.e., differing structures of CBDR service delivery across LGUs) that will 
be using this system.
VI. TB Platforms
Hybrid 
monitoring of 
TBCC using 
regularly
scheduled phone 
calls and onsite 
monitoring/mento
ring visits
Weight: Level 2b
Hybrid monitoring of TBCC is still recommended given the ongoing pandemic, as 
call log sheets can be validated remotely. Current call log sheets should also 
incorporate transactions/communications which used FB messenger to avoid 
creating a "false decrease" of calls/communications with the patient (current log 
sheets only count phone calls). A monitoring tool for use during the scheduled 
phone calls, as well as an orientation for the use of this tool, should be
developed. Provision of forms should be streamlined as well. 
In implementing the TBCC, official protocols/guidelines should be developed to 
better elicit needed data from the patients. TB Platforms should also devise a 
better way to disburse communication allowances for HCWs, preferably in the 
facilities instead of at the province level). 
Ensuring the sustainability of TBCC and its monitoring activities beyond the life of 
the project will entail further investigation of the local context as to where TBCC 
is being implemented (such as number of RHUs implementing TBCC, restrictions 
of TBCC in a certain area). There is a need to look at other facilities interested in 
implementing TBCC and to engage stakeholders such as the RHUs, Municipal 
Health Offices, Development Management Officers (DMOs), and PHOs at the 
local level. Sustainability measures should focus more on the district and 
54
provincial levels . There must first be evidence for sustainability at the local level 
before scaling up up at the national level.
Hybrid 
monitoring/
mentoring for 
FAST 
implementing 
facilities
Weight: Level 1
This is a promising M&E practice, but challenges during implementation would 
suggest that it is not feasible to scale at the moment. Given its potential 
adaptability and sustainability, we provide these recommendations. 
Strengthening the quality of data from the intervention and sustaining its 
implementation will require addressing inadequate human resources and the 
current limitations in the remote monitoring approaches used. Field counterpart 
partners must be able to document and provide data for monitoring to ensure
the quality of data. Short-term assistance may not be sufficient to address the lack 
of human resources on either side of the monitoring activity. Overcoming this 
hurdle may require additional staff focused on M&E (TB Platforms side), and 
retraining current point persons and building capacity of other staff members in 
the FAST implementing facilities (counterpart facilities side). 
Previous iterations for the use of remote approaches should be institutionalized 
as official mandates to improve both processes and data quality. Adding hybrid 
monitoring to the current process guide will improve remote monitoring 
procedures and quality of data obtained from remote approaches. Strengthening 
the quality of data obtained from remote methods may also require additional 
qualitative remote measures for corroboration. In addition, it can be leveraged to 
ease the burden of onsite monitoring. As stated by one of the participants, “We 
have proven that there are certain activities we can do, such as the hybrid 
mechanism, that it can be remotely or virtually done. There are less physical 
activities since most have been conducted virtually.”
Shifting from FTF to remote approaches requires adjusting expectations of what 
successful implementation looks like given the inherent differences between the 
two modes. Having set guidelines for a hybrid approach will help to this end. 
These guidelines can also serve as a baseline process guide to gauge quality 
improvements when remote approaches are used. Findings may then be 
integrated with the internal evaluation of the implementation of FAST strategy as 
whole. The creation of a manual will eventually benefit the organizations which 
intend to continue monitoring FAST implementation beyond the life of TB 
platforms, or other organizations seeking to employ similar remote monitoring 
approaches.
Assessment of 
IPC using Google 
Survey
Weight: Level 0
Before considering the use of different platforms such as Open Data Kit or DHIS, 
the current design must be assessed to mitigate the risk of failure with the use of 
another digital platform. The questionnaire used in the survey should be shorter 
(fewer questions), and other platforms can be considered for open-ended 
55
questions. In any survey, respondent issues such as busy schedules and Internet
connectivity should be considered. In addition to using Google Surveys, paper￾based hard copy or soft copy Word files may be provided as an alternative 
option. With the influx of online surveys, respondent burnout is a risk. Thus, it is 
important to provide feedback to respondents so they understand the value of 
the data being collected.
A clear framework is needed in the pre-implementation phase. There was no 
clarity regarding baseline acceptability of the survey; consultations on the design; 
conduct of validation, pre-testing or piloting; looking at actors/players involved;
and other important considerations at the pre-implementation stage. Preference 
for other platforms aside from Google Survey may have affected planning and 
implementation of this activity. 
The appropriateness of the people providing feedback to the facilities should be 
considered. Feedback may be done in tandem with a DMO who may already have 
regional connections to allow backing. It may have been inappropriate to have 
someone with a background in engineering engaged for the baseline assessment 
of IPC. 
Overall, Google Surveys has great potential due to its innovation, global 
interoperability, and advantages. However, due to poor planning and 
implementation, this was not utilized well. A more in-depth investigation is 
needed to identify for the reasons for failure. An key lesson from this experience 
is the importance of good planning prior to implementation.
PMDT remote 
monitoring and 
mentoring
Weight: Level 2a
Improving the quality of information obtained through remote monitoring will 
involve several adjustments in implementation. This may require additional 
qualitative remote measures for corroboration. The remote monitoring tool can 
also be enhanced to capture more detail for well-informed decision making. 
Regular orientation or a briefer may also be provided to the participating facilities 
on how to answer the monitoring tool to avoid misinterpretation of the 
questions. 
Remote monitoring and mentoring in PMDT facilities is a good alternative when 
in-person activities are not possible ad may also be used in tandem with FTF 
monitoring. Where possible, monitoring/mentoring sessions can be conducted in 
a hybrid manner: some activities such as FGDs and filling out the monitoring tool 
can be done online, while data validation is better done in-person if feasible. 
The protocol for PMDT monitoring/mentoring may be revised to include steps to 
do this remotely or using a hybrid approach. These will guide cases where onsite 
monitoring is not feasible or if a hybrid setup for monitoring is possible. 
56
According to a participant, “Hybrid is good because there are remote and FTF 
activities. Only two representatives per STC are needed. Also since my STC is 
small, FTF monitoring is not advisable if there are a lot of attendees. But it is also 
good for them to see the difficulties of the STC nurses.”
DQCs can also be done in a hybrid manner, especially for those who are not 
provided any accommodation since they reside within the same province but live 
far from the conference venue.
The development of guidelines for remote and/or hybrid monitoring of PMDT 
facilities may improve the quality of monitoring procedures and, by extension, the 
overall quality of data obtained. Development of said guidelines can also serve as 
a baseline process guide to gauge quality improvements when remote or hybrid 
approaches are used. Written guidelines will eventually benefit organizations 
which seek to use similar remote monitoring approaches.
VII. TB IHSS
Remote P&R 
sessions
Weight: Level 3
Regional implementation teams should be trained to conduct and process outputs 
from P&R to decrease the burden on support staff and the SI unit. Addressing 
poor Internet connection in partner areas may improve engagement in remote 
P&R sessions.
Other practices which use an online conferencing format may adopt TB IHSS use 
of online polling, e-games, and other interactive methods to sustain the attention 
of participants.
Remote rapid 
assessment survey
Weight: Level 3
The remote rapid assessment found that remote data collection has been 
successful in the dynamic COVID-19 situation. Factors contributing to the 
implementation’s success include: 1) being mindful of HCWs’ busy schedules by 
designing a short but comprehensive questionnaire, 2) validating and pre-testing 
the questionnaire, 3) providing paper-based forms (that HCWs can take photos 
of) to be sent to the Field Implementation Officer [FIOs]) as well as soft copy 
Word files (can be sent via email), 4) conducting ongoing validation simultaneous 
with data collection to clarify any missing or conflicting information, and 5) 
involving upper to lower management. Complementing these practices was a 
strong commitment of personnel involved,
Because this is considered as a good practice, TB IHSS can start formulating steps 
for quality improvement and process excellence such as having pre￾implementation checks for scaling up (same rigor of planning, collection, and 
analysis), mid-implementation monitoring, and post-implementation M&E. If this is 
well-designed, scale-up of the practice is easier and investments are not poorly 
57
placed since there is acceptability and trialability at the pre-implementation phase. 
USAID should consider the best way to persuade DOH to adopt the practice as 
it has been conducted by TB IHSS. 
If the survey is to be used in the future, the questionnaire will need further 
modifications to ensure it remains relevant to the current situation and the needs 
of DOH NTP. Sample size should be increased (if possible, not limited to the big 
three regions) to thoroughly evaluate reliability of the results and provide better 
analysis of data. More time should be allotted for implementation, and logistics 
should be considered (e.g., communication allowance for Internet connectivity). If 
implemented nationally, NTP should have a department memo/policy on the 
conduct of the surveys to make it easier for the hospitals because they are 
informed of its importance.
58
7. CONCLUSION
The use of remote M&E practices has become the “new normal” in the COVID-19 era. Although 
remote M&E methods are not new to development and humanitarian work, the pandemic has forced 
almost all organizations to increase use of such methods to comply with restrictions. This assessment of 
the remote and hybrid M&E practices within the HP has shown how digital technologies play a vital part 
in responding to the need for remote M&E solutions. The nature of these technologies and the 
complexity of the needed information influenced the levels of effectiveness, adoption, and adaptation. 
Individual and organizational facilitators of implementation include resource allocation, leadership and 
policy support, external party involvement, trust and commitment of implementers, relevant digital skills, 
and adequacy of process orientation (e.g., through SOP). Barriers include complexity of the M&E 
solution, FTF preference or staff resistance, technological failures (e.g. Internet signal), poor data quality 
generated, low participation, overburdened staff, and low leadership support.
Strengthening M&E systems would require digital training, commensurate investments in associated 
technology, use of multiple tools and methods to improve data validity, context understanding, and end￾user studies to improve the intervention. Implementation can be strengthened by reducing complexity 
and generating user confidence in the collected data. Additional studies are recommended to objectively 
and extensively assess the data quality from remote practices when compared to FTF methods, 
particularly in LMIC settings. 
Of the 16 remote and hybrid M&E solutions, three were evaluated as good practices: TB IHSS Remote 
Rapid Assessment Survey, TB IHSS Remote P&R Sessions, and ProtectHealth’s CiTEC Project Regular 
Online Conference Calls. All three demonstrated features of an effective M&E practice, had fidelity in 
the implementation design, were feasible to scale, and had potential for sustainability. Nascent systems 
such as RenewHealth’s CBDR IMS still warrant further observation. Hybrid practices, in particular, 
consistently showed an optimal level of M&E effectiveness and acceptability in the study. It is then 
expected that post-pandemic ways of working in M&E should employ more hybrid practices where 
remote methods will complement FTF activities rather than fully replacing them.
59
8. REFERENCES
Allen JD, Linnan LA, Emmons KM, Brownson R, Colditz G, Proctor E. Fidelity and its relationship to 
implementation effectiveness, adaptation, and dissemination. Dissemination and Implementation 
Research in Health: Translating Science to Practice 2012:281–304.
Brownson RC. Dissemination and Implementation Research in Health: Translating Science to Practice. 
Oxford University Press; 2017.
Center for Public Health Systems Science. Program Sustainability Framework and Domain Descriptions 
2012. https://doi.org/10.7936/frfw-fk62.
CFIR Research Team- Center for Clinical Management Research. The Consolidated Framework for 
Implementation Research 2021. https://cfirguide.org/ (accessed June 24, 2021). 
Constructs n.d. https://cfirguide.org/constructs/ (accessed December 3, 2021).
Dabalen A, Etang A, Hoogeveen J, Mushi E, Schipper Y, von Engelhardt J. Mobile Phone Panel Surveys in 
Developing Countries : A Practical Guide for Microdata Collection n.d. 
https://openknowledge.worldbank.org/handle/10986/24595 (accessed December 6, 2021).
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of 
health services research findings into practice: a consolidated framework for advancing 
implementation science. Implement Sci 2009;4:50.
Dette R, Steets J, Sagmeister E. Technologies for monitoring in insecure environments. Global Public Policy 
Institute, Berlin, Www Gppi 
net/media/SAVE__2016__Toolkit_on_Technologies_for_Monitoring_in_Insecure_ Environments 
Pdf 2016.
Dube T, Ncube SB, Mlotshwa S, Matsika GN, Maonde N. The changing face of monitoring and evaluation in 
the age of COVID-19: practitioners’ field experiences from Zimbabwe. Heliyon 2021;7:e07386.
Fortin D. Remote data collection during COVID-19 2020. https://www.jsi.com/remote-data-collection￾during-covid-19/ (accessed June 25, 2021).
Greenleaf AR, Gibson DG, Khattar C, Labrique AB, Pariyo GW. Building the Evidence Base for Remote 
Data Collection in Low- and Middle-Income Countries: Comparing Reliability and Accuracy Across 
Survey Modalities. J Med Internet Res 2017;19:e140.
Groupe URD, Cartong. Learnings from the COVID-19 Crisis and its Impact on Information Management 
Practices in the Aid Sector - Improving Information Through Inclusive Processes and Capacity 
Building n.d. https://www.alnap.org/help-library/learnings-from-the-covid-19-crisis-and-its-impact￾on-information-management-practices (accessed December 6, 2021).
Hensen B, Mackworth-Young CRS, Simwinga M, Abdelmagid N, Banda J, Mavodza C, et al. Remote data 
collection for public health research in a COVID-19 era: ethical implications, challenges and 
opportunities. Health Policy Plan 2021;36:360–8.
Jessica Z. Understanding CLA 2017. https://usaidlearninglab.org/qrg/understanding-cla-0 (accessed July 16, 
2021). 
Karsh B-T. Beyond usability: designing effective technology implementation systems to promote patient 
safety. Quality and Safety in Health Care 2004;13:388–94. 
https://doi.org/10.1136/qshc.2004.010322.
Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability 
Assessment Tool: a new instrument for public health programs. Prev Chronic Dis 2014;11:130184.
Means AR, Kemp CG, Gwayi-Chore M-C, Gimbel S, Soi C, Sherr K, et al. Evaluating and optimizing the 
consolidated framework for implementation research (CFIR) for use in low- and middle-income 
countries: a systematic review. Implement Sci 2020;15:17.
MEASURE Evaluation PIMA. Monitoring and Evaluation Capacity Assessment Toolkit User Guide 2017.
Pérez D, Van der Stuyft P, Zabala MC, Castro M, Lefèvre P. A modified theoretical framework to assess 
implementation fidelity of adaptive public health interventions. Implement Sci 2016;11:91.
60
Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Implementation research: what it is and how to do it. 
BMJ 2013;347:f6753.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation 
research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment 
Health 2011; 38:65–76. 
PSAT 2019. https://sustaintool.org/psat/ (accessed December 2, 2021).
Raftree L. Getting remote M&E right: ethics, challenges and gaps n.d. https://www.alnap.org/help￾library/getting-remote-me-right-ethics-challenges-and-gaps (accessed December 6, 2021).
Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: Pragmatic application of 
and adaptations to the Consolidated Framework for Implementation Research (CFIR) for 
evaluation of a patient-centered care transformation within a learning health system. Learn Health 
Syst 2020;4:e10201.
Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and 
development of a theoretical framework. BMC Health Serv Res 2017;17:88.
Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, et al. The role of formative 
evaluation in implementation research and the QUERI experience. J Gen Intern Med 2006;21 Suppl 
2:S1–8.
UKaid. Discussion series summary: monitoring and evaluation during the COVID-19 pandemic n.d.
USAID. Discussion Note: Third-Party Monitoring in Non-Permissive Environments 2021.
USAID. Evaluation 2021. https://www.usaid.gov/evaluation (accessed June 22, 2021).
USAID. Guide for Adopting Remote Monitoring Approaches During COVID-19. United States Agency for 
International Development; 2020.
USAID. How to Work with USAID: Monitoring, Evaluation, and Learning n.d. https://www.usaid.gov/work￾usaid/get-grant-or-contract/trainings-how-work-usaid/monitoring-evaluation-learning (accessed 
December 6, 2021).
USAID. IR Tip #1: Introduction to IR: Real-Time Evidence to Save Lives. https://www.harpnet.org/tips-on￾implementation-research/tips/tip-1/. 
USAID. Monitoring Toolkit 2016. https://usaidlearninglab.org/monitoring-toolkit?tab=2&subtab=1 (accessed 
June 22, 2021).
USAID. Monitoring Pathways 2019. https://www.usaid.gov/project-starter/monitor-evaluation-and-learning￾toolkit-pathways/monitoring-pathways (accessed June 22, 2021).
USAID. Project M&E Plan. USAID n.d. https://www.usaid.gov/project-starter/program-cycle/project￾monitor-evaluation-plan (accessed June 26, 2021). 
USAID Learning Lab. Evaluation Toolkit 2015. https://usaidlearninglab.org/evaluation-toolkit?tab=2
(accessed June 23, 2021).
USAID/Philippines Collaborating Learning and Adapting for Improved Health (CLAimHealth) Activity. 
Health Project Performance Data Consolidation: FY 2019 Midyear Report. 2019.
USAID/Philippines Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) Activity. 2020 
Health Project Performance Evaluation Report 2021.
USAID/Philippines Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) Activity. 
Good Practices and Promising Interventions, Technical Series No. 6: Engaging Local Chief 
Executives to Build Local Capacity and Strengthen Health Systems 2021.
USAID/Philippines. USAID/Philippines Health Project 2017-2022 Redacted Project Appraisal Document 
2017.
Women’s Refugee Commission. Guidance on Establishing Remote Monitoring and Management of GBV 
Programming in the Context of the Covid-19 Pandemic. Women’s Refugee Commission; 2020.
World Health Organization Regional Office for Africa. A Guide to Identifying and Documenting Best 
Practices in Family Planning Programmes 2017.
61
World Bank. Mobile cellular subscriptions (per 100 people) n.d. 
https://data.worldbank.org/indicator/IT.CEL.SETS.P2?end=2020&name_desc=true&start=2020&view
=map (accessed December 6, 2021).
62
ANNEX 1: CFIR DOMAINS AND CONSTRUCTS85,86
DOMAIN SHORT DESCRIPTION
Intervention 
characteristics
Aspects of an intervention that may affect implementation success, including its 
perceived internal or external origin, evidence quality and strength, relative 
advantage, adaptability, trialability, complexity, design quality and presentation, 
and cost.
Outer setting External influences on intervention implementation including patient needs and 
resources, cosmopolitanism or the level at which the implementing 
organization is networked with other organizations, peer pressure, and 
external policies and incentives. 
Inner setting Characteristics of the implementing organization such as team culture, 
compatibility and relative priority of the intervention, structures for goal￾setting and feedback, leadership engagement, and the implementation climate.
Characteristics of 
individuals
Individuals' beliefs, knowledge, self-efficacy, and personal attributes that may 
affect implementation.
Process of 
implementation
Stages of implementation such as planning, executing, reflecting and evaluating, 
and the presence of key intervention stakeholders and influencers including 
opinion leaders, stakeholder engagement, and project champions
85 Safaeinili, Nadia et al., 2020. 86 Constructs (n.d.) 
63
ANNEX 2: IDENTIFIED M&E DOMAINS
M&E Domains Description
Overview of the overall 
M&E system
This includes planning, capacity building, adaptive management, and the 
necessary resources for the implementation of the practice or the M&E 
system (e.g., budget, personnel, equipment).
Monitoring System This involves the implementation of set activities (e.g., data collection, 
management, analysis, validation, consolidation, interpretation, reporting 
or dissemination, utilization) in the MEL plan which involve monitoring 
performance data.
Evaluation System This involves plans and processes related to evaluation activities of the 
IPs.
Information management This refers to the systems and processes related to the management of 
collected and stored data. 
Monitoring of risks and 
assumptions 
This refers to the processes involved in the identification and 
monitoring of risks and assumptions of project implementation.
Data quality Assessing the quality of data involves looking into the criteria below:
• Timeliness Data are available at useful frequency, are current, and timely enough to 
influence management decision making
• Reliability Data reflect consistent collection processes and analysis methods over 
time
• Validity Data clearly and adequately represent the intended result
• Precision Data have sufficient level of detail to permit management decision 
making
• Integrity Data have safeguards to minimize the risk of transcription error or data 
manipulation 
64
ANNEX 3: SUSTAINABILITY DOMAINS OF THE PSAT
Domains Description
Environmental Support Having a supportive internal and external climate for your program
Funding Stability Establishing a consistent financial base for your program
Partnerships Cultivating connections between your program and its stakeholders
Organizational Capacity Having the internal support and resources needed to effectively manage 
your program
Program Evaluation Assessing your program to inform planning and document results
Program Adaptation Taking actions that adapt your program to ensure its ongoing 
effectiveness
Communications Strategic communication with stakeholders and the public about your 
program
Strategic Planning Using processes that guide your program's direction, goals, and strategies
65
ANNEX 4: DATA COLLECTION TOOLS 
Annex 4 - Data 
Collection Tools.docx
ANNEX 5: DATA COLLECTION TABLES 
Annex 5 - Data 
Collection Tables.xlsx
66
ANNEX 6: SUMMARY OF IDENTIFIED M&E 
PRACTICES BY CATEGORY
M&E 
Practice
IP Level Facilitating Factors Hindering Factors
I. Phone-based practices
Remote phone 
monitoring
ReachHealth 2 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Cheaper than FTF
● Low acceptability from 
enumerators
● Long questionnaire
● Compromised data quality
● Not appropriate for specific FP 
indicators (e.g., FP stockout rate)
● Low response rate
II. Web-based practices
Remote rapid 
assessment survey
TB IHSS 3 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Multiple modes of data 
collection
● Short yet comprehensive 
questionnaires 
● Conducted pre-testing and 
validation of tools
● Training and manuals provided 
to enumerators
● Communications allowance 
provided to respondents
● Constant follow-up of 
enumerators
● Regular validation of data
● Strong commitment of 
personnel involved
● Involvement of upper to lower 
management
● No policy developed, only 
concept note
● Low sample size
● Difficulty ascertaining reliability 
of responses
● Tight window period
Assessment of 
IPC using Google 
Survey
TB Platforms 1 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Training provided to field staff
● No official SOP or policy
● Poor data quality especially with 
open-ended questions
● Low response rate 
● Discomfort in using Google 
Sheets
● Limited field staff technical 
capacity in analysis and providing 
feedback
● Limited field staff technical 
capacity in providing TA on 
engineering
● Negative experiences of staff in 
giving feedback
67
KoboToolBox BARMMHealth 2 ● Aligned with the general M&E 
system
● Well resourced (staff, money)
● Orientations provided to 
provincial staff
● Automatic generation of tables 
and charts
● Bias measures conducted
● Not updating the online and 
paper-based forms at the same 
time
● Low reporting rates
III. Videoconferencing practices
Blended P&R 
sessions
BARMMHealth 2 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Limited Internet connectivity in 
some areas
● No SOP/guidelines for P&R 
sessions
Remote P&R 
sessions
TB IHSS 3 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Has a licensed platform account
● Had pre-pandemic experience of 
hybrid P&R sessions, with 
interactive activities
● No SOP/guidelines for P&R 
sessions
CiTEC project 
regular online 
conference calls
ProtectHealth 
(and TB IHSS)
3 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the M&E 
system for the CiTEC project, 
which is aligned with the general 
M&E system of ProtectHealth
● Capacity to adapt (flexible 
schedule)
● Communications allowance 
provided
● Added security measure (in 
Microsoft Teams)
● No guidelines on the process
● Repetitive updates
PMDT remote 
monitoring and 
mentoring 
TB Platforms 2 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Potential bias from respondents
● Limited data validation affecting 
perceived data quality
● No guidelines developed
IV. Information management system practices
DHIS 2 ReachHealth 2 ● Aligned with the general M&E 
system
● Well resourced (staff, money, 
equipment) 
● Consultant available
● Automatic features
● Limited system capacity to 
manage volume and complexity 
of information collected
CBDR IMS - still 
being developed
RenewHealth 0 ● Aligned with the general M&E 
system
● Well resourced (staff, money, 
equipment)
● Early stages of the program
● Lack of clear CBDR indicators
● No guidelines developed
68
V. Remote access software practices
Remote access to 
information 
systems (DOH￾PMIS and ITIS)
MTaPS 1 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Real-time updates and fuller 
access to data
● Presence of data map
● Lack of sustainable funding for 
DOH-PMIS and ITIS 
(government) 
● Identified data quality issues in 
DOH-PMIS and ITIS
VI. Hybrid practices in USAID Philippines HP
Hybrid Rapid 
Feedback 
Mechanism - still 
being developed
ReachHealth 2 ● Well resourced (project staff, 
money, equipment) at national 
level
● Well integrated within the 
general M&E system
● Quick turnaround time
● Perceived data quality issues
● Limited field staff in counterpart 
facilities
Hybrid monitoring 
of FP program 
through site visits, 
phone calls, and 
FB messenger
BARMMHealth 2 ● Well resourced (staff, money, 
equipment) at provincial level
● Communications and transport 
support provided at provincial 
level
● Pre-pandemic experience of 
hybrid P&R sessions
● Data validation measures in place
● Limited staff and communication 
allowance at RHU level
● No guidelines on the process
● Limited Internet connectivity of 
some facilities
COVID-19 Special 
Response Project
- hybrid 
monitoring of 
facilities (onsite 
and virtual)
MTaPS 2 ● Well resourced (money, 
equipment) 
● Few data quality issues which 
were easily addressed
● Not aligned with the general 
M&E system
● Challenges with human 
resources
● Dual responsibilities of staff 
(COVID-19 and regular project 
work)
Hybrid monitoring 
of TBCC using 
regular scheduled 
phone calls and 
onsite 
monitoring/mento
ring visits
TB Platforms 2 ● Well resourced (staff, money, 
equipment) 
● Well integrated within the 
general M&E system
● Communications allowance 
provided
● Guide on filling out the forms 
(patient contact form and 
summary log sheet) available 
● Monitoring tool available for the 
facility visits
● Follow-up calls made if late 
submission
● Issues with the form
● Limited availability of facility staff 
for monitoring
● No monitoring tool used for the 
phone calls
● SMS/FB Messenger calls not 
logged
Hybrid monitoring 
and mentoring for 
FAST facilities
TB Platforms 1 ● Well resourced (money, 
equipment) at national level
● Well integrated within the 
general M&E system
● Multiple modes of data collection
● Communications allowance 
provided
● Clear instructions for 
● Limited human resources at field 
and facility level
● No guidelines for remote 
monitoring
● COVID-19 workload of point 
persons 
● Difficulty in data validation 
remotely
69
accomplishing the tools
● Virtual meetings to validate data
● Difficulty in scheduling FTF visits
70
ANNEX 7: IMPLEMENTATION FIDELITY OF EACH 
IDENTIFIED PRACTICE IN THE SELECTED USAID IPs
IP Remote and Hybrid M&E Practice Associated 
with a plan
Progress
ReachHealth DHIS 2 ✓ Delayed
Remote Phone Monitoring ✓ On Track
Hybrid Rapid Feedback Mechanism ✓ Delayed
BARMMHealth
 Hybrid monitoring of FP program through site 
visits, phone calls, and FB messenger x On Track
Blended P&R Sessions ✓ On Track
KoboToolBox ✓ On Track
MTaPS Remote access to information systems (DOH￾PMIS and ITIS) ✓ On Track
COVID-19 Special Response Project- hybrid 
monitoring of facilities (onsite and virtual) ✓ Delayed
ProtectHealth CiTEC project regular online conference calls ✓ On Track
RenewHealth CBDR IMS ✓ Delayed
TB Platforms
 
Hybrid monitoring of TBCC using regular 
scheduled phone calls and onsite 
monitoring/mentoring visits
✓ On Track
Hybrid monitoring/mentoring for FAST 
implementing facilities ✓ On Track
Assessment of IPC using Google Survey ✓ Aborted
Remote monitoring and mentoring of PMDT ✓ Delayed
71
TB IHSS Remote P&R sessions ✓ On Track
Remote rapid assessment survey ✓ On Track
72
ANNEX 8: CONSOLIDATED EVALUATION FRAMEWORK OF THE 
IDENTIFIED REMOTE AND/OR HYBRID M&E PRACTICES
Remote 
and/or 
Hybrid M&E 
Practice
M&E Assessment IR Assessment Weighted Evaluation
I. ReachHealth
DHIS 2 DHIS 2 has adequate financial and human resources in 
place, and it is aligned with the general M&E system. DHIS 
2 has improved data management and expedited certain 
processes in ReachHealth. It has generated good quality 
data for decision making, but not for a significant amount 
of time to reliably conclude effectiveness.
Fidelity: Had delayed implementation which led to 
sequential delays in training, refitting, adoption to 
revised contexts, unclear status of establishment of 
validation rules, dashboard development, migration of 
databases (compounding the effect of delayed 
establishment). DHIS 2 has good feasibility, deemed a 
good system with good organizational championship.
Issues that can become major considerations included 
implementer-level training limitations, radicalness 
(technology mismatch). There is promising 
sustainability, based on good fidelity in terms of general 
adherence to the design of implementation despite 
delays and good feasibility.
Level 2a
DHIS 2 being implemented within 
ReachHealth with adequate 
resources; processes in place for 
analysis and reporting but aligned 
with the general M&E system; 
generated evidence of good 
quality has been used for decision
making but not consistently; good 
fidelity, with adaptation and 
feasible to scale (acceptable); 
some gaps and challenges 
identified on the domains 
observed.
Remote Phone 
Monitoring 
The collection of data using remote phone monitoring is 
well resourced as evidenced by available data collectors 
(project staff) and materials (phone with allocated 
communications allowance). Serving as an alternative to 
in-person facility monitoring, it is integrated well within 
the general M&E system: follows the same data flow, using 
There is good fidelity because adaptation ensures the 
implementation of quarterly monitoring data collection 
despite restrictions under the pandemic. Further 
adaptations were made to ensure regular monitoring 
(i.e., FTF options in case remote is not possible). 
Starting Q2 FY 2020, use of remote phone monitoring 
Level 1 
ReachHealth is implementing 
remote phone monitoring with
adequate resources; processes in 
place for analysis and reporting 
73
the same standard facility monitoring tool, stored and 
managed in the same DHIS 2 database. The data generated 
are analyzed, visualized, and are used by the team 
particularly in reporting to USAID and during reviews with 
partners.
However, there were a few challenges identified, 
particularly on staff acceptability and data quality. Field 
staff are not comfortable making long phone calls with 
high-level officials by asking them many questions, which 
could be addressed by shortening the tool. Also, the 
quality of data collected through remote phone 
monitoring has been compromised in a few indicators, 
specifically the FP stockout rate, where prior to COVID￾19, actual inspection of inventory was the only acceptable 
practice for monitoring. When constraints of COVID-19 
are removed, FTF data collection rather than phone 
interview is preferred by both project and field 
counterpart staff.
We are suspending judgment on the effectiveness of the 
remote approach in comparison to the FTF approach for 
data collection. Despite specific gains in terms of coverage 
(1,250 public health facilities in FY 2020 vs. random 
sampling via FTF prior to Y2Q2) and cost-effectiveness, 
the remote approach may have less reliability in tracking 
some indicators (e.g., stockout rates) as well as issues with 
user acceptability. Unlike for indicators, the standard for 
monitoring stockout rates requires actual onsite 
inspection and, according to a key informant, there are 
studies showing a decline in reliability when remote 
approaches are used. Moreover, there is limited 
information in assessing the effectiveness of the approach 
because some decline in information gathered can be 
supported by the challenges faced during COVID-19 (i.e.,
non-responses due to busy schedules of the partners), not 
necessarily correlated to the ineffectiveness of the phone 
was also able to expand the coverage of quarterly 
monitoring to almost all sites compared to facility visits 
from randomly sampled sites done in previous quarters. 
There may be limited feasibility despite no major issues 
on mandates, budgets, and training specific to 
implementation of phone monitoring. Project staff and 
field counterpart staff both expressed preference for 
FTF data collection. Despite wider coverage in phone 
monitoring, there are some issues in ensuring data 
quality (i.e., reliability of stockout rate). Participants also 
identified several relative advantages of field visits over 
data collections: rapport, immediate TA, reliability, 
seamlessness, and appropriateness (especially with high￾level interviewees). The main factors that limit feasibility 
at the implementer level may be a result of the users' 
lack of familiarity with the remote approaches versus 
FTF approaches. 
With good fidelity and limited feasibility, the practice 
may not be sustainable as regularly maintained activity 
due to the lack of support from project and field 
counterpart staff alike. Reliance on this mode of data 
collection also depends on the situation of the 
pandemic. However, this may still serve as an important 
alternative measure to data collection that 
complements FTF approaches to ensure and enhance 
monitoring in certain cases.
It is worth noting that while the respondents for this 
practice are central office staff members, they were 
able to highlight field implementer-level challenges for 
remote phone monitoring. 
but run aligned with the general 
M&E system; generated evidence 
of good quality has been used for 
decision making but not 
consistently; good fidelity, with 
adaptation, limited feasibility 
(acceptability) on the side of 
ReachHealth; some gaps and 
challenges identified on the 
domains observed.
74
monitoring.
Overall, this can be considered as an efficient alternative in 
cases where in-person facility monitoring visits are not 
possible. This serves to complement FTF data collection 
rather than replace it. 
Hybrid Rapid 
Feedback 
Mechanism 
The hybrid rapid feedback mechanism has adequate 
financial and trained human resources for its 
implementation. Its data collection, analysis, and reporting 
processes are aligned with the general M&E system. 
Implementation of the mechanism has allowed for 
immediate feedback to partners in some facilities, but it is 
challenged by some data quality issues due to its remote 
approach for data collection, and the lack of human 
resources and time (during the pandemic) on the side of 
field partners. It has generated good quality data for 
decision making, but not for a significant amount of time 
to reliably conclude effectiveness. Major aspects of the 
mechanism are still currently being finalized.
Despite delayed implementation, there may be good 
fidelity because adaptations improved data encoding and 
management and due to the data collection tool's ease 
of use and accountability systems for encoding; there 
are only minor deviations due to external factors (i.e., 
unavailability of field partners during feedback). The 
range of adaptations presented has increasing 
integration and acceptance within the system but needs 
further solidification into the common practice 
Considering only USAID ReachHealth's internal 
capacity to implement the rapid feedback mechanism, 
there may be limited to moderate feasibility. Despite 
adequate mandates, budget, and training in place, the 
practice may not be feasible to scale due to 
unresolvable issues related to fulfilling mandates on the 
side of the field partners. Challenges in accomplishing 
rapid data collection and providing feedback at the 
required levels (i.e., lack of field partners' time and 
human resources during the pandemic) are beyond the 
control of USAID ReachHealth. With good fidelity 
(based on general adherence to design of the practice) 
and limited to moderate feasibility (the organization's 
internal capacity to implement the practice), there may 
be promising sustainability for this developing practice, 
especially because it is driven by the recommendation 
of USAID.
Level 2a
Hybrid rapid feedback 
mechanism is being implemented 
within ReachHealth with 
adequate resources; processes in 
place for analysis and reporting 
but aligned with the general M&E 
system; good quality evidence has 
been used for decision making 
but not consistently; good 
fidelity, with adaptation and 
limited to moderate feasibility to 
scale (acceptable) on the side of 
ReachHealth; some gaps and 
challenges identified on the 
domains observed.
II. BARMMHealth
Hybrid Data collection and validation using the combination of There is good fidelity as they continued using the Level 2a
75
monitoring of 
performance 
through site 
visits and 
validation 
through phone 
call and FB 
Messenger
hybrid monitoring approaches (site visits, phone calls, and 
FB messenger) has adequate resources as evidenced by 
available human resources and communication allowance 
at the provincial level. BARMMHealth also provides 
support to the implementation through transportation for 
the PHO staff and allowances (transportation and 
representation) for participants of the monitoring 
activities at the RHU level. However, there may be a need 
to provide support for human resources and 
communication allowances at the RHU level in some 
areas. The hybrid monitoring approaches have been 
employed even before the pandemic and are well 
integrated within the general M&E system of the BARMM 
PHOs. With several data validation measures in place, data 
generated are of good quality. Despite specific challenges 
identified for each mode of data collection/validation, 
hybrid monitoring is seen as more effective than purely 
onsite or remote monitoring, as it allows adaptation to 
the geographical context, access to decent Internet and 
mobile connectivity, and the constantly changing situation 
of the pandemic. Data collected through hybrid 
monitoring approaches have been consistently used for 
monitoring the activities of the FP program. Overall, this 
can be considered as a good practice as it has ensured the 
continuity of monitoring despite the various constraints 
brought about by the pandemic. Implementation of this 
hybrid approach to monitoring is envisioned to continue 
during and even beyond the pandemic.
combination of hybrid monitoring approaches, adapting 
in terms of frequency of mode used, to ensure 
quarterly monitoring activities under the pandemic. 
There may be moderate feasibility because there are no 
major issues on mandates, budgets, and training specific 
to implementation of hybrid monitoring approaches. 
No individual challenges were presented by the 
participants. However, participants at the regional level 
identified challenges in coordination and collaboration 
for activities in partnership with BARMMHealth. Given 
the unique set up of BARMM Region’s health system, 
organizational level support for monitoring activities 
from the regional level is as important as implementer 
level support from the provincial level. In addition, 
some organizational challenges in executing each 
specific mode of data collection may be difficult to 
address because they are tied to wider geographical 
(weather, island areas) and digital limitations (Internet, 
mobile connectivity) across the BARMM PHOs. Despite 
good fidelity and moderate feasibility, the hybrid 
approach to monitoring in these areas may have limited 
sustainability. Taking into consideration that support 
from BARMMHealth is temporary (the project will end 
by 2024), sustainability will be ensured if constant 
coordination, collaboration, and support from MOH￾BARMM is solidified before the project ends.
The hybrid approach to data 
collection and validation is being 
implemented within the BARMM 
PHOs with adequate resources; 
good quality evidence has been 
consistently used for decision
making; good fidelity, with 
adaptation and feasible to scale; 
some gaps and challenges 
identified on the observed 
domains.
Blended P&R 
Sessions
Blended P&R has adequate financial and human resources 
in place, and it is aligned with the general M&E system. 
Blended P&R has contributed to the M&E goals of the 
project as a venue to strategize and learn from well￾performing areas. However, this approach still has limited 
reach because some areas cannot participate in online 
sessions, and it is not guaranteed that BARMMHealth staff 
can conduct in-person P&R due to travel restrictions. The 
blended approach was able to inform project activities but 
It is difficult to assess fidelity of blended P&R because 
there is no tangible protocol to reliably account for its 
performance. Blended P&R also has limited to
moderate feasibility; the hybrid method cannot ensure 
that if participants cannot join online meetings, in￾person sessions can be conducted, and vice versa. No 
individual challenges were stated by the participants. 
With good fidelity in terms of adherence to the Activity 
Monitoring, Evaluation, and Learning Plan (AMELP) and 
Level 2a
Blended P&R is being 
implemented within 
BARMMHealth with adequate 
resources; good quality evidence 
has been consistently used for 
decision making; well-aligned with 
the general M&E system; good 
76
was not able to collect inputs from all involved partners, 
thus having limited effectiveness. 
limited to moderate feasibility, blended P&R may not be 
sustained as a regular activity due to its limited reach. 
However, it may be useful as an alternative measure 
while in-person P&R is not yet possible. 
Decisions are not incumbent upon P&R sessions; thus 
there is no urgency to conduct them. To address this, 
organizational incentives must be clarified to ensure the 
regular conduct of P&R sessions for the improvement 
of M&E processes. 
fidelity, with adaptation and 
limited to moderate feasibility; 
some gaps and challenges 
identified on the observed 
domains.
KoboToolbox KoboToolBox has adequate financial and human resources 
in place, and it is aligned with the general M&E system. 
KoboToolBox has contributed to the M&E goals of the 
project by accelerating data collection, reporting, and 
generation/analysis. It has consistently generated good 
quality data for decision making. KoboToolBox also has 
good effectiveness, having successfully informed project 
decisions such as service delivery as well as identifying
areas/themes for improvement in reporting RHUs.
KoboToolBox has good fidelity: BARMMHealth was 
able to establish a reporting system for use while the 
planned PMIS has not yet been developed. A further 
adaptation includes the conversion of paper-based data 
collection tools into an electronic format to accelerate 
data encoding and validation. KoboToolBox also has 
good feasibility, being deemed a good system for data 
consolidation and analysis at the central office and 
implementer levels. Individual challenges were also well 
addressed, while some are beyond the control of the 
project. KoboToolBox also demonstrated adaptability 
as it can be modified according to project needs, and 
trialability wherein BARMMHealth can easily revert 
back to the original data consolidation process. With 
good fidelity in terms of adherence to the AMELP and 
good feasibility, sustainability is also promising, as no 
major issues were identified for environmental support, 
funding stability, and organizational capacity to 
implement the practice in the long-term, until such time 
that the intended PMIS has been developed for use.
Level 2b
KoboToolBox is being 
implemented within 
BARMMHealth with adequate 
resources; good quality evidence 
has been consistently used for 
decision making; well-aligned with 
the general M&E system; good 
fidelity, with adaptation and 
feasible to scale; some gaps and 
challenges identified on the 
observed domains.
III. MTaPS
Remote access 
to information 
systems (DOH￾Remote access to DOH-PMIS and ITIS is free. MTaPS has 
adequate financial and human resources in place; however, 
this is a challenge on the side of DOH PD, NTP, and 
There is good fidelity, as the adaptation facilitated 
better M&E of one of its major indicators. Despite little 
to no issues on the side of MTaPS in implementing the 
Level 1
Practice is already being 
77
PMIS and ITIS) KMITS for the day-to-day operations of the information 
systems. DOH PD lacks funding support for manpower, 
software, and plug-ins, and is heavily reliant on DOH 
KMITS for its infrastructure needs. DOH KMITS is heavily 
dependent on Global Fund for Internet connection, 
servers, equipment, Apple and Google subscriptions, and 
human resources to operate ITIS. At the facility level, 
issues with human resources, equipment, and Internet
connectivity pose challenges to encoding data into DOH￾PMIS and ITIS. 
Remote access to DOH-PMIS and ITIS is very well￾integrated with the general M&E system of MTaPS, and it 
has improved data analysis and use due to ease of and 
fuller access to the data. However, there were some 
issues with the quality of data in DOH-PMIS and ITIS.
MTaPS is working with NTP to improve the reporting rate 
of its TB Directly Observed Treatment, Short-course 
(DOTS)-providing facilities to ITIS and to further 
strengthen NTP's management and analysis of inventory 
data from ITIS. MTaPS is also working with the DOH’s PD 
to further strengthen their recently upgraded DOH-PMIS. 
Both information systems will improve the reporting and 
data challenges for TB medical products.
remote access to the information systems, there is 
poor feasibility on the side of partners who maintain 
these systems (DOH PD, NTP, and KMITS). There are 
major issues in human resources, inadequacy of funding 
and equipment, server problems, Internet connectivity, 
and data quality. Sustainability of operating these 
information systems is not ensured due to major 
challenges faced by partners in terms of environmental 
support, funding stability, and organizational capacity.
However, MTaPS provides assistance to DOH such as 
quarterly meetings to review quality of DOH-PMIS 
stock data, and conducting analysis and providing 
recommendations on PD's consumption and stock-on￾hand reports and on DOH-PMIS' and ITIS' stock and 
consumption data of TB medicines. Likewise, MTaPS is 
working with NTP to improve reporting rate of its TB 
DOTS-providing facilities to ITIS and to further 
strengthen NTP's management and analysis of inventory 
data from ITIS. MTaPS is also working with the DOH’s 
PD to further strengthen their recently upgraded 
DOH-PMIS. Both information systems will improve 
reporting and data challenges for TB medical products.
implemented within MTaPS with 
adequate resources, but with 
very limited resources on the 
side of DOH to support 
continued use. Collection, 
analysis, and reporting of data is 
well integrated within the general 
M&E system; with evidence of 
poor data quality; evidence on 
data use for decision making; 
good fidelity, but with 
considerable gaps and challenges 
identified on the domains 
observed. 
COVID-19 
Special Response 
Project - hybrid 
monitoring of 
facilities (onsite 
and virtual)
This COVID-19 project of MTaPS is a special activity, with 
a big effort around COVID-19 was in Feb/March 2020, and 
a significant level of government intention to expand 
absorptive capacity of facilities, improve IPC, HCWM, and 
SCM implementation, and mitigate COVID-19 impacts. 
The monitoring effort was built as part of the adaptive 
goals to respond to the pandemic.
There are adequate financial resources and equipment to 
conduct the onsite and virtual facility assessments as well 
as the online training. Some challenges with human 
resources included: 1) resignation of one national 
consultant, 2) delays in hiring of additional field 
Despite some delays in the implementation of their 
activities due to various reasons (e.g. dynamic COVID￾19 pandemic, difficulties creating the materials, and 
rescheduling of facilities for visits), there is good fidelity, 
as adaptations were made and activities indicated in the 
work plan were implemented by the end of the project. 
It also has good feasibility because there is 
transferability (manual of operations/SOPs), trialability, 
access to knowledge and information, financial 
resources, external policies, and leadership engagement, 
with some issues on human resource and use of 
technology (WebEx) on MTaPS’ side. There was high 
support and cooperation from DOH due to the 
Level 2b
Onsite and virtual facility visits 
were implemented by MTaPS 
with adequate resources; data 
collection, storage, and reporting 
do not align with the general 
M&E system, but align with an 
emergent adaptive goal which is 
COVID-19 mitigation. The 
indicators that the special 
COVID-19 response project is 
working on relate to an indicator 
78
consultants, 3) COVID-19 infection in the team, and 4) 
dual responsibilities of MTaPS staff with the COVID 
project and regular MTaPS work. Despite these issues, the 
right skills and previous experiences of the team aided in 
the successful implementation of the activities. 
Its data collection, storage, and reporting generally do not 
align with the general M&E system, but align with an 
emergent adaptive goal of COVID-19 mitigation. The 
indicators that the Special COVID-19 response project is 
working on relate to an HP indicator (number of 
institutions with improved capacity on HCW and 
emergency SCM to assess or address disaster and climate 
change risks supported by USG assistance), but it is not 
really similar. Thus, it is difficult to assess whether the 
project was effective in terms of its original intentions. 
Some significant overlaps with the general M&E system 
included use of Microsoft Excel for data analysis, use of 
WebEx as their main video conferencing platform, and 
conduct of P&R sessions. Moreover, this COVID-19 
project has a component on HCWM and SCM which 
supports MTaPS' objective in building health care facilities' 
resilience on climate risk mitigation.
There were a few data quality issues identified, but these 
were mostly addressed or mitigated. Data gathered from 
the monitoring activities were immediately shared with
the facilities. Likewise, a list of recommendations/areas for 
improvement in the three areas was shared with DOH, 
and data generated will be used by MTaPS and DOH to 
develop capacity building materials for ToT and guidelines 
that can address gaps at the facility level. 
Overall, the COVID-19 response project demonstrated 
successful onsite and virtual facility visits under a dynamic 
pandemic situation, with adaptations made along the 
course of the project. While this monitoring activity was 
concerted effort to prioritize COVID-19, which aligns 
with the intentions of the project. 
There is no need to ascertain whether this activity is 
sustainable because by intention, it was a short-lived 
project with a specific purpose. Whether this should be 
continued will depend upon the COVID-19 situation. 
But in particular, one critical demonstration that would 
have been useful was the virtual facility tours, which 
were unique to this project. Virtual facility tours are 
complex and are difficult to implement, and few 
facilities were assessed this way. This experience has 
shown that virtual tours do have certain requirements 
such as Internet connectivity, high quality videos, and 
the ability to capture the right angles and visualization 
of the particular assessment domain. To date, there is 
insufficient information with this particular effort to 
demonstrate that the virtual facility tours were effective 
and if this is something worth strengthening or 
investigating.
of the HP but are not particularly 
similar. However, there were 
some significant overlaps with the 
general M&E system. Good 
quality evidence has been used 
for decision making; good fidelity, 
with adaptation and feasible to 
scale (acceptable and 
appropriate); some gaps and 
challenges identified on the 
domains observed. Sustainability 
cannot be ascertained because 
this was a short-lived project 
with a specific purpose.
79
just added to USAID activities, it does resemble the 
framework of many COVID-19 emergency response 
efforts which begin with status assessment, immediate 
gaps response, and building capacities. Thus, it can also be 
interpreted based on its effectiveness of 
averting/controlling COVID-19, but this was not 
manifested as part of the indicators.
IV. ProtectHealth
CiTEC Project 
Regular Online 
Conference Calls
The regular conference calls between ProtectHealth and 
TB IHSS are well resourced as evidenced by available 
human (staff time) and financial (communication 
allowance) resources. The general M&E monitors three
indicators related to the CiTEC project: 1) proportion of 
the resource requirements for the CiTEC activities in 
Mambaling and Pasil funded by commitment of partners, 2) 
proportion of TB cases who availed of PhilHealth-provided 
TB-benefits (Intensive Phase), and 3) proportion of 
notified TB cases who availed of PhilHealth TB benefits 
(Intensive Phase). However, the M&E system for the 
CiTEC implementation project is separate from the 
general M&E system of ProtectHealth. The regular CiTEC 
meetings (through conference calls) are well integrated 
within the M&E system for the CiTEC project because it is 
part of five main activities (under Data Analysis and 
Reporting) involved in the database management of the 
CiTEC project as written in the “Protocols Monitoring the 
Progress of Implementation of CiTEC.” The data 
presented during these meetings are used for tracking 
progress of implementation and decision making and are 
documented as part of an end of project/demonstration 
study report. No major challenges with data reporting, 
use, and quality related to the conference calls were 
reported. Overall, the practice is effective in ensuring 
coordination given the unique set-up of the joint project, 
with participants from two different IPs and across 
There is good fidelity, with a range of adaptations made 
to the meetings (whether in mode, scope, or regularity) 
to ensure adequate and efficient coordination between 
the partner projects. The practice's inner characteristic 
of trialability allowed for the reversal of certain 
adaptations (i.e., frequency) to ensure the continuing 
achievement of the project's needs and goals. Although 
there is a decrease in call frequency, the practice itself 
is increasingly being adopted and improved to maximize 
efficiency. There is also good feasibility, with no major 
issues on mandates, budgets, and training. The 
conference calls are well-scheduled (calendar-blocked) 
yet flexible, and are conducted on a reliable platform, 
Microsoft Teams. Participants did not have any 
problems using the platforms despite the lack of 
orientation for these conference calls. This may be 
attributed to their basic technology skills or familiarity 
with modern communication technologies. Participants 
were able to balance the relative advantages between 
remote and FTF modes of meeting by having nuanced 
expectations of the purpose of these conference calls, 
viewing them as venues for coordination and data 
sharing between units from distinct projects and varying 
levels, as opposed to other activities that would be 
benefit more in a FTF set-up (i.e., data collection, first￾time engagement with potential partners). The relative 
advantages of these scheduled conference calls are the 
Level 3 
Practice can be considered as a 
good practice; already showing 
high penetration and is 
sustainable; fewer gaps and 
challenges identified on the 
domains observed; able to 
improve M&E performance. 
There are processes in place for 
analysis and reporting; and high 
quality evidence has been 
consistently used for the teamʼs 
decision making and 
improvement of project 
performance.
80
geographically separate units (central and field level). The 
M&E practice's effectiveness is also evidenced by its 
capacity to adapt (i.e., in terms of frequency) to continue 
facilitating timely informed decisions/actions on activities 
in the achievement of set outcomes of the HP and IP 
(both ProtectHealth and TB IHSS).
following: time-saving, logistical convenience, ease of set
up, and commitment of higher-ups. The only downside 
is the lack of regular representation allowances for food 
during meetings. These are only allowed under specific 
circumstances such as events involving external 
stakeholders or for meetings longer than two hours. 
With no major issues in environmental support, funding 
stability, and organizational capacity, the practice has 
promising sustainability. While there are no concrete 
plans yet for changes when the project reverts to an 
office setup, these online conference calls between the 
two projects and across central and field staff may 
continue and be adapted until the end of the CiTEC 
project.
V. RenewHealth
CBDR IMS The CBDR IMS is well resourced as evidenced by available 
human resources and budget to maintain and further 
revise the system. RenewHealth staff also have adequate 
equipment to use the system, which only requires a 
computer/mobile device with access to Internet
connection. Data collection, analysis, and reporting 
processes are aligned with the general M&E system. Once 
operational, the CBDR IMS will expedite data collection 
and analysis for RenewHealth M&E staff by removing the 
burden of data encoding of LGU forms and automating the 
process of summarizing and generating disaggregated data 
for the project’s performance monitoring. The CBDR IMS 
is intended to address gaps in the current data collection, 
management, and sharing of CBDR information from 
LGUs to government agencies. LGUs who will be using 
these have staff who are familiar with information systems. 
They will also receive equipment (e.g., devices and 
Internet connection). Orientation and a manual for the 
CBDR IMS will be provided. Partner agencies will also 
have access to disaggregated data generated by the IMS. 
The development, implementation, and capacity-building 
activities for the CBDR IMS are delayed. Despite having 
adequate budget, the mandates for implementation and 
training are still being resolved and are dependent on 
factors beyond the control of RenewHealth (i.e., 
technical working group matrix of essential data 
components influencing system design and orientation 
materials as well as approval of USAID and partner 
government agencies). However, the practice is still in 
its pre-implementation phase and there may still be 
good fidelity, as further adaptations were already made 
and will still be made to the system to address the 
needs of the LGU and government agency partners, 
which is the main intention behind the IMS. There may 
be limited to moderate feasibility in implementing the 
CBDR IMS, as foreseeable issues in important domains 
have mitigation measures in place. Once CBDR IMS is 
approved for implementation, RenewHealth has 
adequate internal mandates and budget to 
operationalize the system. There are enough human 
Level 0 
Practice is still in pre￾implementation stage but has 
adequate resources allocated for 
it; processes in place for analysis 
and reporting will be aligned with 
the general M&E system; will 
have capacity to generate 
evidence of good quality for 
decision making; may have good 
fidelity, with adaptation and may 
be feasible to scale (acceptable 
and appropriate); some 
foreseeable gaps and challenges 
identified on the domains 
observed. 
81
As an early stage practice, CBDR IMS is challenged by 
planning hurdles. Setting up the system is hindered by 
theoretical bottlenecks such as the need to finalize 
performance measures for CBDR services. At this stage, 
the system is still under the review of USAID and partner 
government agencies, which will mean further refinements 
to the latest version of the system. Having yet to graduate 
into the stage of implementation, the effectiveness of the 
IMS in aiding RenewHealth’s M&E as well as in attending to 
the LGU and partner government agencies’ needs cannot 
be fully assessed at this point. The system has not yet 
generated high quality evidence which has been 
consistently used for the teamʼs decision making and 
improvement of project performance. However, as a 
comprehensive yet complementary information system 
envisioned to manage data across the cascade of CBDR 
services, there is great potential to accelerate processing 
and improve data quality of CBDR information on the side 
of RenewHealth, LGUs, and partner government agencies.
resources to maintain the system. Additional 
consultants can be hired to support the one staff 
member who can provide system orientation to users, 
and online materials will be made available in the CBDR 
Resources domain of the CBDR portal. LGU users are
willing to implement the IMS. Technical and 
technological barriers in using the system will be 
mitigated by provision of FTF and online orientation, 
manual for the IMS, and necessary equipment (e.g., 
devices and Internet connection). Despite potentially 
good fidelity and limited to moderate feasibility in 
implementing the CBDR IMS during the life of the 
project, there may be a challenge in sustaining the 
system beyond the project given issues in the three 
major domains (environmental support, funding 
stability, and organizational capacity). The system will 
have limited sustainability if key partner agencies do not 
obtain increased support for the IMS, and if no agency is
identified as the recipient of the system by the end of 
the project. Funding stability may become a problem 
once the system is turned over and will definitely 
become a problem if it fails to be turned over. 
Sustained use in LGUs will also be challenged by 
constant staff turnover.
VI. TB Platforms
Hybrid 
monitoring of 
TBCC using 
regularly
scheduled phone 
calls and onsite 
monitoring/ 
mentoring visits
Hybrid monitoring of TBCC using phone calls and facility 
visits is well-resourced as evidenced by available staff and 
materials (mobile unit, SIM cards, and communication 
allowances) on the side of TB Platforms and the 
counterpart facilities. It is also well-integrated within the 
general M&E system of TB Platforms. 
This can be considered as an effective M&E practice 
because it is simple enough while at the same time, 
monitoring activities generate the necessary information 
Fidelity is high, as all activities are on track, with several 
adaptations made: provision of mobile unit and use of 
FB messenger (instead of phone calls) to improve 
implementation of TBCC, adjusted frequency of phone 
calls based on the needs of the facilities, revision of 
forms, and creation of a data analysis tool to facilitate 
monitoring of TBCC. 
Feasibility is good with minor issues with: 1) the lack of 
mandate/policy on the implementation and monitoring 
Level 2b
Practice is being implemented 
with adequate resources; 
processes in place for analysis 
and reporting but aligned with 
the general M&E system; good 
quality evidence has been used 
for decision making; high fidelity, 
with adaptation and feasible to 
82
to identify challenges and improve implementation of 
TBCC. Data gathered from the monitoring activities were 
also used to advocate for the adoption of TBCC to other 
facilities/LGUs. However, there were a few challenges 
identified, particularly the availability of staff in facilities for 
monitoring (during the pandemic), and some issues with 
the forms. Respondents prefer the current hybrid set up 
of monitoring TBCC due to the current COVID-19 
pandemic. Overall, hybrid monitoring using phone calls 
and facility visits is a good practice, having ensured 
monitoring of TBCC during the pandemic generating good 
quality data for decision making. Type of information 
gathered fits well with the platform being used. However, 
small improvements and changes are needed.
of TBCC, 2) human resource strains of the counterpart 
facilities (dual responsibilities, quarantine) causing delays 
in submission and validation of reports and unavailability 
for monitoring, and 3) with the forms/monitoring tool 
(delayed provision of forms, confusion, and non￾inclusion of transactions which used FB messenger, lack 
of monitoring forms for the scheduled phone calls). 
Another contributor to good feasibility is baseline 
acceptability of the TBCC practice. Before facilities 
were engaged, sites were first identified based on their 
status of TB contact screening and investigation, 
receptivity to the practice, support of LGU, and 
availability of manpower and logistics. Online meetings 
were held to advocate the establishment of TBCC, and 
facilities were informed beforehand of the required 
monitoring and completion of forms. Furthermore, 
implementation of TBCC maximized existing structures 
that allowed for easy uptake, contributing to 
organizational feasibility. There is no overreaching of TB 
Platforms due to respect on control with the 
management level. There is seamless integration in how 
the RHUs work, with TB Platforms supporting what the 
RHU needs.
Sustainability is also good, with no major issues on 
environmental support, funding stability, or 
organizational capacity. Monitoring of TBCC will 
continue so long as the TBCC is implemented. 
Sustainability of the implementation of TBCC at the 
local level should continue as long as there are partners 
willing to carry it out.
scale (acceptable and 
appropriate); some gaps and 
challenges identified on the 
domains observed.
Hybrid 
monitoring/ment
oring for FAST 
implementing 
facilities
The hybrid monitoring and mentoring of FAST 
implementing facilities has available resources in terms of 
budget, equipment and orientation. However, there is a 
lack of adequate time and human resources on both TB 
Platforms field project and facility counterpart sides to 
There is good fidelity: adaptations were made to ensure 
the continuity of the monitoring and mentoring 
activities in FAST implementing facilities throughout the 
pandemic. The adaptability and trialability of the 
practice is shown in the shift from depending on 
Level 1 
Practices being implemented 
within an activity with adequate 
resources; processes in place for 
83
adequately implement this. The remote approaches to 
monitoring and mentoring are integrated within the 
general M&E system, following the same data flow and 
monitoring tool used in onsite monitoring/mentoring 
visits. The team analyzes, visualizes, and uses data from 
these monthly data collections in much the same way as 
before. Data are useful for feedback and mentoring to 
improve implementation in FAST facilities. However, a few 
challenges were identified with both remote and onsite 
monitoring approaches during the pandemic. While 
remote monitoring approaches allow for data collection 
during some periods in pandemic, TB platforms project 
staff had difficulty not only in obtaining all data needed 
remotely, but also in validating data that were provided to 
them. Scheduling of onsite remote monitoring was also 
more difficult due to changing circumstances related to 
COVID-19. Across both approaches, FAST point persons 
had difficulties facilitating the monitoring due to added 
COVID-19 responsibilities. It is difficult to fully assess the 
effectiveness of a hybrid monitoring approach versus a 
purely FTF approach because several limitations in 
conducting the hybrid approach are attributed to the 
circumstances brought about by the pandemic. There are 
specific advantages in using remote approaches, such as 
ensuring the continuity of monitoring despite mobility 
restrictions. However, the remote approach may be less 
reliable given the challenges in validating data. The 
participants have expressed the relative advantage of an
FTF approach when it comes ensuring quality for process 
checks and data collection. Currently, they are doing more 
FTF than remote methods of monitoring. However, they 
are open to sustaining hybrid practices for monitoring, as 
remote approaches may not only be useful in 
contingencies but also in complementing onsite 
monitoring (i.e., online platforms used for additional 
qualitative validation).
remote approaches during the earlier parts of the 
pandemic to returning to FTF approaches once 
restrictions eased, and other adjustments to the 
pandemic have been made. There is limited feasibility in 
implementing the hybrid setup for monitoring and 
mentoring in FAST implementing facilities. Despite no 
major issues on mandates and budgets, implementation 
hurdles stem from the lack of human resources and 
capacity-building practices on both the provincial 
project and facility counterpart staff levels. The 
complexity of the tool and the process also contributes 
to limited uptake. Even prior to the pandemic, TB 
platforms staff would often fill out the data collection 
tools, which ideally should be accomplished by point 
persons in FAST implementing facilities. Given the 
added burden of COVID-19 responsibilities, facility 
counterpart staff find it even more difficult to remotely 
accomplish data collection tools themselves or provide 
all necessary data to the provincial project staff. This is 
the case even with additional support from short-term 
data collectors and encoders from TB platforms. Onsite 
monitoring is also dependent on the changing 
circumstances related to COVID-19, such as mobility 
restrictions or positive cases in the facility. Despite 
these limiting factors beyond the control of TB 
platforms, external mandates (i.e., the need to report 
to USAID or regional policies supporting FAST 
implementation) drive the continuous implementation 
of the monitoring activities. There is sustainable 
environmental support and funding stability for FAST 
implementation, especially in areas where TB platforms 
was able to establish regional and local policy support. 
However, there is limited sustainability for the practice 
of hybrid monitoring and mentoring in FAST facilities 
due to insufficient human resource capacity at TB 
Platforms and the partner facilities. This also impacts 
the quality of data generated, especially when remote 
analysis and reporting and aligned 
with the general M&E system; 
good quality evidence has been 
used for decision making but not 
consistently; high fidelity, with 
adaptation and feasible to scale 
(acceptable); some gaps and 
challenges identified on the 
domains observed.
84
monitoring approaches are employed. There may be 
potential sustainability when these challenges are 
addressed. The assessment is on the hybrid monitoring 
and mentoring approach for FAST facilities and not the 
implementation of the FAST strategy itself, which 
showed good uptake and promising sustainability. 
Respondents for this practice were primarily field 
project and counterpart staff, which may skew the 
analysis toward a focus on field-level implementer 
challenges. In addition, there is no objective evidence 
that data collected and validated through remote 
approaches were poorer than data collected from FTF 
approaches. The perception of lower data quality using 
remote approaches and the preference for FTF 
approaches must be understood in context. Factors 
such as familiarity with FTF approaches as well as 
perceived limitations and uneven expectations for 
remote approaches may have hindered adoption of 
remote approaches.
Assessment of 
IPC using Google 
Survey
There were adequate financial resources and equipment 
to implement the Google Survey. Despite adequacy of 
personnel, technical capacity of the field staff to perform 
analysis as well as to provide TA and feedback were a 
challenge. There were plans to hire an external consultant 
(engineer) to provide short-term TA to develop simple 
guidelines for IPC, but they have not found one yet. The 
use of Google Survey is well-integrated within the general 
M&E system of TB Platforms. However, several data 
quality issues were identified with the use of Google 
Survey forms: 1) need to clean the data due to numerous 
duplicates, 2) incomplete responses even for required
fields, 3) missing results even if respondents accomplished 
the survey, and 4) inability to further probe responses.
Results of the survey were fed back to the regions and to 
DOH NTP, but not to the majority of the facilities 
because of hesitancy among field staff to provide feedback 
Although adaptations were made along the way, there 
is low fidelity because some activities were delayed and 
aborted. Despite having an adequate budget, there is 
low feasibility due to major issues with human 
resources (gap in technical capacity of field staff and 
unavailability of an external consultant). Likewise, there 
are major issues with the quality of data collected using 
Google Surveys as well as with issues due to the 
preference of using other platforms. There is low 
sustainability given major issues in organizational 
capacity. Future assessments on IPC will use IPConnect 
instead.
Level 0
Practices already being 
implemented within an activity 
with adequate equipment and 
financial resources but with 
issues on human resources; it is 
well integrated within the general 
M&E system; with evidence of 
poor data quality; some evidence 
on data use for decision making; 
low fidelity, with some 
adaptations; considerable gaps 
and challenges identified on the 
domains observed.
85
due to a lack of technical expertise and negative 
experiences in previous feedback (i.e., facilities were 
defensive after scoring low). This then limited the use of 
the baseline findings at the facility level, but nonetheless 
were used by the regions in strengthening health worker 
surveillance programs and supportive supervision systems 
and by TB Platforms in developing a comprehensive TA 
package for IPC. Overall, it was difficult to assess the 
effectiveness of Google Surveys from the M&E standpoint 
due to implementation issues.
PMDT remote 
monitoring and 
mentoring
Remote monitoring/mentoring has adequate financial and 
human resources in place, and it is aligned with the general 
M&E system. Remote monitoring/mentoring has 
contributed to the M&E goals of PBSP in the sense that 
facilities were made aware of the directive to resume TB 
services during the pandemic. However, respondents 
expressed mistrust of the quality of the data. For the 
central office staff, there is a possibility of bias from the 
monitoring/mentoring participants because questions are 
sent to them beforehand. Central office staff and field 
counterpart staff agree that limited data validation 
undermines data quality. Because of these factors, 
effectiveness of remote/monitoring is difficult to assess, 
although respondents have indicated that the practice was 
helpful in improving the implementation of PMDT.
Despite delayed implementation of remote 
monitoring/mentoring in terms of the AMELP, there is 
still promising fidelity because adaptations (i.e., shift to 
remote approaches) were made to ensure continuity 
and improve monitoring (enhancements in remote 
monitoring tool) despite mobility restrictions during the 
pandemic. In addition, only poor-performing facilities 
will require constant follow up. Remote 
monitoring/mentoring has limited to moderate 
feasibility; there are significant individual challenges at 
both the central project and field counterpart staff 
levels, mostly regarding the perceived quality of data 
collected. While participants highlighted the relative 
advantage of in-person monitoring in ensuring data 
quality, they also acknowledged the benefits of remote 
approaches to monitoring: less preparation needed, less 
intimidating, and appropriate for smaller facilities that 
cannot accommodate facility visits with numerous staff 
members. Field staff are also having difficulty balancing 
remote monitoring/mentoring with other duties. With 
promising fidelity and limited to moderate feasibility, 
sustainability is questionable given the limited 
acceptability of purely remote monitoring/mentoring 
among central and field staff; however, this may be 
sustainable when used in tandem with FTF monitoring 
activities.
Level 2a
Remote monitoring/mentoring is 
being implemented with adequate 
resources; with limited evidence 
of good quality data used for 
decision making; well-aligned with 
the general M&E system; 
promising fidelity, with adaptation 
and limited to moderate 
feasibility; considerable gaps and 
challenges identified on the 
observed domains.
86
VII. TB IHSS
Remote P&R 
sessions
Remote P&R has adequate financial and human resources 
in place, and it is well integrated within the general M&E 
system. Remote P&R has contributed to the M&E goals of 
the project as a venue to strategize and learn from well￾performing areas. It has consistently generated good 
quality data for decision making.
Remote P&R has good fidelity: TB IHSS was able to 
ensure the continuity of conducting P&R sessions by 
adapting a previous hybrid activity into a fully remote. 
Remote P&R also has good feasibility: it demonstrates 
adaptability by addressing challenges with adaptive 
measures such as utilizing online tools for learning and 
engagement. Remote P&R also demonstrated trialability 
because data collection tools were piloted within the 
team and revised as needed. Remote P&R participants 
also indicated that the practice had impact. With good 
fidelity in terms of adherence to the AMELP and good 
feasibility, sustainability is also promising given no major 
issues in environmental support and funding capacity 
and only minor issues in organizational capacity. 
Currently, the SI team can conduct and process outputs 
from P&R; it will be beneficial if the regional teams are 
also capacitated to do so. Additionally, there are 
already plans of integrating this process in the toolkits 
of the innovations that will be turned over to the 
project's counterparts.
Level 3
Remote P&R is being 
implemented within TB IHSS with 
adequate resources; good quality 
evidence has been consistently 
used for decision making; well￾aligned with general M&E system; 
good fidelity, with adaptation and 
feasible to scale; some gaps and 
challenges identified on the 
observed domains that have been 
well addressed.
Remote rapid 
assessment 
survey
Remote rapid assessments were well-resourced as 
evidenced by available budget, staff (Chief of Party, 
technical teams, Operation Research Specialist, Regional 
Implementation Managers, and FIOs), and equipment 
(laptop and communication allowance). It is well￾integrated within the general M&E system of TB IHSS. A 
concept note was developed for the implementation of 
the three surveys, but there was no policy or mandate 
provided by DOH NTP. 
The team ensured that the questionnaire was short but 
comprehensive, and it was validated and pre-tested before 
deployment. Data were collected through online Google 
Survey Forms, paper-based hard copies, or soft copy 
There is high fidelity. All planned activities were 
implemented, and necessary adaptations were made 
such as: 1) having the option of responding to the 
survey with paper-based forms or through soft copy 
(Word file), in consideration of HCWs busy schedules
and Internet connectivity, 2) extending the data 
collection period to give HCWs more time to respond 
to the survey, 3) providing HCWs a communication 
allowance after the first survey, (4) conducting a follow 
up survey after six months to evaluate practice, and 5) 
expanding coverage to regions beyond the planned 
scope.
Feasibility is high with no major issues on budget, 
Level 3
Remote rapid assessment surveys 
can be considered as a good 
practice; already showing high 
penetration and is sustainable; 
fewer gaps and challenges 
identified on the domains 
observed; able to improve M&E 
performance. Collection of data 
is well resourced, and practice is 
integrated well within the general 
M&E system; processes in place 
for analysis and reporting; and 
87
Word Files, in consideration of HCWs’ busy schedules
and Internet connectivity. However, there were difficulties 
with obtaining responses from HCWs in the hospitals due 
to the COVID-19 response work. Responses were 
validated simultaneously with the data collection; any 
missing or conflicting information identified by the central 
staff was relayed to the FIOs for validation with the 
hospitals. Data were analyzed through STATA. There was 
no official plan for specific data analysis because analysis 
depended on partner and technical team needs. There 
were minor issues with data quality such as the low 
sample size, reliability of responses, and tight window 
period. 
Results of the assessments were presented to DOH NTP, 
regional NTP, and hospitals and helped to identify 
mitigation measures and recommendations to improve TB 
care during the pandemic. Results of the assessments 
were also used as one of the decision points for the NAP, 
while results of the third assessment helped in the 
implementation of FASTPlus. Overall, the remote rapid 
assessment is effective and has demonstrated the 
successful conduct of remote data collection under a 
dynamic COVID-19 situation, with adaptations made along 
the way. Experience from this activity paved the way for 
TB IHSS' subsequent survey development and data 
collection on the ground.
training, and mandates. Although DOH NTP did not 
provide memos on the conduct of all three surveys, 
high-level agreements between USAID and DOH as 
well as established partnership and rapport between TB 
IHSS and the hospitals aided in the rollout of the 
surveys. Other minor issues include: 1) difficulty of data 
collection during the pandemic because HCWs in the 
hospitals were busy with COVID response, 2) Internet
connectivity, 3) low response rate and reliability of data, 
and 4) tight timeline for development of questionnaire, 
orientation, pre-testing, data collection, and data 
analysis. Nonetheless, strong commitment and technical 
capacity of TB IHSS personnel, involvement of the 
upper to lower management, willingness of facilities to 
answer the survey, and provision of STATA and 
communication allowances by TB IHSS were enablers in 
the implementation of the remote rapid assessment 
surveys.
Sustainability is high, with no major issues in 
environmental support, funding stability, or 
organizational capacity on the side of USAID. The 
remote rapid assessments are already being scaled up, 
with interest of an external partner (DOH NTP) to 
adopt it. Though there are no plans to conduct follow 
up surveys, experience from this activity paved the way 
to develop subsequent surveys and data collection 
activities.
good quality data have been 
consistently used for TB IHSS' 
decision making and 
improvement of project 
performance.