CLAimHealth:
COLLABORATING, LEARNING, AND 
ADAPTING FOR IMPROVED HEALTH 
ACTIVITY
Evaluating the Effectiveness of After-Action Reviews for
Adaptive Management in Selected USAID/Philippines 
Health Project Interventions 
March 16, 2022
DISCLAIMER
The views expressed in this report do not necessarily reflect the views of the United
States Agency for International Development or of the United States government.
This report was produced for review by the United States Agency for International Development. It was prepared by 
Panagora Group for the USAID/Philippines Collaborating, Learning and Adapting for Improved Health (CLAimHealth) 
Activity, IDIQ No. AID-OAA-I-15-00025, Task Order No. 72049218F00001 as a technical deliverable per Task 
Order Section F.6 (b) and therefore uses USAID branding as outlined in the CLAimHealth Branding and Marking 
Plan. 
Recommended citation: USAID/Philippines Collaborating Learning and Adapting for Improved Health 
(CLAimHealth) Activity. Evaluating the Effectiveness of After-Action Reviews for Adaptive Management in Selected 
USAID/Philippines Health Project Interventions. March 2022.
Panagora Group Contact: 
Mary Ann Lansang, MD
Chief of Party
11/F, Ramon Magsaysay Center – Tower 
Manila1004, Philippines
ACKNOWLEDGMENTS 
Panagora Group and CLAimHealth gratefully acknowledge the contributions of the Planetary and Global 
Health Program of the St. Luke’s Medical Center College of Medicine – William H. Quasha Memorial
(SLMCCM – WHQM) in producing this report. The team is composed of Ramon Lorenzo Luis R. 
Guinto, MD DrPH (Principal Investigator), Rafael Jairah D. Matoy, Jr., MPH, RN (Research Fellow), and 
Michael B. Salise, MHSS, RND (Research Associate).
We also acknowledge the contributions of the following resource persons from the three implementing 
partners that participated in this implementation research:
ReachHealth
• Dr. Laurentiu Stan, Chief of Party
• Dr. Easter Dasmariñas, Deputy Chief of Party
• Dr. Ophelia Mendoza, MERLA Director
• Mr. Glenn Bede Benablo, Regional Manager
• Ms. Algin Gultia, Program Officer for Luzon, Social and Behavior Change & Gender
RenewHealth
• Dr. Regina Hechanova-Alampay, Chief of Party
• Dr. Ivanhoe Escartin, Deputy Chief of Party
• Ms. Aimee Malinga, Monitoring, Evaluation and Learning Director
• Dr. Jason Manaois, Learning and Development Specialist
TBIHSS
• Dr. Soliman Guirgis, Chief of Party
• Dr. Lalaine Mortera, Deputy Chief of Party
• Mr. Anup Kumar, Strategic Information Lead
• Ms. Maria Althea Sabrina Perez, Monitoring and Evaluation Specialist
• Dr. Bea Barbara Carrascal, Public-Private Mix Specialist
The SLMCCM – WHQM team also wishes to express their gratitude to Dr. Pilar Ramos-Jimenez and 
Ms. Justine Co of CLAimHealth for their generous collaboration.
TABLE OF CONTENTS
ACKNOWLEDGMENTS ............................................................................................................................................... iii
ACRONYMS....................................................................................................................................................................... ii
EXECUTIVE SUMMARY.................................................................................................................................................. 1
1. INTRODUCTION........................................................................................................................................................ 4
2. LEARNING QUESTIONS AND OBJECTIVES...................................................................................................... 5
2.1. Primary Learning Question........................................................................................................................... 5
2.2. Secondary Learning Questions..................................................................................................................... 5
2.3. Objectives ......................................................................................................................................................... 5
3. AFTER-ACTION REVIEWS: A BRIEF REVIEW..................................................................................................... 6
3.1. Forms of AAR........................................................................................................................................................ 6
3.2. The AAR Process.................................................................................................................................................. 7
3.3. Evaluating AARs..................................................................................................................................................... 8
3.4. Pause and Reflect Among Different USAID Implementing Partners........................................................ 8
4. METHODOLOGY......................................................................................................................................................10
4.1. Overall Research Design...................................................................................................................................10
4.2. Selection of Cases...............................................................................................................................................10
4.3. Data Collection....................................................................................................................................................10
4.4. Data Analysis........................................................................................................................................................13
4.5. Analytical Framework.........................................................................................................................................13
4.6. Limitations of the Study.....................................................................................................................................16
4.7. Ethical Considerations .......................................................................................................................................16
5. RESULTS .......................................................................................................................................................................17
5.1. Implementing Partner 1: Improved Health for Underserved Filipinos: Family Planning and 
Maternal and Neonatal Health Innovations and Capacity Building Platforms (ReachHealth)...................17
5.2. Specific Intervention 1: ReachHealth’s Family Planning (FP) Ayuda Express...................................19
5.3. Implementing Partner 2: RenewHealth - Expanding Access to Community-based Drug 
Rehabilitation................................................................................................................................................................25
5.4. Specific Intervention 2: RenewHealth’s Motivational Interviewing Training ...................................28
5.5. Implementing Partner 3: TB Innovations and Health Systems Strengthening (TBIHSS) Project 31
5.6. Specific Intervention 3: TBIHSS’s Philippine Private Sector Diagnostics Consortium (PPSDC) 35
6. DISCUSSION...............................................................................................................................................................39
6.1. The 7 P’s of AARs in the IPs.............................................................................................................................39
6.2. Achievement of Implementation Outcomes Related to the Implementation of AARs......................41
6.3. AARs as Applied to Specific IP Interventions...............................................................................................42
6.4. Impact of AARs on Implementation Outcomes Related to the Intervention.......................................43
7. CONCLUSION...........................................................................................................................................................45
8. RECOMMENDATIONS............................................................................................................................................46
8.1. For USAID ............................................................................................................................................................46
8.2. For Implementing Partners...............................................................................................................................46
8.3. For Implementation Researchers....................................................................................................................46
REFERENCES....................................................................................................................................................................47
ANNEX A. SURVEY QUESTIONNAIRE ..................................................................................................................49
General Questionnaire About AAR Practices.....................................................................................................49
Questionnaire About the Specific Intervention...................................................................................................52
ANNEX B. LIST OF DOCUMENTS REVIEWED....................................................................................................54
i
LIST OF TABLES AND FIGURES
Table 1. AAR Formats...................................................................................................................................................... 7
Table 2. Group-Based Pause and Reflect Interventions........................................................................................... 9
Table 3. Selected USAID/Philippines HP Activities, Their Justification, and Specific Interventions.............11
Table 4. AAR Effectiveness Evaluation Framework Based on Widely-Used Implementation Outcomes..15
Table 5. AAR Good Practices and Resulting Intervention Adaptations Across Three IPs............................43
Figure 1. AARs as a Type of P&R activity .................................................................................................................... 8
Figure 2. Study Design....................................................................................................................................................10
Figure 3. Seven P’s Framework ....................................................................................................................................14
Figure 4. AAR Planning Roadmap ................................................................................................................................14
Figure 5. FP Ayuda Express’ Implementation Phases..............................................................................................20
Figure 6. Social Media Cards from FP Ayuda Express............................................................................................21
Figure 7. Sample Chat Inquiry in Facebook Messenger..........................................................................................21
Figure 8. AAR Sessions Held During FP Ayuda Express’ Implementation Period ...........................................23
Figure 9. Sample Content Based on Client Testimonials.......................................................................................24
Figure 10. General Phases of KKDK-CBDR Training, Including MI ....................................................................28
Figure 11. Steps in the AAR Process for MI Training .............................................................................................29
Figure 12. AAR Sessions Held During the Implementation of MI Training .......................................................30
Figure 13. Eisenhower Decision Matrix .....................................................................................................................34
Figure 14. AAR Sessions Held During PPSDC’s Implementation PeriodError! Bookmark not defined.
ii
ACRONYMS
7P’s people, preparation, process, performance, purpose, period, and place
AAR after-action review
AMELP Annual Monitoring, Evaluation, and Learning Plan
CBDR community-based drug rehabilitation
CLA collaborating, learning, and adapting
CLAimHealth Collaborating, Learning and Adapting for Improved Health
DOH Department of Health
DOH-CHD Department of Health Center for Health Development
FP family planning
HP Health Project
IP implementing partner
KKDK Katatagan Kalusugan at Damayan ng Komunidad
L&D learning and development
LGU local government unit
M&E monitoring and evaluation
MI motivational interviewing
MEL monitoring, evaluation, and learning
MERLA monitoring, evaluation, research, learning and adapting
NTP National Tuberculosis Control Program
NTRL National Tuberculosis Reference Laboratory
OH Office of Health
P&R pause and reflect
PACT Partnership among Academes and Communities: Teaming up
for Improved Family Planning Services in Bicol
PDSA Plan-Do-Study-Act Cycle
PhilCAT Philippine Coalition Against Tuberculosis
PhilSTEP Philippine Strategic Tuberculosis Elimination Plan
iii
PIR program implementation review
POPCOM Commission on Population and Development
PPM public-private mix
PPSDC Philippine Private Sector Diagnostics Consortium
PWUD people who use drugs
ReachHealth Improved Health for Underserved Filipinos: Family Planning and Maternal and 
Neonatal Health Innovations and Capacity Building Platforms
RenewHealth Expanding Access to Community-Based Drug Rehabilitation in the Philippines
TBIHSS Tuberculosis Innovations and Health Systems Strengthening
UHC Universal Health Coverage
URC University Research Co.
1
EXECUTIVE SUMMARY
Through its Office of Health (OH), the United States Agency for International Development (USAID) is 
implementing a project in the Philippines to enhance the capacity of the national government, local 
government units (LGUs), and the private sector to provide quality, efficient health service delivery. 
USAID OH utilizes a collaborating, learning, and adapting (CLA) approach to ensure that USAID￾financed activities are on track to achieve their objectives. One of the tools in the CLA approach that
USAID recommends is the After-Action Review (AAR), which is an assessment conducted after a 
major activity or after an identifiable event within a project or major intervention. It allows team 
members and leaders to discuss what happened and to reassess direction; review successes and 
challenges; and identify action steps to meet goals.
USAID’s Collaborating, Learning, and Adapting for Improved Health activity (CLAimHealth), one of 
the eight ongoing activities of the USAID/Philippines Health Project, aimed to document and assess AAR 
processes across USAID/Philippines activities. This report presents the findings of implementation 
research that examined AAR practices and experiences among three of USAID’s implementing partners 
(IPs): 
1. Improved Health for Underserved Filipinos: Family Planning and Maternal and Neonatal Health 
Innovations and Capacity Building Platforms (ReachHealth)
2. Expanding Access to Community-Based Drug Rehabilitation in the Philippines (RenewHealth) 
3. TB Innovations and Health Systems Strengthening (TBIHSS) Project.
The objectives of this implementation research project are as follows:
1. Document the AAR processes of three IPs under the USAID/Philippines Health Project, 
highlighting what is working well and what is not working well, and showing how they contribute 
to adaptive management (i.e., generating short learning loops or activating the Plan-Do-Study￾Act (PDSA) cycle).
2. Generate evidence on how AAR practices contributed to adaptive management and the further 
improvement of a specific intervention implemented by the IP over time.
3. Assess the effectiveness of AAR practices in creating learning and improving performance, using 
six implementation outcomes: fidelity, adaptation, accountability, integration, inclusivity, and 
sustainability.
4. Provide recommendations on how AARs can be more effective in improving the Health 
Project’s interventions.
This implementation research team adopted an exploratory, retrospective, embedded multiple case 
study research design in which IPs are the main cases and a specific intervention from each IP serves as 
the ‘case within a case.’ They studied the following specific interventions: 
• ReachHealth’s Family Planning (FP) Ayuda Express
• RenewHealth’s Motivational Interviewing Training
• TBIHSS’s Philippine Private Sector Diagnostics Consortium (PPSDC)
2 
To develop the individual case studies, the research team used three data collection methods: holding 
preliminary meetings with IPs, providing survey questionnaires to key IP representatives, and reviewing
documents obtained from the IPs. The team conducted data analysis in two stages: ‘within-case’ analysis 
and ‘between-case’ analysis. These were informed by three frameworks: 
1. The Seven P’s Framework, which looked at the anatomy of the AAR process (people, 
preparation, process, performance, purpose, period, and place) 
2. The AAR Planning Roadmap, which depicts the different phases of the AAR process (pre-, 
during-, and post-AAR phases)
3. The AAR Effectiveness Evaluation Framework, which is based on six widely-used implementation 
science outcomes (fidelity, adaptation, accountability, integration, inclusivity, and sustainability).
The case studies of ReachHealth, RenewHealth, and TBIHSS showed that projects are conducting AARs 
in different forms, reflecting their diversity in terms of period, place, people involved, preparation, and 
process. They include stand-alone pause and reflect (P&R) sessions, embedded P&R sessions in routine 
meetings (such as program implementation reviews), and post-session debriefings after a particular 
training event. Despite the different terminology and methodologies that IPs use to conduct AARs, , 
their AAR activities share a common purpose of strengthening and continuously improving project
implementation. 
AARs involve internal team members and external partners in identifying and assessing enabling factors 
as well as barriers, gaps, and challenges in implementation. Projects then use such valuable information 
to plan and execute course correction strategies and manage emerging challenges. The repeated AAR 
activities are meant to eventually help the IP achieve its ultimate goal through continuous process 
improvements. As a sign of adaptation, AAR activities underwent a notable shift from in-person to 
virtual sessions in response to restrictions imposed by the COVID-19 pandemic. AAR sessions also 
helped enhance the sense of accountability among team members. There are additional indications that 
these AAR practices promoted inclusivity through their participants and processes; were integrated into 
CLA plans and routine operations; and were adequately resourced to ensure they could be sustainably 
conducted over time.
When applied to the three specific interventions that the selected IPs implemented, AARs and the 
learnings they generated were an integral part of improving processes and implementation. AARs served 
as a major knowledge sharing opportunity for both internal and external stakeholders to assess the 
implementation of each intervention, identify gaps and challenges that require adaptive actions, and 
determine best practices and lessons learned. The research team used different types of AARs to review
the studied interventions: P&R sessions for ReachHealth’s FP Ayuda Express; post-session debriefings for 
RenewHealth’s Motivational Interviewing Training; and Governing Council Meetings for TBIHSS’s 
PPSDC. AARs enabled teams to successfully address concerns in a timely manner such as increasing 
responsiveness to clients in FP Ayuda Express; improving the content and delivery in motivational 
interviewing training; and expanding the membership of PPSDC. Central to the usefulness of AARs is the 
considering feedback from end-users to development actions for modifying and improving interventions.
Moreover, applying AARs in the specific interventions also helped create a sense of ownership and 
accountability among different stakeholders, promote inclusivity, and develop strategies for eventual 
sustainability and integration of the intervention as part of the health system.
The three case studies in this research (i.e., the three IPs and their specific interventions) demonstrated 
that there are different understandings of AARs and of the ways to conduct them. This is not necessarily 
a problem, provided that these activities still have the key elements of an AAR and fulfill the same 
purpose. The cases that the research team examined demonstrated how adopting AAR practices helps 
3 
in adaptive management, which is the basic purpose of AARs and the reason for their and effectiveness. 
Activating rapid learning cycles allows the IPs to immediately integrate lessons learned from failures and 
successes into implementation. This an essential ingredient for iteration, improvement, and innovation. 
The AAR practices documented in the case studies also revealed numerous other benefits: enhancing 
engagement between different stakeholders (both internal and external); promoting accountability and 
ownership for the intervention, which is vital for its long-term sustainability especially beyond the IP’s 
implementation period; and enhancing the responsiveness of interventions by incorporating the voices of 
clients and users.
Based on the lessons generated from this research, CLAimHealth makes the following
recommendations: 
For USAID
• Clarify the definition of AAR, its types and dimensions, and existing tools and guidelines. 
• Explicitly and actively promote the use of AARs as part of the CLA approach to raise awareness 
and understanding, and increase uptake. 
• Create platforms for joint learning about AAR practices across IPs. 
• Encourage and support continuous documentation of AAR practices and their effectiveness 
through case study development and implementation research. 
For implementing partners
• Adopt existing AAR design templates to ensure quality, enhance repeatability, and ease 
documentation and reporting. 
• Consider the use of existing tools for prioritization (e.g., the Eisenhower Decision Matrix), 
tracking progress (e.g., the Responsibility Matrix), and building accountability. 
• Improve documentation of AAR practices, agreements, and outcomes, including through case 
study development and implementation research. 
• Share good practices on conducting AARs with other IPs. 
For implementation researchers
• Examine similar IPs/organizations/interventions, but with different exposures to AAR practices; 
and consider including a control that does not practice AAR at all. 
• Conduct more quantitative investigations to build evidence on the causal impacts of AAR 
practices on the ultimate outcomes of activities and interventions. 
• Gather the perspective of external stakeholders on how important and useful to them are the 
AARs conducted by the IPs (especially external AARs)
4 
1. INTRODUCTION
Through its Office of Health (OH), the United States Agency for International Development (USAID), is 
implementing a project in the Philippines to enhance the capacity of the national government, local 
government units (LGUs), and the private sector to provide quality, efficient health service delivery. The 
USAID Health Project (HP) contributes to the attainment of the Philippines’ National Objectives for 
Health and is carried out under a Development Objective Agreement with the Government of the 
Philippines. Through activities that its implementing partners (IPs) manage, the HP contributes to health 
system strengthening (HSS) by raising demand for and access to family planning (FP), adolescent 
reproductive health (ARH), maternal and child health services, tuberculosis (TB) treatment and 
prevention, and community-based drug rehabilitation (CBDR).
To ensure that USAID-financed IPs are on track to achieve their objectives, the USAID OH utilizes a 
collaborating, learning, and adapting (CLA) approach. The CLA approach informs the implementation 
framework that IPs employ and enables them to be responsive to challenges as they emerge. One of the 
tools in the CLA approach that USAID recommends is the After-Action Review (AAR). According 
to USAID, an AAR is an assessment conducted after a major activity, or after an identifiable event within 
a project or major activity. It allows team members and leaders to discuss what happened and why to 
reassess direction; review successes and challenges; and identify action steps to meet goals (USAID, 
2006). AARs foster a culture of learning, knowledge sharing, collaboration, and adaptive management, 
which is defined as “an intentional approach to making decisions and adjustments in response to new 
information and changes in context” (USAID, 2021).
In terms of organizational learning, AARs provide the opportunity for teams to continuously assess their 
performance and learn from successes and failures. In general, AARs follow a sequence of four steps: 
planning, preparation, implementation, and follow-through. As such, AARs are intended to generate 
short learning loops for adaptive management among OH and IPs, functioning to facilitate the iterative 
plan-do-study-act (PDSA) cycle, and facilitate the use of data and documentation over time to inform 
next steps and follow-through (Taylor, et al., 2014). There are different types of AARs, including formal 
reviews that are regularly scheduled, more informal pause-and-reflect (P&R) sessions, and individualized 
meetings in the field (e.g., spot inquiries), and others (WHO, 2019).
USAID’s Collaborating, Learning, and Adapting for Improved Health activity (CLAimHealth), one of 
eight activities in the USAID/Philippines Health Project, intended to document and assess AAR 
processes across USAID/Philippines activities. This report presents the findings of implementation 
research that examined AAR practices and experiences of three IPs: 
1. Improved Health for Underserved Filipinos: Family Planning and Maternal and Neonatal 
Health Innovations and Capacity Building Platforms (ReachHealth) 
2. Expanding Access to Community-Based Drug Rehabilitation in the Philippines 
(RenewHealth)
3. TB Innovations and Health Systems Strengthening (TBIHSS) Project.
5 
2. LEARNING QUESTIONS AND OBJECTIVES
2.1. Primary Learning Question
Are the after-action reviews that USAID/Philippines’ Health Project implementing partners conduct 
effective in generating short learning loops for adaptive management, and quality improvement of health 
interventions and technical assistance? What is working well and what can be enhanced? 
 
2.2. Secondary Learning Questions
• How are government and local implementing stakeholders actively involved in the AAR process, 
including adaptive management? Should AARs involve appropriate representatives from national 
and local partners, or ensure their inputs are solicited?
• In terms of AAR follow-through, how are agreed-upon adaptive management actions brought 
down to the level of the right partners for action and implementation? 
2.3. Objectives
The objectives of this implementation research project are as follows: 
1. Document the AAR processes of three (3) IPs under the USAID OH Philippines Health Project, 
highlighting what is working well and what is not working well, and showing how they contribute 
to adaptive management (i.e., how well they generate short learning loops or activate the PDSA 
cycle). 
2. Generate evidence on how AAR practices contributed over time to adaptive management and 
further improvement of a specific intervention implemented by the IP. 
3. Assess the effectiveness of AAR practices in creating learning and improving performance, using 
six implementation outcomes: fidelity, adaptation, accountability, integration, inclusivity, and 
sustainability. 
4. Provide recommendations on how AARs should be implemented by other IPs. 
6 
3. AFTER-ACTION REVIEWS: A BRIEF REVIEW
Given the complexity of health problems, it is imperative to ensure that health interventions are 
responsive to needs. Considering the dynamics of public health issues, practitioners must constantly 
capture learnings within their organizations to find creative ways to develop interventions that address 
challenges as they emerge. One example of activities that harvest lessons learned during program 
implementation is after-action reviews (AAR). An AAR is an assessment conducted after a major activity 
or an identifiable event within a project or intervention. It allows team members and leaders to track 
progress, correct unintended impacts, and ensure that planned outcomes are achieved (USAID, 2006; 
UNDP, 2007). Originally, an AAR was conducted at the end of military training for leaders to reflect on 
and quickly adopt critical analysis provided to them by observers (National Police Foundation, 2020). 
Nowadays, it fosters a culture of learning, knowledge sharing, collaboration, and adaptive management in 
organizations because it is a way for them to reflect and learn while performing (Baird, et al., 1999).
AARs are structured around one basic analytical methodology that aims to respond to four main 
questions (WHO, 2019):
1. What was supposed to happen?
2. What actually happened?
3. Why was there a difference?
4. What can we learn from this?
The simplicity of this framework allows various organizations to assess the processes of their activities, 
analyze how these processes performed compared to their planned assumptions, and identify reasons 
for variance or determine actions for improvement (WHO, 2019; CLIC, nd)
3.1. Forms of AAR
There are three forms of AARs that various organizations usually practice: formal, informal, and 
personal/individual. Formal AARs are resource-intensive and usually involve planning, coordinating, and 
preparing supportive training aids, an AAR site, support personnel, and time for facilitation and report 
preparation (WHO, 2019). Informal AARs are usually carried out by the people who are responsible for 
an activity. If necessary, the discussion leader or facilitator can be identified beforehand or chosen by the 
team itself (WHO, 2019). Finally, individual or personal reviews usually concern individual reflection on 
the course of action or activities of immediate past events (USAID, 2013). The type of AAR to be used 
may vary depending on the activity to be reviewed, the level of various team members’ participation in 
the project, the scope of the project, and a general assessment of which approach might work best 
(Hengeveld-Bidmon, 2015).
AARs may also be conducted through different formats, which offer collective learning and operational 
improvement after a public health response (WHO, 2019). Table 1 lists different formats that can be 
used in conducting AARs.
7 
Table 1. AAR Formats (WHO, 2019)
Format Description Best Used For
Debrief An informal facilitator-led discussion with no 
more than 20 participants focusing on a limited 
number of functional areas (3 or less), taking 
place over half a day with more focus on 
learning within a team
Smaller responses/activities
Working group An interactive format that consists of guided 
group work and plenary sessions with up to 50 
participants focusing on more than 3 functional 
areas, taking place over 2 to 3 days
Responses that involve multiple 
sectors
Key informant 
interview
A longer and more in-depth review of an event 
that begins with a literature review and 
feedback surveys, followed by semi-structured 
interviews and focus group discussion, taking 
place over several weeks
Complex and larger responses where 
those involved can no longer be 
brought together, or where 
confidentiality and non-attribution are 
necessary for honest and open 
feedback
Mixed-method A blended approach of the above three formats 
consisting of a working group AAR with 
contents supplemented from key informant 
interviews
Responses with a large scope where 
the majority (but not all) of those 
involved can be brought together
3.2. The AAR Process 
In general, AARs follow a sequence of four steps: planning, preparation, implementation, and follow￾through (Serrat, 2008). As such, an AAR is intended to generate short learning loops for adaptive 
management among organizations or institutions, facilitate the iterative Plan-Do-Study-Act (PDSA) 
Cycle, and encourage the use of data and documentation over time to inform next steps and follow￾through (Taylor, et al., 2014).
Guidelines for conducting an AAR are available, such as the ones developed by the World Health 
Organization (WHO) and USAID. Sexton and McConnan (2003) also proposed a checklist that is 
structured similarly to the WHO guidelines. That checklist is divided into five components: 1) before the 
AAR event; 2) during and after the event; 3) disseminating event findings; 4) monitoring and follow-up; 
and 5) development activities. Each component lists several tasks accompanied by a subtle reminder to 
ensure smooth task execution and achievement of objectives.
Having a database is also useful to connect AAR experiences for future improvement. Effectively
managing data generated by AAR activities helps stakeholders understand the context and mechanisms 
that drive successful and unsuccessful practices; identify and share best practices; and drive individual 
and organizational improvement. Weak data management structures, data sharing restrictions, and 
concerns about staff privacy and job security are challenges to collecting information from AAR 
processes. Hence, it is necessary to offer incentives that encourage teams and their individual members 
to report and document AAR findings. It is also important to frame reporting systems as more than a 
way to ensure accountability, but also to facilitate organizational learning and system improvement, as is 
the case with the aviation industry's critical incident registry (WHO, 2019).
8 
3.3. Evaluating AARs 
While an AAR itself has an evaluative purpose, there is limited literature on how to evaluate the AAR 
process and its outcomes. One comparative case study by Sexton and McConnan (2003) studied the 
AAR practices of three organizations: World Vision International (WVI), the British Red Cross Society 
(BRCS), and the Joint Emergency Food Aid Programme (JEFAP). Drawing lessons from an extremely 
small sample of past AAR-type events, the study found out that even the simplest approach to learning 
is difficult to execute because of an environment with a complex arrangement. The AARs’ most critical 
components—which are reflective and deeper probing work around identifying critical issues--tend to 
be overlooked more often than not. 
Another study used a bare-bones meta-analysis approach to examine the effectiveness of AARs (Keiser 
and Arthur, 2021). Based on 61 studies, it showed that AARs led to an overall improvement in multiple 
training evaluation criteria. The study suggested that a highly structured AAR is more effective than a 
less structured AAR in the military, but either highly- or lowly-structured AARs displayed comparable 
effectiveness in the healthcare setting. 
During the COVID-19 era, a more recent study in a tertiary care hospital by Sorbello et al. (2021) 
evaluated the AAR process by conducting an AAR itself. Overall, the hospital’s COVID-19 response was 
deemed to be successful and timely, underscoring the importance of three components: governance, 
health personnel, and a multidisciplinary approach. Meanwhile, the participants in the AAR cited poor 
communication management as a major factor that negatively affected the hospital's response.
3.4. Pause and Reflect Among Different USAID Implementing Partners
As mentioned earlier, USAID is endorsing the incorporation of CLA practices into IP activities (USAID 
Learning Lab, nd). Pause and reflect (P&R) is one CLA method that USAID promotes, emphasizing that 
pausing and reflecting regularly identifies what is working and what needs adapting. It also allows 
organizations to consider the impact of changes in the operating environment or context (USAID, 
2018). The USAID CLA Toolkit for Adaptive Management suggests that P&R can be done individually or 
in groups. Individual-based P&R can be done through journaling or blogging. Meanwhile, examples of 
group-based P&R sessions, which include AARs, are listed in Table 2. Because AARs are seen as one 
type of P&R activity, the formats for AAR (according to WHO) and the types of P&R (according to 
USAID) can be visualized in the manner shown in Figure 1. 
Figure 1. AARs as a Type of P&R activity
(Adapted from USAID, 2018 and WHO, 2019)
9 
Table 2. Group-Based Pause and Reflect Interventions
Activity Description
Portfolio Review A periodic review of all aspects of a USAID Mission/Office's assistance objective, 
projects, and activities, often held prior to preparing the annual Joint Operational Plan
After-Action Review An assessment originated in the U.S. military, conducted after a project or major 
activity to allow team members and leaders to discover (learn) what happened and why, 
reassess direction, and review successes and challenges
Hot Wash A more informal equivalent of an AAR also originating from the U.S. military,
comprising a facilitated discussion immediately following an exercise to capture 
feedback on issues, concerns, and proposed improvements from implementers
Liberating Structures A series of innovative approaches to help groups tap into their collective know-how 
and creativity
Appreciative Inquiry A change management approach that focuses on identifying what is working well, 
analyzing why it is working well, and then doing more of it
Communities of Practice A group that shares a common interest and interacts often (face-to-face or online) to 
learn from each other and advance their work
10 
4. METHODOLOGY
4.1. Overall Research Design
The implementation research team adopted an exploratory, retrospective, embedded multiple case 
study research design in which IPs were the main cases and a specific intervention from each IP serves 
was the ‘case within a case.’ The team employed widely-accepted case study research principles and 
methods as described in Yin (2018). Below is a diagram summarizing the overall design of the research 
project (Figure 2).
Figure 2. Study Design
4.2. Selection of Cases
The study team selected three IPs for this implementation research: ReachHealth, RenewHealth, 
and TBIHSS (Table 3). They were chosen for several reasons: they expressed interest and willingness 
to participate in the research; their participation ensured diversity among participating IPs in terms of 
thematic focus (public health domain or disease area), size, and age of the project; and they had 
implemented a specific intervention (a ‘case within a case’) that depicted significant change or 
improvement over time.
4.3. Data Collection
The research team used three data collection methods to develop the individual case studies of the 
three IPs and their specific interventions: 
1. Preliminary meetings with IPs to gain an overview of their work, CLA approaches, and AAR 
practices. 
2. A survey questionnaire for key IP representatives, which included specific questions pertaining to 
various elements of the 7 P’s framework, the AAR Planning Roadmap, and the AAR Effectiveness 
Evaluation Framework (all discussed below). See Annex A for the questionnaire.
3. A review of documents obtained from the IPs to validate and supplement the findings from 
preliminary meetings and survey questionnaires. The list of documents can be found in Annex B.
11 
Table 3. Selected USAID/Philippines HP Activities, Their Justification, and Specific Interventions
Selected USAID 
HP IP (Case)
Brief Description of the IP Justification for Selection Specific Intervention
(Case within a Case)
ReachHealth Begun in 2018, ReachHealth is a five-year 
project that aims to strengthen and improve 
access to critical health services in the 
Philippines by addressing unmet need for 
family planning services, teen pregnancy, and 
maternal and neonatal health. Implemented by 
RTI International, the project supports the 
country by identifying and responding to local 
root causes of poor family planning and 
maternal and neonatal health outcomes. 
ReachHealth continues to address gaps and 
challenges related to family planning and 
maternal and neonatal health services as well 
as provide technical assistance on 
interventions and innovations that contribute 
towards achieving health targets.
One of the ‘older’ 
USAID/Philippines OH Project 
projects, ReachHealth 
demonstrated extensive experience 
in CLA, particularly in the 
implementation of AARs and P&R. 
ReachHealth is also a big project
covering many project sites 
nationwide. Its components range 
from family planning to COVID-19 
response and are primarily focused 
on delivering interventions to local 
communities. Hence, this is a good 
case of a big project with multiple 
delivery-oriented components. 
Their monitoring, evaluation, 
research, learning and adapting 
(MERLA) team also demonstrated 
great enthusiasm to participate in 
this research and shared 
substantive knowledge during the 
first meeting.
FP Ayuda Express. In response to 
COVID-19 restrictions, ReachHealth 
conducted a series of P&R sessions to adapt 
Enhanced Usapan, a community-based group 
discussion with immediate on-site service 
provision of methods suitable to clients’ 
goals of limiting or spacing childbirths. The 
intervention was then converted into Ayuda 
Express, which uses existing popular social 
media platforms among Bicolanos to 
communicate FP information and services. 
12 
Table 3. Selected USAID/Philippines HP Activities, Their Justification, and Specific Interventions (cont.)
Selected USAID 
HP IP (Case)
Brief Description of the IP Justification for Selection Specific Intervention
(Case within a Case)
RenewHealth Launched in 2019, RenewHealth is a five-year 
project implemented by University Research 
Co. (URC) that seeks to expand access to 
quality community-based drug rehabilitation 
(CBDR) and encourage voluntary drug 
demand reduction in the Philippines. The 
project’s theory of change is as follows: If 1) 
there is increased support for CBDR and 
help-seeking among persons who use drugs 
(PWUDs); 2) evidence-based and culturally 
appropriate treatment tools and interventions 
are created; and 3) the capacity, policies, 
systems and resources of communities and 
government agencies to provide community￾based drug recovery services are enhanced, 
then PWUDs and their families will access 
treatment, thus reducing drug dependence in 
the country.
One of the ‘youngest’ 
USAID/Philippines OH Project 
projects, RenewHealth is also the 
first USAID project of its kind in 
the world that focuses on mental 
health and substance abuse. This 
means that there was no foreign 
model to emulate or existing 
guidelines to adopt when the 
project began. Within a short 
period of time, the project was able 
to roll out innovative interventions 
for different parts of the treatment 
cascade, most especially 
community-based drug 
rehabilitation. This was the result 
of a continuous learning process, 
including repeated P&R sessions.
Motivational Interviewing Training.
This training for community facilitators 
evolved over time through repeated P&R 
sessions. Future rounds of the training 
produced significant changes in trainee 
outcomes (i.e., post-test results).
TBIHSS Launched in 2018, TBIHSS is a five-year project
implemented by FHI360 that is designed to use 
a dynamic, strategic, and cutting-edge approach 
to accelerate the fight against tuberculosis (TB) 
and to institute health processes and systems 
to help the Philippines achieve TB elimination 
targets by 2035. It assists the Department of 
Health’s National TB Control Program (NTP) 
in actively identifying, developing, testing, and 
scaling up innovative technologies and 
approaches across the TB continuum of care. 
It also helps maximize programmatic outcomes 
and strengthen TB health systems.
USAID/Philippines OH and 
CLAimHealth recommended adding 
a TB-focused project given TB’s 
national importance. Moreover, as 
one of the longest-running 
USAID/Philippines OH projects, 
there are many lessons from 
failures and successes that can be 
learned in relation to the 
implementation of AARs, especially 
when it comes to developing 
innovations. 
The Philippine Private Sector 
Diagnostics Consortium (PPSDC) is a 
group of private laboratories and hospitals 
that aims to ensure quality TB diagnostics 
are made available at concessional prices to 
accredited laboratories and hospitals that 
will in turn ensure these tests are made 
available to the public at affordable prices. 
AAR sessions have helped PPSDC evolve 
and grow over time.
13 
Initially, the study team planned to make key informant interviews, participant observation of upcoming 
AAR sessions, and review of recordings of previous AAR sessions a part of the research methodology. 
However, they had to adjust or modify these data collection tools (for instance, through greater use of 
the survey questionnaire and reliance on existing documents) due to timing limitations brought about by 
the COVID-19 pandemic, year-end workload borne by the IPs, unavailability of recordings and upcoming 
events, and other factors. To enhance the validity of the case studies, each IP was also given the chance 
to review the case study draft and provide additional inputs and corrections if necessary.
4.4. Data Analysis
The team analyzed data in two stages, using ‘within-case’ analysis and ‘between-case’ analysis, which are 
usually done in multiple case study research. 
● The ‘within-case’ analysis focused on individual IPs as well as the specific intervention in each 
IP. This was done by writing an individual case study for each IP using: a general description of an 
IP’s AAR practices using existing frameworks (see below); documentation of the specific 
intervention with a summary of events and discussion of lessons learned; and insights related to 
the implementation outcomes enumerated in the AAR Effectiveness Evaluation Framework.
● The ‘between-case’ analysis examined the three IPs and their specific interventions 
collectively and comparatively. Using information and insights generated through the ‘within￾case’ analysis, the study team identified common themes across the three IPs and their specific 
interventions, including AAR characteristics, barriers and challenges encountered, and lessons 
learned in relation to the implementation outcomes enumerated in the AAR Effectiveness 
Evaluation Framework. The team summarized common themes as well as shared and unique 
lessons from all cases in a case study database using Microsoft Excel.
4.5. Analytical Framework
To systematically analyze the data, the study team used the following frameworks, which also informed 
development of the survey questionnaire (see Annex A):
The Seven (7) P’s’ Framework (Figure 3; adapted from Graves, 2017) describes the anatomy of the AAR 
process. The 7 P’s are: people, preparation, process, performance, purpose, period, and place. They 
resemble the five W’s and one H: who, what, where, when, why, and how. This framework is the basis 
for developing a general description of each IP’s AAR practices.
The AAR Planning Roadmap (Figure 4; WHO, 2019) depicts the different phases of the AAR process: pre-, 
during-, and post-AAR phase. This framework is essential for tracing the entire journey of an 
organization and/or intervention on the road to learning and continuous improvement, and if AAR 
practices contributed to goal achievement. This framework from WHO also resembles USAID’s AAR 
Technical Guidance (USAID, 2006) and another checklist of AAR good practice (Sexton & McConnan, 
2003) that was discussed earlier in the literature review.
14 
Figure 3. Seven P’s Framework
(Adapted from Graves, 2017)
Figure 4. AAR Planning Roadmap
(WHO, 2019)
15 
The AAR Effectiveness Evaluation Framework (Table 4) is developed by the research team to allow 
assessment of the IPs and their AAR practices based on widely-used implementation science outcomes. 
Effectiveness simply pertains to the extent to which an intervention achieved its goal. For an AAR, being 
effective means that it has enhanced organizational learning and also resulted in improvements to an
intervention. In this study, effectiveness is assessed in two dimensions: 1) an ‘internal’ dimension, which 
pertains to the improvement of organizational learning, including the effective implementation of AARs 
themselves; and 2) an ‘external' dimension, which pertains to how AAR practices contributed to the 
improvement of the intervention or the activity as a whole. To assess these two dimensions, the study 
team selected six widely-used implementation outcomes. They applied internal dimension learning 
questions to general AAR practices in the IP and used external dimension learning questions to identify 
the contribution of AAR sessions to implementation improvement in the three specific interventions. 
Table 4. AAR Effectiveness Evaluation Framework Based on Widely-Used Implementation 
Outcomes
Implementation 
Outcomes
Definition Internal Dimension 
Learning Question
External Dimension 
Learning Question
Fidelity The degree to which an 
intervention was 
implemented in relation 
to the way it was 
designed in an original 
protocol, plan, or 
policy
Does the AAR process 
adhere to the original 
process designed by the 
management and/or defined 
in national or international 
guidelines/practices?
Does the AAR process help 
the activity/intervention stick 
to its original implementation 
design?
Adaptation The ability to revise an 
existing intervention to 
suit new circumstances
Does the AAR process adapt 
to changing circumstances, 
for instance shifting to an 
online platform?
Does the AAR process 
facilitate the 
activity’s/intervention’s ability
to adjust to changing 
circumstances when 
necessary?
Accountability The process is able to
generate a sense of 
responsibility and 
ownership among 
stakeholders, who then 
feel compelled to 
execute agreed-upon 
adaptive actions in a 
timely manner
Does the AAR process have 
mechanisms to ensure 
follow-up and dissemination 
of agreed-upon adaptive 
actions in a timely manner?
Does the AAR process 
compel or motivate the staff 
to execute agreed-upon 
adaptive actions in a timely 
manner? 
Integration The process is 
embedded in everyday 
operations or 
organizational 
structures rather than 
fragmented and siloed
Is the AAR process 
embedded or tied into the 
organization’s structures and 
operations?
Does the AAR process aid in 
ensuring incorporation of the 
activity/intervention into the 
health system (i.e., local, 
provincial, regional, national)?
16 
Table 4. AAR Effectiveness Evaluation Framework Based on Widely-Used Implementation 
Outcomes (cont.)
Implementation 
Outcomes
Definition Internal Dimension 
Learning Question
External Dimension 
Learning Question
Inclusivity The process allows 
relevant stakeholders 
to participate (rather 
than excluding them); 
hence their voices are 
heard
Does the AAR process allow 
the participation of different 
personnel within the 
organization as well as other 
external stakeholders?
Does the AAR process 
promote the inclusion of 
diverse stakeholders in 
implementation of the 
activity/intervention? 
Sustainability The process can be 
executed over time 
with adequate 
resources and even 
when there is a change 
in leadership or 
personnel
Has the AAR process been 
continuously performed 
since it was first introduced 
and are there indications for 
its continuous 
implementation in the future?
Does the AAR process help 
ensure continued 
implementation and adoption 
of the activity/intervention 
even beyond the activity’s 
timeframe?
4.6. Limitations of the Study
While this study was extensively conducted, it still has some limitations. First, because of the major 
mobility restrictions imposed by the COVID-19 pandemic, the study team had to rely only on 
information and data it could retrieve by digital means. Second, while the study’s retrospective design 
generated a wealth of insights about the way IPs conduct AARs, a prospective approach (for instance, by 
observing actual AAR sessions or conducting intervention research that uses AAR methods) could have 
also produced rich insight through first-hand observation. Time and resource limitations also forced the 
research team to examining only three IPs and a specific intervention from each. Finally, throughout the 
short project period, timing limitations experienced by both the investigators and IPs—including 
limitations related to COVID-19 and numerous holidays—greatly reduced the possibility of conducting 
additional interviews and meetings with the IPs. However, the team addressed this limitation through 
close liaison with the IPs to ensure that information presented in the case studies is as accurate as 
possible. 
4.7. Ethical Considerations
This implementation research did not require ethics review approval as it mainly utilized survey 
questionnaire responses based on facts and not the personal views of select staff members from each IP, 
as well as documents submitted to the research team.
17
5. RESULTS
5.1. Implementing Partner 1: Improved Health for Underserved Filipinos: Family 
Planning and Maternal and Neonatal Health Innovations and Capacity Building 
Platforms (ReachHealth)
Overview of AAR Practices
To ensure its project goals are achieved, ReachHealth has utilized different CLA tools such as AARs.
ReachHealth conducts several CLA activities that embed AAR components like the Mid-Term Technical 
Review and Program Implementation Review (PIR). It also conducts dedicated P&R sessions, which can 
be held in groups as well as one-on-one sessions with persons-in-charge of activities. The 
documentation revealed examples of internal P&R sessions where the senior management team 
reviewed the status of family planning (FP) in the implementation of a hospital program as well as the 
postpartum family planning (PPFP) performance of hospitals. These AAR practices allow the team to 
identify enabling factors, gaps, and challenges in implementation, and agree on priority actions for 
addressing the gaps determined during the exercise.
Before the AAR Event
Technical staff. No particular individual officer is assigned as being responsible for preparing and 
organizing AAR activities. The organizer depends on the activity for which the AAR is being conducted. 
For example, the MERLA director is in charge of implementing the Mid-Term Technical Review, while 
the regional managers are responsible for regional PIRs.
Budget allocation. There is no explicit line item in the budget that mentions AAR, but ReachHealth has an 
allocated budget for CLA/MEL activities, which also include AAR/P&R sessions. The shift to online 
meetings during the COVID-19 pandemic may have also led to reduced spending on CLA and AAR 
sessions. 
Venue selection. The selection of the venue usually depends on the location of the intervention being 
reviewed. However, during the COVID-19 pandemic, these AAR/P&R sessions have been shifted to 
virtual meetings.
● Before the pandemic, all regional PIRs were held in areas nearest to where most staff members 
were. The cities of Cebu and Davao hosted the Visayas and Mindanao teams respectively, while 
the IP's national office in Ortigas hosted the Luzon team. The senior management team traveled 
to the regions to conduct the PIR. 
● During the pandemic, ReachHealth shifted to utilizing online video conferencing platforms (i.e.,
Zoom) to conduct AARs and other activities as a way of ensuring the safety of participants and 
team members.
Timing/scheduling. Some AARs are conducted as part of routine CLA activities such as the Mid-Term 
Technical Review and PIRs, which are already pre-scheduled at regular intervals. Meanwhile, AAR/P&R 
sessions that are dedicated to a specific activity to be reviewed are held right after the event to ensure 
the attendance of participants.
AAR design. There is no predetermined AAR design because it is usually developed by the assigned 
responsible person for the activity. The assigned team member who developed the first draft AAR 
design would share it with the senior management team (for a national-level AAR) or with the regional 
18
manager (for regional AAR) for review. There are also instances in which participants were tasked to do
preliminary activities in preparation for a P&R session. Participants were clustered into small groups and 
given guide questions for group discussions. These preparatory activities allowed the participants to 
review their data and to generate learnings that would be presented at the P&R session afterward.
Agenda setting. The process followed for preparing the agenda is the same as the process for preparing
the design, with the staff member responsible for conducting the activity being the one who prepares
the first draft that will later be reviewed by a higher body. Usually, the topics included in the agenda are 
issues or concerns emerging from a certain implementation period, learnings from routine monitoring 
data, lessons learned from a period of implementation, root cause analysis, adaptive responses that were 
identified, and translation of learning into actions.
Participant selection. Selecting participants for AAR activities depends on the type of AAR being
conducted and the activity being reviewed. For example, for the Mid-Term Technical Review, all project 
technical staff members were asked to attend and participate in all preliminary activities; PIRs would 
usually include USAID and other IPs; and implementation reviews of grantee projects that are about to 
end would include USAID and relevant partners.
● Internal AARs usually include ReachHealth’s expanded senior management team, technical 
advisors and regional advisors, MERLA national team, regional MERLA staff, regional 
representatives, and communications staff. Participants whose attendance is required are usually 
indicated in the AAR design.
● External AARs usually include ReachHealth’s team, representatives from partner government 
agencies such as the Commission on Population (POPCOM) and the Department of Health 
Center for Health Development (DOH-CHD), the Provincial Health Office, focal persons from 
partner health facilities, and family planning coordinators.
During the AAR Event
Facilitation. It is not standard practice for ReachHealth to prepare trigger questions during an AAR. 
Discussions and reflections just naturally arise, resulting from questions asked by senior management 
team members (often addressed to the regional technical staff) or by USAID representatives. In the case 
of the Mid-Term Technical Review for example, each regional team was even asked to prepare a root 
cause analysis of their performance.
Participation. The responsibility of time management is given to the activity facilitator/moderator. In many 
instances, the original time allotted for the activity is extended when the team feels that more in-depth 
discussion is needed to thresh out the issues at-hand. This allows greater participation of more 
attendees to ensure that diverse insights and reflections are captured.
AAR content. P&R sessions usually revolve around reviewing the status of a specific activity within a 
period of implementation; identifying items that went well and those that need improvement; and 
sharing and discussing experiences and learnings from both internal and external stakeholders. 
Participants discuss enabling factors for good practices, gaps and challenges, and opportunities. They are 
then asked to identify adaptive solutions that will address the challenges that the team encountered.
For example, in one P&R session for makeshift teleconsultation installation and implementation, 
participants were asked to share experiences and identify strengths and challenges encountered in 
different types of facilities. Participants were given guide questions (“What were the challenges 
encountered?”; “What are the concerns that we still need to improve on?”; “What are the proposed solutions to 
19
address these?”; and “What activities can we undertake to ensure sustainability of the teleconsultation 
mechanism?”) that allowed them to generate action points for improving access to and utilization of 
teleconsultation services, and to give recommendations for sustainability measures.
Prioritization. The team prioritizes action points in an iterative manner (as opposed to using certain tools 
or frameworks), but often based on the need, importance, and urgency of addressing an issue. 
Task assignment. The team identifies who is responsible for a certain task based on the nature of the 
action to be taken. For example, if the action is about data, then the MERLA officer will be in charge; 
and if it is an action for a regional team, then the corresponding regional manager will be responsible.
Documentation. The last step before closing an AAR session is presenting agreements and 
recommendations made during the meeting. For future reference, the team documents the highlights of 
the AAR, discussion and action points made, agreements reached, and next steps agreed upon. The 
team’s assigned moderator usually prepares the written report and reviewed by a member of the senior 
management team. A photo of the attendees is usually taken as proof of the activity.
Post-AAR evaluation. There is no indication that AAR sessions are also evaluated, either immediately or 
later.
After the AAR Event
Follow-up and monitoring. ReachHealth monitors the progress of recommendations and action points in 
several ways. In the case of agreements resulting from the Mid-Term Technical Review for example, the
IP has a learning tracker that is updated every quarter. In most instances, the technical person-in-charge 
monitors the progress of recommendations and action points.
Dissemination of AAR outcomes. ReachHealth disseminates the outcomes of AAR activities to key partners 
through other external and internal P&R sessions, as well as through regular staff meetings and senior 
management team meetings. 
Progress reporting. In reporting back to key partners on progress, the project team compares key 
indicators from previous quarters to present quarters in accordance with the outcomes of the P&R 
sessions.
5.2. Specific Intervention 1: ReachHealth’s Family Planning (FP) Ayuda Express
Background of the Intervention
FP Ayuda Express is a virtual adaptive solution that has used existing popular social media to 
communicate and disseminate family planning-related information and services in the Bicol Region during 
the COVID-19 pandemic. Clients can ask questions on topics related to family planning and avail 
themselves of family planning-related services by sending queries through FP Ayuda Express Chat via 
Facebook Messenger or by calling the POPCOM Helpline. Chat moderators or call respondents then 
link these clients to the nearest health facilities and ensure they receive their family planning method of 
choice.
This virtual platform is a result of the joint implementation of PACT (Partnership among Academes and 
Communities: Teaming up for Improved FP Services in Bicol,) consisting of the Family Health Cluster 
Division of DOH-CHD Bicol Region, POPCOM Bicol Region, seven academic partners in Albay and 
Camarines Sur, and USAID’s ReachHealth Project. The intervention is part of ReachHealth’s plan to 
20 
integrate its Enhanced Usapan program in the community health education curriculum of select academic 
institutions in the Bicol Region. 
Originally, select academic institutions with midwifery and nursing courses were to be developed as 
training centers in conducting Usapan sessions among partner communities. However, the COVID-19 
pandemic disrupted implementation of face-to-face demand generation activities and house-to-house 
tracking of unmet needs of the community. The situation called for innovative solutions to ensure 
continuous delivery of family planning-related services that can be implemented with limited face-to-face 
interactions without compromising the health of the community and the team. As part of ReachHealth’s 
adaptive response, the program shifted to online delivery and launched FP Ayuda Express, a virtual 
platform that serves as a teleconsultation facility for family planning services in the region.
The implementation of FP Ayuda Express was divided into four phases (Figure 5). Phase 1 focused on 
designing the intervention and engaging partners. In this phase, the concept was presented to the 
regional CHD, regional POPCOM, and academic institutions. Through a series of consultation meetings 
with key stakeholders, the team was able to develop the design of FP Ayuda Express and conducted pre￾testing activities to ensure the intervention would respond to the community’s needs. Further, this 
phase was also the time when the team selected and trained students as chat moderators, created the 
chat guide and algorithm for chat moderators, conducted an orientation on FP Ayuda Express for 
students and clinical instructors, and formulated the content plan for its Facebook page. Phase 1 began in 
May 2020 and was completed in June 2020.
Figure 5. FP Ayuda Express’ Implementation Phases
21 
Phase 2 comprised the launch and implementation of the intervention. It focused on developing social 
media cards for the FP Ayuda Express Facebook page and addressing inquiries from clients. Figure 6
shows two examples of social media cards uploaded on Facebook that inform clients what the 
intervention is and how to access it. Once clients send their inquiry via Facebook Messenger, they are
given topics that may be of interest to them. Sample topics include family planning services, family 
planning methods, and specific information for mothers practicing the lactation amenorrhea method 
(LAM). Figure 7 shows how the FP Ayuda Express Chat works. Phase 2 was completed in December 
2020. 
Figure 6. Social Media Cards from FP Ayuda Express
Figure 7. Sample Chat Inquiry in Facebook Messenger
22 
Phase 3 involved monitoring the intervention. To ensure the intervention could meet its objectives, the 
FP Ayuda Express team documented implementation and implemented a referral and feedback 
mechanism. Insights from the documentation and feedback collected from this mechanism were 
discussed during monthly P&R meetings with key partners. This phase ran from July 2020 to April 2021. 
Finally, Phase 4 of the intervention involved sustaining and scaling-up FP Ayuda Express. In this phase, the 
team developed a playbook on FP Ayuda Express and distributed it to partners. The team also developed 
its sustainability plan, conducted an orientation on scaling up the intervention, and identified potential 
scale-up locations. Moreover, ReachHealth proposed actions for 2022, including expanding and 
developing the new FP Ayuda Express platform for private lying-in clinics (PLICs) in Central Luzon. 
After more than a year of implementation, FP Ayuda Express has tallied a total of 2,908 followers on its 
Facebook page and provided family planning information to over 850,000 individuals online. In 2021 
alone, the platform directly reached more than 500,000 people and over 5 million via local radio.
After-Action Review as Applied to the Intervention 
For FP Ayuda Express, AARs are conducted through the intervention’s monthly M&E meetings in the 
form of a P&R session. These sessions serve as a venue for the ReachHealth team and key stakeholders 
to discuss updates on project implementation, and raise emerging issues and concerns for the team to 
address. They also allow the team to share lessons learned during a period of implementation, which is 
helpful for general improvement of the project. As one member during a session shared, “The regular 
meeting is very valuable to troubleshoot certain issues and to improve services for the good of the 
people.”
The AAR is usually called to order by a presiding officer of the day and includes the participation of the 
ReachHealth team, academic partners, representatives from DOH CHD Region V and POPCOM Region 
V, and student volunteers. Generally, during the implementation of an AAR, the team follows a structure 
that involves presenting and discussing implementation updates on FP Ayuda Express Chat and Call as 
well as the latest analytics on FP Ayuda Express Chat, which is vital for improving the service delivery of 
the intervention. After the presentation and discussion of updates and analytics, each partner will then 
share experiences, updates, and accomplishments. Discussions on issues and challenges follow, and 
agreements or recommendations are then identified. The AAR is conducted monthly unless a specific 
date is agreed upon by the team.
As shown in Figure 8, AARs provide an opportunity for the project team to discuss what went well and 
what needed improvement. The team was able to identify specific gaps through AARs, which led them 
to revise their chat guide, change their medium of language into Tagalog, and modify the automatic 
response on Facebook Messenger. The team was also able to develop a client tracking mechanism to 
record those who have been successfully referred to facilities and received appropriate family planning 
services, and identify appropriate family planning-related messages to increase engagement and reach a 
wider audience. 
23 
Figure 8. AAR Sessions Held During FP Ayuda Express’ Implementation Period
Impact of AARS on Implementation Outcomes
Fidelity. Initially, the intervention was slated to be implemented in-person, but had to be adapted to 
online mode because of the COVID-19 pandemic. Thus, it could be said that fidelity to the
intervention’s original design significantly decreased. However, while the delivery mode may have 
changed, the actual purpose and content of the intervention remained intact. The monthly P&R sessions 
could have helped come up with adaptive actions in response to changing demands and circumstances, 
and also ensure that the team and the intervention remained faithful to the overall goals that were set at 
the outset.
Adaptation. As expected, regular AAR sessions have helped the implementation team develop adaptive 
actions in response to changing circumstances and client demands. AARs serve as a platform for 
deliberating insights gathered from clients (in this case, the users of the online platform). For instance, 
the team learned that testimonials previous clients posted on the Facebook Page achieved the highest 
reach and engagement. This resulted in the creation of more content based on stories of successful 
family planning clients to motivate and encourage behavior change among women with unmet needs for 
FP. Figure 9 shows an example of content based on a successful client’s testimony that garnered 1,500 
likes and an inquiry from a follower.
24
Figure 9. Sample Content Based on Client Testimonials
During one of the P&R sessions, the team also discussed feedback that some clients may be intimidated 
by the use of English on the online platform. This led to a shift to conversational Tagalog as the 
platform’s primary medium for its automatic responses to ensure that clients can easily understand the 
questions and are comfortable in communicating with chat moderators. During a P&R session, the team 
also discussed the fact that the platform constantly received non-family planning-related health topics. 
To address this concern, the team devised a mechanism that would properly refer clients to appropriate 
agencies and organizations.
The monthly P&R session also provided an opportunity for the team to revisit the online platform’s
algorithm. However, there have been ongoing discussions in modifying the algorithm to accommodate 
other relevant audiences such as male members of households, as well as other topics related to 
adolescence and gender-based violence. Documentation of P&R sessions indicate that the platform is 
continuously evolving in response to expanding client needs.
Accountability. As part of the IP’s P&R system, the team also developed an FP Ayuda Express responsibility 
matrix to keep track of interventions throughout implementation. It includes the person or partner 
organization that is responsible for a particular task and the target date for completion. The team 
utilizes the matrix to determine the status of each activity assigned to a specific team member or 
partner organization. Subsequent P&R sessions are used to revisit previous agreements and check 
whether the team already implemented tasks and incorporated recommendations. 
One occasion where a P&R session helped enhance accountability occurred when the team found out 
that there were clients who prefer not to disclose personal information in the chat; that there were 
unclear queries; and that there were reports of delayed responses from dispatchers. To address these 
findings, the team assigned specific team members to fix the identified issues, which later led to 
enhanced efficiency and confidentiality on the online platform. 
Integration. The monthly P&R sessions included internal staff members as well as representatives from 
the DOH-CHD, who are familiar with the state of the local health system. Hence, these external 
stakeholders can help the implementing team identify ways to address challenges faced by clients in 
25
accessing family planning services in rural health units and barangay health centers. Moreover, their 
participation allows stakeholders to explore ways to incorporate the intervention into the health system 
or the school system, rather than implementing it only on an ad hoc basis.
Inclusivity. The monthly updates facilitated knowledge sharing among PACT members, allowing them to 
identify opportunities for engaging more potential family planning clients. PACT members included 
representatives from partner government agencies such as POPCOM Region V and DOH CHD Bicol 
Region, as well as responsible parenthood and family planning (RPFP) experts, clinical instructors from 
seven partner academic institutions in Bicol Region, chat moderators, and call responders. The team 
regularly conducted P&R sessions with diverse PACT members to garner their inputs and feedback, as 
well as to ensure their commitment to executing specific action points. For instance, specific feedback 
from chat moderators and clinical instructors allowed the team to revisit its algorithm for addressing 
chat inquiries and revise the chat guide to properly respond to these inquiries.
Sustainability. The involvement of academic partners, regional DOH and POPCOM, and students during 
monthly P&R sessions is important to promote inclusivity and to also generate a sense of ownership, 
which is important for sustaining and supporting the intervention beyond the project implementation 
period. Since these partners were actively included in both the implementation of the intervention and 
accompanying AAR sessions, there is a higher chance they will continue implementing the intervention, 
track progress, and adapt to changing circumstances, even with little support from USAID and 
ReachHealth. Lessons learned and best practices documented in the AARs during the implementation 
period can also be useful for future implementers as the FP Ayuda Express continues to expand.
5.3. Implementing Partner 2: RenewHealth - Expanding Access to Community-based 
Drug Rehabilitation
Overview of AAR Practices
AARs are part of RenewHealth’s processes for learning and developing collaborative solutions. They 
allow the project to realign its interventions (e.g., trainings) with changing circumstances and improve 
rapidly. RenewHealth conducts AARs in the form of post-session debriefings. Such AAR sessions 
provide an opportunity for staff members, partners, and communities to critically assess whether 
interventions and innovations are being implemented properly and impacting the project’s performance 
indicators (for instance, in relation to community-based drug rehabilitation). 
In general, the AAR process in RenewHealth comprises the following components: a post-activity 
evaluation with participants using a questionnaire; discussion of the results of the post-activity evaluation 
by internal staff; a post-session debriefing with internal staff, allowing them to share general experiences
with an intervention (i.e., what went well and what could be improved) ; and progress sharing with 
external stakeholders, including AAR outcomes.
Before the AAR Event 
Technical staff. Under the guidance of the chief of party (COP), the learning and development (L&D) 
officer manages RenewHealth’s training programs. The L&D officer is also responsible for conducting 
AAR activities after training activities, monitoring their progress, and sharing outcomes with the internal 
staff and external stakeholders. Meanwhile, since the M&E officer normally administers the online survey, 
they are also responsible for presenting the results and leading the discussion to analyze them.
Budget allocation. There is a budget allocated for CLA activities, which include workshops, field focus 
group discussions (FGDs), and conferences. While AAR activities are not mentioned explicitly, it can be 
assumed that they are also covered by the CLA budget. Because most face-to-face activities such as CLA 
26
activities were adjusted to online due to the COVID-19 pandemic, some of the budgets are also 
allocated to online technologies such as Zoom accounts, which are also used for AAR sessions.
Venue selection. The COVID-19 pandemic has had a substantial impact on AAR sessions. All AAR 
activities that were normally held in person have been switched to virtual meetings.
● Before the pandemic, AARs typically occurred shortly after a session or event had ended. Such 
was done for everyone's convenience and to ensure that feedback was obtained as soon as 
possible before staff members forget the specific details from the intervention that was being 
reviewed.
● During the pandemic, all AAR activities have been conducted using Zoom, an online platform 
that is widely used among organizations. The AAR activities generally take place during an 
allotted time, usually lasting for an hour. A copy of the recording is kept for future reference 
and is also shared internally.
Timing/scheduling. AARs are carried out immediately after an intervention has finished. It is essential to 
obtain feedback and insights from internal staff members while the memory of the activity is still fresh, 
allowing participants to recall what happened.
AAR design. Although there is no predetermined AAR design and written guidelines have not been 
developed, there is a generic structure that RenewHealth follows during post-session debriefings. The 
first part is a discussion of the post-activity evaluation result. Staff members are encouraged to comment 
on issues and challenges collected from the evaluation. This is followed by a discussion on what went 
well and what can be improved. Each staff member is asked for insights. The post-session debriefing 
wraps up with the identification of next steps (e.g., action points for improving the next activity), all of 
which are collectively agreed upon by the team.
Agenda setting. Since an AAR session is conducted right after a particular training event, there is no 
specific agenda prepared beforehand. However, a post-session debriefing usually follows the generic 
structure earlier described. It is a free-flowing discussion that mainly tackles results of the post-activity 
evaluation that is regularly collected by the end of an intervention. During the post-session debriefing, 
the team assesses the intervention in terms of challenges, issues, and improvements. Everyone is 
encouraged to give their insights during the discussion.
Participant selection. As described above, the AAR sessions are primarily internal staff meetings where 
results of the post-activity evaluation survey that was administered with training participants are 
deliberated. The only participation of external stakeholders such as training participants is through the 
survey, which is a one-way feedback process. Hence, there is no opportunity for a more interactive 
discussion or bidirectional exchange between project staff members and training participants. 
During the AAR Event
Facilitation. For the post-session debrief, internal staff members are normally asked a set of questions, 
including what went well, what didn't go well, and what else could be done better. Post-activity 
evaluation results are complemented with a presentation of data relevant to the intervention such as the 
number of invited participants who attended a training, and the number of drop-outs or those who 
were unable to complete the training. Reasons for such occurrences are explored during the discussion.
27
Participation. Post-session debriefings are conducted immediately after an intervention, while all staff 
members engaged in the intervention are still present and can participate in the discussions. A facilitator 
usually asks the participants to voluntarily share their insights on the evaluation results. To stimulate 
lively discussions, the facilitator utilizes follow-up questions and paraphrasing techniques. Sometimes, the 
facilitator calls on participants who they believe have not participated yet in the discussion.
AAR content. Generally, the results of the post-intervention evaluation survey administered to training 
session participants indicate what is working and what is not. Hence, these responses provide the basis 
for internal team discussions on what needs to be changed and improved for subsequent training 
interventions.
Prioritization. While there is no straightforward approach to prioritizing action points, suggestions from 
the participants are given a lot of attention and the utmost importance. For instance, the team was quick 
to adjust the session duration with speakers from one hour to two hours based on feedback from 
participants in motivational interviewing training. Meanwhile, other suggestions may require a longer 
time to be implemented such as requests for the development of materials using the local language. The 
time required for suggestions to materialize is also taken into account when prioritizing action points.
Task assignment. Particular tasks that emerge from the post-session debriefing are assigned to specific 
team members who possess the relevant skills and/or are responsible for the relevant task area (e.g., 
refining information, education, and communications or IEC materials, or addressing technical issues 
related to the CBDR online portal).
Documentation. The AAR session discussions are included in the minutes of the meetings. The lessons 
learned and highlights of the discussion are written in separate sections for easy reference. The draft is 
shared with the members of the team for refinement, recontextualization, and clarification. Once 
everyone’s inputs are garnered, the document is then finalized. 
Post-AAR evaluation. There is no indication that AAR sessions are also evaluated, either immediately or 
later.
After the AAR Event
Follow-up and monitoring. The L&D officer is responsible for monitoring progress in terms of the 
execution of action points that were agreed upon during the post-session debriefing.
Dissemination of AAR outcomes. While agreements made during the AAR session are shared with the 
internal staff, it is not a practice to share the results of an AAR activity with external stakeholders. 
Instead, the changes to the intervention (e.g., training) that were decided upon are automatically 
implemented and observed by future participants. Documentation of AAR outcomes are only kept 
internally as reference for further improvement of subsequent activities.
Reporting of progress. The L&D officer is primarily responsible for sharing progress with the team on 
implementing post-session debriefing action points. Occasionally, progress reports are also shared with 
external partners. These reports are normally prepared by the program development officer and 
approved by the L&D officer. For instance, the project team updated the Bureau of Jail Management and 
Penology (BJMP) regarding implementation of a facilitators training. The report included discrete 
information such as a list of participants who participated in the training as well as a list of those who did 
not complete the training. It also contained a discussion of the limitations of online simulation as well as 
feedback from trained coaches regarding what went well and what needed to be improved.
28 
5.4. Specific Intervention 2: RenewHealth’s Motivational Interviewing Training
Background of the Intervention
RenewHealth spearheaded the Motivational Interviewing (MI) Training project to assist LGUs in 
providing evidence-based treatment to people who use drugs (PWUD). MI is a counseling skill designed 
to evoke change talk among clients. This skill is vital for community facilitators implementing 
community-based drug rehabilitation (CBDR) interventions as they interact with their clients/PWUDs. 
Facilitators implementing the intervention must undergo training/simulation, where they need to 
demonstrate basic competence in MI and facilitation skills.
Currently, there are no internationally-accredited MI trainers in the Philippines. This project aims to 
build a cadre of MI coaches and facilitators. RenewHealth developed a learning management system to 
deliver the MI course. The course consists of four modules:
1. Introduction to Motivational Interviewing 
2. Motivational Interviewing Fundamental Skills: Questioning and Reflective Listening 
3. Motivational Interviewing Fundamental Skills: Reflective Listening & Rolling with Resistance 
4. Motivational Interviewing Fundamental Skills: Evoking Change Talk & Readiness for Change
MI was initially embedded in the five-day training program, named KKDK-CBDR (Katatagan Kalusugan at 
Damayan ng Komunidad). Training was delivered face-to-face before the COVID-19 pandemic, which 
subsequently restricted travel across sites and limited the size of meetings and trainings. These 
restrictions made it difficult for project team members to go to sites to pilot-test, train, and observe 
community facilitators. As a result, MI training is currently delivered via the CBDR Learning Portal, 
developed by RenewHealth. The program utilized a blended design with four hours of asynchronous 
training and four hours of synchronous training. The asynchronous modules are designed to deliver 
knowledge, whereas the synchronous training is used to enable participants to practice skills in MI. The 
roll-out strategy is depicted in Figure 10. 
Figure 10. General Phases of KKDK-CBDR Training, Including MI
MI training has four levels of evaluation. This ensures the quality of training and allows for gauging if 
there is a need to incorporate adaptive changes for the next training . Levels 1, 2, and 3 are conducted 
immediately after each training.
● Level 1: Reactions - This level is composed of quantitative and qualitative components. For the 
quantitative part, participants are asked to use a Likert scale to rate the training’s usefulness, 
ease of learning, visual impact, and ease of use. On the other hand, the qualitative component 
collects opinions from the participants on which content is most useful and asks if they have any 
suggestions for improvement.
● Level 2: Learning - This level gauges the acquisition of knowledge by comparing pre- and post￾tests taken by participants.
29 
● Level 3: Behavior - This level checks the confidence level of the participants based on a self￾assessment before and after the training.
● Level 4: Outcomes - This level is conducted three months after the training to check participants’ 
retention of knowledge and skills, as well as if they have effectively applied what they learned in 
the real world.
After-Action Review as Applied to the Intervention
The AAR conducted for MI training is a post-session debriefing. After the end of the training, an online 
survey is immediately sent to the participants. The survey essentially covers levels 1, 2, and 3 of the MI 
evaluation. It is an anonymous survey to encourage everyone to participate and provide honest 
feedback, which is later used for the post-session debriefing with internal staff.
The post-session debriefing usually takes place right after the end of training. It generally follows the
structure shown in Figure 11. The M&E officer leads these sessions. Attendance at the training is first 
discussed. Specific indicators are presented, including the number of invitations sent, the number of 
people who actually participated, the drop-out rate, and the completion rate. This is followed by a 
review of results from the post-evaluation activity. Staff members are encouraged to provide comments 
and suggestions on how to address the issues and challenges that were identified from the survey 
results. Lastly, a general discussion on the activity is triggered by a question: “What went well and what 
can be improved?” All the staff attendees are encouraged to give their insights on what could have been 
done better and how this can be applied to the next training session. 
Figure 11. Steps in the AAR Process for MI Training
Implementation of AARs in the form of post-session debriefings that utilize results from post-evaluation 
surveys contributed to the gradual improvement of MI Training. Figure 12 shows some milestones in the 
training events and their accompanying AARs, as well as the changes instituted in subsequent trainings. 
For instance, in batches 1 and 2, the need to increase interaction time with the lecturer was repeatedly 
mentioned in survey feedback. This feedback was no longer observed in batches 3, 4, and 5, which may 
indicate that the issue had been addressed by then. 
30
Figure 12. AAR Sessions Held During the Implementation of MI Training
While several issues were resolved through repeated AARs, a few concerns also remained and were 
repeatedly raised in all batches through the survey. Examples of these concerns included the need to 
provide more realistic scenarios and the need for a Tagalog version of the video material to make it 
easier to understand. These issues may have not been successfully addressed right away given that there 
is only a one-month period between training batches. Revising the learning materials comprises a 
complex process of drafting and testing before fully integrating a new version into the CBDR online 
portal for training use. This requires time, effort, and resources
Impact of AARs on Implementation Outcomes
Fidelity. Initially, the intervention was slated to be implemented in-person, but because of the COVID-19 
pandemic, it had to be adapted to online mode. Thus, it can be said that that fidelity may have 
significantly decreased in relation to the intervention’s original pre-pandemic design. However, while 
delivery may have changed, the actual purpose and content of the intervention remained unchanged. The 
online version is still patterned after the original KKDK-CBDR training. This version was gradually 
improved to suit the demands of training participants through repeated post-session debriefings on the 
results of post-evaluation surveys. Nonetheless, the post-session debriefings also ensured that the team 
and the intervention remained faithful to the intervention’s initial overall goals. Moreover, despite 
unpredictable circumstances, the training series proceeded as scheduled. 
Adaptation. The shift from in-person to online mode presented new challenges that were unfamiliar to 
training participants. Hence, AARs are essential to ensuring the intervention adapts to changing 
circumstances. One major modification that was made based on the results of the AAR/post-session 
debriefing pertains to the length of the practice session. The session was stretched to two hours from 
its original one-hour length. After the practice session, an open forum was also added to ask participants 
about issues related to the online portal and immediately troubleshoot their concerns. Furthermore, 
orientation on the use of and access to the portal was also lengthened to allow a detailed walk-through 
with the participants. 
31
Accountability. The post-session debriefings also motivate the team members to be responsive to the 
demands and needs of their clients (i.e., training participants). Responsiveness is a hallmark of 
accountability and a reflection of a commitment to quality. For example, the team heeded the 
participants’ suggestion to extend the practice session from one to two hours. AAR sessions and 
participant surveying aimed at improving training processes also likely had a positive effect on trainers’ 
motivation to recognize and pursue ways to improve their personal effectiveness. Notably, facilitators 
consistently obtained scores of 4.5 out of 5 in all training batches. 
Integration. Community facilitators are the primary health system agents for delivering CBDR services to 
the community. Through repeated AAR sessions and evaluations by participants, the MI training has 
been continuously improved over time. The hope is that better training will produce better community 
facilitators, who are embedded in the local health system. However, since AARs in this case do not 
include participants from government partners, there may be little discussion conducted regarding how 
to eventually integrate the MI training program into the local government’s CBDR programs.
Inclusivity. After each training, the team holds post-training huddles or debriefings to discuss what went 
well and what needed improvement. While the feedback from participants is utilized during the post￾session debriefings, they are not included in the actual discussions by the team. Other local government 
partners are also not included.
Sustainability. Similar to integration, the instructional design that has been improved over time through 
repeated AAR sessions will hopefully produce well-equipped community facilitators who will continue to 
provide CBDR services in the community. However, the absence of local government partners in the 
AAR sessions may limit the potential for the training to be sustained and owned by them in the long 
term. It is important to note though that RenewHealth has more than two years remaining; thus 
discussions about long-term ownership and sustainability can still be initiated.
5.5. Implementing Partner 3: TB Innovations and Health Systems Strengthening 
(TBIHSS) Project
Overview of AAR Practices
AAR processes in TBIHSS are embedded in the entire project life cycle. Two types of AARs are 
practiced in TBIHSS: topic or theme-based P&R sessions, and periodic P&R sessions that are held as part 
of routine internal team meetings.
The topic- or theme-based AARs are triggered, for instance, by policies that are being introduced by the 
government. According to TBIHSS staff members, the passage of the Universal Health Care (UHC) Law 
in 2019 was a major trigger event, pushing the project to organize P&R sessions. This is important 
because the IP must align its agenda and activities with the government's emerging priorities. On the 
other hand, internal review meetings trigger the implementation of P&R sessions to assess progress, 
achievements, and challenges. In this case, the purpose of an AAR is to brainstorm strategies to further 
strengthen progress already achieved while planning and executing course correction strategies to 
manage emerging challenges. 
Before the AAR Event 
Technical staff. The strategic information (SI) team is responsible for conducting AAR activities. It leads 
the coordination of TBIHSS' P&R sessions, for both internal and external events. The team consists of 
representatives from three units: the M&E unit; the operations research unit; and the knowledge 
management unit. There is no single person responsible for conceptualizing the P&R sessions. Instead, 
team members collaboratively work to plan, manage, and coordinate the sessions.
32
Budget allocation. AAR activities are budgeted during the annual work planning process. The budget 
includes a contingency fund for unplanned events as well as all pre-planned AAR events for the financial 
year. The budget includes an allocation for software and other technologies for enabling remote 
meetings, publications for documenting P&R sessions, field trips for project monitoring, and fringe 
provisions such as lunch for in-person P&R sessions. 
Venue selection. The COVID-19 pandemic significantly affected venue selection. Generally, all P&R 
sessions that are usually held in-person needed to be changed to virtual meetings beginning in early 
2020. 
● Before the pandemic, the paramount consideration in choosing a venue for AAR events was 
finding a location that could guarantee the attendance and safety of participants. Sometimes, the 
participants were also consulted on their preference for the venue. There were times when 
AAR activities were held in the TBIHSS office, DOH central and regional offices, and rented 
facilities where interventions were being conducted.
● So as to continue conducting AARs during the pandemic, session organizers made two types of 
adaptions that address mobility restrictions: (1) investing in digital tools such as Zoom and 
online polling application software, and (2) orienting and building the capacity of partners and 
staff members on using remote technologies.
Timing/scheduling. The scheduling of AAR activities adopts a more iterative approach. Rather than a 
“one-size-fits-all” approach, AAR scheduling is based on the stage of activities under review. This 
includes considering whether an immediate review is pertinent (i.e., if there is an issue that needs to be 
urgently addressed) and the level of invited participants (e.g., including busy senior officials may make 
scheduling more difficult).
AAR design. A predetermined template is used for TBIHSS’s monthly technical team review-cum-planning 
meeting, an internal meeting that has a strong AAR component. Other teams in the organization
participate in these meetings. Having such a template allows structured and standardized documentation 
of the agenda and the proceedings. In the meeting minutes they reviewed, the CLAimHealth study team 
found that TBIHSS technical team leads are expected to provide updates, discuss issues and challenges, 
garner comments and suggestions, and arrive at decision points. 
TBIHSS uses another template for an AAR involving external partners. It consists of six main 
components: rationale background, learning objectives, methodology, expected participants, expected 
output, and session flow. The learning objective directs the overall discussion of the meeting. It contains 
key learning questions and sub-questions that must be answered during the meeting. Documentation is 
crucial for following up and monitoring the action points that were agreed upon during the review and 
planning meetings.
Agenda setting. AAR session agendas are developed in consultation with different stakeholders such as 
internal project staff members, and local and national government partners. The overall aim of these 
sessions is to share knowledge and strategies on how to strengthen linkages in TB programs at various 
levels (i.e., national, regional, local, community). Some preparatory work is initiated to inform the AAR 
session’s agenda. For instance, an online assessment form hosted on Google Forms or Survey Monkey is
sent to participants of a recently-concluded training. The questions are both quantitative and qualitative. 
The qualitative portion elicits feedback on what the participants liked the most about the training and 
suggestions for further improvement.
33 
Participant selection
● For internal AAR sessions, invitees are usually the TBIHSS project staff, particularly relevant 
technical teams.
● For external AAR sessions, the following partners and counterparts are usually invited: 
Department of Health, particularly the National Tuberculosis Program, as well as other 
relevant bureaus such as the Bureau of Local Health Systems and Development (BLHSD), 
the Disease Prevention and Control Bureau (DPCB), the Epidemiology Bureau (EB), the 
Health Facility Development Bureau (HFDB), the Health Policy Development and Planning 
Bureau (HPDPB), and the Health Promotion Bureau (HPB); the Philippine Health Insurance 
Corporation (PhilHealth); DOH’s Regional Offices/Centers for Health Development (CHD); 
Provincial Health Offices (PHO); USAID/Philippines’ Office of Health; implementing partners
of other USAID/Philippines OH activities; and private sector partners such as professional 
medical societies and the Philippine Coalition Against Tuberculosis (PhilCAT); manufacturer 
of TB diagnostics (e.g., Cepheid)
During the AAR Event
Facilitation. Two approaches are often used to stimulate discussion during the AAR activity: open-ended 
trigger questions and presentation of quantitative data. These approaches ensure an in-depth discussion 
for exploring the root cause of a problem during the review.
Open-ended trigger questions are prepared and used during the plenary sessions to encourage 
participants to open up and share their thoughts and opinions. For more fact-based deeper discussions, 
quantitative data is used to trigger discussions and reflection on current performance. Such data-based 
information encourages participants to dig deeper into the topics. During the COVID-19 pandemic, 
AAR sessions needed to shift to virtual meetings. Hence, participants were actively engaged using online 
polls for open-ended discussions during plenary sessions.
Participation. Part of the AAR’s organization plan is to assign a timekeeper so that the session adheres to
schedule. It was noted that timekeeping during in-person events was relatively better controlled than 
virtual sessions. Proper time allocation ensures that all participants can actively participate to express 
their thoughts and opinions, and obtain valuable information from others. In addition and as a backup 
plan, online platforms such as a Google Sheets are opened during the plenary session for individuals who 
prefer to offer their opinions in writing. Such virtual technologies also help the AAR session remain on￾schedule. 
AAR content. Conducting AARs is part of TBIHSS’s commitment to building a culture of learning within 
the organization. Central to this is reviewing lessons learned from both successes achieved and 
challenges encountered. Hence, implementation failures and the factors behind them are continuously 
assessed in AAR sessions. Lessons learned from adversities as well as their underlying reasons were 
carefully analyzed based on solid evidence and rather than mere anecdotes. This approach helps inform 
the development of adaptive measures to avoid adverse consequences and improve project 
performance. Sustained documentation of AAR sessions, including main discussion points and key 
decisions, has led to the creation of a repository of good practices and lessons learned that also feeds 
into subsequent AAR activities.
Prioritization. The Eisenhower Decision Matrix (Figure 13) is used to prioritize the long list of action 
points that come from an AAR session by categorizing them according to their urgency and importance.
34
Figure 13. Eisenhower Decision Matrix
Task assignment. Minutes of external AAR sessions did not clearly indicate who was responsible for 
action points and next steps. Meanwhile, for internal AAR sessions, tasks were assigned according to the 
individual or team's specialty. For instance, tasks related to community engagement and demand 
generation of TB testing from one AAR were assigned to the demand generation team.
Documentation. As part of the AAR organization plan, a dedicated person is pre-assigned to document
the salient points from the session. To support this process, informed consent is obtained at the start of 
the event to audio-record the sessions.
Post-AAR evaluation. There is no indication that AAR sessions are also evaluated, either immediately or 
later.
After the AAR Event
Follow-up and monitoring. Individual teams regularly monitoring specific action points. Meanwhile, overall 
progress is monitored and discussed during subsequent meetings. Additionally, the M&E team monitors 
the quantitative aspect of implementation progress, comparing data about past performance and current
status.
Dissemination of AAR outcomes. A few days after the event and AAR session, documentation is shared 
with participants through e-mail or web link. When AARs are published, a printed copy of documents 
such as case studies or communication materials is delivered to the participants.
Reporting of progress. As a standard practice, progress in implementing action points from AAR sessions 
is routinely reported back to stakeholders. The progress report is initially shared for comments from 
stakeholders in advance of the next activity and its accompanying AAR session. This practice makes
them aware of the project's status and helps them prepare pertinent points they would like to raise 
during a future governing council meeting.
35
5.6. Specific Intervention 3: TBIHSS’s Philippine Private Sector Diagnostics Consortium 
(PPSDC)
Background of the Intervention
In 2018, the Philippines made a commitment to find and treat around 2.5 million Filipinos with TB by 
2022. Currently, one in three Filipinos with TB prefers to seek testing and care from the private sector. 
However, the costs of private sector TB services are catastrophic for Filipino households with an 
average income. GeneXpert testing, one of the globally recommended primary diagnostic tests for TB, 
costs an average of $150 when performed by private facilities. To address this gap in health services, 
TBIHSS spearheaded the creation of the Philippine Private Sector Diagnostics Consortium (PPSDC) in 
2019, a platform that allows TB testing to be made available to the general public at a cost that is 70
percent below commercial pricing.
Convened by the Philippine Coalition Against Tuberculosis (PhilCAT), the PPSDC comprises various 
private hospitals and laboratories, suppliers, distributors, and procurement agencies in the Philippines. 
The aim of PPSDC is to ensure quality TB diagnostics are made available to the general public at 
affordable prices, particularly at concessional pricing through pooled procurement with accredited 
laboratories and hospitals. To ensure PPSDC’s success of in achieving its objectives, the Governing 
Council was established to oversee the consortium. 
Formalized through an MOU, PPSDC has so far invited Cepheid and Macare to join the Consortium for 
GeneXpert systems and cartridges, and established Philippine Pharma Procurement, Inc. (PPPI) as the 
default procurement agency. It also has an MOU with the DOH and the Research Institute of Tropical 
Medicine (RITM), which oversees the National TB Reference Laboratory (NTRL). This has served to 
formalize recognition of PPSDC members as members of the country’s TB laboratory network, and as 
members of their respective local laboratory networks at the regional and provincial/city levels. 
After-Action Review as Applied to the Intervention
The Governing Council conducts a periodic meeting, which TBIHSS considers to be the PPSDC’s regular 
AAR activity. The schedule of the Governing Council’s meeting (i.e., AAR session) is set regularly on a 
quarterly basis. It is attended by the PPSDC’s members, which include both the private and public 
sectors. Since the time of PPSDC’s inception, eight council meetings have been conducted. These 
meetings provide an opportunity for members to look at current project activities and critically assess 
them to identify key challenges and gaps. The meetings serve as a platform for each member and partner 
to contribute evidence and provide criticism for further improvements.
Meetings follow a general structure that involves the following: checking attendance; presenting the 
agenda; giving a brief recap on the previous council meeting; giving updates on membership, 
procurement, and the number of tests conducted; discussing other concerns; and wrapping up by 
presenting the next steps that were identified during the meeting. As the meeting progresses, questions 
and suggestions are encouraged from the participants to stimulate active discussion.
36 
Figure 14 shows PPSDC’s implementation period, which includes Governing Council meetings 
conducted since the beginning of the project. The findings from Governing Council meetings were 
helpful, particularly for increasing the number of members. During the second council meeting, it was 
noted that a strategy for membership expansion needs to be developed. Immediate actions agreed upon 
during these meetings included setting the criteria for targeted membership expansion, and mapping the 
level 2 and 3 hospitals and laboratories in Regions 3 and 4 to be recruited. Over time, new members 
have been added at every council meeting. By the end of 2021, PPSDC membership had increased to 21 
private hospitals and laboratories. Fourteen (14) PPSDC members participated in nine pooled 
procurements for 18 GeneXpert machines and 16,450 cartridges. These initiatives helped lower cost in 
relation to the regular commercial price, ultimately making TB testing more affordable to indigent 
patients in the country.
Within this framework, PPSDC members participate in training on GeneXpert systems, biosafety, and 
recording reporting procedures. They also participate in post-training competency assessments as well 
as data quality assessment activities organized by the NTRL in coordination with the regions. To date, 
136 personnel from member labs have been trained. Of these, 35 staff members from seven different 
laboratories achieved a 100 percent score on the competency evaluation. One data quality check 
(DQC) event was organized to identify challenges, best practices, and thematic areas for improvement. 
Impact of AARs on Implementation Outcomes
Fidelity. No significant deviation from PPSDC’s original vision and design was observed over time. Hence, 
it can be said that there is a high degree of fidelity in this intervention. The changes that occurred in the 
Consortium could be viewed as enrichment to the original design rather than a major pivot since the 
actions that were agreed upon during Governing Council meetings were geared towards PPSDC’s 
sustainability and expansion. 
Adaptation. Since the PPSDC is still in its infancy, its internal team and external stakeholders agree that
organizational and process improvements will be required, especially to achieve the Consortium’s goal 
of expanding its membership. Documentation from Governing Council meetings indicates numerous 
adjustments that were agreed upon in order to recruit more partners and adapt to changing 
circumstances. This included, for example, shifting GeneXpert training from in-person to online delivery 
due to COVID-19 restrictions.
Accountability. During the third Governing Council meeting, the roles and responsibilities of PPSDC 
members were clearly defined and delineated, which has helped enhance accountability within the 
Consortium. The attendance of Consortium members is also checked at every meeting because their 
presence and participation are key expectations. Additionally, members actively participate in discussions 
and give their insights, which demonstrates their commitment to exercising their roles. All action points 
are carefully documented, indicating the individuals and organizations that are responsible for carrying 
out agreed-upon tasks, which is vital for accountability building.
Integration. The initiative to form the PPSDC began as part of TBIHSS’ technical assistance to increase 
support to the private sector in addressing TB in the country. One of the main goals is to ensure the 
initiative operates in conjunction with the National TB Program’s goals. The PPSDC’s operations are 
anchored in the Philippine Strategic TB Elimination Plan (PhilSTEP) as well as the Public-Private Mix 
(PPM) national action plans. Because of this, reaching out to the local health system is easier during the 
implementation process. The MOU between PhilCAT and DOH to formally introduce PPSDC members 
to the different regions is another key agreement that aids in enhancing integration. 
37
Figure 14. AAR Sessions Held During PPSDC’s Implementation Period
38
Inclusivity. Since the project's inception, PPSDC has been constituted by hospitals and laboratories from 
the public and private sector. Individually, they have no power to negotiate directly with suppliers and 
distributors to lower commercial pricing for GeneXpert machines and cartridges. During the third
Governing Council meeting, a strategy was developed for membership expansion in level 2 and 3 
hospitals and laboratories in Regions 3 and 4A to join PPSDC.
Sustainability. The council meetings always take into account PPSDC’s sustainability beyond the TBIHSS 
project. One aspect that has been considered is directly involving members in operations while the 
project is ongoing. The four committees created as part of the Governing Council included 
representatives of different laboratories and hospitals. In this manner, members develop a sense of 
ownership towards PPSDC during project implementation, increasing the chance of sustainability in the 
long run. By design, 2 percent of each cartridge’s purchase price is used to subsidize the Consortium’s
operational budget, thus serving as seed money to sustain its functions beyond project support.
39 
6. DISCUSSION
USAID has promoted the CLA approach by developing a design for conducting AARs through the 
Agency’s Innovation Lab. The design is available to all IPs (USAID, 2006) but is not a strict requirement. 
IPs are allowed to design their own approach or use other AAR designs such as the ones their 
respective organizations have developed. For instance, TBIHSS is guided by the AAR designs that its 
implementer, University Research Company (URC), developed.
Generally, AARs are conducted in different forms because IPs do not abide by a standard definition of an 
AAR. In fact, not all IPs are familiar with the term ‘AAR.’ However, when AARs were described to the
IPs in this study, they recognized them as a familiar process that their organization conducts in various 
ways. For example, TBIHSS and ReachHealth conduct AARs in the form of P&R sessions, while 
RenewHealth utilizes post-session debriefings after each activity. AAR styles also differ across the three 
IPs. Although TBIHSS and ReachHealth conduct P&R sessions, each has its own way of executing them. 
TBIHSS conducts P&R sessions that can either be topic- or theme-based, and are either triggered by 
new government policies or are periodic (i.e., prompted by routine internal meetings). Meanwhile,
ReachHealth P&R sessions come in the form of Mid-Term Technical Reviews and Program 
Implementation Reviews; monitoring and evaluation meetings for specific interventions (such as FP Ayuda
Express); or one-on-one review sessions with the person-in-charge of interventions. Moreover, 
RenewHealth AARs gather feedback from training participants/attendees, which the IP’s team 
subsequently analyzes through post-session debriefings. 
Despite differences in terminology and methodology, IPs’ AARs share the common purpose of
strengthening and continuously improving the implementation of activities and interventions. AARs 
involve different partners in identifying and assessing enabling factors as well as barriers, gaps, and 
challenges. They then allow the IPs to plan and execute course correction strategies to manage 
emerging challenges. The repeated AARs also eventually help achieve a project’s ultimate goal through 
continuous process improvements. Even if there are different interpretations of what an AAR is, the 
different interventions reported by and observed in the three IPs remain faithful to the definition and 
characteristics of AARs, as described in existing toolkits.
6.1. The 7 P’s of AARs in the IPs
Period. The timing of AARs across the three IPs differs from immediate to routine. On some occasions 
such as a training event, an AAR is conducted immediately afterwards because all key participants are 
present, and can provide feedback and insights while their memories are still fresh. Meanwhile, a success 
factor for more routine AAR sessions is the availability of key partner stakeholders, especially those 
who hold key positions in organizations and government agencies.
Place. Before the pandemic, organizers selected AAR venues based on convenience to the participants to 
better guarantee their attendance. TBIHSS often considered the preference of the participants, while 
ReachHealth and RenewHealth often held their AARs in the venue where the intervention being 
reviewed had been held. However, venue selection was significantly affected by the emergence of the 
COVID-19 pandemic in early 2020. Generally, all AAR sessions that were usually held in-person needed 
to be moved to virtual meetings, utilizing video conferencing platforms such as Zoom and Google Meet 
to ensure the safety of participants and team members.
People. Responsibility for the planning and implementation of AARs varies across the three IPs. 
RenewHealth specifically assigns this task to the learning and development officer. In TBIHSS, the 
strategic information team is responsible for preparing all internal and external AAR activities. The 
40 
members of the team collaboratively plan, manage, and coordinate the sessions. For ReachHealth, the 
responsibility for conducting the AAR goes to the leader of the intervention being reviewed.
When it comes to participants or attendees, AARs are not limited to the internal staff. A diverse group 
of people are invited, including representatives from various partner stakeholder organizations who are 
actively engaged in the activity or intervention being reviewed. However, the extent of their 
involvement varies across the IPs. In RenewHealth, participants in an intervention like a training event 
were asked to answer a survey, which the internal team later analyzed during their AAR session. When 
conducting a PIR with an AAR component, ReachHealth only invited those who played a key role in 
implementing the activity. 
Purpose. In general, AAR activities in all three IPs tend to focus on identifying challenges encountered 
during a certain implementation period; identifying potential solutions to known problems; and distilling 
lessons learned. The three IPs have slight differences in terms of developing specific agenda items. 
TBIHSS consults with different stakeholders about specific items that need to be added to the agenda 
prior to the AAR session. In RenewHealth, while there may be a general topic to be discussed, the 
actual discussion tends to be more free-flowing and unanticipated topics may also emerge. ReachHealth, 
on the other hand, observes a more structured hierarchical process. Usually, officers (e.g., the MERLA 
officer and provincial technical officer) will prepare the first draft of the agenda, which will be later 
reviewed by a higher body (e.g., a member of the senior management team or a regional officer). 
Preparation. The budget for AAR activities is taken from allocated funding for approved CLA/MEL 
activities. TBIHSS includes a budget for its AAR activities during the annual work planning process, as 
well as contingency funds for unplanned events. RenewHealth and ReachHealth have no specific budget 
line allocation for AAR activities. RenewHealth did not usually need a dedicated budget for AARs 
because they conducted them immediately after events. During the COVID-19 pandemic, all budgets for 
CLA activities were reduced. Moreover, budgets for CLA activities including AAR were focused on 
procuring software and other technologies needed for remote meetings.
IPs do not have written guidelines on how to conduct AARs. TBIHSS has different templates for both 
internal and external AARs, which are reflected in the documentation of past sessions. While 
RenewHealth and ReachHealth don’t use predetermined designs for their AARs, these IPs tailor designs 
to the activity or intervention they are reviewing. Common topics include issues or concerns, routine 
monitoring data, and lessons learned from a specific period of the intervention’s implementation.
Process. Proper use of time during an AAR session is crucial to ensuring active and inclusive engagement 
of the participants. In TBIHSS and ReachHealth, a timekeeper is assigned to ensure proper time 
allocation for the different parts of the AAR session. This practice helps ensure that every participant is 
given equal time to express their thoughts. To stimulate the discussion, different approaches are utilized 
such as open-ended questions and root cause analysis. A structured discussion on what went well and 
what didn’t go well is usually included in the agenda.
One good practice across the three IPs is using data during the AAR session. A presentation of data 
related to the activity of intervention being reviewed is initially used as a starting point for discussion. 
For instance, in RenewHealth, team members jointly analyze data generated from post-intervention
evaluations. This allows them to distill lessons learned from implementation shortcomings. Another type 
of data that is often used is learnings from previous AAR sessions, which help participants recall 
previous realizations and agreed-upon actions, and monitor progress over time. Using these different 
types of data, participants are then asked to come up with adaptive and innovative solutions, and identify 
risks to avoid adverse consequences in the future. 
41 
All discussions and agreements made during the AAR sessions are carefully documented. In TBIHSS, a 
dedicated documenter is identified in the AAR organization plan. Meanwhile, for RenewHealth and 
ReachHealth, a draft of the documentation is shared with team members for review and refinement, 
ensuring the accuracy of the data and insights captured.
Performance. The ‘performance’ of an AAR session (i.e., outcomes such as agreed-upon action points) is 
reflected in its main outcomes. In all three IPs, AAR sessions generate lessons learned and action points 
for improving implementation and solving problems. Since these outcomes may be numerous, IPs 
prioritize them according to their importance and urgency, and the level of effort required; often in an 
iterative and unstructured fashion. However, TBIHSS is the only IP that uses a matrix (the Eisenhower 
Decision Matrix) to identify the action points that are most urgent and require the highest priority. 
Another way to measure an AAR’s performance is by evaluating its actual implementation, and if
participants found it useful and satisfying. However, a post-AAR evaluation is not common practice 
among the three IPs, perhaps because it may be deemed unusual to ‘review a review.’
An AAR’s performance is also followed-up and monitored by a responsible team across the three IPs. 
For instance, in ReachHealth uses an AAR monitoring tracker to quickly check the progress of approved 
action points. Typically, IPs also report progress on implementing these action points to other external 
stakeholders, for instance by sharing a soft copy of the AAR session’s outcome. However, in the case of 
RenewHealth, AAR outcomes are only kept internally as a reference for further improvement of 
subsequent activities.
6.2. Achievement of Implementation Outcomes Related to the Implementation of AARs 
Fidelity. Among the three IPs, only TBIHSS has a pro forma template to guide AAR session flow. This 
helped their AAR session stick to the overall design they initially set. Similarly, the other two IPs’ AAR 
sessions closely adhere to the initial design, even though there is no predetermined template. When 
compared with existing guidelines such as the WHO’s AAR Planning Roadmap, the most apparent 
missing step in the AAR process of all IPs is post-AAR evaluation. The IPs do not typically conduct this 
step upon completion of an AAR session. 
Adaptation. During the COVID-19 pandemic, AAR interventions underwent a notable shift from in￾person to virtual sessions. Aside from that, IPs reported that they sometimes adjusted the time allotted 
for particular sessions to accommodate participants' insights. For example, in some instances, 
RenewHealth extended the post-session debriefing to allow more time for discussion and obtain more 
insights from the internal staff about the participants' suggestions for improvement, as reflected in the 
post-activity evaluation survey.
Accountability. All of the IPs carefully document AAR sessions, which allows them to record agreed-upon 
action points, including the names of staff members who are responsible or assigned to execute them. 
The progress of implementing action points is usually discussed at the subsequent AAR session. 
ReachHealth in particular developed an AAR monitoring tool for easily monitoring action points.
Integration. Rather than being an ad hoc activity, the AAR process is integrated into the IPs’ project cycle. 
The AMELPs of all three IPs also mention AARs (or P&R), embedding the practice in the organization's 
CLA system. Moreover, regular implementation of AAR-like sessions, either as part of an initiative with 
routine activities or as automatic aftermath of a specific event, may also indicate the level by which 
AARs are integrated into an IP’s operations.
42 
Inclusivity. While internal AARs include staff members who are relevant to the intervention under 
review, external AARs involve various people from outside the team; especially key external 
stakeholders such as partners from the government, private sector, and civil society. For example, 
TBIHSS usually has internal general meetings involving the organization’s different teams (e.g., Monthly 
Technical Team Review-cum-Planning Meetings, which have a strong AAR component). This allows the 
teams to discover for themselves what happened and why, and how to build on strengths and improve 
on areas of weakness, as well as explore ways by which they might collaborate more effectively. In terms 
of external AARs, ReachHealth conducts regional P&R sessions (combined with a harmonization 
workshop) that include representatives from partner government agencies.
Sustainability. AMELPs from the three IPs indicate that interventions for monitoring, evaluation, research, 
and learning are planned and budgeted. Budget is not necessarily specified for AARs, though it is 
presumably included in the overall AMELP. The presence of documentation of past AARs that were 
conducted repeatedly or routinely may also be reflective of the three IPs’ commitment to the sustained 
implementation of AARs as part of their CLA approach for continuous improvement.
6.3. AARs as Applied to Specific IP Interventions 
The case studies showed that, in the three specific interventions implemented by the selected IPs, 
incorporating learnings generated through AARs was an integral part of the process and of improving
implementation (Table 5). AARs also served as a major knowledge-sharing opportunity. Internal and 
external stakeholders assessed the implementation of each intervention (e.g., to gauge whether it was
aligned with the intervention’s original design), identify gaps and challenges that require adaptive actions, 
and determine best practices and lessons learned. Different forms of AARs were used to review the 
studied interventions: P&R sessions for ReachHealth’s FP Ayuda Express; post-session debriefings for 
RenewHealth’s motivational interviewing training; and Governing Council Meetings for TBIHSS’s PPSDC.
In all three interventions studied, the implementation of repeated or routine AAR activities allowed the 
implementing teams to address issues and concerns that emerged during certain implementation 
periods. Most concerns were successfully addressed by the team because adjustments that were agreed 
upon during an AAR session were accomplished within the agreed timeline. For example, improvements 
in the content and delivery of RenewHealth’s motivational interviewing training were immediately 
realized because of the team’s commitment to the adaptive actions that were decided upon during AAR 
sessions. Specifically, in the first and second batches, participants repeatedly mentioned a need to 
lengthen the time with master coaches for interactions. In response, the team extended the duration of
a particular session from one hour to two hours. This issue did not reappear in subsequent AAR 
sessions. Moreover, project teams also allocated time for discussion in subsequent AAR sessions to 
recall agreements made in previous sessions. This practice allows IPs to ensure that the adjustments that 
were previously agreed upon were successfully incorporated into the intervention. Furthermore, it 
helped monitor and assess whether these adjustments were effective in improving interventions. 
43 
Table 5. AAR Good Practices and Resulting Intervention Adaptations 
Across Three IPs
IP and specific 
intervention
ReachHealth’s Family 
Planning (FP) Ayuda 
Express
RenewHealth’s 
Motivational 
Interviewing Training
TBIHSS’s Philippine 
Private Sector 
Diagnostics 
Consortium (PPSDC)
AAR good practices Conduct P&R sessions 
with internal staff 
members and external 
stakeholders utilizing 
analytics obtained from 
online platforms 
Hold internal team post￾session debriefings to 
analyze post-training 
evaluation surveys 
Convene regular 
Governing Council 
meetings where 
Consortium members can
discuss the status of 
interventions and related
challenges
Intervention 
improvements/ 
adaptations
The language used in the 
chat guide was changed 
from English to Tagalog, 
and content was 
improved to highlight 
success stories
The length of orientation 
and practice sessions 
were extended in 
response to participants’ 
request
Improved strategies and 
criteria were agreed upon 
to continuously expand 
membership
It is also important to note that there were instances when issues and concerns re-emerged after being 
addressed. For example, the FP Ayuda Express team often received inquiries on non-FP related topics 
that were not covered by the virtual platform during its early implementation run. These issues and 
concerns were discussed during subsequent AAR sessions. Eventually, this led to the creation of a 
referral system to link these clients to relevant offices that could assist them. Therefore, it is important 
to continuously revisit the implementation process through AARs and assess which aspects need further 
improvement so similar issues and concerns no longer re-emerge.
6.4. Impact of AARs on Implementation Outcomes Related to the Intervention
Fidelity. Based on the case studies of the three specific interventions examined, there seemed to be no 
major deviations from the original vision of the interventions. Rather, the changes that have been 
instituted as a result of AAR activities are better viewed as enrichment to the original design. The AAR 
sessions also served as regular checks on whether the intervention is faithful to the initial plan and is 
achieving the goals that were established at the onset.
Adaptation. Common concerns of end-users were the primary consideration for modification and 
improvement. For FP Ayuda Express, for instance, testimonials from previous clients posted on the 
Facebook Page achieved the highest reach and engagement, which resulted in producing more content 
based on successful family planning clients. For motivational interviewing training, the team lengthened 
its allocated time for practice sessions to cover more topics. This was done in response to participant 
feedback that was collected through post-evaluation surveys. The COVID-19 pandemic also called for 
adaptive measures to continue implementation despite limitations and restrictions. For example, PPSDC 
shifted its interventions online by integrating its training platform to its Consortium website. 
Accountability. Team members and external stakeholders involved in an intervention are assigned tasks 
and responsibilities, and there are various mechanisms to ensure accountability and monitor progress 
44 
throughout implementation. For instance, FP Ayuda Express developed a responsibility matrix to track 
the execution of agreed-upon actions. This matrix includes the person or organization responsible for a 
particular action and the target date for completion. In PPSDC, the PPM specialist oversees overall AAR 
interventions (i.e., Governing Council meetings), including design and execution. In motivational 
interviewing training, facilitators constantly receive high marks on post-evaluation surveys, indicating 
their responsiveness to training participants’ needs. 
Integration. Securing the participation and support of local and national government partners helps 
ensure that these specific interventions are integrated into the health system. For instance, reaching out 
to the local health system is easier for PPSDC because its operations are anchored to the Philippine 
Strategic Tuberculosis Elimination Plan (PhilSTEP) and the Public-Private Mix (PPM) national action plans. 
It was also supported by an MOU between PhilCAT and DOH that introduces PPSDC members to 
different regions in the country. Improvement to motivational interviewing training will hopefully 
capacitate more community facilitators, who will be integrated into local health systems to manage 
substance abuse clients. In FP Ayuda Express, the consistent participation of representatives from 
regional DOH and POPCOM aids the implementing team in identifying ways to address challenges 
related to accessing family planning services, and in incorporating the intervention into the health 
system.
Inclusivity. Internal staff members and external stakeholders participated in AARs for specific 
interventions, emphasizing shared responsibility towards achieving common goals. For example, FP 
Ayuda Express’ P&R sessions served as an opportunity for knowledge-sharing across different 
stakeholders, and resulted in improvements to that intervention’s algorithm and mechanism. For 
motivational interviewing training, the feedback generated from training participants allowed the 
implementing team to identify points of improvement that were later applied to subsequent training 
events. For PPSDC, the inclusion of more private hospitals and laboratories was made possible by 
developing an inclusive strategy for membership expansion.
Sustainability. AARs are helpful in transitioning ownership for interventions to the government and 
securing needed resources, particularly budgets for continuous implementation. In FP Ayuda Express, 
lessons learned and best practices that were documented during the implementation—including the P&R 
sessions—are useful for expanding and developing similar platforms in new locations. For motivational 
interviewing training, local government has yet to be included in long-term sustainability discussions (e.g., 
via AAR sessions), but it is hoped that newly-equipped trainees will provide services to PWUDs and 
strengthen CBDR in the local health system. PPSDC’s AAR activities (i.e., Governing Council meetings) 
are always anchored to sustainability. With this mindset, members are directly involved in operations, 
allowing them to develop a sense of ownership towards the Consortium.
45 
7. CONCLUSION
This study of AAR practices among three IPs implementing the USAID/Philippines Health Project 
contributes to a deeper awareness and understanding of the AAR process, which is an essential 
component of the CLA approach. This study also documented examples of how AARs helped create a 
culture of learning within USAID/Philippines and helped improve the performance of public health 
interventions. 
IPs have different understandings of AARs and different ways of conducting them. This is not necessarily 
a problem, provided these activities still have the key elements of an AAR and fulfill the same purpose. 
The cases that the study team examined demonstrated how adopting AAR practices facilitates adaptive 
management, which is the basic purpose of AARs and the reason behind their effectiveness. Activating
rapid learning cycles allows IPs to immediately integrate lessons learned from failures and successes into 
implementation. This is an essential ingredient of iteration, improvement, and innovation. 
The AAR practices documented in these case studies revealed additional benefits beyond direct effects 
on adaptive management, learning, and process and performance improvement. One additional benefit is 
that AAR processes also help enhance engagement between different stakeholders, both internal and 
external. In the examples examined across the IPs, stakeholder engagement manifested in diverse ways, 
including garnering participant feedback through post-training surveys, co-designing meeting agendas, and 
active participation in consultative meetings. As a result, the outcomes of these AAR practices generally 
reflect the insights and experiences of a wide range of actors, which is a good indicator of inclusivity.
Thanks to their inclusive nature, AAR processes also help promote accountability and ownership for 
interventions, which is vital for its long-term sustainability, especially beyond a project’s implementation 
period. AAR sessions provide a platform for discussing how interventions can be integrated into the 
health system. Finally, the practice of AARs among IPs aids in enhancing the responsiveness of 
interventions to clients and users. These activities provide an opportunity for the voices of clients and 
users to be considered in reaching agreements on how to improve the delivery of public health 
interventions. 
46 
8. RECOMMENDATIONS
8.1. For USAID
USAID plays a crucial role in ensuring that AAR practices help strengthen the implementation of public 
health interventions by activating the PDSA cycle and enhancing adaptive management within 
implementing organizations. CLAimHealth recommends the following to optimize the adoption of AAR 
practices across USAID and its IPs:
● Clarify the definition of AAR, its types and dimensions, and existing tools and guidelines. 
● Explicitly and actively promote the use of AARs as part of the CLA approach to raise awareness 
and understanding, and increase uptake. 
● Create platforms for joint learning about AAR practices across IPs. 
● Encourage and support continuous documentation of AAR practices and their effectiveness 
through case study development and implementation research. 
8.2. For Implementing Partners
IPs are not ‘starting from scratch’ when it comes to adopting and practicing AARs. In fact, they are 
practicing them in different ways. To further expand and enhance the use of AAR among IPs, 
CLAimHealth recommends that IPs: 
● Adopt existing AAR design templates to ensure quality, enhance repeatability, and ease 
documentation and reporting. 
● Consider the use of existing tools for prioritization (e.g., the Eisenhower Decision Matrix), 
tracking progress (e.g., the Responsibility Matrix), and building accountability. 
● Improve documentation of AAR practices, agreements, and outcomes, including through case 
study development and implementation research. 
● Share good practices on conducting AARs with other IPs. 
8.3. For Implementation Researchers
The study generated rich insights about how different IPs conduct AARs and how AARs differ for
specific interventions. There are other arenas that this study only partially covered, which future 
implementation research can address. To advance scholarship on AARs, CLAimHealth recommends that 
implementation researchers: 
● Examine similar IPs/organizations/interventions, but with different exposures to AAR practices; 
and consider including a control that does not practice AAR at all. 
● Conduct more quantitative investigations to build evidence on the causal impacts of AAR 
practices on the ultimate outcomes of projects and interventions. 
● Gather the perspective of external stakeholders on how important and useful to them are the 
AARs conducted by the IPs (especially external AARs)
47 
REFERENCES
Baird, L.S., Holland, P., & Deacon, S. (1999). Learning from action: Imbedding more learning into the 
performance fast enough to make a difference. Organizational Dynamics, 27, 19-32. 
https://doi.org/10.1016/S0090-2616(99)90027-X\
CLIC. (nd). After action review (AAR) – evaluate all activities (CPS toolkit). Cumbria, United Kingdom: 
Cumbria Learning and Improvement Collaborative (CLIC).
https://www.theclic.org.uk/resources/toolkits/toolkit-2/appendices/tools-in-detail/a-module￾1/after-action-review
Graves, T. (2017). How to build continuous-learning into architecture-practice.
https://www.slideshare.net/tetradian/how-to-build-continuouslearning-into-architecturepractice
Hengeveld-Bidmon, E. (2015). An After-Action Review of a Research Project Team: A Teaching Case. 
Mack Center on Nonprofit and Public Sector Management in the Human Services. University of 
California, Berkeley. 
https://mackcenter.berkeley.edu/sites/default/files/an_after_action_review_of_a_research_project
_team.pdf
Keiser, N. L., & Arthur, W., Jr. (2021). A meta-analysis of the effectiveness of the after-action review (or 
debrief) and factors that influence its effectiveness. Journal of Applied Psychology, 106(7), 1007–
1032. https://doi.org/10.1037/apl0000821
National Police Foundation. (2020). How to conduct an after-action review.
https://www.policefoundation.org/publication/how-to-conduct-an-after-action-review/
Serrat, O. (2008). Conducting After-Action Reviews and Retrospects. Washington, DC: Asian 
Development Bank. https://www.adb.org/sites/default/files/publication/27570/conducting-after￾action-reviews.pdf 
Sexton, R. and McConnan, I. (2003). A Comparative Study of After-Action Review (AAR) in the Context 
of the Southern Africa Crisis. United Kingdom: ALNAP.
https://reliefweb.int/report/lesotho/comparative-study-after-action-review-aar-context-southern￾africa-crisis
Sorbello, S., Bossi, E., Zandalasini, C., Carioli, G., Signorelli, C., Ciceri, F., Ambrosio, A., Zangrillo, A., & 
Odone, A. (2021). After Action Reviews of COVID‐19 response: Case study of a large tertiary 
care hospital in Italy. The International Journal of Health Planning and Management, 36(5), 1758–
1771. https://doi.org/10.1002/hpm.3258
Taylor, M. J., McNicholas, C., Nicolay, C., Darzi, A., Bell, D., & Reed, J. E. (2014). Systematic review of 
the application of the plan–do–study–act method to improve quality in healthcare. BMJ Quality & 
Safety, 23(4), 290. https://doi.org/10.1136/bmjqs-2013-001862
UNDP. (2007). Knowledge management toolkit for crisis prevention and recovery practice area. United 
Nations Development Programme. https://reliefweb.int/report/world/knowledge-management￾toolkit-crisis-prevention-and-recovery-practice-area
48 
USAID. (2006). After-Action Review Technical Guidance. Washington, DC: USAID.
https://pdf.usaid.gov/pdf_docs/PNADF360.pdf
USAID. (2013). Pre-action review in Jordan. United States for International Development. 
https://jordankmportal.com/resources/ldlyl-l~-lmrj-at-lty-tsbq-tkhdh-ljrt
USAID. (2018). Facilitating Pause and Reflect. United States Agency for International Development. 
Retrieved from
https://usaidlearninglab.org/sites/default/files/resource/files/cla_toolkit_adaptive_management_facil
tiating_pause_and_reflect_final_508.pdf
USAID, (2021). ADS Chapter 201, Program Cycle Operational Policy. Retrieved from 
https://www.usaid.gov/sites/default/files/documents/201.pdf
USAID Learning Lab. (nd). Understanding CLA. https://usaidlearninglab.org/qrg/understanding-cla-0
WHO. (2019). Guidance for After Action Review (AAR). Copenhagen: WHO Europe.
https://apps.who.int/iris/rest/bitstreams/1212880/retrieve
Yin, R. (2018). Case study research and applications: design and methods. 6th Edition. USA: Sage 
Publications.
49 
ANNEX A. SURVEY QUESTIONNAIRE
General Questionnaire About AAR Practices
Questions Answers
AARS IN GENERAL
1 Can you describe the different types of AAR activities that are being 
done by your IP?
2 What kinds of activities trigger an AAR activity?
3 Can you share with us a list of all AARs you did for the past year, and 
kindly indicate the activity reviewed, date of both activity and AAR, and 
venue (including zoom)?
 
Kindly share the agenda or minutes of the meeting for each as well.
Kindly share photos of previous AAR activities as well.
 
4 Can you briefly share some experiences when your conduct of AARs 
led to successes?
 
How about failures – do you have experiences when, despite the 
conduct of AARs, a project or intervention cannot anymore be 
salvaged?
AAR PREPARATION
5 Is there a permanent individual or team that is always responsible for 
the conduct of AAR activities?
 
If yes, who is this individual, or who are the people usually included in 
the team?
 
If no, who leads the preparations for the AAR activity?
6 Do you allocate a regular budget for the conduct of AAR activities? Is 
this included in the annual budget?
 
If yes, can you share a copy of the budget?
 
If no, how are AAR activities budgeted?
7 What are your considerations in choosing a venue for the AAR activity, 
especially before the pandemic?
 
How about now during the pandemic – are your AAR activities all 
virtual? Briefly describe the situation
8 Are the AARs you conduct sufficiently timely and immediate to the 
event being reviewed?
50 
Questions Answers
If no, when is the AAR conducted relative to the activity being reviewed?
9 Do you have a predetermined AAR design based on your internal 
guidelines?
 
If yes, can we have a copy of the AAR design or guidelines? Perhaps 
you can also share some written agenda of previous AAR activities?
 
If yes, are there any instances when you cannot adhere to the design 
planned (before and during the COVID-19 pandemic)? Kindly share 
some examples – why did that happen?
 
If no, can you briefly describe how the AAR design is developed?
10 How is the agenda prepared prior to the conduct of AAR? Can you 
briefly describe it?
 
Can we ask for a copy of the AAR’s agenda – perhaps samples from 
previous AARs?
11 Who are usually invited to these AAR activities – any external 
stakeholders? How do you determine the set of participants to be 
invited for the AAR activity?
 
Can we have copies of attendance sheets from previous AARs?
ACTUAL CONDUCT OF AAR
12 Can you describe the overall design of your IP’s AAR activities? What 
happens during the AAR activity?
 
Kindly share copies of the agenda of previous AARs.
13 How do you stimulate discussion and reflection during the AAR? Are 
there trigger questions prepared on hand?
14 Does the AAR look beyond surface issues and encourage participants 
to keep ’digging’ to discover the real root cause of a problem? Kindly 
describe.
 
15 How do you manage the time for the AAR ensuring that all participants 
could actively participate and obtain valuable information?
16 How do you prioritize the action points/recommendations generated 
from the AAR?
 
17 During the AAR activity, how do you distill lessons from both successes 
and failures?
18 How do you check through the results of past AARs to identify any 
lessons that might be useful to the current AAR?
19 During the AAR activity, how do you identify who is responsible for 
which actions?
51 
Questions Answers
20 How do you document the salient points discussed during the AAR 
activity?
 
If yes, kindly share some sample documentation of AAR activity 
reports/minutes.
21 How do you assess the AAR event? Do you conduct a quality or audit 
check or elicit feedback from participants?
 
Kindly share documentation of feedback or evaluation of past 
AAR activities, if any.
AFTER THE AAR
22 How do you monitor progress on the recommendations and action 
points?
23 Do you report back to participants and stakeholders on progress made 
during follow-up? How?
24 How do you disseminate the outcomes of the AAR activity to other 
staff and stakeholders?
52 
Questionnaire About the Specific Intervention
Questions Answers
General Questions
1 Tell us a bit about this intervention – what is it all about? What 
are its components and activities?
2 How was the intervention conceived and designed?
3 How was the intervention implemented? What is its current 
status and outcomes so far?
4 Kindly share background documents about the intervention 
– examples include concept note and workplan (including 
original and updated versions), reports pertaining to 
implementation, reports regarding status and outcomes
AARs as Applied to the Intervention
5 This intervention was selected for this study because it 
underwent AARs throughout its implementation.
 
Kindly share agenda/reports/minutes of AAR activities
conducted throughout the implementation of this intervention.
6 Do you recall the first AAR conducted during the 
implementation of this intervention?
 
What were the key findings and recommendations of the AAR? 
Were the suggested changes incorporated into the 
intervention? 
 
Kindly share agenda/reports/minutes of this AAR activity
7 Do you recall the subsequent (i.e., second, third, etc.) AAR 
activities conducted during the implementation of this 
intervention? 
 
What were the key findings and recommendations of the AAR? 
Were the suggested changes incorporated into the 
intervention? 
 
Kindly share agenda/reports/minutes of this AAR activity
Assessing the Role of AARs in Achieving Implementation Outcomes
8 Since we are interested in AARs, can you tell us how the 
conduct of AARs modified or improved this intervention? What 
adjustments were made? How did your AAR process facilitate 
the adjustments for your intervention?
9 Meanwhile, can you tell us how your AAR process also helped 
the intervention to stick to its original implementation design?
53 
Questions Answers
10 Can you tell us how your AAR process compelled or motivated 
the staff to execute the agreed-upon adaptive actions? How 
about the other stakeholders in the health system?
11 In the conduct of AAR processes for this intervention, were 
diverse stakeholders within and outside the project included? 
Who are they? Kindly describe the experience.
12 Can you tell us how your AAR process also aided in ensuring 
the incorporation of the intervention into the health system 
(i.e., local, provincial, regional, national)?
13 Can you tell us how the AAR process helped ensure the 
continued implementation and adoption of the 
project/intervention even beyond the project’s timeframe? 
What are the indications? (i.e., budget, policy, manpower 
capacity, interest)
54 
ANNEX B. LIST OF DOCUMENTS REVIEWED
Document USAID OH Implementing Partner
ReachHealth RenewHealth TBIHSS
Annual Monitoring, Evaluation and Learning 
Plan (AMELP)
3 3 3 
Annual Report 3 5 2 
2019 Progress Report 2 1 0 
2020 Progress Report 3 3 2 
2021 Progress Report 3 3 3 
Work Plan 4 4 3 
Case Study 0 0 3 
Fact Sheets 0 0 5 
Activity Design / Agenda of AAR 13 0 1 
Health Project (HO) Research Undertaking 0 0 1 
TBHISS evaluation on Online Training for 
GeneXpert
0 0 1 
FP Ayuda Express Reach (Facebook, Chat 
Inquiries, POPCOM Helpline)
1 0 0