USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | i EVALUATION: USAID/PHILIPPINES HEALTH PROJECT MULTIYEAR WHOLE-OF-PROJECT PERFORMANCE EVALUATION (2018–2021) FEBRUARY 2022 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of USAID or of United States government. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | ii Cover Photo: The COVID-19 pandemic disrupted local health systems, including the provision of family planning (FP) services. The United States Agency for International Development (USAID) Health Project pivoted to help service providers develop adaptive solutions to address the challenges of this unprecedented time. This included providing FP informational material for health workers going house to house to reach more people who need FP supplies. Source: USAID/RTI International This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Orville Solon, Ma. Soledad Antonio, Frances Mamaril, Mario Festin, Kathryn Roa, Julienne Baldo-Cubelo, Ermi Amor Figueroa-Yap, Fatima Verzosa, Noel Juban, Oscar Picazo, Arturo Ongkeko Jr., and Chamuel Michael Joseph Santiago. USAID Contract No. IDIQ No. AID-OAA-I-15-00025, Task Order No. 72049218F00001 Period of Performance: July 2021–February 2022 Contracting Officer’s Representative (COR): Helen Hipolito USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | iii ACKNOWLEDGMENTS We express our gratitude to those who shared their expertise, experience, and ideas, which were instrumental in helping us complete this evaluation report. We appreciate their patience and willingness to participate in our virtual interviews. We also extend our thanks to the officials of the Department of Health and its attached agencies, the local government units, United States Agency for International Development /Philippines, and other development partners for facilitating this evaluation by helping us schedule the key informant interviews, allowing us access to reports and data, and providing us insightful comments and suggestions that helped shape this report Finally, we thank the patients and service providers who participated in the focus groups for sharing their valuable experiences, expectations, and personal thoughts about the country’s health care delivery system. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | iv PREFACE The whole-of-project evaluation (WOPE) of the United States Agency for International Development (USAID)/Philippines Health Project (HP) comes at an important juncture: The Philippines—indeed the entire world—is still reeling from the COVID-19 pandemic, which has severely disrupted health services and derailed ongoing health reforms and innovations, including the 13 USAID-supported health activities covered under the WOPE. Meanwhile, a new government will take over in June 2022, after the national elections in May. The findings of the WOPE, therefore, are expected to provide insights on how USAID should configure its assistance to the sector in the remaining life of the activities and for its new program cycle (2024–2026), and also offer an independent assessment of the host government on how USAID-supported interventions have performed. I. Setting the Scene The COVID-19 pandemic has overwhelmed the Philippine health system and economy. As a middle-income country, the Philippines has struggled to bring universal health care to its people, and the grim impact of COVID-19 on service delivery and financing will make this dream even more challenging. Despite this cloudy outlook, strengths and opportunities exist that could allow the health sector to regain the traction that was lost as a result of the pandemic. First, a new government will be installed in June 2022, after the national elections in May. Second, the Mandanas-Garcia ruling is expected to infuse more budgetary resources into local government units (LGUs). Third, the economic recovery is expected ensure this year, and prospects for growth in the country are good. Fourth, the insurgency problem, which has bedeviled many rural areas and halted local health sector development and citizens’ access to basic health services, is predicted to end soon, or at least be severely mitigated, bringing normalcy to formerly strife-torn communities. Although strengths and opportunities exist for the health sector to be revitalized and made more resilient, long-term weaknesses and threats that have hindered health sector development should get more attention, principally from the national government and ultimately from all health sector stakeholders. First, institutional development and staff skills have lagged behind the requirements of a modern health system that a middle-income country (and hoped-for upper-middle-income country) requires. Important skills in health leadership, health policy analysis and dialogue, program management, and key highly technical areas are increasingly in short supply. Technical areas include health technology assessment; social health insurance; impact and evaluation; health planning, budgeting, and costing; information systems; informatics; and data analytics. Second, organizational development, especially at the Philippine Health Insurance Corporation (PHIC or PhilHealth), has lagged behind because of frequent institutional disruptions. PhilHealth is envisioned to be the principal funder of the health system in the future and, therefore, a potential major player, especially in interventions including tuberculosis (TB) treatment, some family planning, and reproductive health (FP/RH) procedures, drug rehabilitation, and treatment of infectious diseases such as COVID-19. Third, the uncertainty and lack of clarity over who pays for what services, commodities, and health personnel has been a lingering issue for years and continues to result in suboptimal allocation of resources, especially at the health facility level. Fourth, a few critical policies and program impediments remain that directly inhibit wider, more sustainable USAID HP performance. For instance, the Responsible Parenthood and Reproductive Health limits the USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | v teaching of RH and access to contraceptives among adolescents (below the age of 18 years) without parental consent. These upstream (higher-level) issues need to be resolved or at least eased before the Philippines can realize sustained, downstream (lower level) results. These obstacles directly impinge on the performance of the USAID HP’s 13 activities that the WOPE team evaluated. Although these factors are outside the scope of the WOPE, we invoke them as reminders that performances do not happen in a vacuum and that any theory of change (ToC) has critical assumptions that need to hold for sustained positive project performance to occur. The USAID HP’s ToC posits that “strengthening key aspects of the health system will contribute to the health of underserved Filipinos, and the overall health profile of the country will improve.” There are two key assumptions of the ToC that USAID did not fully meet. First, the Project Approval Document assumed that “public sector funding levels at the central and regional levels will be maintained at current levels and increased throughout the life of the project.” Budget appropriations to the health sector were indeed maintained and even increased, but disbursement rates left much to be desired, especially in key interventions. The disbursement rate of the National TB Program in 2021, for instance, was only 54 percent. Also, USAID assumed that “health services will be continued or resumed during periods of natural disaster or political unrest.” However, the global COVID-19 pandemic proved to be far more serious than any health disaster ever imagined. Thus, although the project hypothesis remains generally valid, USAID must adjust a few key elements to make future engagements more attuned to changing realities. II. Conclusions on the WOPE We can draw seven conclusions from the WOPE findings. First, although there were significant project achievements, there were also significant performance gaps compared to the life-of-activity (LOA) targets. The USAID HP met only around half of its LOA targets. Low performance (less than 50 percent of achievement of LOA target) was reported for: couple-years of protection in project sites (47 percent), number of new FP acceptors (28 percent), and number of individuals reporting exposure to FP/RH messages (34 percent), TB treatment coverage rate (9–12 percent), drug-resistant TB notification rate (29–45 percent), functional adolescent-friendly service delivery points (17 percent), and number of health workers who received in-service training in nontraditional platforms (30 percent). In contrast, high performance (65 percent or more achievement of LOA target) was reported for: the treatment success rate for multidrug-resistant TB (60–67 percent t); treatment success rate for drug￾sensitive TB (91 percent); the number of community health workers (CHWs) providing information, referral, and services (90 percent); service delivery sites providing FP/RH services (89 percent); adolescents availing of FP/RH services (140 perc, and pulmonary TB bacteriological service coverage (65–77 percent). Second, natural, and human-made disasters hampered the implementation and rollout of USAID HP activities, causing the use of USAID services to stall or fail in project sites. As of February 2, 2022, 3.5 million Filipinos have been infected with COVID-19, and 54,000 have died (3.4 million have recovered, and 160,300 remained active cases). The epidemic hit hardest in the USAID HP’s three focus regions—the National Capital Region, Region III, and Region IV-A—as they are the densest and most urbanized. COVID-19 crowded out other public health services as the government redeployed health assets (facilities, staff, beds, ICUs, equipment) to control the epidemic. As a result, the patient/client backlog in services such as TB, FP, RH, maternal, and child health has increased. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | vi The Marawi Siege in 2017 between government security forces and militants affiliated with the Islamic State and local jihadist groups became the longest urban battle in modern Philippine history. It caused the displacement of 200,000 people in the city of Marawi and neighboring areas. All of Mindanao was placed under martial law, with attendant population mobility controls. Third, USAID HP targets were generally too ambitious, and data problems constrained proper performance assessment. A review of the LOA targets in relation to performance achievements indicates that most targets might not be met. Key informant interviewees noted that many of these targets were set too high, perhaps to encourage partner institutions to achieve more. Some of the targets, however, rested on assumptions that were made before the COVID-19 pandemic struck or were based on limited data available. Fourth, the plethora of USAID HP’s activities makes it easy to lose sight of the forest for the trees. In the laser-sharp focus on project implementation, it is easy to get fixated on urgent matters and lose sight of non￾urgent but important ones. In interventions that rely heavily on sectoral factors—financing, human resources, local leadership, and management—looking at these peripheral issues is just as important as managing the technical service delivery requirements of the HP activities. Technical relevance is necessary but is not sufficient to achieve and sustain performance. More to the point, the process of innovation, prototyping and scale-up focuses too much on technical aspects and often glosses over the availability of leadership, management, financial, and coordination inputs. Fifth, although the linear process of innovation, prototyping, and scale-up of HP interventions is logically sound, multiple partnerships and input sources often challenge sequencing during actual implementation. As a result, although certain innovations and models have been scaled up, some have not. Models that have earned unanimous or an official endorsement can be stalled by supply problems (logistics, HR, funding), but others can be stalled by demand problems (for instance, low utilization due to weak messaging of the intervention or lack of promotion). And even an innovation that a previous DOH administration has endorsed, like service delivery networks, can be neglected because the subsequent administration has different priorities or is swamped by more urgent concerns. Sixth, the planning and implementation horizon of some USAID HP activities, especially in the area of strengthening health systems, is too short, and the assumed implementation sequence is linear with minimal risks. But evaluations of similar efforts worldwide show that it is necessary to have long-term engagement. In the Philippines, the short-term tenure of elected officials (three years for LGU executives) often leads to policy and program slippage. PHIC changes leadership frequently (once every two years on average), and programs such as Konsultasyong Sulit at Tama (or Konsulta, a primary benefit package) have been renamed with each leadership change. Thus, at every implementation level (LGU, Department of Health, PHIC), there are risks that should be accounted for. Seventh, the complex USAID HP activities and the equally complex environments they are implemented in require more intensive and strategic inputs from USAID staff. USAID HP activities include prototyping/innovation to scale up platforms, health policy and regulation, health technology, health financing in various modalities, public-private partnerships (PPPs), and health informatics and data analytics. Implementation settings involve the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM), devolved LGUs, metropolitan governments, social health insurance, and corporate social responsibility programs. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | vii HP activities introduced a wide range of adaptations to allow interventions to proceed under challenges posed by the pandemic, including, for example, the conduct of rapid assessment for TB, FP, and ARH services, development of the National TB Control Program Adaptive Plan, development of guidelines for continuous FP/ARH services, and adoption of alternative training modalities for healthcare service providers. However, the WOPE team did not have sufficient time to assess the effectiveness of these adaptive measures. Various adaptive management and interventions were introduced in the past two years and a separate assessment was done.1 The above findings show the critical contextual factors that contributed to USAID HP’s varied performance in achieving its LOA targets. III. Looking to the Future The Philippine health sector is in transition, with much remaining policy and programmatic constraints as well as impending challenges brought about by an evolving economy and changing burden of disease. In addition to highlighting the unfinished public health agenda in traditional areas, the COVID-19 pandemic has overburdened the country with a new infectious disease, creating a massive challenge of an old service backlog plus new service demands. The new political administration that will emerge from the May 2022 elections will have its health priorities, which are expected to be couched not only in terms of response to COVID-19 (as it moves from a pandemic to an endemic) but also in terms of the long-term economic trajectory (from crisis to recovery to evolution into a middle-income and, eventually, an upper-middle￾income country). The envisioned higher economic status also has implications on the type of health system and financing the country needs and the institutional setup and management skills required to run such a system. The next planning horizon, therefore, will be more complex and challenging. The current program pillars remain valid, but USAID must adjust them to fit the health sector’s evolving needs. ● Health service delivery: Operational measures of care quality should be re-introduced so that they can be monitored and linked to input use (e.g., use of clinical vignettes). Also, moving most in-service training to pre-service programs should continue rapidly, because pre-service programs are more cost-effective and lead to better outcomes. ● Health workforce: Future USAID programming should distinguish between service delivery workers (also known as “frontliners”)—which has been the focus of traditional USAID training—and those involved in health service integration, health economics, and financing, health technology assessment, and other new skills needed by a modernizing health system that is increasingly devolved, privatized or under PPP, and funded by third-party payors and LGUs. The further professionalization of CHWs (from volunteers to paid workers, with the requisite training, the scope of work, and stature) should continue. Substantial training investment, management, and supervision are needed to make this cadre of CHWs provide consistently high-quality care at their level. 1 USAID/Philippines. Joint Learning on Adaptations in the Era of COVID- 19: An Assessment of Intervention Adaptations and Adaptive Management Processes among USAID Health Projects. March 2022. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | viii ● Health information system: Evidence-based decision-making requires the continuation of USAID support in increasing the reliability of the Field Health Service Information System, increasing the interoperability of multiple data systems, and conducting big data analytics on social media engagement. ● Medical products, vaccines, and technologies: Continuing USAID support is needed in this area to evaluate and use new TB and COVID-19 testing and treatment regimens, especially because of the rapid pace at which these are being developed and introduced to the market. Institutional support is also necessary (including staff skilling) to implement specific provisions of the Health Technology Assessment Law that are relevant to USAID-supported interventions. ● Health financing: The USAID HP could focus on continuing policy and programmatic reforms in this area, including: (a) under PhilHealth, scale-up of the Konsulta primary care package to as many service delivery points as possible, plus full PhilHealth coverage of TB treatment, including multidrug-resistant TB; (b) under the national government budget, full financing of the National TB Control Program and consideration of providing social-care support (Ayuda) to TB patients whose treatment regimen is long; (c) under LGUs, assisting health teams in provinces, cities, and municipalities to advocate, budget, plan for, and spend expected additional local resources under the Mandanas-Garcia ruling and wisely using the 5 percent of LGU budget appropriated for Gender and Development to support women’s and children’s health concerns; (d) for both national government and LGUs, budgeting for COVID-19 response (new facilities, vaccines, treatment, and social support). ● Health governance: The PPP response to COVID-19 has shown that PPP can be done on a large scale on other health interventions, such as TB, FP/adolescent reproductive health (ARH), and community-based drug rehabilitation. Doing so would require determined governance efforts that USAID should support. Stronger health regulation, especially in health technology assessment (TB, COVID-19, and other infectious disease interventions) is another key area USAID could support. Finally, the Philippines needs assistance in knowledge management in health, especially the documentation and impact evaluation of interventions. The magnitude of unfinished work in FP/RH and TB services and the backlog caused by COVID-19 are large, and the demand for technical assistance will be significant. Two years of intermittent lockdowns, travel restrictions, economic slowdowns, staffing, and funding redeployment to address the pandemic, and other coping mechanisms enforced by the government have led to supply restrictions of non-COVID health services. A massive crowding out has occurred, leading to a short supply of FP/ARH, maternal and child health, and other essential health services. Demand for these services, especially elective/non-emergency care, has also shrunk because households became hesitant to go to health facilities full of COVID-19 patients. The Philippine health sector needs technical assistance to reprogram these services and bring them back to normal—or the “new normal.” USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | ix ABSTRACT This whole-of-project evaluation (WOPE) report assesses the performance of the United States Agency for International Development (USAID)/Philippines Health Project (HP) from 2017 to 2022 in 13 discrete activities clustered around three service-delivery efforts: family planning and adolescent reproductive health, tuberculosis control, and community-based drug rehabilitation. The activities spanned three cross-cutting areas: health systems strengthening/governance, social and behavior change communication, gender equality and women’s empowerment. The WOPE team reviewed documents and performance indicators, analyzed secondary data, conducted key informant interviews, and held focus group discussions. The report is organized by USAID HP contributions to (a) improved social norms and behaviors, (b) continuous quality improvement in care service delivery, (c) the strengthening of health systems and the operationalization of universal health care in the Philippines, and (d) improving health outcomes among the underserved. The evaluation team then considered investments needed to achieve desired health outcomes and sustainable systems changes in the Philippines. The WOPE team found that to date, USAID HP achieved only around half of its life-of-activity targets. Key constraints were: (a) the prolonged COVID-19 pandemic, which stalled project activities because of frequent lockdowns and the redeployment of the nation’s health assets; (b) the lingering policy and programmatic obstacles—budget and financing, human resources, management, and coordination—characteristic of a long￾term health system reform program; and (c) institutional constraints at the Department of Health, the Philippine Health Insurance Corporation, and devolved local government units. The report then recommends changes in the portfolio’s priorities, including input on training and technical assistance. TABLE OF CONTENTS2 Acknowledgments.............................................................................................................................................................iii Preface .................................................................................................................................................................................iv Abstract...............................................................................................................................................................................ix List of Tables.......................................................................................................................................................................x List of Figures......................................................................................................................................................................x Acronyms and other abbreviations...............................................................................................................................xi Executive Summary............................................................................................................................................................1 What is WOPE? .................................................................................................................................................................7 Evaluation Objectives....................................................................................................................................................7 Evaluation Questions....................................................................................................................................................8 WOPE Methodology ....................................................................................................................................................9 Methodological Limitations.................................................................................................................................. 10 Data Collection Activities.................................................................................................................................... 10 Context of the current HP portfolio ......................................................................................................................... 12 How have the USAID HP interventions improved social norms and behaviors among the underserved seeking treatment and prevention services?............................................................................................................. 14 Overall findings in the assessment of performance indicators for improved social norms and healthy behaviors....................................................................................................................................................................... 15 Overview of interventions and innovations supported by the HP to strengthen healthy behaviors..... 18 On interventions for behavioral change on FP/ARH..................................................................................... 18 On interventions for behavior change on TB ................................................................................................. 20 On interventions for behavior change on CBDR........................................................................................... 21 Role of CHWs............................................................................................................................................................. 22 How has the USAID HP led to continuous quality improvement in service delivery? .................................. 23 Overall findings in the assessment of performance indicators to improve quality of service delivery.. 23 On FP/ARH.............................................................................................................................................................. 23 On TB....................................................................................................................................................................... 25 Overview of interventions and innovation supported by HP to improve quality of service delivery.... 26 On HP interventions to improve quality of FP/ARH services..................................................................... 26 On HP interventions to improve quality of TB care..................................................................................... 27 On HP interventions to improve quality of CBDR services ....................................................................... 28 How has the USAID HP supported the strengthening of health systems and the operationalization of UHC in the Philippines?................................................................................................................................................. 29 Overall findings in the assessment of performance indicators for HSS/G.................................................... 29 On stockout rates of contraceptive commodities and anti-TB drugs and TB laboratory tracer commodities ................................................................................................................................................................ 30 Overview of HSS/G interventions and innovations supported by HP........................................................... 32 On HSS reforms implemented and their impact on health outcomes.......................................................... 32 On HP interventions that have improved the effectiveness of DOH and other stakeholders in delivering their mandates.......................................................................................................................................... 33 2 Supplementary Materials A to L have been submitted to USAID/Philippines and are available upon request: (A) Family Planning and Adolescent Reproductive Health, (B) TB Control, (C) Community-Based Drug Rehabilitation, (D) Social Behavior Change and Communication, (E) Health Systems Strengthening and Governance, (F) Gender Equality and Women’s Empowerment, (G) An Approach to Conduct an Independent Whole-of-Project Evaluation of the USAID Health Project, (H) Initial Findings from Documentary Review of the Whole-of-Project Evaluation of the USAID Health Project, (I) Review of Performance Indicators, (J) Report on Secondary Data Analysis; (K) Comparison of Health Project Performance Indicators Baseline, End-of-Project Targets, and Accomplishments as of Q2 FY2021; and (L) Inventory of HP Outputs. On HP interventions that have improved national and local health system capacity ................................ 35 On HP interventions that helped expand and build provider capacity and improve supporting supervision and mentoring................................................................................................................................... 36 On HP interventions that helped ensure health commodity security nationally and locally ............... 37 On HP support to improve health information systems.............................................................................. 37 How has the USAID HP contributed to improving health outcomes among the underserved?................. 38 Overall findings............................................................................................................................................................ 38 On FP/ARH .................................................................................................................................................................. 39 On critical state-of-the-art training........................................................................................................................ 41 On TB............................................................................................................................................................................ 41 On CBDR..................................................................................................................................................................... 44 On challenges and barriers....................................................................................................................................... 45 For the next USAID program cycle (2022–2026), what type of support should the HP invest in to achieve desired health outcomes and sustainable systems changes in the Philippines? ................................. 45 Nature of future HP support................................................................................................................................... 45 On priority health systems pillars........................................................................................................................... 46 On technical assistance for FP/ARH, TB, and CBDR......................................................................................... 48 On mechanisms for delivery of support—working with partners ................................................................. 51 Summary of key recommendations........................................................................................................................ 53 General references......................................................................................................................................................... 55 Documents related to USAID/Philippines Health Project................................................................................ 57 Philippine Government Laws/Administrative Orders........................................................................................ 58 Online Resources....................................................................................................................................................... 59 Annex A. Scope of Work.............................................................................................................................................. 60 Annex B. Evaluation Frame, Analytical Protocol, Data Collection Methodology and Tools........................ 80 Annex C. All Sources of Information .......................................................................................................................132 Persons Interviewed ................................................................................................................................................132 Annex D. Consolidated Report on Key Informant Interviews..........................................................................136 General Findings...................................................................................................................................................136 1. On Partnerships with Stakeholders...................................................................................................136 2. ReachHealth.............................................................................................................................................136 3. TB Innovations........................................................................................................................................136 4. DOH..........................................................................................................................................................136 5. LGUs .........................................................................................................................................................137 On facilitators and enabling factors identified by key informants ............................................................137 On challenges and hindering factors ...............................................................................................................137 6. Monitoring................................................................................................................................................137 7. Availability of Data .................................................................................................................................137 8. Issues with Partners...............................................................................................................................137 Program-specific findings....................................................................................................................................138 9. FP Program ..............................................................................................................................................138 10. TB Program.........................................................................................................................................139 11. CBDR ...................................................................................................................................................140 12. Health Systems Strengthening ........................................................................................................141 13. Gender Equality and Women’s Empowerment..........................................................................142 Synthesis of Key Insights ....................................................................................................................................143 14. Social Behavior Change ....................................................................................................................143 15. Effect of Gender in Supply Chain Management..........................................................................143 16. Monitoring of Funds at the LGU Level.........................................................................................143 17. Approaches Used ..............................................................................................................................143 18. Sustainability and Future Activities and Projects........................................................................143 Annex E. Disclosure of Any Conflict of Interest...................................................................................................144 Certification No. 1...............................................................................................................................................144 Certification No. 2...............................................................................................................................................145 Certification No. 3...............................................................................................................................................146 Certification No. 4...............................................................................................................................................147 Certification No. 5...............................................................................................................................................148 Certification No. 6...............................................................................................................................................149 Certification No. 7...............................................................................................................................................150 Certification No. 8...............................................................................................................................................151 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | x List of Tables Table 1. Number of new FP acceptors in USG-assisted sites...................................................................................... 15 Table 2. Number of individuals in the target population reporting exposure to USG-funded FP messages through radio, television, electronic platforms, community group dialogue, interpersonal communication, or in print (by channel/# of channels) ................................................................................. 16 Table 3. Number of USAID-assisted CHWs providing FP information, referrals, or services during the year in USG-assisted sites................................................................................................................................................... 16 Table 4. TB detection rates (TB treatment coverage)................................................................................................... 16 Table 5. Drug-resistant TB notifications............................................................................................................................ 17 Table 6. Childhood TB notifications................................................................................................................................... 17 Table 7. Percentage of community contribution to TB notification ........................................................................... 17 Table 8. Percentage of USG-assisted SDPs providing FP counseling and/or services ............................................ 23 Table 9. Number of functional adolescent-friendly SDPs.............................................................................................. 23 Table 10. Number of adolescents using FP/SRH services in supported adolescent-friendly SDPs..................... 24 Table 11. Percentage of USG-assisted DOH regional offices and LGUs with functional ICV compliance committee/monitoring team.............................................................................................................................. 24 Table 12. Bacteriological diagnosis coverage (pulmonary TB) ..................................................................................... 25 Table 13. Private-sector TB notifications......................................................................................................................... 25 Table 14. Percentage of new and relapse TB patients tested using a WHO-recommended rapid test at the time of the diagnosis, USG-assisted sites........................................................................................................ 25 Table 15. Average stockout rates of contraceptive commodities, disaggregated by method at USG-assisted FP service delivery points in USG-assisted sites........................................................................................... 30 Table 16. Reported average stockout rate of contraceptive commodities, disaggregated by method, at USG￾assisted FP service delivery points in USG-assisted sites (MTaPS FY 2021 AMEL/PITT) ................... 31 Table 17. Average stockout rates of selected anti-TB drugs and TB laboratory tracer commodities at USG￾supported NTP facilities in USG-assisted sites.............................................................................................. 32 Table 18. CYP in USG-supported programs.................................................................................................................... 39 Table 19. Number of health workers who received in-service training using nontraditional learning platforms in FP and TB in USG-assisted sites................................................................................................................... 41 Table 20. Treatment success rates for MDR/RR-TB cases, CY 2019–2021............................................................. 41 Table 21. DS-TB treatment success rate, CY 2019–2021 ............................................................................................ 42 List of Figures Figure 1. HP log frame (2017 PAD).................................................................................................................................... 10 Figure 2. CYP growth rates, 2018–2020 ........................................................................................................................... 40 Figure 3. TB treatment success rates, 2018–2020.......................................................................................................... 43 Figure 4. Percentage of provinces and cities that reached TB treatment success rates, 2018–2020................. 44 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | xi Acronyms and other abbreviations 4Ps Pantawid Pamilyang Pilipino Program AI artificial intelligence AO administrative order ARH adolescent reproductive health BARMM Bangsamoro Autonomous Region in Muslim Mindanao BARMMHealth Bangsamoro Autonomous Region in Muslim Mindanao Health Capacity-Building BHW barangay health worker BIHC Bureau of International Health Cooperation, DOH BNS barangay nutrition scholar CBDR community-based drug rehabilitation CBMS community-based monitoring system CDCS Country Development Cooperation Strategy CHD Center for Health Development CHW community health worker CiTEC Citywide TB Elimination Campaign CLA collaborating, learning, and adapting CLAimHealth Collaborating, Learning, and Adapting for Improved Health CMSU2 Community Maternal, Neonatal, Child Health & Nutrition Scale-up CQI continuous quality improvement CSE comprehensive sexual education CY calendar year CYP couple-years of protection DBM Department of Budget and Management, GPH DDB Dangerous Drugs Board, GPH DepEd Department of Education, GPH DID difference-in-difference DILG Department of the Interior and Local Government, GPH DOAg Development Objective Agreement DOH Department of Health, GPH DOTS directly observed treatment, short course (for TB) DPCB Disease Prevention and Control Bureau, DOH DR-TB drug-resistant tuberculosis DS-TB drug-sensitive tuberculosis DSWD Department of Social Welfare and Development, GPH eLMIS electronic logistics management and information system EO executive order USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | xii EQ evaluation question FAB fertility-awareness-based FAST Find Actively, Separate Safely, Treat Effectively FDA Food and Drug Administration FGD focus group discussion FHSIS Field Health Service Information System FP/ARH family planning/adolescent reproductive health FPCBT family planning competency-based training FY fiscal year GAD Gender and Development GBV gender-based violence GEWE gender equality and women’s empowerment GPH Government of the Philippines HCPN health care provider network HHRDB Health Human Resources and Development Bureau, DOH HP Health Project, USAID/Philippines HPB Health Promotion Bureau, DOH HPPE Health Project performance evaluation HRH human resources for health HRH2030 Human Resources for Health 2030/Philippines HSS/G health systems strengthening/governance HTAC Health Technology Assessment Council ICV informed choice and voluntarism IEC information, education, and communication IHLGP Institutionalization of Health Leadership and Governance Program IMAP Integrated Midwives Association of the Philippines IP implementing partner IRR implementing rules and regulations ITIS Integrated Tuberculosis Information System IUD intrauterine device KII key informant interview KMITS Knowledge Management and Information Technology Service, DOH KRA key result area LGU local government unit LOA life of activity M&E monitoring and evaluation MDR/RR-TB multidrug-resistant/rifampicin-resistant TB USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | xiii MEL monitoring, evaluation, and learning MNCHN maternal, neonatal, child health, and nutrition MOH Ministry of Health, BARMM MRL Muslim religious leader MTaPS Medicines, Technologies, and Pharmaceutical Services Program NCR National Capital Region NTP National Tuberculosis Control Program, DOH NTRL National TB Reference Laboratory OH Office of Health, USAID/Philippines PAD Project Approval Document PFG Partnership for Growth PhilCAT Philippine Coalition Against Tuberculosis PhilHealth or PHIC Philippine Health Insurance Corporation PhilSTEP1 Philippine Strategic TB Elimination Plan, Phase 1 PITT performance indicator tracking table POPCOM Commission on Population and Development PPP public–private partnership PRM Program Resource Management Office, USAID/Philippines ProtectHealth Health Equity and Financial Protection Platform PSI progestin-only subdermal implants PWUD people who use drugs RA Republic Act ReachHealth FP/MNH Health Innovations and Capacity-Building Platforms RenewHealth Expanding Access to Community-based Drug Rehabilitation Program in the Philippines RHU rural health unit SBCC social and behavior change communication SBIRT screening, brief intervention, and referral to treatment (for CBDR) SCM supply chain management SDM standard days method (for FP) SDN service delivery network SDP service delivery point SHF Special Health Fund SRH sexual and reproductive health TB tuberculosis TB IHSS TB Innovations and Health Systems Strengthening TB LON TB Local Organizations Network TB Platforms TB Platforms for Sustainable Detection, Care, and Treatment USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | xiv ToC theory of change TPT TB preventive treatment TSR treatment success rate UHC universal health care UIS UHC integration sites USAID United States Agency for International Development USG United States government WHO World Health Organization WISN Workload Indicators of Staffing Need WOPE whole-of-project evaluation WRA women of reproductive age USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 1 EXECUTIVE SUMMARY I. Purpose of the Report Using a whole-of-project evaluation (WOPE) approach, this report examines the extent to which the U.S. Agency for International Development (USAID)/Philippines Health Project (HP) has achieved its purpose in four areas: (a) improving Philippine health outcomes among the underserved, (b) improving social norms and behavior change in health, (c) establishing and nurturing innovations in continuous quality improvement (CQI) in service delivery, and (d) operationalizing health systems’ support for and governance toward universal health care (UHC). The evaluation team analyzed three areas of USAID’s interest: family planning and adolescent reproductive health (FP/ARH), tuberculosis (TB) prevention and treatment, and community￾based drug rehabilitation (CBDR). They used findings from these analyses to identify priorities for USAID’s next program cycle (2024–2026). This WOPE relied on document reviews, review of implementing partners’ project performance data, secondary data analysis, key informant interviews between October 14 and December 2, 2021, and focus group discussions among FP acceptors and TB patients in Tanza, Cavite, on November 15, 2021. However, because of the COVID-19 pandemic, the evaluation team did not conduct extensive field visits. Moreover, sites supported by the United States government (USG) were compared to non-USAID sites, and there are inherent shortcomings in these direct comparisons. II. HP Contributions in Improving Philippine Health Outcomes among the Underserved Although the USAID HP made significant achievements in reaching intermediate outputs (e.g., trained workers, innovations started and nurtured), performance outputs—in terms of FP acceptors, couple-years of protection (CYP), and TB case finding—fell short of life-of-activity (LOA) targets. Although major policy reforms have been achieved at the macro level, in part due to USAID support—such as the Responsible Parenthood and Reproductive Health (RPRH) Law, the UHC Law, the TB Law, and their implementing rules and regulations (IRR) and manuals of operations—difficult implementation challenges remain, including the Government of the Philippines’ (GPH) partially budgeted activities, staff shortages, and weaknesses in management and coordination. At best, USAID HP has met 50 percent of its LOA targets. This checkered performance happened during the COVID-19 pandemic, which stalled critical activities in the last two years and severely affected people’s utilization of services. In hindsight, many LOA targets were overly optimistic, and some implementation challenges were under-appreciated; scaling up is difficult in the context of unfinished devolution with multiple stakeholders. The planning and implementation of institutional changes, such as the Department of Health (DOH) management of human resources, also require horizons further than what was envisioned in the HP. Many project activities happen within the complications of bureaucracy and local politics, with their inherent incentive issues and problems of short political tenure, which we must make allowances for, even with well￾intentioned managers in place. Finally, the Philippines faces a lingering scarcity of leadership and management in key public health interventions and policy, planning, programmatic, and evaluation requirements, which must be dealt with. The progress achieved during 2017–2022 reflects an incremental achievement in health reform and program agenda in USAID’s focus areas of FP/ARH, TB, and CBDR, even with the emergence of new challenges and USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 2 threats, such as COVID-19 and its economic and sector impact. This incremental progress also sets the stage for what the new Philippine government administration will do to advance these focus areas starting in mid-2022. FP/ARH: The USAID HP has provided technical assistance to more than 2,000 service delivery points (SDPs) providing FP services, which is 86 percent of the LOA target. However, the number of functional adolescent￾friendly health SDPs that the HP assisted was only 36 in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM), which is 65.5 percent of the LOA target, and only 18 in the rest of the country, which is 6.8 percent of the LOA target). The HP also provided institutional support to the DOH and local government units (LGUs) for FP/ARH. HP’s fiscal year (FY) 2021 midyear report shows USAID sites have reached close to nine million CYPs, representing only around half (48 percent) of its LOA target. Moreover, 2018–2020 data show that HP￾assisted cities and provinces had the same CYP as those that did not receive HP assistance. Finally, statistical analysis of the performance of USG sites versus non-USG sites over time did not show significant differences between the two groups. In BARMM, the use of adolescent-friendly health SDPs has been low, owing in part to the region’s cultural views of teenage pregnancy. Although these findings may appear weak traction of USAID assistance, non-USG sites could very well be receiving other forms of FP/ARH assistance from other sources, which would clearly demonstrate the shortcomings of directly comparing sites. Note also that USG sites were specifically chosen for their higher burden of disease with extremely low baseline performance. There are also questions about the reliability of Field Health Service Information System data used in the comparison analysis and about the over-optimism of the FP/ARH LOA targets, given the ensuing COVID-19 pandemic, with the epicenter in USAID’s “big three” regions: the National Capital Region, Region III, and Region IV-A. More analyses are also needed on the “dosage effect” (program intensity) of USAID HP interventions. Researchers should undertake a more careful impact evaluation and cost-effectiveness analysis of USAID interventions receiving significant funding and management attention. Key challenges in reaching the LOA targets are resolving the status of barangay (community) health workers (CHWs) regarding professionalization, full budgeting of primary package (Philippine Health Insurance Corporation [PhilHealth or PHIC], LGUs), further reducing contraceptive stockouts, and resolving the legal impediments regarding teenage access to contraceptives under the RPRH Law. The program introducing FP in hospitals reached sufficient maturity for broader adoption. TB: At the regional level, for quarter two (Q2) FY 2021, success rates for treating drug-sensitive TB (DS￾TB) cases in USAID regions were 90–91 percent, the same as the National TB Control Program (NTP) target of 91 percent. Success rates for treating multidrug-resistant/rifampicin-resistant TB were 60–67 percent, compared to NTP’s 66 percent target. It is harder to establish comparative trends at provincial and city levels because of the highly differing baseline burdens of disease. In terms of program innovations, the Find Actively, Separate Safely, Treat Effectively (FAST) Plus program for TB, implemented in 59 hospitals (54 in the NCR and five in other USG-supported regions), has been effective. Also, the Philippine Private Diagnostics Consortium (with 20 members), established with USAID support, ran 5,000 tests, and detected 1,100 DS-TB cases. However, despite these programmatic achievements, the TB detection rate, even in USAID sites, remains low. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 3 In light of the large funding gaps in implementing the TB Law, the key challenge in reaching the LOA targets is the creation of national and local government TB budgets. Fund disbursement also remains a big problem, with only 54 percent of funds being spent for the calendar year (CY) 2019 and CY 2020. Sustaining the city￾led TB campaigns is crucial, as TB is largely found in dense areas. CBDR: As of Q2 FY 2021, only 723 people who use drugs (PWUD) were reported to have undergone CBDR in USAID sites, representing 3.5 percent of the LOA target of 20,926. Although COVID-19 has definitely slowed CBDR activities, the underlying challenge is the intricate multi-agency CBDR framework involving the Philippine Drug Enforcement Agency, Dangerous Drugs Board, DOH, Department of Interior and Local Government, and host LGUs. Much more can be done in this area, but the stigma remains strong and the capacity of CBDR workers is still limited. Moreover, due to the enormous needs and operation gaps at the community levels, the current project appears to be spreading itself too thinly across many activities, and multiple stakeholders have not yet agreed on a common set of CBDR indicators that link performance to national CBDR goals. III. HP Contributions in Improving Social Norms and Behavior Change in Health FP/ARH: In Q2 FY 2021, USAID HP reported: (a) 900,000 new FP acceptors, representing 27.9 percent of the LOA target, and (b) 12.3 million people exposed to FP/RH messages through USAID’s various platforms, representing 34.2 percent of the LOA target. These seemingly low-performance achievements may be due to underreporting, as the most measurement is being done in traditional platforms (direct patient dialogue) but is not yet comprehensively done in social media, which has gained popularity. Compared to FP/RH clients, USAID HP was more successful in reaching health workers, with 90 percent of the LOA target of 89,735 community health workers (CHWs) met. Of the 13 USAID interventions in this area, seven have buy-in from the government (i.e., DOH, the Commission on Population and Development, and the Department of Education). The DOH Health Promotion Bureau has deemed USAID’s support, in terms of upgrading skills and formulating communication strategies, effective. Muslim religious leaders (MRLs) have been tapped in BARMM to endorse FP and help clarify pertinent issues. Usapan (conversation) sessions and their variants have reached almost 6,000 people, but interventions oriented to male clients (e.g., Katropa and Usapang Maginoo) need to be replicated. Social media campaigns such as “It’s OK to Delay,” “Konektado Tayo,” and “Usap Tayo sa FP” have reached millions of online users. However, converting messages into behavior change and utilization of services—and the impact measurement of this process—remains challenging. Expanding male involvement in FP remains difficult because of the scarcity of male FP counselors and other providers. The Lunas Collective, a feminist initiative supporting women experiencing gender-based violence, although innovative, has attracted only 272 clients. TB: For Q2 FY 2021, TB detection rates in USG-supported regions were 7–9 percent, lower than baseline rates and the 12 percent LOA target. For DR-TB notifications, the 6,841 cases detected in USG-supported regions represent only 35.9 percent of the LOA target. Childhood TB case notification posted a higher achievement of 67.5 percent of the LOA target of 64,506. Through the HP, USAID supported DOH in developing the National TB Health Promotion and Communications Strategy 2020–2023. Among the key interventions were the social media campaign #TBFreePH, the Tibay ng Dibdib storybook campaign for children, and the Ayos Ka Lungs campaign involving entertainment personalities. However, there has not been much analysis of the performance and USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 4 effectiveness of these engagements using appropriate metrics. Data analytics of Facebook, Twitter, and YouTube TB engagements would shed light on the reach of these programs. Feedback mechanisms that answer clients’ queries also need to be built into these activities. CBDR: Testing of the Lusog Isip (Healthy Mind) mobile app for mental health and substance abuse help will be rolled out in 2022. In USG sites,100,772 PWUD (64 percent of target LOA) have sought help. IV. HP Contributions Leading to Continuous Quality Improvement in Service Delivery FP/ARH: In areas outside BARMM, a Q2 FY 2021 survey of clients in 33 USAID-assisted sites showed that 92 percent gave a rating of 3–4 out of 5 for quality factors involving providers’ interpersonal skills and the quality of service. In BARMM, a Q1 FY 2021 survey in four provinces showed an average satisfaction rating of 3.8 out of 5. This increased to 4.8 out of 5 in a separate survey in Q2 FY 2021 in five provinces. These numbers confirm that, in general, service quality is acceptable in USG-assisted sites. USAID has supported 32 LGUs in setting up their CQI initiatives. The FP in Hospital Program, currently running in 162 public and private hospitals nationwide, has built-in CQI elements. Despite the COVID-19 pandemic, this program continues in 46 facilities through the Makeshift Teleconsultation platform. In BARMM, the HP supported provincial health offices in establishing 68 CQI sites and conducted CQI training for health providers. TB: The 2020–2021 pilot survey among patients showed an average satisfaction score of 4.6 out of 5, indicating excellent quality of TB care in USG-supported sites. The coverage of bacteriological diagnosis of pulmonary TB in USG-supported regions ranged from 39 percent in NCR to 46 percent in Region III (or 65 percent and 77 percent of LOA targets, respectively). Technology-based innovations abound in USAID￾supported TB interventions. With USAID assistance, DOH introduced the all-oral short-term regimen for MDR-TB. The HP is also validating the accuracy of the new Xpert MTB/RiF Ultra for TB diagnosis. Digital adherence tools such as Video Observed Treatment and ConnecTB improve treatment compliance. CBDR: In June 2020, the New Client Flow for PWUD was launched, which standardized screening tools, placing them in the appropriate treatment mode according to risk level. As of Q3 FY 2021, 50,635 PWUD have been screened, representing 64 percent of the LOA target. The HP conducted online webinars and training-of-trainers programs as part of the rollout of the CBDR screening tool. For example, 248 screening providers have been trained on screening, brief intervention, and referral to treatment. V. HP Contributions in Operationalizing Health Systems Strengthening/Governance (HSS/G) The HP measures overall HSS/G performance using three indicators: (a) the percentage of people in USAID￾assisted areas enrolled in PhilHealth, which is high at 88 percent; (b) PhilHealth support values, which measure financial protection but are not available for FP and TB services; and (c) budget disbursement rates (FY 2020), which were low for both TB (54 percent) and FP (16 percent) because of disruptions by the COVID-19 pandemic. Key activities in this area are supply chain management (SCM), implementing UHC and strengthening local health systems, and improving the management of training programs in health. SCM: Stockout rates of key FP commodities in USG-assisted sites are generally better than those in non￾USG-supported sites, and performance in the USG-assisted sites is close to meeting the LOA targets. The USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 5 LOA target for stockout rate at the USG-assisted sites was set at 7 percent across all FP commodities; by Q2 FY 2021, stockout rates for pills, injectables, and intrauterine devices (IUDs) had gone down to 8.5–10.0 percent. Only condoms, progestin-only subdermal implants (PSI), and standard days method beads continue to experience high stockouts. The HP also appears headed to achieve its stockout targets for TB commodities, except for pediatric TB drugs and drugs for TB preventive treatment. USAID helped the DOH secure an open-source electronic logistics management and information system (eLMIS), which is being tested in DOH warehouses, selected regional DOH offices, and LGUs. USAID expects the total number of test sites to reach 170 by June 2022. UHC Implementation and Local Health Systems Strengthening: At the policy level, USG helped the government draft the IRR of the new UHC Law. UHC is being rolled out nationwide. At the operational level, USAID helped PHIC develop the Konsulta primary care benefit package (which now covers TB treatment, IUD insertion, and PSI placement) being pilot tested among 130 providers as of September 2021. The HP supports LGUs in UHC integration. As of November 2021, 35 out of 58 LGUs in UHC integration sites have signed a memorandum of understanding. However, because of COVID-19, most of these sites have not yet set up their organizational key result areas (KRAs) to achieve such integration, and half the sites have yet to start KRAs in human resource management. Managing State-of-the-Art Training in Health: USAID assisted in setting up and operationalizing the DOH Academy, particularly its e-learning portal. However, the number of health workers who went through in-service training using nontraditional learning platforms reached only 7,706 participants or 29.8 percent of the LOA target. VI. Summary of Key Recommendations On HP support for the DOH: (a) size up and cost out the FP and TB backlog due to the COVID-19 pandemic; (b) help develop a strategy for the regularization of CHWs/BHWs/barangay nutrition scholars (BNS); (c) link CQI with budgets and incentives; (d) rationalize the Gender and Development (GAD) budget; (e) undertake capacity-building for the newly organized DOH Disease Prevention and Control Bureau (DPCB); (f) build on the momentum generated by COVID-19 containment, especially private-sector engagement; and (g) organize a region-wide UHC implementation lab using TB and FP-maternal, neonatal, child health, and nutrition (MNCHN) as tracers (possibly within the “big three” regions). On current HP interventions for testing, documentation, and replication: (a) Citywide TB Elimination Campaign; (b) Philippine Private Diagnostics Consortium; (c) eLMIS; (d) Workload Indicators of Staffing Need; (e) CBDR screening, brief intervention, and referral to treatment; (f) ConnecTB; and (g) Catastrophic TB Package in PhilHealth. On HP interventions for scaled implementation: (a) FP in hospitals, (b) MRLs as public health leaders, (c) FAST Plus, (d) Institutionalization of Health Leadership and Governance Program in regions, (e) Konsulta package, (f) Katropa, (g) comprehensive sexual education-ARH Convergence Program, (h) adolescent-friendly health facilities, and (i) men’s reproductive health. In future HP activities: (a) one TB implementing partner (IP) with distinct components on innovation design, testing, replication, and planning for institutionalization; (b) a distinct BARMM component within an umbrella USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 6 FP IP; (c) independent monitoring, evaluation, and learning/collaborating, learning, and adapting IP, with the mandate to collect, analyze, and monitor the performance of HP activities and to share lessons with the DOH; and (d) additional role for an HSS/policy/finance IP that will validate institutionalization and scale-up strategies for HP interventions. Moreover, technical capacity, perhaps by way of a facility similar to an indefinite-quantity contract, may be considered to augment HP management’s ability to supervise increasingly complex and challenging interventions. Such additional capacity may be needed in sorting and assessing the design, technical soundness, approaches to replication, and institutionalization for eventual scaled implementation. An approach that ties all these recommendations together in a way that allows for scaled implementation is to package HP support and focus its activities to help the DOH implement UHC at a regional level. Support for a UHC regional laboratory should involve two or more provinces containing consumers and providers of health services representing a continuum of care from disease prevention to primary curative care to treatment and rehabilitation in tertiary and specialty facilities. Replication and eventual institutionalization might be easier at the regional level, given existing administrative and budgetary mechanisms at the regional level through the DOH, PhilHealth, the Commission on Population and Development, and other institutions. It may be in the context of supporting regional UHC labs, where the LGU pivot initiated by current HP activities might become more meaningful. The regional level is also where USAID could concentrate future efforts to build capacity to manage increasingly complex financing and delivery system. A particular concern is the need to expand and strengthen the number and availability of middle-level managers at regional and LGU health offices. USAID also needs to complement this proposed expansion of health management capacity with the appropriate enhancement of health management information systems. Finally, following the experience with efforts to contain COVID-19, it may be easier to engage the private sector and civil society organizations at levels that involve both consumers and providers of the continuum of care. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 7 WHAT IS WOPE? Evaluation Objectives The evaluation team uses a whole-of-project evaluation (WOPE) approach to examine the extent to which the United States Agency for International Development (USAID) Health Project (HP) has achieved its intended purpose and sub-purposes. Toward this end, the WOPE is expected to: 1. Inform future strategic directions and approaches of the USAID HP 2. Recommend adaptations in the design of current interventions that could be implemented in the remaining years 3. Determine whether currently supported activities contribute to achieving HP goals 4. Determine facilitating and hindering factors affecting the achievement of HP goals 5. Determine the validity of the HP theory of change (ToC), value perceptions of stakeholders on USAID assistance, and unintended results that affect HP outcomes The mid-cycle WOPE covers the 13 HP activities organized by the USAID/Philippines Office of Health (OH) since 2018. These activities have asynchronous project start-up and closeout dates, but the end of the current HP program cycle is in 2024.3 The HP focuses on family planning and adolescent reproductive health (FP/ARH); tuberculosis (TB) control; and community-based drug rehabilitation (CBDR), as well as three cross-cutting activities covering health systems strengthening and governance (HSS/G), social and behavioral change communication (SBCC), and gender equality and women’s empowerment (GEWE). This report is intended for the information and use of the leadership and technical staff of the USAID/Philippines OH, USAID/Philippines Program Resource Management (PRM) Office, and the Development Objectives Agreement (DOAg) management committee and steering committee. 3 Health Project Activities 2018–2024: (1) Community Maternal, Neonatal, Child Health & Nutrition Scale-up (CMSU2) (August 2016–December 2019); (2) FP/MNH Health Innovations and Capacity Building Platforms (ReachHealth) (December 2018–November 2023); (3) Bangsamoro Autonomous Region in Muslim Mindanao Health Capacity Building (BARMMHealth) (February 2019–February 2024); (4) Treat TB: Supporting MDR-TB Activities in the Philippines (September 2016–March 2019); (5) TB Innovations and Health Systems Strengthening (TB IHSS) (February 2018–February 2023); (6) TB Platforms for Sustainable Detection, Care and Treatment (TB Platforms) (April 2018–April 2023); (7) TB Local Organizations Network (TB LON) (October 2020–September 2023); (8) Institutionalization of the Health Leadership and Governance Program (IHLGP) (July 2017–September 2020); (9) Health Equity and Financial Protection Platform (ProtectHealth) (March 2019–March 2024); (10) Medicines, Technologies, and Pharmaceutical Services Program (MTaPS) (September 2018–September 2023); (11) Human Resources for Health 2030/Philippines (HRH2030) (October 2017–June 2020); (12) Expanding Access to Community-based Drug Rehabilitation Program in the Philippines (RenewHealth) (May 2019–May 2024); and (13) Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) (March 2018–March 2022) USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 8 Evaluation Questions The scope of work for the WOPE specifies 14 main evaluation questions (EQs), with corresponding sub-questions to clarify the intent of the main questions. In subsequent meetings with USAID/Philippines, five questions were deemed most critical and essential in meeting the evaluation objectives. These are: EQ1. How have the USAID HP interventions improved social norms and behaviors among the underserved seeking treatment and prevention services? ● What demand-generation platform and messaging were most effective for: men, adolescent youth, urban poor, women with unmet needs for FP, and people who use drugs (PWUD)? ● What HP interventions have improved health-seeking behavior and treatment adherence? EQ2. How has the USAID HP led to continuous quality improvement in care service delivery? ● How has the HP improved the skills of FP/ARH health providers? ● How has HP expanded FP and other FP services? ● What were the TB projects’ key contributions to TB control in the Philippines, and to what extent have the projects addressed key TB issues and gaps? ● What are the three most effective packages of TB and FP interventions that could be scaled up nationwide? EQ3. How has the USAID HP supported the strengthening of health systems and the operationalization of universal health care (UHC) in the Philippines? ● What HSS reforms are in place because of USAID interventions? How are these HSS reforms impacting health outcomes? ● What HP interventions have improved the effectiveness of the Department of Health (DOH) (Disease Prevention and Control Bureau [DPCB], Health Promotions Bureau [HPB], Pharmaceutical Division, Health Human Resources and Development Bureau [HHRDB], Dangerous Drugs Abuse and Prevention and Treatment Bureau, its information systems); the Ministry of Health (MOH) of the Bangsamoro Autonomous Region of Muslim Mindanao (BARMM); the Philippine Health Insurance Corp. (PHIC or PhilHealth); and the Dangerous Drugs Board (DDB) in delivering their mandates? ● What HP interventions have improved local health system capacity for policy formulation, budget planning and execution, inter-local cooperation, program implementation, and monitoring and evaluation (M&E)? ● What HP interventions have increased national and local capacity in health systems management in sustainably supporting TB and FP/ARH programs? ● What HP interventions have helped expand and build health provider capacity for quality health care delivery for TB, CBDR, and FP/ARH programs? ● What HP interventions have helped in improving supportive supervision and mentoring of human resources for health (HRH)? ● What HP interventions have helped ensure TB and FP/ARH health commodity security at the national and local levels? USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 9 ● How has the HP supported the improvement of the TB, FP/ARH, and CBDR information systems? EQ4. How has the USAID HP contributed to improving health outcomes among the underserved? ● To what extent have HP activities contributed to achieving the targets (i.e., the extent of performance relative to the annual targets and end-of-project targets)? ● What were the critical enabling factors associated with achieving HP targets? ● What were the challenges and barriers to achieving HP targets? ● What innovations contributed to improving health outcomes? EQ5. For the next USAID program cycle (2024–2026), what type of support should the HP invest in to achieve desired health outcomes and sustainable systems changes in the Philippines? ● Which health system pillars should be prioritized? ● What type of technical assistance should be provided for TB, HSS/G, FP/ARH, and CBDR? ● What mechanisms should be used for the delivery of support? The evaluation team addressed the remaining questions and the factors contributing to specific HP performance in various parts of the evaluation. We adopted the final set of questions underscored in Technical Direction No. 2 provided by the USAID/Philippines PRM Office. In our approach to this WOPE, we view the USAID/Philippines OH as the principal body that exercises both technical and administrative leadership over the HP. The OH is responsible for HP’s design and implementation strategy. It is ultimately accountable for the performance. Additionally, the DOH, local government units (LGUs), and other entities are implementing partners (IPs) of the HP that are engaged at various levels. The IPs are contracted or have cooperative agreements with the OH to implement the HP and produce deliverables at specified levels of effort with specified resources. We assumed the IPs exercise due diligence by engaging the OH in ways that would allow necessary adjustments for better implementation of HP tasks. Using this perspective, the evaluation team delineated the roles of these key HP players. WOPE Methodology The evaluation team employed four inter-related methods of analysis given the available information, time, and resources. The team: 1. Compared and explained the differences between the original HP design and the implementation strategy described in the Project Approval Document (PAD) of 2017 2. Explained changes in performance over time from the start of HP activities (baseline measures) to life-of-activity (LOA) performance measures 3. Explained variations between targets and accomplishments 4. Compared the differences in program performance between sites where HP interventions were introduced against those of sites where HP interventions were not present (specifically at the province and highly urbanized city levels). USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 10 We used the logical framework in the 2017 PAD to compare the HP design with its actual implementation (Figure 1). Key sections in the 2017 PAD that served as the yardstick for the evaluation were the comparative advantages of USAID built by predecessor projects and the ToC for HP, particularly the recommendations referred to as big shifts. The detailed evaluation method is discussed in Annex B. Figure 1. HP log frame (2017 PAD) Methodological Limitations Ideally, the evaluation should be able to determine outcomes and attribute these to specific interventions that the HP introduced. However, because the HP was not designed with a built-in evaluation frame, complete with sufficient controls, we implemented an approach that approximated a full difference-in-difference (DID) analysis using selected data from the Field Health Service Information System (FHSIS) and the Integrated TB Information System (ITIS). A full description of the DID methodology is in Supplementary Material J. Ideally, the analysis should have been conducted at the municipal level, where, according to reports, current HP interventions have gained traction. However, because of issues with data quality and the availability of reliable FHSIS and ITIS data at the municipal level, we limited our secondary data analyses to the provincial level and highly urbanized cities. Data Collection Activities The evaluation included a review of relevant project documents such as the IPs’ annual reports and work plans and the yearly HP performance evaluations (HPPEs); a review of performance data (baseline, target, annual, and LOA); analyses of secondary data from the FHSIS and ITIS; key informant interviews (KIIs) with USAID/OH, IPs, government counterparts, and other stakeholders; and focus group discussions (FGDs). Document Review. During the incipient stage of the evaluation, we conducted an initial review of the 2017 PAD, annual reports submitted by IPs, HPPEs, and other background documents related to the project. This review provided the team with deeper insights into HP operations to date. It also allowed us to finalize the evaluation design, data collection tools, and relevant probing questions to help us dig USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 11 deeper into specific issues. The initial document review for this evaluation is provided in Supplementary Material H. Review of IP Reports and Performance Data. The team examined the performance indicator tracking tables (PITTs) submitted by IPs and compared the baseline measures with targets and actual accomplishments in the years, regions, provinces, cities, and municipalities for which data are available. We took note of the explanations of observed differences that the IPs and partners reported in official documents. The team also performed an in-depth analysis of several initiatives and innovations implemented by the HP that are expected to have a high impact, have taken root, or have gained traction where they were introduced and have the potential for being scaled up and sustained beyond the HP. The full review of performance data (baseline, targets, and accomplishments) and an assessment of performance indicators are provided in Supplementary Material K. Secondary Data Analysis. We selected measures available in the FHSIS and ITIS to determine HP’s contribution. In evaluating the impact on its target goals, we analyzed changes across sites (HP-assisted and non-HP-assisted provinces and cities) and across time (fiscal year [FY] 2018–2020) using the DID method. The DID approach compared the changes between HP-assisted and non-HP-assisted provinces and cities using various regression models (e.g., logit, fractional logit, and ordinary least squares) to evaluate the statistical significance of the observed differences between these groups through time while controlling for the provincial and independent/highly urbanized cities’ internal revenue allotment. However, it should be noted that the DID analysis provides an estimate of how selected performance indicators vary between sites supported by the United States government (USG) and non-USG-assisted sites over the years covered in the report. The analysis found no statistically significant differences between sites and over time, but this should not be taken as evidence that HP interventions are ineffective. In other sections of this report, we note that HP interventions have only started to gain traction and cannot be expected to have affected outcomes at scale. It should also be considered that HP-assisted sites were chosen particularly for having a high disease burden and unmet needs. While we have been unable to explicitly account for interventions by other donor agencies, such influences may be accounted for to the extent that they vary across provinces, cities, and regions. What we cannot discern from the analysis would be the joint influence of other donors in sites also supported by the HP. The full report on the secondary data analysis of select FHSIS and ITIS indicators is provided in Supplementary Material J. KIIs. The team completed 75 online KIIs between October 14, and December 2, 2021. Annex C lists the persons we interviewed and their institutional affiliations. We have not identified respondents by name to maintain confidentiality. FGDs. We conducted two FGDs among FP acceptors and TB patients in Tanza, Cavite, on November 15, 2021. The evaluation team was led by Dr. Orville Solon, professor and former dean of the School of Economics, University of the Philippines Diliman. He is supported by: Dr. Ma. Soledad Antonio, ex-officio member and director of the DOH Bureau of International Health Cooperation; Ms. Frances Mamaril, ex￾officio member and ad interim director, DOH Health Policy Development Program Bureau; Dr. Mario USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 12 Festin (FP/ARH specialist), professor at the College of Medicine, University of the Philippines Manila and a former medical officer at the World Health Organization (WHO) Headquarters in Geneva; Dr. Kathryn Roa (TB specialist), infectious disease specialist at the Davao Doctors Hospital and Southern Philippines Medical Center; Dr. Julienne Baldo-Cubelo (CBDR/SBCC specialist), professor and former chairperson of the University of the Philippines Mass Communication Research Department; Ms. Ermi Amor Figueroa-Yap (HSS/G specialist), economist and former consultant of USAID’s Collaborating, Learning, and Adapting for Improved Health (CLAimHealth); Ms. Fatima Verzosa (GEWE specialist), former gender advisor in USAID/Philippines, USAID/Afghanistan and USAID/Burma (Myanmar); Dr. Noel Juban† (data collection specialist); Mr. Oscar Picazo (technical editor); Mr. Arturo Ongkeko, Jr. (evaluation associate); and Mr. Chamuel Michael Joseph Santiago (evaluation coordinator/technical assistant). The main report describes the team’s overall findings across all three program areas and cross-cutting themes answers the five key EQs and provides strategic recommendations for consideration. Individual team members prepared the following supplementary materials: Family Planning and Adolescent Reproductive Health, TB Control, Community-Based Drug Rehabilitation, Social Behavior Change and Communication, Health Systems Strengthening and Governance, and Gender Equality and Women’s Empowerment. Other supplementary materials include the following: An Approach to Conduct an Independent Whole-of-Project Evaluation of the USAID Health Project, Initial Findings from Documentary Review of the Whole-of-Project Evaluation of the USAID Health Project, Review of Performance Indicators, Report on Secondary Data Analysis, Comparison of Health Project Performance Indicators Baseline, End-of-Project Targets and Accomplishments as of quarter two (Q2) FY2021, and Inventory of HP Outputs. Supplementary materials will be made available upon request. The following required annexes are attached to the main report: WOPE Scope of Work; Evaluation Frame, Analytical Protocol, Data Collection Methodology, and Tools; Source of Information; Consolidated Report on Key Informant Interviews; Statement of Difference; and Disclosure of any Conflict of Interest. CONTEXT OF THE CURRENT HP PORTFOLIO USAID/Philippines’ Country Development Cooperation Strategy (CDCS) served as the guiding framework in implementing the current HP. The HP straddles two CDCSs. From 2013to –2019, the focus was on the Partnership for Growth (PFG) to attain the goal of a more stable, prosperous, and well-governed nation. Building on the gains of 2018 and 2019, the current HP support is guided by the new CDCS (2019–2024), which focuses on good governance and self-reliance to achieve the goal of the Philippines becoming “A Well-Governed and More Self-reliant Indo-Pacific Partner.”4 The three development objectives of the CDCS are: (1) democratic governance strengthened; (2) inclusive, market-driven growth expanded; and (3) environmental and community resilience enhanced. The CDCS also emphasizes three cross-cutting strategies: increased private-sector engagement, enhanced gender and social inclusion, and civil society strengthened. 4 USAID Philippines. (October 28, 2019). Country development cooperation strategy 2020–2024. Available at https://www.usaid.gov/sites/default/files/documents/1861/Philippines_CDCS_2019-2024.pdf USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 13 In line with the CDCS, the purpose of the USAID/Philippines HP 2018–2024 is “Improved Health for Underserved Filipinos.” In keeping with the PFG, the Health Portfolio aligns with the Philippine Health Agenda 2016–20225 of DOH and was developed with guidance from the USAID/Philippines Health Portfolio Evaluation in August 20166 and the PAD.7 Three sub-purposes contribute to the HP purpose: (1) health behaviors strengthened, (2) quality of services fortified, and (3) key health systems bolstered and institutionalized. The HP consists of four primary health programs: (1) FP/ARH, (2) TB control, (3) CBDR, and (4) HSS. Under each program, USAID/Philippines funds one or more mechanisms also referred to as activities. The HP focuses on geographic areas where the health burden is the greatest—in other words, where the TB disease burden is the highest, where unmet needs for FP are the highest, where there are high teenage pregnancy rates, and where there is a high evidence-based need for drug demand reduction services. This is in addition to the inherent structural weaknesses of a devolved public health system, resulting in fragmented service delivery and financing, high transaction costs of stakeholder engagement, and widening inequity in access to quality public and private health care.8 The Mandanas-Garcia ruling, while expected to increase LGU funding by close to 30 percent, may not necessarily lead to increased investments in health. Moreover, outstanding issues remain, especially regarding the implementation of milestone health legislation, as discussed below. The enactment of the UHC Law (Republic Act [RA] 11223) in July 2018 led the HP IPs to refocus some of their resources to support the DOH, PhilHealth, and USG-assisted sites in establishing the necessary soft infrastructures to improve access to health services for the underserved. However, delays in the development and approval of RA 11223 implementing rules and regulations (IRR) led to postponements in the implementation of planned activities. In terms of health financing, PhilHealth provided coverage for 98 percent of the population in 2018. However, it is hounded by charges of inefficiency and corruption. By the end of 2019, PhilHealth owed approximately 174 million Philippine pesos (PHP) for primary care benefit payments. Budget planning and execution, as well as procurement and supply chain management, remain problematic, even with the provision of training and bureaucratic remedies. While there is a steady rise in the DOH’s annual budgets, delays in budget approval and the persistent problem of low disbursement of funds at the DOH have led to delays in implementation of activities, hindrance of the procurement process, and stockouts of drugs and commodities. The Comprehensive TB Elimination Plan Act of 2016 mandated the adoption of a multisectoral response to TB. This law paved the way for the 2017–2022 Philippine Strategic TB Elimination Plan, Phase 1 (PhilSTEP1). However, the fragmentation of service delivery and financing has prevented the effective 5 Department of Health, Republic of the Philippines. Philippine Health Agenda 2016–2022. 6 USAID Philippines. (August 2016). Health portfolio evaluation. 7 USAID Philippines. Project appraisal document: USAID/Philippines Health Project 2017–2022. 8 Panelo, C. I. A., Solon, O. J. C., Ramos, R. M., Herrin, A. N. (2017). The challenge of reaching the poor with a continuum of care: A 25-year assessment of Philippine health sector performance. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 14 control of TB. Despite efforts based on the globally accepted directly observed treatment, short-course (DOTS) strategy, there has been no significant decline in TB burden since 2007, as confirmed by the findings of the 2016 National TB Prevalence Survey. The Responsible Parenthood and Reproductive (RPRH) Law provided a fresh mandate to centrally finance and deliver FP services. However, the late disbursement of funds, combined with the lack of storage space and disruptions in delivery due to Food and Drug Administration (FDA) requirements, exacerbated stock-outs of drugs and commodities. Furthermore, a moratorium on the implementation of the RPRH Law and a global shortage of progestin-only subdermal implants (PSI) have aggravated the resupply issue. COVID-19 and Private Sector Engagement. In 2020, the COVID-19 pandemic diverted the time and resources of the Philippine’s national and local governments away from other public health programs like TB and FP. It overburdened the country with reemerging infectious diseases, such as polio. Lockdowns, travel restrictions, and distancing protocols disrupted the provision of essential health services such as immunization for children and antenatal care. Health providers employed adaptive measures to mitigate the effects of the mobility restrictions, such as reducing the number of health staff and transport constraints of clients. However, even with adaptive measures, we can expect significant backlogs in non-COVID-19 health programs. Despite these challenges, the pandemic demonstrated unprecedented private-sector participation. The private sector was forced to take a more proactive role in supporting the health system to thwart the threat to business continuity and survival. Actions have included mass testing and procurement of personal protective equipment needed by those on the front line, especially health care workers. The private sector collectively stepped up and did more than its fair share in addressing this crisis and supporting communities at large. At present, the public and private sectors are working on the scaled and swift rollout of the country’s COVID-19 vaccination program. Several companies are contributing financial resources, contacts and networks, intellectual capital, and logistics assets to help the government meet its vaccination objectives. HOW HAVE THE USAID HP INTERVENTIONS IMPROVED SOCIAL NORMS AND BEHAVIORS AMONG THE UNDERSERVED SEEKING TREATMENT AND PREVENTION SERVICES? Although SBCC is understood as the framework for improving health behavior through communication, it is hardly given any significant discussion in HP activity work plans and reports. Much of the HP work on SBCC continues to be measured in terms of participation in or attendance at face-to-face communication sessions. Often, meetings or conversations on SBCC among government agencies, providers, and beneficiaries are deemed as evidence of strengthened health behavior. In lieu of “reach” metrics, USAID may want to consider pilot-testing exit survey interviews to get a sense of the narratives and sentiments of the clients on HP-supported services and activities. This can be done by way of reception analysis, which is a qualitative method that explores the extent to which the client makes sense of media content. In addition, this can be complemented by a textual analysis, which refers to the review of content in media and popular culture. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 15 There is a divide between two mindsets: SBCC as delivery of services and SBCC as capacity-building. SBCC as delivery of services refers to the provision of information to targeted clients to influence health care utilization. SBCC as capacity-building refers to the provision of training and related assistance to health care providers so that they can relay the information their clients need. Given limited resources, the current HP portfolio was designed to prioritize building the capacity of health providers and health program managers to design and provide SBCC. The trade-off is to limit reach for now in exchange for greater reach when capacitated providers can act as force multipliers. Overall findings in the assessment of performance indicators for improved social norms and healthy behaviors On FP/ARH The USAID HP reported having achieved more than 900,000 new acceptors in Q2 FY 2021. This represents 28 percent of its LOA target (Table 1). During the same period, HP reported that 12.3 million people were exposed to HP-funded messages through various platforms (Table 2). This gap in numbers indicates more work is needed to get people to reflect on and establish the link between SBCC strategies and health outcomes. To do this, social media, and on-ground SBCC activities ideally must include direct, individualized follow-up discussions on particular topics that can likely influence individuals to make better health care decisions. This can be in the form of a phone or social media hotline where an adolescent can talk to a peer advocate to clarify and follow up on information. In terms of establishing metrics to link SBCC strategies to outcomes, USAID may want to consider using “engagement” metrics such as a brief survey tool that simply asks questions such as: What prompted you to use the service? What health information have you come across lately that helped you in your decision to come here? These questions can be incorporated into the facility intake form. In addition, the HP can measure the number of times an SBCC material has been shared. Because the indicators refer to individuals reporting exposure to messages, the HP needs to address whether 12 million individuals were asked questions. HP reports should indicate if such reports are based on estimates. Although the target platform is radio, social media use can be measured or estimated using the number of views and likes. The HP should also include other methods to improve the reporting of reach of USAID-supported messages. Table 1. Number of new FP acceptors in USG-assisted sites Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished Health Project 754,672 3,419,909 952,509 27.9% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 16 Table 2. Number of individuals in the target population reporting exposure to USG-funded FP messages through radio, television, electronic platforms, community group dialogue, interpersonal communication, or in print (by channel/# of channels) Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished Health Project 836,162 36,116,740 12,341,645 34.2% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet However, it is notable that the HP reached 90 percent of its target, representing more than 80,000 community health workers (CHWs) trained to provide FP information, referrals, and services (Table 3). The HP must continue to harness the skills of the CHWs to reach more women of reproductive age (WRA) with unmet needs for FP. CHWs are on the front lines, and USAID may want to explore with the DOH and the LGUs how to professionalize and support them. The HP can assist the DOH and LGUs in reviewing RA 7883 or the Barangay Health Workers (BHWs) Benefits and Incentives Act of 1995. USAID should explore how the current training programs for BHWs may be integrated into the Barangay Health Services NC2 vocational courses of the Technical Education and Services Development Authority. USAID should also consider supporting the National Confederation of BHWs of the Philippines. Table 3. Number of USAID-assisted CHWs providing FP information, referrals, or services during the year in USG-assisted sites Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished Health Project 52,166 89,735 80,771 90.0% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet On TB TB detection rates reported in USG-assisted sites remain low. For FY 2020, detection rates for the National Capital Region (NCR), Region III, and Region IV-A were 7 percent, 7 percent, and 9 percent, respectively. These rates are lower than what was reported at baseline in 2018. The detection rates are no more than 12 percent of LOA targets for these regions (Table 4). Table 4. TB detection rates (TB treatment coverage) Baseline LOA Target Accomplishment as of FY2020 Percentage Accomplished NCR 12% 75% 7% 9.3% Region III 10% 75% 7% 9.3% Region IV-A 13% 75% 9% 12.0% USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 17 Source: HP PITT_210701.xlsx, TBR1 sheet In terms of drug-resistant TB (DR-TB) notifications, USG-assisted regions in Q1 of calendar year (CY) 2021 reported the cumulative number of cases notified representing 44.5 percent, 35.5 percent, and 28.7 percent of LOA targets for NCR, Region III, and R VI-A, respectively (Table 5). Table 5. Drug-resistant TB notifications Baseline LOA Target Accomplishment as of Q1 CY 2021 Percentage Accomplished NCR 1,701 6,235 2,776 44.5% Region III 785 5,582 1,984 35.5% Region IV-A 968 7,261 2,081 28.7% Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet Childhood TB cases notified are over half the LOA targets. In Region III, the number of childhood TB cases notified reached over 90 percent of LOA target. It is important to carefully document this accomplishment and determine what underlying factors may be used in other sites (Table 6). Table 6. Childhood TB notifications Baseline LOA Target Accomplishment as of Q1 CY2021 % Accomplished NCR No baseline 20,987 13,472 64.2% Region III No baseline 18,866 17,273 91.6% Region IV-A No baseline 24,653 12,766 51.8% Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet From these data, one gets the impression that the TB program has focused on multidrug￾resistant/rifampicin-resistant TB (MDR/RR-TB) and childhood TB cases. It is important to review the strategies that worked in these interventions and find ways that they can be applied to improve drug￾sensitive TB (DS-TB) case detection. In terms of percentage of community contribution to TB notification for NCR, Region III, and Region IV￾A, the HP reported 8 percent, 10 percent, and 11 percent, respectively (Table 7). Table 7. Percentage of community contribution to TB notification Baseline LOA Target Accomplished as of Q1 CY2021 Percentage Accomplished NCR No baseline Indicator not carried over to 8% N/A Region III No baseline 10% N/A USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 18 Table 7. Percentage of community contribution to TB notification Baseline LOA Target Accomplished as of Q1 CY2021 Percentage Accomplished Region IV-A No baseline Updated PhilSTEP1. 11% N/A Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet Overview of interventions and innovations supported by the HP to strengthen healthy behaviors The HP can track how many people SBCC interventions have reached, but the final effects on health￾seeking behaviors have yet to be observed. The HP needs to translate the number of people reached into the number of people using health services and test this mechanism in a local setting. The COVID-19 prevention and control experience in the Philippines offers opportunities to determine how messages and communications convert to actual behaviors (e.g., mask wearing, vaccination) and subsequently on disease incidence. The HP work on FP/ARH, which has directly engaged the DOH’s HPB, regional DOH offices, and the Commission on Population and Development (POPCOM), is in the best position to sum up these lessons and help translate, for the DOH and LGUs, how these might be used to meet SBCC goals for FP/ARH, TB, and CBDR. DOH officials point out that HP support to the HPB has been useful and effective, and helped build staff skills in developing communication materials. HP also helped develop communications strategies of programs for health schools and the Healthy Pilipinas campaign. Many of the interventions to strengthen healthy behaviors are expected to indirectly affect output targets, with medium- and long-term gains to be realized. For FP/ARH, of the 13 interventions, seven have buy-in from the DOH, POPCOM, and the Department of Education (DepEd). For TB, 20 of 25 interventions have buy-in from health facilities, LGUs, regional offices, and private provider groups. Many of these interventions include help designing and evaluating possible pilot and demonstration activities. In these activities, there are no explicit expectations on how much such interventions will translate into utilization of services for FP/ARH and TB. For CBDR, gains are expected to be realized mostly in the long term. The WOPE team finds these interventions to be sound in design and in the form of assistance piloting or demonstrating activities (see Supplementary Material L). The DOH also appreciates HP support in assessing the needs of various population groups, which allows for tailored communications strategies. An inventory of interventions directed at improving social norms and behaviors that lead to seeking treatment and prevention services is in Supplementary Material L. On interventions for behavioral change on FP/ARH Engagement of Muslim religious leaders (MRLs) and other religious leaders USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 19 MRLs or Ulama were engaged in BARMM as active partners in addressing misconceptions of FP in Islam and promotion of FP use. MRLs clarify questions raised by potential users. Because MRLs also indicate which FP methods cannot be used, the HP should focus on the methods that are endorsed, especially those with available supply, and focus its monitoring on these. For men, avenues for discussing FP, especially about their roles, show potential for improved health￾seeking behavior. The Usapan (conversation) sessions with MRLs provide a venue to discuss masculinity, family leadership, and responsibility and how they relate to spirituality. The MRLs as spiritual leaders help dispel myths and outdated notions about FP. The MRLs’ facilitation in these sessions provides a sense of safety around traditionally difficult topics to discuss among men. Little progress has been made on dialogue with Catholic leaders. These have been difficult, as they involve spiritual, moral, and philosophical concepts that may not lead to easy conclusions and may lead to more confusion. However, USAID may want to explore other Christian denominations that are more progressive in their views on FP. Usapan sessions (and its variants) Usapan is a group communication technique developed to directly link clients to FP service delivery. USAID succeeded in getting buy-in from the DOH and POPCOM on this activity. However, it has not been scaled up and is not well known among FGD participants. The challenge for the HP is to scale up this intervention by attaching it to social networks such as the regular family development session of 4Ps (Pantawid Pamilyang Pilipino Program) or the conditional cash transfer program. Opportunities for Usapan scale-up include integrating it with the regular clinic sessions (like antenatal classes) and putting forth a deliberate effort to engage special groups like out-of-school youth, older clients, men, and post-pregnancy clients. As pointed out by FGD participants, the main source of information on FP are rumors or casual conversations among neighbors and peers, typically involving details that are false. In engaging these special groups, it may be worthwhile for the HP to identify where these conversations take place and explore how to integrate correct and valid information. Usapan is a good demand-generation tool. However, it targets those who are already open to using FP. It is a service-oriented FP dialogue designed to allow clients to listen to concerns from people who are less aware or are ashamed to ask questions. To address concerns that only those who are about to decide, or have already decided, to use FP are likely to join these sessions, Usapan should be monitored systematically, including follow-up with attendees. Usapan sessions need to balance discussions on decisions, preferences, and choices, which can be discussed privately during individual counseling sessions. Social media campaigns and IEC materials for FP/ARH The HP effectively used social media to create demand for FP/ARH. Social media campaigns such as “It’s Okay to Delay,” “Konektado Tayo,” and “Usap Tayo sa FP” reached millions of potential beneficiaries. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 20 However, we cannot ascertain the effectiveness of these online engagements in translating reach into actual behavior change. The HP also developed FP videos with songs and messages to address fear and anxiety about FP, such as the “Sama-sama Tayong Mag FP” video with English and Tagalog subtitles, with 30- and 60-second versions for TV and radio spots, plus infographics and flyers. These IEC materials are best used if lodged in well-planned SBCC platforms with specific message strategies. Visible SBCC work also reinforces community-level efforts, especially in indirect and peripheral routes to expanding and changing mindsets. If media content is available, health care providers can not only point their clients to this content, but they themselves can continually upgrade and update their health education. On interventions for behavior change on TB Social media campaigns for TB The HP supported the DOH in developing the National TB Health Promotion and Communication Strategy 2020–2023, but there is a gap in terms of systematic implementation of interventions. Various approaches were used, such as the #TBFreePH campaign via social media; “Tibay ng Dibdib,” a storybook campaign for children; and the “Ayos Ka Lungs?” campaign, which included engagements with entertainment celebrities. The effectiveness of these campaigns can only be partially determined. For example, for the Tibay ng Dibdib campaign, the number of subscribers for each platform of the media partner was mentioned in one IP’s annual report, but there were no metrics to determine engagements. The report mentioned 249,000 YouTube subscribers for Net 25 TV. These reach metrics should be substituted with engagement metrics, such as a brief survey asking the clients what prompted them to use the service and their sources of service information. The creation of the Tibay ng Dibdib campaign can be considered an empowering spontaneous SBCC innovation. However, there is a need to establish a more systematic feedback mechanism from the health care providers/implementers. The impact of SBCC, although often long-term in fruition, can provide windows for provisional appraisal through the stories from the ground noted by the health care providers. This implies the importance of IP staff having a strong SBCC mindset and experience. They are the ones who could be quick to pick up these experiences and transport them into creative ideas for a scalable SBCC campaign. The #TBFreePH FB page/group is a gold mine for SBCC data generation that has potential for the creation of both specific and universal messaging for SBCC. Inside the online support group, there are frequently asked questions showing common myths about TB treatment and recovery; narratives of coping and treatment; and even stories of personal encounters with health care providers. This can be strengthened by a supplemental social media portal that gives compact media content that can be passed around without requiring membership. Group membership can still appear as an important commitment to many, therefore deterring potential TB patients from moving closer to seeking treatment. An open￾to-all social media page can be a one-stop shop for health care providers and TB patients and their families. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 21 HP-supported online patient support groups for TB can be a platform for the analyses of patient preferences and sentiments to improve healthy behaviors. An example that can be used in TB is “social listening,” which the DOH uses to correct publicly provided information on COVID-19. HP support of active case finding The HP supported active case finding in communities and enhanced facility-based case finding. These are not demand-generation activities but are effective alternatives to passive case finding. As of FY 2021, Find Actively, Separate Safely, and Treat Effectively (FAST) Plus is being implemented in 54 hospitals in the NCR, four hospitals in Regions IV-A and VIII, and eight hospitals in Region III.9 FAST Plus is an effective strategy for identifying, isolating, diagnosing, and managing patients coupled with infection prevention and control and health care worker surveillance for TB. HP support of TB advocacy Although the TB Local Organizations Network (TB LON) supports the Philippine Alliance to Stop TB, which has a strong focus on TB patients, it is not clear whether TB LON differentiates itself from similar work by the Philippine Coalition Against Tuberculosis (PhilCAT), given that PhilCAT counts community service organizations, individual providers, and public agencies as members and seems to be doing the same activities. Cooperation and coordination between these two organizations can lead to scaled effects. TB LON does advocacy work to increase TB funding at the national and local levels. This needs to be connected to activities that ensure programs are accessible to, and addressing health behaviors of, the underserved. TB LON should seek out more nontraditional and non-TB organizations representing vulnerable groups, such as farmer cooperatives, public transport driver associations, and other organizations representing marginalized sectors. TB LON should conduct research on stigma to determine its pervasiveness and its patterns across population groups so that the right interventions can be designed. It is also important to determine SBCC strategies to promote primary care practices (e.g., hand washing, cough etiquette, distancing, use of masks) that are effective for a class of communicable diseases, including TB. On interventions for behavior change on CBDR Although HP support focuses on the client journey to recovery and reintegration, many LGUs still subscribe to “fear mode” on the rehabilitation of PWUD. A crucial step is for HP and its government partners to finalize and agree on a common set of CBDR indicators linked to national CBDR program goals. Formative research on PWUD influenced the branding of CBDR programs, including Katatagan Kontra Drogas sa Komunidad (Resilience against Drugs in the Community) and Katatagan, Kalusugan at Damayan 9 These figures are from the FY 2021 Annual Report of TB Innovations. The Q1 FY 2022 report of TB Platforms state that 219 hospitals have been covered. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 22 ng Komunidad (Resilience, Health, and Care in the Community) to minimize stigma and highlight drug use as a health issue. Lusog Isip (Healthy Mind) is an online app that encourages seeking help and facilitating self-awareness and self-love among PWUD. It has been pilot-tested and will be rolled out in FY 2022 through the DOH mental health program. The HP’s work on CBDR shows that at the end of FY 2021, a total of 100,772 PWUD have sought help for their substance abuse problem (or 64 percent of the life-of-project target). The HP’s work on CBDR, specifically on the SBCC paradigm, is manifold. It is working toward changing many behaviors in a general sense. USAID should sift through the identified behaviors that PWUD are encouraged to change and zero in on more specific aspects of health behaviors that are obstacles to change. For example, from the general target of removing stigma against addiction, CBDR should sharpen the focus on collective care as the pathway to recovery. In this pathway, HP can magnify the role of health care providers, communities, and the families of PWUD. Materials are available in government portals and the HP IP’s portal informing stakeholders of the role of community in CBDR. However, a strong SBCC program that targets specific behaviors and mindsets—among PWUD as well as the general population—has yet to be drafted. Role of CHWs CHWs (and MRLs, in the case of BARMM) play a key role in supporting FP and TB service provision. CHWs have successfully assisted the community by providing preventive and primary health care services, particularly during the pandemic. As residents of geographically isolated and disadvantaged areas in BARMM are less receptive to FP, CHWs work in tandem with MRLs to provide information based on fatwa. In Lapu-Lapu City, 90 percent of CHWs are treatment partners for TB. A recurring concern is that CHWs are overworked and underpaid. CHWs, who are predominantly women, serve as the backbone of the health system at the lowest organizational level and carry on their shoulders the success of FP/ARH and TB interventions. However, they are often shifted to do multiple tasks. Before 2018, a ratio of 1:20 households per CHW was the basis for tasking for all FP and TB concerns of adults, children, and adolescents. Now, the assignment is over 1:100 households, with at least two days of task-sharing duty at rural health units (RHUs) for client intake, registration, and cooking. CHWs are key to reaching adolescents and engaging male clients. CHWs with the barangay and youth organizations are also effective in reaching adolescent out-of-school youth. There are very few male CHWs, although the MOH in BARMM estimates that 30 percent of its CHWs are male. Some gay males were cited as effective in connecting with both men and women in barangays. Legislation to support CHWs has been pending for years and has not been considered a priority. Most CHWs get an allowance of only PHP 500 to PHP 3,000 per month. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 23 HOW HAS THE USAID HP LED TO CONTINUOUS QUALITY IMPROVEMENT IN SERVICE DELIVERY? a. How has the HP improved the skills of FP/ARH health providers? b. How has the HP expanded FP services? c. What were the TB project’s key contributions to TB control in the Philippines, and to what extent has HP addressed key TB issues and gaps? d. What were the three most effective packages of TB and FP interventions that could be scaled up nationwide? There were fewer interventions directed at continuous quality improvement (CQI) initiatives in health centers compared to public and private hospitals. Moreover, while USAID wanted to shift the HP away from health service delivery to HSS activities (training, policy, logistics), many LGUs still express a preference for continued USAID support in providing additional staff and a steady supply of commodities and educational materials, especially for FP and TB. But efforts to institutionalize CQI have been stalled in many sites that were closed during the COVID-19 pandemic, including those for adolescents and postpartum women and TB cases. The HP needs to support the DOH and LGUs in developing a catch-up strategy to address delays and disruption owing to the pandemic. Overall findings in the assessment of performance indicators to improve quality of service delivery On FP/ARH The HP provided technical assistance to more than 2,000 service delivery points (SDPs) providing FP counseling and services. This represents 85.6 percent of its LOA target (Table 8). However, in terms of functional adolescent-friendly SDPs, HP support in non-BARMM areas achieved only 6.8 percent of the LOA target. In BARMM, around 36 facilities, or about 65.5 percent of the HP’s LOA target, was achieved (Table 9). Table 8. Percentage of USG-assisted SDPs providing FP counseling and/or services Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished Health Project 75.6% 97% 83.1% 85.6% Numerator 1,643 2,622 2,126 Denominator 2,173 2,698 2,557 Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet Table 9. Number of functional adolescent-friendly SDPs Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished ReachHealth N/A 266 18 6.8% USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 24 BARMMHealth N/A 55 36 65.5% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet Although there are fewer functional adolescent-friendly health SDPs in non-BARMM-supported areas, the HP catered to 148,568 adolescents, or 184.2 percent, of its LOA target (Table 10). The low utilization in BARMM may be associated with adolescents’ cultural and social norms and family influence in the region. Given this, USAID may want to promote a wide range of services offered in adolescent￾friendly facilities, such as mental health support and nutrition, and not just associate adolescent services in the facilities with sexual and reproductive health (SRH). Table 10. Number of adolescents using FP/SRH services in supported adolescent￾friendly SDPs Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished ReachHealth 69,350 80,650 148,568 184.2% BARMMHealth 0 27,055 2,489 9.2% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet The HP established only one informed choice and voluntarism (ICV) compliance committee (out of the LOA target of 32) in the HP-assisted LGUs, excluding BARMM. At the regional level, there was none reported among the 11 regional offices supported by the HP. In BARMM, three of six ICV committees, or half of the LOA target, are already in place. Because this is a requirement by law and is mandated by DOH Administrative Order (AO) No. 2011-0005, HP may want to shift its support to the validation and monitoring teams and ensuring quality standards for FP service delivery. Table 11. Percentage of USG-assisted DOH regional offices and LGUs with functional ICV compliance committee/monitoring team Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished ReachHealth 0.0% CHDs 100% 0.0% 0.0% Numerator 0 Denominator 11 LGUs 0.0% 100% 3.1% 3.1% Numerator 1 Denominator 32 BARMMHealth 0.0% 100% 50.0% 50.0% Numerator 3 Denominator 6 Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 25 On TB HP support of bacteriological diagnosis coverage for pulmonary TB achieved 39 percent in NCR, 46 percent in Region III, and 45 percent in Region IV-A (Table 12). We suggest the reporting of actual numbers in addition to percentages to better assess the scale and coverage of continuing assistance. Table 12. Bacteriological diagnosis coverage (pulmonary TB) Baseline LOA Target Accomplishment as of Q1 CY2021 Percentage Accomplished NCR 60% 39% 65.3% Region 3 60% 46% 77.0% Region 4A 60% 45% 74.8% Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet In terms of private-sector TB notifications, the HP achieved close to 50,000 in NCR, more than 66,000 in Region III, and more than 89,000 in Region IV-A (Table 13). These accomplishments may have to be compared with the burden of TB cases in these regions to assess the scale and coverage of support provided. Table 13. Private-sector TB notifications Baseline LOA Target Accomplishment as of Q1 CY2021 Percentage Accomplished NCR No data 49,406 N/A Region 3 No data 66,106 N/A Region 4A No data 89,217 N/A Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet For the percentage of new and relapse TB patients tested using a WHO-recommended rapid diagnostic test for NCR and Regions III and IV-A, the HP reported 27 percent, 40 percent, and 35 percent actual accomplishment, respectively, as of Q1 CY 2021. Table 14. Percentage of new and relapse TB patients tested using a WHO￾recommended rapid test at the time of the diagnosis, USG-assisted sites Baseline LOA Target Accomplishment as of Q1 CY2021 Percentage Accomplished NCR 30% 70% 27% 39.0% Region 3 15% 70% 40% 56.6% Region 4A 11% 70% 35% 50.7% Source: HP PITT_210701.xlsx, TB indicators-CY2021 Q1 sheet Average client satisfaction score of clients who came for FP/RH and TB services As of Q2 FY 2021, the HP work on FP/ARH (excluding BARMM) included surveying 1,520 clients in 33 HP-supported sites. About 92 percent of clients gave an average rating of 3–4 (out of 5) for each of the following: health facility factors, interpersonal skill of providers, and service delivery factors. In BARMM, USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 26 in a survey conducted during Q1 2021, the average satisfaction reported was 3.8 out of 5 among randomly selected WRA in four HP-supported provinces. In Q2 FY 2021, the HP surveyed 43 clients in five HP-assisted provinces, who gave an average satisfaction rating of 4.8 out of 5. It is useful to analyze feedback from both clients and nonclients and pay attention to the comments from those who gave low feedback, because these indicate areas where services were deficient or unsatisfactory. Feedback can also be taken from the health providers regarding their perceptions of the program’s strengths and weaknesses. For TB, the HP pilot-tested client satisfaction from Q4 FY 2020 to Q1 FY 2021. The survey reported an average satisfaction score of 4.6 (out of 5) among the 550 TB patients surveyed. USAID must make efforts to expand measurement of client satisfaction ratings to inform current and future needs of FP/ARH, TB, and CBDR clients. The HP has yet to report its actual accomplishment on contact investigation coverage for pulmonary TB, TB preventive treatment coverage, and the number of people with improved access to services. Overview of interventions and innovation supported by HP to improve quality of service delivery On HP interventions to improve quality of FP/ARH services The HP conducts regular CQI initiatives in public and private hospitals. Key characteristics the HP looks at are client satisfaction, privacy, cleanliness of clinical areas, efficiently used waiting times, and respectful and knowledgeable providers. In BARMM, the HP supported the provincial health offices in establishing 68 CQI sites and trained 195 health care providers. Meanwhile, USAID also supported 32 LGUs outside BARMM in establishing their CQI initiatives. These CQI methods may be adapted for health centers, although quality indicators and parameters need to be defined within the context, workflow, and scope of services offered in a primary health care setting. There are several CQI frameworks and indicators, which include key quality indicators, certain infrastructure standards such as privacy and cleanliness of facilities, client satisfaction, and availability of commodities. Analysis of quality indicators in the HP results framework will provide information on areas that need improvement. The analysis should involve the DOH and LGUs as part of the process in determining standard quality indicators for FP/ARH. In health care, it is especially important to pay close attention to CQI monitoring to make timely local adjustments. The FP in Hospital Program is another HP intervention that has reached sufficient maturity for broader adoption. The program acknowledges the key role of hospitals in increasing demand for FP, providing quality FP services, and creating an enabling and respectful environment for FP/RH. The program is running in 162 public and private hospitals nationwide. While the WOPE team recognizes that hospital￾based providers are more than capable of providing counseling services, especially for teenage mothers, the challenge is to implement the program without overburdening hospital staff. Despite the COVID-19 pandemic, the FP program in hospitals continues to run in 46 health facilities through the Makeshift Teleconsultation platform developed with HP support. “Makeshift” connotes using a basic laptop, mobile phone, and Internet access sans more sophisticated computers and complex USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 27 databases. The core component of this package is the manual of operations, which specifies the requirements, workflows, and processes to conduct the teleconsultation. The WOPE team encourages hospitals implementing teleconsultation to provide it as a permanent alternative service for those unable or afraid to attend an in-person visit because of the pandemic. The WHO published two global guidance documents on task shifting10 and task sharing.11 Using these guidelines, USAID should explore how it can expand training on Family Planning Competency-Based Training (FPCBT) Levels I and II for health providers such as nurses and midwives. The training includes management of complications (a referral process to a high-level cadre or specialist care). However, because of the limitations brought about by the COVID-19 pandemic, no FPCBT training was held during the lockdowns, particularly from March 2020 to August 2021. The HP started migrating the FPCBT training online and will scale its implementation at the DOH e-Learning Academy. The HP tapped the services of BHWs to deliver FP information and commodities, especially during the initial onslaught of the COVID-19 pandemic. The HP provided support in developing materials and tools for training the BHWs on basic FP service delivery tasks. Coverage of these materials and tools includes information dissemination, mapping of WRA, providing condoms and pills, addressing missed pill concerns, helping clients correctly use the lactational amenorrhea methods, and transitioning to a more effective FP method. On HP interventions to improve quality of TB care With the HP’s assistance, DOH introduced and implemented an all-oral short-term regimen nationally for MDR-TB. This support is expected to ease administration and shorten treatment duration, enabling better adherence and improving treatment outcomes. This was a rare instance when the pilot phase of innovation leapfrogged to the scale-up phase. Another example is the video technology to undertake DOTS, which selected LGUs had already introduced and employed. The HP supported the establishment of the Philippine Private Sector Diagnostics Consortium, which addressed the high cost of GeneXpert testing in the private sector, especially because most patients prefer to seek diagnostic and treatment services from private providers. As of August 2021, the consortium, which currently has 20 members, performed 5,798 tests, and detected 1,161 DS-TB cases and 101 RR-TB cases using the Xpert MTB/Rif test. The National TB Control Program (NTP) wants to continue this consortium, with commitment from PhilCAT to host it when the HP ends. Technology-based innovations abound in many of the HP-supported activities on TB. The challenges are (a) how to pick which ones are high yielding versus “boutique”-type interventions, (b) how to identify difficulties in introducing these at scale, (c) how national and local partners can pay for and sustain these innovations, and (d) how to resolve operational issues (such as legal and privacy issues). 10 World Health Organization. (2008). Task shifting: Global recommendations and guidelines. World Health Organization. 11 World Health Organization. (2017). Task sharing to improve access to Family Planning (No. WHO/RHR/17.20). World Health Organization. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 28 The HP also currently supports validation of the new Xpert MTB/Rif Ultra, which is more sensitive than Xpert MTB/Rif; however, validation has been done only for certain specimens. There is also a possible issue of getting many trace results with Xpert MTB/Rif Ultra if it is used in a highly endemic setting. The HP and DOH should consider these possible challenges with Xpert MTB/Rif Ultra rollout plans. Regarding improving compliance to treatment, some technology-based innovations include digital adherence tools such as the Video Observed Treatment and ConnecTB. Results from the 414 patients enrolled in ConnecTB are promising thus far, showing over 95 percent adherence, with improved reporting and management. These new technologies introduced by the HP are likely to realize desired outputs well beyond project life. Because many HP innovations being introduced are still in the pilot and evidence-generation phase, the challenge is for these innovations to generate substantial benefits during the remainder of the HP’s life. The operational costs involved in developing and implementing these technologies may be helpful for future evaluations by the DOH Health Technology Assessment Council (HTAC). On HP interventions to improve quality of CBDR services The DOH had already unfolded its program for PWUD before the HP’s CBDR program, hence USAID￾supported activities and inputs had to work their way through an existing approach which is facility￾based rehabilitation and mindsets (fear factor in Operation Tokhang). USAID’s CBDR program has made its presence felt in DOH and DDB communication/social media portals and in providing technical assistance in capacity-building. In June 2020, the New Client Flow process was launched, providing a clearer path forward for CBDR implementation in its second year. New Client Flow standardized screening tools and placement of the PWUD in the appropriate treatment mode depending on their risk level. This was the HP’s most substantial early project support in CBDR, as it cemented its presence through this collaborative relationship with the government. The HP provided training inputs (through the DOH Academy), including videos, webinars, client primers, and a service directory. The HP’s support of CBDR is perceived as model-building and, therefore, indirectly includes providing services to the community’s PWUD and their families. It equips LGU and health care provider stakeholders with tools for CBDR processes. As of Q3 FY 2021, the LGU project sites have screened 50,635 PWUD in LGU partner sites, which is 64 percent of its LOA target. As part of its culturally appropriate screening training provision, the HP conducted online webinars and train-the-trainers’ sessions as part of the rollout of CBDR screening tools. To date, the HP has trained 248 screening providers on screening, brief intervention, and referral to treatment (SBIRT). Partner agencies have validated these technical assistance inputs on screening as helpful. In FY 2019, the CBDR program also conducted formative research, needs analyses with PWUD, baseline surveys, and consultations, and advanced the policy environment for community-based rehabilitation. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 29 HOW HAS THE USAID HP SUPPORTED THE STRENGTHENING OF HEALTH SYSTEMS AND THE OPERATIONALIZATION OF UHC IN THE PHILIPPINES? a. What HSS reforms are in place because of USAID interventions? How are these HSS reforms impacting health outcomes? b. What HP interventions have improved the effectiveness of DOH (DPCB, HPB, Pharmaceutical Division, HHRDB, Dangerous Drugs Abuse and Prevention and Treatment Bureau, its information systems); MOH BARMM; PHIC; and DDB in delivering their mandates? c. What HP interventions have improved local health system capacity for policy formulation, budget planning and execution, inter-local cooperation, program implementation, and M&E? d. What HP interventions have increased national and local capacity in health systems management in sustainably supporting TB and FP/ARH programs? e. What HP interventions have helped expand and build health provider capacity for quality health care delivery for FP/ARH, TB, and CBDR? f. What HP interventions have helped in improving supporting supervision and mentoring of HRH? g. What HP interventions have helped ensure TB and FP/ARH health commodity security nationally and locally? h. How has the HP supported improvement of the TB, FP/ARH, and CBDR information systems? Overall findings in the assessment of performance indicators for HSS/G Three sets of indicators are used to track performance related to HSS/G. The first set refers to the extent that financing for TB and FP services are being secured via social health insurance and health budgets: (a) percentage of people covered under financial protection programs in HP-assisted sites, (b) ratio of total claims paid over benefits claimed for FP and TB, (c) proportion of domestic financing for TB, (d) disbursement rates of government budgets for TB and ITIS, and (e) proportion of domestic financing for TB. As of Q2 FY 2021, 88 percent of people in HP-assisted sites have been enrolled in PhilHealth. This figure varies by source of estimates. PhilHealth records tend to show higher figures, but there is an ongoing need to regularly update enrollment data to account for births, changes in membership status, and deaths. PhilHealth has attempted to clean up its membership data and keep it up to date. A more effective approach would be to condition the release of national government-funded premium subsidies to verified PhilHealth members, starting with matching indigent program members with those in the 4Ps roster of DSWD. The ratio of claims paid over total claims filed (or support value) is a measure of the extent of PhilHealth’s financial protection. Support value data for FP and TB are not available. Key informants reported the budget disbursement rate for TB in 2020 to be 54 percent and the budget disbursement rate for the Family Health and Responsible Parenthood Unit for 2020 to be 18 percent. Budget disbursement rates are likely to remain low, especially with delays and disruptions due to the COVID-19 pandemic. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 30 As of 2021, NTP program managers reported the domestic financing for TB to be 26 percent. Key informants have raised concerns that foreign assistance for TB may be a preferred funding source due to its less complicated and more transparent procurement processes. On stockout rates of contraceptive commodities and anti-TB drugs and TB laboratory tracer commodities The second set of HSS indicators being monitored refers to the efficiency and effectiveness of the logistics and supply chain management (SCM). In the case of FP commodities, reported stockout rates in HP-assisted sites are mostly better than baseline, and some (pills, injectables, intrauterine devices [IUDs]) are close to LOA targets. But the reported performance in BARRM needs to be carefully examined. Key informants from USAID and the DOH reported that these figures may not reflect actual conditions. The reported low stockout rates in USG-assisted sites do not jibe with the key informants’ concern about the need to improve the supply chain for commodities. The HP collects two sets of FP commodities stockout data, but only one is reported in all three HPPE annual reports and the HP PITT (Table 15). For example, the HP Medicines, Technologies, and Pharmaceutical Services (MTaPS) reports higher stockout rates across commodities at the USG-assisted sites (Table 16), which are closer to the rates the DOH reports. Table 15. Average stockout rates of contraceptive commodities, disaggregated by method at USG-assisted FP service delivery points in USG-assisted sites Baseline LOA Targets Actual LOA average rates as of Q2 FY2021* ReachHealth Pills 23.4% 7% 8.5% Injectables 18.4% 7% 9.9% Condoms 27.8% 7% 19.9% IUD 13.9% 7% 8.5% PSI 23.9% 7% 33.3% Standard days method (SDM) beads 49.8% 7% 32.7% BARMMHealth Pills 4.2% <10% 0% Injectables 7.5% <10% 16.6% Condoms 7.5% <10% 18.9% IUD 0% <10% 0% PSI 0% <10% 8.8% SDM beads 27.5% <10% 0% Sources: ReachHealth data are based on total USGA SDPs reporting stockouts in USGA sites as of Q2 FY2021 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 31 Table 15. Average stockout rates of contraceptive commodities, disaggregated by method at USG-assisted FP service delivery points in USG-assisted sites BARMMHealth data are based on total USGA SDPs reporting stockouts in USGA sites at midyear FY 2021 In the case of stockout rates, the use of USAID’s traffic light system for assessing performance does not appropriately show performance. Stockout rate is a ceiling (i.e., an upper limit threshold), so if the actual rate is higher than the target, it is not a positive accomplishment. Table 16 shows that the HP missed the full year FY 2021 target stockout rate for FP injectables by 53.3 percent ([23 percent-15 percent/15 percent] x 100). The HP missed the target for FP implants by only 5 percent, but it exceeded the target for FP male condoms by 16.7 percent. There are wide variations in the baseline stockout numbers, and the LOA targets for some can be considered ambitious—for example, the baseline rate for progestogen-only oral contraceptive pills is 69 percent, yet the LOA target is 15 percent; the same can be said for SDM beads. The HP should include a more in-depth assessment of the drivers of utilization of each commodity—which can be collected from facility intake data—to inform commodity quantification, allocation, and distribution and program development Table 16. Reported average stockout rate of contraceptive commodities, disaggregated by method, at USG-assisted FP service delivery points in USG-assisted sites (MTaPS FY 2021 AMEL/PITT) FP Method Baseline FY 2021 Q4 FY 2021 Full Year LOA Target Actual Target Actual Target Injectable 30% 15% 31% 15% 23% <10% Subdermal implant 53% 40% 39% 40% 42% <15% Combined oral contraceptive pill 26% 10% 25% 10% 14% <10% Progestogen-only contraceptive pill 65% 25% 21% 25% 24% <15% IUD 37% 30% 42% 30% 42% <15% Male condom 39% 30% 20% 30% 25% <15% SDM beads 75% 30% 73% 30% 79% <15% Except for TB medicines for pediatric cases and drugs for TB preventive treatment (TPT), the HP seems well on its way to achieving LOA targets for stockout rates. The discussion below raises similar questions that were raised for FP: What are the incremental gains from efforts to reform the supply chain and logistics for TB drugs? Is there a system problem, or are the reported stockout rates specific to drugs for childhood TB? Because the state-of-the-art system that was planned to solve SCM problems has not been operationalized—and its effective institutionalization by the end of LOA is uncertain—the impact of the planned reforms cannot be ascertained at this time. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 32 Table 17. Average stockout rates of selected anti-TB drugs and TB laboratory tracer commodities at USG-supported NTP facilities in USG-assisted sites Baseline FY 2021 Targets Actual as of Q2 FY2021 Accomplishment as of Q2 FY 2021 TB first-line drugs, 4 drug regimen 41% 25% 22% Exceeded target by 12% Pediatric TB, 4 drug regimen 91% 45% 46% Missed target by 2.2% TPT drugs 64% 50% 79% Missed target by 58% Levofloxacin 500 mg 49% 10% 18% Missed target by 80% Moxifloxacin 400 mg 4% 10% 6% Exceeded target by 40% Linezolid 600 mg 20% 10% 8% Exceeded target by 20% Bedaquiline 21% 10% 8% Exceeded target by 20% Xpert cartridge 15% 15% 1% Exceeded target by 93.3% Overview of HSS/G interventions and innovations supported by HP Over 130 interventions, innovations, and policies and regulations were introduced with support from the HP. Several interventions are expected to directly benefit quality of service delivery at scale but can be expected to yield impacts over the medium- and long-term. An example is HP support of local procurement and SCM with its act-on-site mechanism. Interventions with large-scale but indirect effects over the medium term include policy advisories on the effects of the COVID-19 pandemic and recommendations on how these might be mitigated. Advisories on the development of PhilHealth benefit packages (e.g., Konsultasyong Sulit at Tama or Konsulta) have potential scale effects but can be realized only in the long term. The HP needs to consider alternative indicators that reflect the true nature of progress in these HSS/G areas. Thus, USAID might need to focus the HP on fewer sets of HSS/G interventions, especially those with large-scale and direct effects on target outcomes such as the support provided in the development and rollout of Konsulta and the eLMIS. An inventory of these HSS/G-related interventions is provided in Supplemental Material L. On HSS reforms implemented and their impact on health outcomes HP supported the development of the UHC Law’s IRR and the Memorandum Circular 2021-0001 by the Department of Budget and Management (DBM), the DOH, DILG, and PHIC, which established the guidelines for allocation, utilization, monitoring, and accountability for the Special Health Fund (SHF). The HP also helped formulate more than 50 policy guidelines in the UHC integration sites (UIS). These enabling policies are crucial in ensuring that poorer LGUs have the capacity to continue providing the health services that will be developed but likely without the equivalent local fiscal resources, even with the Mandanas-Garcia ruling. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 33 Moreover, the HP assisted PHIC in developing the Konsultasyong Sulit at Tama (Konsulta) package. This new primary care benefit package under the UHC Law is being pilot tested in more than 130 providers as of September 30, 2021. Konsulta was built on the expanded primary care package and now includes coverage for screening and diagnostics for TB that are used in public health facilities. Future integration of TB-DOTS services in the Konsulta package is one way to increase access and utilization of services through the participation of more public and private providers. While FP services are still not formally covered under the Konsulta package, accredited providers can provide services such as IUD insertion, PPIUD, and PSI implant and can file claims through the usual PhilHealth claims process. The USAID HP was also instrumental in establishing the DOH Academy, particularly its e-learning portal, which was helpful during the pandemic, when in-person training was suspended. While we cannot ascertain competencies gained, there has been an explosion of access to the online courses. The effectiveness of these nontraditional learning platforms remains to be assessed. The HP also assisted the DOH in securing an open-source electronic logistics management and information system (eLMIS). The DOH will soon complete the system’s configuration. User acceptance is being tested among DOH warehouses, select Centers for Health Development (CHDs) and LGUs, and health facilities. Once this is completed, the DOH can roll out the eLMIS to more CHDs, LGUs, and facilities, with a goal of around 170 sites by June 2022. It is expected that by June 2023, the system will be operational in all DOH warehouses, CHDs, and public facilities and in LGUs that have signified concurrence or commitment. However, the eLMIS is only as good as the skills of the people entering and analyzing the data and the ability to supply the required commodities. Without concurrent improvements in these two areas, there is no assurance that the system will be fully functional or that its use will continue. Finally, USAID supported the DOH in establishing the HTAC as mandated by the UHC Law. The HTAC is expected to improve assessment and fast-track the approval process for medical technologies. This was particularly highlighted during the pandemic, when emergency use applications for COVID-19- related technologies like testing kits had to be quickly processed. On HP interventions that have improved the effectiveness of DOH and other stakeholders in delivering their mandates The assistance in budget analysis and financial planning has contributed to improving the use of funds at the DOH. This has multiple positive effects on health outcomes beyond FP and TB, as a more efficient budget utilization improves all aspects of DOH operations. Training and mentoring support for the DOH from previous HPs and the current HP contributed to increasing the budget obligation rate to 94 percent in 2018 before it went down to 88 percent in 2020. However, disbursement rates went from 58 percent in 2018 to 83 percent in 2020. For the TB program, the NTP has been constantly hovering around PHP 1 billion (from 2014 to 2021), with obligation rates rising from 65 percent in 2014 to 99 percent in 2020 and disbursement rates increasing from 26 percent in 2014 to 54 percent in 2020 (zero USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 34 disbursement in 2018 despite obligating 61 percent, due to procurement issues). On the other hand, the National Family Planning Program budget has been increasing but disbursement rates remain low, reportedly at only 18 percent in 2020. DOH program officers particularly appreciated the budget analysis assistance because they could track funds utilization as they received regular updates from the HP. To sustain the gains in this area, HP should continue assisting the DOH in budget analysis and financial management. In addition, it should extend more intensive training and mentoring of staff on procurement planning, particularly in taking advantage of multiyear budgeting, pooled procurement, and contracting or outsourcing. According to some informants, inadequate human resource staffing, staff transfers, and the inability of trained staff to share knowledge and skills all contribute to persistent procurement planning issues. The reorganization of the DPCB might enable this sharing of technology by requiring staff members to share knowledge and skills in their new assignments. In addition to budget analysis, the HP helped with cost estimates for responding to COVID-19 requirements, which was instrumental in fund allocation for the Bayanihan 1 and 2 Laws. Moreover, estimates of COVID-19 cases were used in case rates implemented by PhilHealth in setting claims. Multiyear financing grants/contracts and pooled procurement affords the DOH the advantages of economies of scale and allows it to use its size and volume to negotiate for lower costs and better services. The HP facilitated the development of a Technical Advisory on the Guidelines and Procedures in Implementing Framework Contracting and Pooled Procurement for Drugs, Medical Devices and Supplies. The use of multiyear financing agreements was also presented to local executives, including in BARMM, as a viable option for procuring essential goods and services. However, this financing mechanism, implemented by the DBM, was not used in 2020 due to lack of time to complete requirements and processes. In light of the Mandanas-Garcia ruling, the DOH should start coordinating and negotiating with the LGUs the management (rational quantification, purchase, distribution) of supplies using the scale advantages afforded by pooled procurement and multiyear financial agreements. Technical assistance on pharmacovigilance, health technology assessments, and FDA approvals is crucial in ensuring a capable and efficient health system that provides high-quality goods and services. Upgrading the pharmacovigilance system ensures quality of drugs, although partners would appreciate the support more if the system upgrade would apply to all drugs and medicines. The support in streamlining the FDA assessment and approval process was especially useful during the pandemic, when emergency use applications for medicines were enforced. The challenge is to ensure that staff members in the relevant offices can apply the knowledge, skills, and technologies learned to other concerns that are not the HP’s primary focus, and that they can make sure the results of the technical assistance and interventions are sustained. At subnational and local levels, the HP also helped develop tools that enabled POPCOM, PhilHealth, the DOH, and other partners to deliver services during the pandemic. Notable tools included: (a) an Excel spreadsheet to quantify FP commodities for subnational allocation, (b) models to estimate COVID-19 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 35 case management costs as a basis for PhilHealth case rates and to conduct budget analysis; and (c) a mobile application (ITIS Lite) to encourage providers to report cases to ITIS. Future HP efforts to help LGUs get facilities certified and accredited should include advocating for amendments in the PhilHealth requirements and addressing delays in processing and reimbursements to expand services and encourage more lying-in centers, health centers, and providers. On HP interventions that have improved national and local health system capacity Some LGUs have been receptive to assistance provided to local chief executives and health officials through the HP’s Institutionalization of Health Leadership and Governance Program (IHLGP). One CHD has partnered (through memoranda of agreement and scholarship funds) with local universities to conduct the program for local health executives and integrate the program into the universities’ regular courses for health professionals. The IHLGP could help more local chief executives and health officials if it is institutionalized in regional academic and training institutions or national institutions such as the Development Academy of the Philippines, Local Government Academy of the DILG, or the UP National College of Public Administration and Governance. HP is actively supporting 12 out of 58 LGUs in planning for the UHC integration. As of November 2021, 35 out of 58 (60 percent) of the UIS have signed a memorandum of understanding with the Department of Health, although the remaining UIS have not completed their preparatory key result areas (KRAs) on strategic and investment planning for health. Almost all (98 percent) of the sites have yet to achieve organizational KRAs in financial management. The UIS have three years to prepare and set up all necessary KRAs, but the HP can accelerate these preparations, particularly in establishing SHFs and strengthening the capacity of provincial/city health boards to manage provincewide and citywide health systems. Half of the UIS have yet to start organizational and functional KRAs in HRH management. The HP can employ the Workload Indicators of Staffing Need (WISN) method to help the LGUs respond to this UHC integration requirement, but local HR managers should be given sufficient training to implement WISN. In terms of information management, 22 UIS (38 percent) have already achieved preparatory KRAs, indicating they are ripe for possible engagement as participants in the eLMIS rollout. The COVID-19 pandemic has affected the ability of many of the UIS to achieve the KRAs because their attention has been diverted to infection prevention. Moreover, there appears to be a lack of commitment from some LGUs to achieve UHC integration because they lack a sense of ownership of the KRA results. The HP can focus some of its resources to address these challenges and respond to UIS areas that need support. The design and establishment of service delivery networks (SDNs) through the previous HPs under the Health Sector Reform Agenda was taken up in the design of the Sin Tax Law (RA 10351), the RPRH Law (RA 10354), and the UHC Law (RA 11223). It was expected that the DOH would scale up SDNs with existing AOs (DOH AO 2017-0014 and 2018-0014), but this effort may have been given a low priority due to competing demands. Although the HP funds are earmarked for TB, FP, and CBDR activities, USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 36 technical assistance to expand the SDNs need not be constrained by these earmarks, because SDNs are system wide interventions supportive of TB, FP, and CBDR. For TB, the HP facilitated the development of TB advisories in LGUs, which led to secured funding commitment for TB. The HP obtained PHP 50 million local TB funds from 56 LGUs, according to WOPE team calculations from TB IPs’ annual reports. HP also supported the creation of policies and plans for the National TB Reference Laboratory (NTRL). The HP should take a closer look at how these plans are being implemented at the local level. The Citywide Tuberculosis Elimination Campaign (CiTEC) initiated by the HP is not expected to generate a significant number of diagnosed cases because the coverage area is too small (only two barangays out of 80). However, this can be a rich learning ground to inform future expansion to other areas. For CBDR, although the current activities are in their infancy, an important milestone was achieved when the DDB passed a resolution citing Katatagan, Kalusugan at Damayan ng Komunidad (Resilience, Health, and Care in the Community) as a model for community-based treatment (DDB Board Resolution 6, 2021). For SBCC, there is no strong indication from key informants that local partners have been capacitated to develop their own SBCCs. However, the HP must educate local partners on the proper use or optimization of SBCCs developed at the national level and made available online. On HP interventions that helped expand and build provider capacity and improve supporting supervision and mentoring A significant HP contribution is the institutionalization of the DOH Academy eLearning portal, where training modules and orientation programs have been made more accessible to public and private providers. Enrollment and visits to the portal dramatically increased during the COVID-19 pandemic. Enrollment in the Adolescent Health Education and Practical Training module rose from 77 in 2018 to 5,059 in 2021. There was also a big jump in enrollment (mostly in NCR) in the e-learning modules “Introduction to Seven Major Recommendations to Prevent TB Transmission” and “Overview of TB and Xpert MTB/Rif Assay.” The establishment of SDNs and health care provider networks (HCPNs) ensures clients receive the full range of services from a network of accredited providers, but capacities across LGUs are uneven, with few able to reach UHC maturity level for full UHC integration. Participation in the Konsulta package, developed with HP support, requires accreditation and certification that incentivize providers to upgrade equipment and facilities, enhance skills, and perform better quality services to secure a viable client population. In FP/ARH, training programs such as FPCBT are needed to ensure competent delivery of FP/ARH services. However, when in-person training was suspended and health providers were asked to assist in the COVID-19 response, training was moved online, which some saw as inadequate. In terms of supervision and mentorship, the Integrated Midwives Association of the Philippines (IMAP), through the continuous support from the HP, can now offer courses and supportive supervision and mentoring of USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 37 midwives. This can be expanded with additional HP support. The same model may be explored to provide remote or online supportive supervision and mentorship for FPCBT programs. The HP supported the training of 7,708 public and private providers on TB topics such as adaptive strategies to COVID-19, the sixth edition of the NTP Manual of Operations, mandatory notifications, and the use of the ITIS Lite mobile application. Another 1,546 providers were trained on the FAST Plus strategy. For SBCC, the HP provided scholarships to four DOH and POPCOM personnel for social media training conducted by the Johns Hopkins University. The training had an immediate effect on the POPCOM and DOH staff’s competency to design and carry out SBCC campaigns. HP-supported training programs in CBDR include coaching and counseling as well as accreditation of medical doctors for drug screening and the SBIRT/e-SBIRT training program. On HP interventions that helped ensure health commodity security nationally and locally This year, an eLMIS provider has been selected, and pilot implementation is set to commence with HP assistance. Although there were delays in the procurement of eLMIS, activities to enhance the capacity of staff for its efficient operation have been conducted at the DOH central and regional offices and even at selected public-facility pharmacies. However, the planned pilot implementation has yet to test its full functionality. Because of ongoing COVID-19 restrictions and possible delays due to the elections in May 2022, the institutionalization could be pushed beyond the LOA. USAID should consider extending support beyond the current LOA to ensure full operationalization. Concurrently, the HP supported the FP program, NTP, POPCOM, and LGUs to ensure availability of commodity and drug supplies, especially during the COVID-19 pandemic. These entities used POPCOM warehouses, created a list of local logistics providers to work around transport constraints, and instituted alternative delivery mechanisms to ensure supply availability and treatment adherence. On HP support to improve health information systems The Pharmaceutical Management Information System will be integrated into the eLMIS to harmonize data encoding and inform SCM. However, interoperability with the DOH’s many other information systems has not been ensured. One of the concerns raised is the harmonization of data systems to reduce burden on health workers at the facilities and to generate standard consistent data from one credible source. For example, the RPRH annual reports, the DOH Family Health Office, and the HP report different FP stockout rates for the same period. In the case of the DOH Knowledge Management and Information Technology Service (KMITS), it is important to determine the basic driver for using data, knowledge, and information. For the information systems to be relevant and useful, HP support to TB and FP may have to be delivered in a way that would benefit the entire DOH information system, which covers a wide range of programs. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 38 For purposes of policy reform and program implementation, indicator definitions and data should be standard, consistent, comparable across time and space, and readily accessible. One local partner suggested the use of the Community-Based Monitoring System (CBMS) established by law (RA 11315 in April 2019 with IRR approved in May 2020) as the primary monitoring and data source for community indicators, including those of FP, TB, HIV, CBDR, and PhilHealth membership and availment. The CBMS uses the cities and municipalities as the primary data processing authorities. Thus, it would be advantageous to use this information system as the consolidated system for LGUs in their local health investment planning and HCPN management. Key informants from the DOH and LGUs appreciated HP￾supported training on measuring and interpreting data indicators, which increased their understanding of the principles of data collection and analyses and why and how data are used. HOW HAS THE USAID HP CONTRIBUTED TO IMPROVING HEALTH OUTCOMES AMONG THE UNDERSERVED? a. To what extent have the HP activities contributed to achieving the targets (i.e., extent of performance relative to annual targets and end-of-project targets)? b. What were the critical enabling factors associated with achieving HP targets? c. What were the challenges and barriers in achieving HP targets? d. What innovations contributed to improving health outcomes? HP contributions to health outcomes are monitored using the following performance indicators: for FP/ARH, couple-years of protection (CYP) and total fertility rate; for TB, treatment success rates (TSRs) in DS-TB cases and MDR/RR-TB cases; and for CBDR, number of PWUD who completed treatment. Performance progress is tracked relative to baseline and end-of-project targets. The team reviewed this using data as of Q2 FY 2021, published in the PITT maintained by CLAimHealth. Overall findings At best, HP activities are close to meeting 50 percent of LOA targets. Many of the interventions have completed their design and initial implementation phases but have yet to gain enough traction and scale to register in regional-, city-, and provincial-level outcome indicators. The average contributions over time and across sites are verified using DOH FHSIS and ITIS data. The team performed a DID analysis using data on selected FP- and TB-related outcome measures from 2018 to 2020 across different provinces and cities. The team did not detect significant differences over these years and across sites. This is consistent with the observation that many of the HP-supported interventions remain in early implementation phases. Lockdowns and travel restrictions during the COVID-19 pandemic disrupted and delayed the implementation of interventions, many of which had just been introduced and were beginning to gain traction at the time. Interventions have gained traction where the HP effectively engaged national, subnational, and local health authorities and built on the gains of previous HP activities. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 39 HP interventions deemed ready for scaled implementation because of their initial traction and sustained support from partners include FP in hospitals; MRLs as public health leaders and SBCC champions; FAST Plus; IHLGP at DOH CHDs; Konsulta; Men’s reproductive health, including Katropa; the Comprehensive Sexual Education (CSE) ARH Converge program; and adolescent-friendly health facilities. There are also HP interventions that may have to be tested further and replicated to more priority sites so we can learn from the contextual nuances of implementation. These are: CiTEC, the Philippine Private Diagnostics Consortium, eLMIS, WISN, SBIRT, ConnecTB, and PhilHealth’s Catastrophic TB Package. The HP can benefit from diligent documentation of design concepts, iterations, and analyses of different interventions introduced and expanded by USAID support. Addressing issues regarding definitions, data sources, delays in data submission, and other inconsistencies could improve measurement of performance outcomes. This is critical if an impact evaluation of key HP interventions would be undertaken. Findings related to data and measurement gaps in the PITT are presented in detail in Supplementary Material I. On FP/ARH Based on the HP Midyear Report for FY 2021, HP-supported sites have reached close to 9 million CYP, representing 48 percent of the LOA target (Table 18). Table 18. CYP in USG-supported programs Baseline LOA Target Accomplishment as of Q2 FY2021 Percentage CYP in USG-supported programs 2,865,360 18,053,938 8,614,086 47.7% Source: HP PITT_210701.xlsx, FP indicators-FY2021_MY sheet Analysis of FHSIS data from 2018 to 2020 showed that sites in HP-assisted cities and provinces have the same CYP performance as non-HP-assisted sites. The CYP was calculated by multiplying the quantity of the FP method distributed to clients (available and as reported in the FHSIS) by a conversion factor. This suggests that HP contributions are not yet substantial enough to be detected by service utilization data at the provincial and city levels. A full analysis of this indicator, including a description of the methodology, can be found in Supplementary Material J. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 40 Figure 2. CYP growth rates, 2018–2020 The 2020 annual FHSIS report published by the DOH already includes data on adolescent birth rate. The highest birth rates are recorded in Region X (47 percent) and Region XI (49 percent). HP may want to consider assisting subnational and local health authorities in these regions to see how this can be addressed. The questions regarding the reliability of FHSIS data remain. Hence, performance, especially on adolescent fertility, may be effectively assessed only once the results of the 2022 National Demographic and Health Survey are available. Meanwhile, the HP relies on intermediate inputs as custom indicators to track performance. These are (a) the number of functional adolescent-friendly SDPs, (b) the number of adolescents using FP/ARH services in adolescent-friendly SDPs, and (c) the number of adolescents exposed to HP-supported FP messages. It would be helpful to know how these measures ultimately affect adolescent fertility rates. In addition to these proxy indicators, the HP is conducting sentinel monitoring of adolescent births in selected hospitals in HP-supported regions and cities. The critical enabling factors that may be associated with the reported HP contributions include buy-in from the DOH, POPCOM, and selected LGUs for interventions, such as the establishment of adolescent-friendly SDPs, FP in hospitals, and various training programs for FP/RH providers. Another enabling factor may be sharing knowledge and lessons from the implementation of those interventions, which leads to replication in other sites. While the Mandanas-Garcia ruling would have major advantages in the allocation of local resources for the LGUs’ health needs, this should include the assurance that FP or contraception services would continue. The experience of one major metropolitan city where the mayor prohibited FP methods other than natural, or fertility-awareness-based (FAB) methods, should not be repeated anywhere. While there can be FAB methods that are highly effective, this would be a narrow range of options and would deny clients the opportunity to use other highly effective methods that they prefer. Technical assistance to LGUs on the use of various contraceptive methods would be needed along with enhancing the role of the private sector to provide services in areas with limited or restricted FP services. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 41 However, there were reported barriers in the implementation of various interventions that need to be addressed, including difficulties in managing implementation in BARMM due to peace and order conditions, geography, special population needs, gaps in capacities of local health authorities, and reported delays and disruptions due to the COVID-19 pandemic. Performance in the FP indicators is encouraging, and there should be efforts to ensure sustainability of the programs—for example, online teleconsulting for regular checkups, use of BHWs in the information drive and distribution of pills and condoms, and use of the SDNs/HCPNs for referral of clients experiencing problems associated with contraception use. On critical state-of-the-art training Halfway through the LOA, the number of health workers who have gone through in-service training using nontraditional learning platforms is a third of LOA targets (Table 19). Unless efforts are ramped up, it is unlikely the LOA targets will be achieved. More importantly, from a system-strengthening perspective, it is essential to find out whether those who have been trained can provide sufficient evidence of improved competency levels that have led to better service delivery and evidence that this alternative platform is as effective as traditional learning platforms and is readily scalable. Table 19. Number of health workers who received in-service training using nontraditional learning platforms in FP and TB in USG-assisted sites LOA Target Accomplishment as of Q2 FY2021 Percentage Accomplished Health Project 25,863 7,706 29.8% Source: HP PITT_210701.xlsx, HSS indicators-FY2021_MY sheet On TB The indicators used to monitor HP contributions to TB control are TSRs in DS-TB cases and MDR/RR￾TB cases. For Q1 CY 2021, DS-TB TSRs in NCR, Region III, and Region IV-A exceeded NTP targets (Table 21). During the same period, the reported TSRs for MDR/RR-TB cases ranged from 60 percent to 67 percent, representing over 70 percent of NTP targets (Table 20). The TSRs would have to be examined together with other indicators, such as case detection, to get a better picture of the continuum of TB prevention and control. Table 20. Treatment success rates for MDR/RR-TB cases, CY 2019–2021 CY 2019 CY 2020 CY 2021 Target Accomplishment as of Q1 CY 2021 Percentage National 56% 66% 55% 85% 66% 78% Numerator 2,642 3,564 3,564 Denominator 4,718 5,376 5,376 NCR 48% 60% 55% 85% 60% 71% USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 42 Table 20. Treatment success rates for MDR/RR-TB cases, CY 2019–2021 CY 2019 CY 2020 CY 2021 Target Accomplishment as of Q1 CY 2021 Percentage Numerator 657 1,010 ,010 Denominator 1,369 1,672 1,672 Region III 43% 62% 53% 85% 62% 73% Numerator 118 488 488 Denominator 275 784 784 Region IV-A 53% 67% 50% 85% 67% 78% Numerator 489 281 281 Denominator 924 422 422 Source: HP PITT_2021-0714.xlsx, TB indicators-CY2021 Q1 sheet Table 21. DS-TB treatment success rate, CY 2019–2021 CY 2019 CY 2020 CY 2021 Targe t Accomplishment as of Q1 CY 2021 Percentage National 91% 91.0% 53% 90% 53% 101% Numerator 94,694 293,986 117,851 117,851 Denominator 324,297 323,158 220,772 220,772 NCR 90% 89.7% 60% 90% 60% 100% Numerator 45,874 47,398 17,180 17,180 Denominator 50,870 52,865 28,745 28,745 Region III 91% 90.8% 54% 90% 54% 101% Numerator 31,590 32,456 13,206 13,206 Denominator 34,855 35,733 24,516 24,516 Region IV-A 92% 90.8% 53% 90% 53% 101% Numerator 46,982 44,583 15,974 15,974 Denominator 51,341 49,112 30,177 30,177 Source: HP PITT_2021-0714.xlsx, TB indicator-CY2021 Q1 sheet Analysis of ITIS data at the city and provincial levels shows that the TSRs of non-USG-supported sites for all forms of TB did not change from 2018 to 2020, at 89 percent (Figure 3). On the other hand, USG-supported sites started at the same TSRs for all forms at 90 percent but dropped to 80.9 percent in 2020. The following explanations can be offered: The areas supported by the HP are where TB is highly prevalent, and the “big three” regions, where HP support for TB is focused, were badly hit by COVID-19. While we can possibly link private-sector mandatory TB notification as a contributing factor to the TSR decline, treatment outcomes from the private sector are not routinely reported, and reports on TSRs are not organized based on source of contribution (community vs. private). USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 43 In the case of MDR/RR-TB, the average TSR significantly improved from 2018 to 2020, by 20 percentage points regardless of location. A full analysis of this indicator and other relevant TB indicators in ITIS is included in Supplementary Material J. It should be noted, however, that by design, USAID selected sites estimated to have large numbers of undetected or untreated persons with TB. Figure 3. TB treatment success rates, 2018–2020 In terms of provinces and cities hitting the target TSR for all forms of TB, 74 percent of non-USG￾supported sites achieved at least 90 percent TSR in 2018–2020 (Figure 4). On the other hand, among USG-supported provinces and cities, 74 percent reached the 90 percent target in 2018 only. By 2019, only 22.4 percent of the HP-supported areas had a TSR of 90 percent for all TB forms. In 2020, this nearly doubled to 39.4 percent but was still only a little over half of the areas that reached the 90 percent goal two years prior. Regarding treatment of MDR/RR-TB, seven out of every 10 provinces and cities were able to reach the target 66 percent TSR regardless of HP support. The number of provinces and cities reaching the target 66 percent nationwide remained unchanged from 2018 to 2019 but improved by a third in 2020. The full report on the secondary analysis of TB data from ITIS is provided in Supplementary Material J. Although this could be a result of the adoption of short-course treatment regimens, we cannot validate this possibility because TSRs are not disaggregated by type of treatment received. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 44 Figure 4. Percentage of provinces and cities that reached TB treatment success rates, 2018–2020 The reported performance levels need to be compared with the number of TB cases detected and then treated. The starting point of the TB cascade of care is detecting cases, and this is what a catch-up plan for TB prevention and control should focus on. But for active and enhanced case-finding approaches to be employed at scale, the HP needs to address funding and staffing issues at the NTP, as well as the challenge of engaging the private sector. SBCC is a vital component of TB prevention and control. Rather than implementing a unidirectional approach where clients merely receive information, the HP can sustain engagements or calls to action by providing an online or physical space for continuing advice and guidance to link clients to services. There are several promising interventions (active case finding in high-risk communities, intensified case finding in facilities, FAST Plus, shorter all-oral regimen for MDR-TB) that the HP will need to implement at scale for improvements in TB outcomes to be detected at the subnational and provincial levels. Many of these interventions are resource intensive, so sustainability becomes a critical concern. Another critical concern is how to sustain community-level active case finding and the use of more effective diagnostic technologies. Follow-up HP activities should focus on helping DOH align national health budgets and PhilHealth benefit payments with public and private resources at the local level to sustain the entire TB cascade of care. On CBDR The LOA number of PWUD completing evidence-based drug rehabilitation interventions in USG￾supported sites is 20,926 persons. The baseline measure in the same sites is 9,811 persons. As of Q2 FY 2021, it was reported that 723 PWUD had undergone USG-supported CBDR. This accomplishment represents only 3.5 percent of target, suggesting that HP interventions in this area have only started and that USAID is still trying to navigate the intricate structural framework of CBDR. However, HP engagement with national and local authorities operating the government’s CBDR program has gained USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 45 ground, according to the USAID HP IP on CBDR. The initial “handshakes” allowed the HP to advocate its mandates by presenting an alternative CBDR approach. On challenges and barriers The significant shift in focus to COVID-19 response efforts caused displacement, disruptions, delays, and demand reduction, which have resulted in considerable backlogs in achieving FP, TB, and CBDR targets. The DOH DPCB might need assistance in determining the extent of the backlogs. There were several notable HP supports to UHC, but conceptual, operational, and financial difficulties in implementing UHC remain at the local level. The challenge for USAID is how to cascade support to USG-assisted LGUs that are not among the UHC UIS. The Mandanas-Garcia ruling is both a challenge and an opportunity. The increased funding to LGUs may not necessarily lead to increased investments in health, as much depends on LGU capacity to program and use these resources. Meanwhile, the RPRH Law’s restrictive provisions on the need for parental and spousal consent to use FP services discourage adolescents to fully exercise their sexual and reproductive rights. HP work on Konektado Tayo may need expansion and strengthening. Studies have shown that adolescent girls’ connectedness to parents contributes to delayed first sex.12,13 Numerous KIIs have pointed out the funding gaps from TB Law and in achieving the UN General Assembly High-Level Meeting on TB targets. However, reported TB budgets are not fully used (not more than 54 percent disbursement for CY 2019 and CY 2020). FOR THE NEXT USAID PROGRAM CYCLE (2022–2026), WHAT TYPE OF SUPPORT SHOULD THE HP INVEST IN TO ACHIEVE DESIRED HEALTH OUTCOMES AND SUSTAINABLE SYSTEMS CHANGES IN THE PHILIPPINES? a. Which health system pillars should be prioritized? b. What type of technical assistance should be provided for TB, HSS/G, FP/ARH, and CBDR? c. What mechanisms should be used for the delivery of support? Nature of future HP support The challenge for the next USAID HP cycle is implementation at scale. Previous and current HP support have contributed to the enactment of new laws and regulations, have introduced and tested innovations, and have helped develop mechanisms to improve health systems. But for impacts to reach intended beneficiaries, all these interventions and innovations need to be institutionalized, made operational and more inclusive, and funded with an explicit bias for the health of poor families. 12 Karofsky, P. S., Zeng, L., & Kosorok, M. R. (2001). Relationship between adolescent–parental communication and initiation of first intercourse by adolescents. Journal of Adolescent Health, 28(1), 41–45. 13 Weinman, M. L., Small, E., Buzi, R. S., & Smith, P. B. (2008). Risk factors, parental communication, self, and peers’ beliefs as predictors of condom use among female adolescents attending family planning clinics. Child and Adolescent Social Work Journal, 25(3), 157–170. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 46 The drive to implement at scale proven approaches to improve access to quality health services will be met with difficulties, some of which are inherent to the way the Philippine health system is organized and financed. Such difficulties arise from an overly decentralized public health delivery system, a large private sector with considerable but mostly untapped potential to contribute to priority public health goals, and multiple fairly uncoordinated sources of public financing for health that have yet to be used to improve overall health performance. But potential drivers and platforms for scaled implementation are in place, established in part with support from USAID. Possible drivers include the Konsulta primary care package, premium subsidies, national health budgets, and health regulations determining quality standards, clinical practice, and professional training. Platforms include the DepEd’s CSE program and the DSWD’s 4Ps program. Potential partners for scaled implementation include the Philippine Obstetrical and Gynecological Society, PhilCAT, Employees Confederation of the Philippines, Philippine Business for Social Progress, and foundations associated with large corporations (e.g., Shell, Zuellig, Ayala, Aboitiz). In addition, the following partners may also have to be engaged: IMAP, Philippine League of Government and Public Midwives, and Philippine Nurses Association. The common thread among these partners, in addition to their interest in advancing public health goals, is that they operate at a scale that can be sustained over extended periods. HP-supported interventions with high potential for scaled implementation include the widespread use of Xpert in diagnosing a range of infectious diseases, sustained promotion of ARH through the CSE-ARH convergence program in high schools, SCM outsourcing, piggybacking CBDR via the DSWD’s 4Ps program, and using the 5 percent allotment of the GPH budget for GAD to sustain health care programs for adults and adolescents. Future HP support must also deal with challenges that will continue to affect developments in the health sector in the medium term. The COVID-19 pandemic caused significant backlogs in FP, TB, and CBDR targets, as well as an increase in cases of gender-based violence (GBV). It is important that future HP support is informed by the magnitude and gravity of these consequences. The HP must address the difficulties (conceptual, operational, and financial) and transaction costs in implementing elements of UHC that are key to advancing FP and TB goals. While supporting UHC implementation at the local level, the HP must find a balance between assisting LGUs in promoting FP and TB in the face of specific local conditions and at the same time generating evidence and lessons general enough to be useful for scaling up in other localities. This challenge may even be more difficult with the Mandanas-Garcia ruling, which is expected to reduce national health budgets and subsidies but may not necessarily lead to increased LGU investments in health. Many senior DOH officials support the view that HP support might be more valuable if directed at LGUs. However, they also emphasize that this be done in coordination with the DOH, especially with subnational DOH offices. On priority health systems pillars USAID must maintain support for all the health system pillars but would have to focus on key elements within each. In service delivery, for example, there may be a need to introduce operational measures of USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 47 quality of care so that these can be monitored and linked to inputs like training, budget, and incentives.14 Results of a randomized social experiment in the Philippines show that by explicitly linking insurance payments to concrete measures of the quality of care being delivered, patients’ health outcomes, measured in terms of biomarkers, anthropometrics, and personal ratings, are significantly improved. For FP/ARH, USAID can start with analyzing quality indicators in the HP results framework to get a sense of what areas to improve on. The analysis should be conducted with the DOH and LGUs as part of the process in determining standard quality indicators for FP/ARH. Due to the highly contextual nature of health care, it is important to pay close attention to CQI monitoring. For TB, competent and high-quality care means all patients are managed according to the latest evidence-based guidelines. We recommend continuing USAID’s support in training health care workers on the latest TB Manual of Procedures. Another area in which to ensure quality care is diagnostic accuracy that will lead to immediate initiation of appropriate treatment regimens. The HP is currently addressing this by monitoring the coverage of bacteriological diagnosis. Future HP activity to consider includes conducting a survey of lag time from when the client seeks their first consult to treatment initiation. Survey results can provide insight on the factors of treatment delays. More research is also needed to determine the best indicators and approaches to improve a client’s care experience. Factors USAID can consider include accessibility, ease of use, affordability, and patient values. Quality education and training of health professionals are important prerequisites for quality care. USAID’s work with the Professional Regulation Commission, particularly with the Board of Midwifery, facilitated the inclusion of the Mentoring and Monitoring Midwives Program within the bachelor’s degree program for midwifery and its link to the Career Progression of Midwives program. This initiative demonstrated a continuum of learning from preservice to in-service training, which can lead to graduates who are competent at the time of entry into the workforce and enable continuous professional development at various stages of their careers. Given this experience, USAID should continue working with and expanding its network of influence with the Professional Regulation Commission and professional organizations. It may be beneficial for USAID to scale up its comprehensive portfolio of training programs for FP/ARH, TB, CBDR, SBCC, HSS/G, and GEWE. On the health workforce, it might be worthwhile to make a distinction between those who are involved in service delivery and those who are involved in managing systems. Future USAID support might also have to include investigating and addressing gaps in the capacity of the health system managers, particularly in light of health system integration to implement UHC. The support provided by the BHWs or CHWs has proved effective in extending the reach of needed essential health services in the communities amid the pandemic, particularly in FP/ARH. However, in a systematic review by Lehmann and Sanders (2007)15, the BHWs—even though they can provide 14 Peabody, J., Shimkhada, R., Adeyi, O, et al. (2017). Quality of care. In D. T. Jamison, H. Gelband, S. Horton, et al. (Eds.), Disease control priorities: Improving health and reducing poverty (3rd ed., pp. 185–214). The International Bank for Reconstruction and Development/The World Bank. https://www.ncbi.nlm.nih.gov/books/NBK525309/ doi: 10.1596/978-1-4648-0527-1_ch10 15 Lehmann, U., & Sanders, D. (2007). Community health workers: what do we know about them. The state of the evidence on programmes, activities, costs, and impact on health outcomes of using community health workers. World Health Organization. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 48 adequate support to improve health outcomes—deliver inconsistent quality of service, hence the need for substantial investments to support training, management, supervision, and logistics. USAID might want to consider reviewing the RA 7883 (the BHW Benefits and Incentives Act of 1995) and see how BHWs may be professionalized and provided with secure compensation and continuous professional education and development. On health information systems, although a lot of data are being collected, they are not easily analyzed and are often not disaggregated by sex and age. By intensifying the analyses of existing data, the HP can help stimulate further development of the information system. An example is the national health accounts developed with USAID assistance in the mid-1990s and the information and communication directed at young people on how to deal with TB and unwanted pregnancies. A key observation is that the elements of a health information system that are used for payment, budget planning, and routine decision making tend to be more developed. Government procurement and PhilHealth payments need to be seen as drivers to improve data quality at SDPs. On medical products, vaccines, and technologies, the HP must ride on the widespread use of modern diagnostics for COVID-19, which are beneficial for TB prevention and control. Once COVID-19 is under control, the HP should be able to help the DOH and LGUs redirect excess diagnostics capacity to address primary care issues, including TB. The HP needs to carefully study how interagency managers of the COVID-19 pandemic worked around existing gaps and weaknesses in the public health system’s procurement and supply chain management. The HP must pay attention to procurement systems outside the DOH; active participation of the private sector, including big business; and the engagement of scientific and research capacities in public and private universities. On financing, HP should support PhilHealth to focus on insurable cases who need financial risk protection, such as for the treatment of MDR/RR-TB, which has catastrophic financial consequences. A possible approach for the next HP is to address operational challenges of pursuing benefits delivery to a specific set of beneficiaries. For example, the HP can drive the needed changes in operations by increasing the TB treatment support value and widening the reach of the Konsulta package. In addition, the next HP should provide technical assistance to build PHIC mid- and top-level management capacity for strategic and policy development of UHC. Finally, on governance, the HP might focus on promoting broad-based public–private partnerships for FP/ARH and TB at a similar scale as with COVID-19, which spans decision making, funding, and service delivery. On technical assistance for FP/ARH, TB, and CBDR Although HP support is expected to remain focused on FP/ARH and TB, UHC implementation must become the platform through which HP support is delivered. From a “program” lens, USAID’s FP and TB support might be coursed through support for the financing and delivery of a primary care package. By supporting FP/ARH and TB via an inclusive package, USAID can address health system gaps and bottlenecks better, especially budget execution and SCM. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 49 USAID might also consider consolidating its support in one or two regions with high unmet needs for TB and FP/ARH services. We can refer to these as “UHC laboratories,” which should be large enough so that the challenges of implementing UHC at scale are better understood and enough evidence is generated to test implementation solutions. By proposing that the HP support UHC labs, the HP is in effect better able to leverage its finite resources by showing better, more effective ways for national and local health authorities to exercise their mandates and apply their resources. These UHC labs might also serve as drivers to upgrade and promote interoperable health information systems, especially as demand for more and better information is needed to advance inclusive reform initiatives. In UHC labs, support to strengthen health systems and governance mechanisms can be consolidated to allow for tested interventions for TB, FP/ARH, and CBDR to be scaled up. USAID may want to tap ideas and lessons from the WHO Global Learning Laboratory for Quality UHC, which is a platform for sharing knowledge, experiences, and ideas.16 Within the UHC lab, the HP can renew its strategic focus on the “big three” regions and can focus FP/ARH on two regions (including BARMM). It also can continue to assist the CHDs and LGUs in all aspects of the UHC integration—including health information systems; HRH capacity-building; local health investment planning; budget and SCM; and engagement of private-sector facilities, pharmacies, and service providers. The proposal for the HP to consider having one activity for FP that covers BARMM can allow joint use of limited expertise and resources. In the case of TB, the HP may gain better coordination at the technical and administrative levels if the two interdependent functions of innovation and scale-up are managed under a single activity. The next generation of USAID support for TB and FP/ARH will come in the wake of massive efforts to contain the COVID-19 pandemic. We expect a continued focus on COVID-19, in terms of policy decision making, health personnel, and investments. Although lessons and opportunities from the campaign to control the pandemic relevant to TB and FP/ARH need to be derived and applied, it is equally important for the DOH and the HP to be able to present a credible estimate of the backlog in the utilization, financing, and delivery of other priority public health programs, including FP/ARH and TB. These estimates of the non-COVID-19 public health backlog are critical in supporting national and local efforts to secure or reclaim pre-pandemic investment levels in public health. Currently USAID might consider helping the DOH and selected LGUs estimate the backlog created by COVID-19 on priority programs like FP/ARH and TB. Two years of intermittent lockdowns, travel restrictions, economic slowdown, and redeployment of HRH and funds to address the pandemic have reduced demand and supply of essential health services. There is a need to estimate the backlog, identify its location, and determine how HP can support efforts by the DOH and LGUs to address the backlog. Meanwhile, the government is likely to invest further in disease surveillance, public health diagnostic laboratories, and logistics and SCM to ensure that present and future pandemics are held at bay. It is 16 World Health Organization. (n.d.). WHO Global Learning Laboratory for Quality UHC. https://www.who.int/initiatives/who-global-learning￾laboratory-for-quality-uhc USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 50 important for USAID to seize the opportunity to ensure investments in such systems are broad enough to address similar weaknesses and gaps that apply to FP/ARH and TB. The nationwide implementation of ARH-friendly facilities and FP in hospitals might be integrated in that larger campaign to reduce teenage pregnancy. USAID support needs to focus greater on reducing teenage pregnancy, given its individual and social burden. Such support might have to emphasize SBCC interventions to effectively reach both in-school and out-of-school adolescents of all genders. POPCOM recommends that support be given to assist developing social protection and a comprehensive plan for teenage mothers, including an M&E system for monitoring adolescent pregnancies. HP support for FP/ARH might include: (a) SBCC for adolescents in school and out of school, with adolescent males as a target, including educating parents to deal with ARH; (b) a push for the nationwide implementation of an integrated FP program in hospitals, including a mechanism for the DOH to leverage local government and private-sector participation and grants; and (c) mainstream use of Katropa and a scale-up Usapan under POPCOM. Although SBCC and CSE may be important in reaching out to adolescents, they are not enough to prevent unplanned pregnancies and sexually transmitted infections, including HIV/AIDS. Interview data showed that sexually active teens are effectively reached by CHWs. Hence, USAID may conduct more intentional outreach and counseling of adolescents to avoid teen pregnancy. USAID may also give attention to providing safe and quality access to post-abortion services to reduce death and suffering from unsafe abortions. Moreover, there is a need to monitor and evaluate the implementation of the RPRH mandate to generate data in support of policy development to expand reproductive and health services for adolescents and services for GBV, as well as to bolster the implementation of the new Republic Act No. 11596, also known as An Act Prohibiting the Practice of Child Marriage and Imposing Penalties. The only SBCC intervention that is targeting childhood TB is the Tibay ng Dibdib, an SBCC strategy integrated in all public elementary schools in NCR to raise public awareness and promote positive health-seeking behavior. Future support for TB might have to emphasize childhood TB, especially given the challenge of more effective diagnosis. This is also an urgent concern, given that childhood TB may lead to latent infection, maintaining the pool of infected people. HP can consider linking the campaign to actual services at various levels of care. TPT is another area that needs stronger USAID support. USAID supported the development of the TPT Roadmap and implementation guide. Health care providers and eligible individuals will need to be informed on this to increase uptake and adherence. HP can consider providing technical assistance to LGUs in conducting contact investigation activities. Moreover, future USAID support for TB control may still focus on NCR, Region III, and Region IV-A, where cases remain high. Using the 4Ps platform to intensify childhood TB awareness, prevention and treatment can be tested at scale in these regions. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 51 Although the Philippine Private Diagnostics Consortium expanded access to Xpert MTB/Rif by decreasing the procurement cost, the cost of the cartridge17 in the market is still widely variable—from PHP 2,000 to PHP 9,000—driving private sector partners to express a need for public subsidies that might be based on performance and service delivery quality. HP should ensure that these GeneXpert machines are strategically deployed in areas with high TB burdens. Other technical assistance for TB includes developing and executing national and local health budgets for TB and intensifying TB awareness and health-seeking through the 4Ps platform. For both FP and TB, USAID might consider an activity designed to help the DOH use its budget and commodities to increase LGU performance and private-sector participation or engagement. For CBDR, USAID may want to continue on strengthening counselor training that targets working not only with PWUD but working with their families and significant others. In a controlled trial conducted by Kirby and colleagues (1999)18, counseling sessions with PWUD that included family members and concerned significant others increased the likelihood that PWUD enter treatment and decreased the drug use of PWUD. Additionally, USAID may want to intensify its efforts to engage faith-based organizations in CBDR work. According to Grim and Grim (2019),19 these organizations are effective in community mobilization and are well versed in facilitating quality group interactions focusing on overcoming past experiences, which are often drivers of emotional and spiritual despair that feed mental illness and substance abuse. Technical assistance in the area of HSS includes: (a) reforms in logistics management and SCM; (b) implementation research on primary care packages mandated under UHC to ensure sufficient coverage for priority programs, including FP/ARH and TB; and (c) revitalization of the DOH Field Epidemiology Training Program and graduate programs in UHC, health economics, or health systems integration, including reforms in the information systems to support effective surveillance and knowledge sharing. On mechanisms for delivery of support—working with partners Key informants appreciated the HP support but perceived the HP as operating as vertical programs. As the DOH attempts to integrate programs, technical and administrative supervision over priority public health concerns other than COVID-19 could be neglected. Success in introducing and sustaining innovations is also limited by local partners’ capacity to adopt and scale up innovations. The basis and prospects for future success of USAID and DOH collaboration may need to be reviewed in the light of the COVID-19 pandemic, UHC, and other new mandates. The coordination of various government agencies is a serious challenge. 17 Note: this is for the cartridge only (consumable). The machine itself is expensive. Sometimes, labs just lease or borrow this from Cepheid as long as a quota of purchased cartridges is met. 18 Kirby, K. C., Marlowe, D. B., Festinger, D. S., Garvey, K. A., & LaMonaca, V. (1999). Community reinforcement training for family and significant others of drug abusers: A unilateral intervention to increase treatment entry of drug users. Drug and alcohol dependence, 56(1), 85– 96. 19 Grim, B. J., & Grim, M. E. (2019). Belief, behavior, and belonging: How faith is indispensable in preventing and recovering from substance abuse. Journal of Religion and Health, 58(5), 1713–1750. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 52 Program-level data like the ITIS and FHSIS are important for M&E, yet gaps remain in data collection and quality, analysis, and interpretation, including gathering gender-sensitive data. HP can support further development of these data sources by engaging with partners to conduct analyses to address policy questions. The HP may have to further improve how it engages partners. Coordination of multiple activities can help deepen partner involvement and buy-in. As an example, managing four functionally differentiated TB activities is difficult. Similar issues have been raised regarding the effectiveness of having a separate HP activity for BARMM. FP work in BARMM continues to face challenges in project management and in dealing with geographic distances, coordination difficulties, special population needs, and social norms and religious beliefs on FP and early marriage. We recommend creating a single umbrella project for FP with a distinct BARMM component to take advantage of shared capacities, especially in technical supervision over complex interventions or innovations. New start-up activities should be given sufficient time to germinate. Work on CBDR has a comprehensive profile of interventions, but some are expected to have less impact than others. Different government agencies oversee different services in the treatment and support of PWUD: DILG and its Bureau of Jail Management and Penology for law enforcement, DOH for facility-based rehabilitation, DSWD for aftercare support, and CBDR for LGUs. As a unifying project for PWUD, the HP can provide technical assistance in streamlining and integrating the care cascade, which can eventually facilitate smooth transition of PWUD from one part of the cascade to another. HP activities that engage experts have been found to be highly effective. HP work in SBCC related to FP/ARH has made a difference because of the presence of a team of SBCC experts. Such expertise is needed in dealing with partners who are conversant with SBCC principles but are able only to translate these operationally as “knowledge translations and communication,” “communications and dissemination activities,” and “materials development.” USAID should intensify engagement of partners from private corporate institutions with interests in public health (e.g., Ayala, Aboitiz, MVP, Razon, Shell). Their track record of participation in the prevention and control of COVID-19 provides the momentum for them to do the same for TB and FP/ARH. USAID should consider engaging a separate independent M&E outfit tasked to (a) collect and analyze data to track progress of priority programs, and (b) transfer the technology of an independent, evidence-based M&E platform to national and local health entities. Several key informants raised questions about how best to organize HP activities in view of the challenge of managing BARMM activities and the reality that different entities manage functionally different activities (e.g., platform vs. innovation). An important consideration is how to optimize USAID capacity to provide both administrative and technical supervision of HP activities. The push for national and local actions and innovations requires more intensive and strategic input from USAID technical staff. In the next program cycle, USAID may want to set up one project for TB with distinct components on USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 53 innovation design, testing, replication, and planning for institutionalization. This may solve coordination issues among different USAID staff and IPs. At the national level, consider focusing assistance to the DOH DPCB and PhilHealth, where most USAID assistance is concentrated. Designating a USAID focal point at the DOH may be beneficial to streamline coordination of all matters related to USAID support to the GPH and reduce transaction costs. The DOH should be preparing a transition program for a new administration in July 2022—it would be useful for USAID to participate in these proceedings, partly as a way of determining priorities that might be included in the DOAg for the next round of HP activities. The transition program should have a clear catch-up strategy to close the gap in FP, ARH, TB, and CBDR targets brought about by the COVID-19 pandemic. Summary of key recommendations On HP support for the DOH: (a) size up and cost out the FP and TB backlog due to the COVID-19 pandemic; (b) help develop a strategy for the regularization of CHW/BHW/BNS; (c) link CQI with budgets and incentives; (d) rationalize the GAD budget; (e) undertake capacity-building for the newly organized DOH DPCB; (f) build on the momentum generated by COVID-19 containment, especially private-sector engagement; and (g) organize a region-wide UHC implementation lab using TB and FP￾MNCHN as tracers (possibly within the “big three” regions). On current HP interventions for testing, documentation, and replications: (a) CiTEC, (b) Philippine Private Diagnostics Consortium, (c) eLMIS, (d) WISN, (e) SBIRT, (f) ConnecTB, and (g) a Catastrophic TB Package in PhilHealth. On HP interventions for scaled implementation: (a) FP in hospitals, (b) MRLs as public health leaders, (c) FAST Plus, (d) IHLGP in regions, (e) Konsulta package, (f) Katropa, (g) CSE-ARH Converge, (h) adolescent-friendly health facilities, and (i) men’s reproductive health. On future HP activities: (a) one TB IP with distinct components on innovations design, testing, replication, and planning for institutionalization; (b) a distinct BARMM component within an umbrella FP IP; (c) independent monitoring, evaluation, and learning/collaborating, learning, and adapting IP with a mandate to collect, analyze, and monitor performance of HP activities and share lessons with the DOH20; and (d) an additional role for an HSS/policy/finance IP that will validate institutionalization and scale-up strategies for HP interventions. Moreover, technical capacity, perhaps by way of an indefinite quantity contract–like facility, may be considered to augment HP management’s ability to supervise an increasingly complex and challenging set of interventions. Such additional capacity may be needed in sorting and assessing the design, technical soundness, approaches to replication, and institutionalization for eventual scaled implementation. 20 It may be argued however, that the IPs must be given the opportunity to learn, to collect the data, and to analyze them—for real-time feedback and adaptation. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 54 An approach to tie all these recommendations together in a way that allows for scaled implementation is to package HP support and focus its activities to assist the DOH in implementing UHC at a regional level. Support for a UHC regional laboratory should involve two or more provinces containing consumers and providers of health services representing a continuum of care from disease prevention to primary curative care, up to treatment and rehabilitation in tertiary and specialty facilities. Replication and eventual institutionalization might be easier to facilitate at the regional level, given administrative and budgetary mechanisms available through DOH, PhilHealth, POPCOM, and other institutions. It may be in the context of supporting regional UHC labs, where the LGU pivot initiated by current HP activities might become more meaningful. It is also at the regional level where future efforts to build the capacity to manage an increasingly complex financing and delivery system may be concentrated. A particular concern is the need to expand and strengthen the number and availability of middle-level managers at regional and LGU health offices. This proposed expansion of health management capacity also needs to be complemented with the appropriate enhancement of health management information systems. Finally, following the experience with efforts to contain COVID-19, it may be easier to engage the private sector and civil society organizations at levels that involve both consumers and providers of the continuum of care. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 55 GENERAL REFERENCES Agrawal, P., Barton, I., Bianco, R. D., Hovig, D., Sarley, D., & Yadav, P. (2016). Moving medicine, moving minds: Helping developing countries overcome barriers to outsourcing health commodity distribution to boost supply chain performance and strengthen health systems. Global Health: Science and Practice, 4(3), 359–365. https://doi.org/10.9745/ghsp-d-16-00130 Banzon, E., & Mailfert, M. (2018). Overcoming public sector inefficiencies toward universal health coverage: The case for national health insurance systems in Asia and the Pacific. ADB Sustainable Development Working Paper Series No. 53. Asian Development Bank. https://www.adb.org/publications/overcoming-public-sector-inefficiencies-universal-health Cagayan de Oro Local Investment Plans for Health 2020–2022. (n.d.). Retrieved December 2021, from Carey, J. W. (2019). Communication as culture: Essays on Media and Society (Rev. ed.). Routledge. Center for Health Market Innovations (n.d.). eLMIS selection guide. Retrieved December 2021 from https://healthmarketinnovations.org/sites/default/files/electronic%20Logistics%20Management%20 Information%20System%20%28eLMIS%29%20Overview.pdf Center for Health Market Innovations. (2012). Logistics management information systems (LMIS). https://healthmarketinnovations.org/sites/default/files/electronic%20Logistics%20Management%20 Information%20System%20%28eLMIS%29%20Supplementary%20Information.pdf Giddens, A. (1991). Modernity and self-identity: Self and society in the late modern age. Polity Press. Greenwell F. & Salentine S. (2018). Module 4: Logistics management information system. Health Information System Strengthening: Standards and Best Practices for Data Sources. https://www.measureevaluation.org/resources/hisdatasourcesguide/module-4-logistics￾management-information-system Grim, B. J., & Grim, M. E. (2019). Belief, behavior, and belonging: How faith is indispensable in preventing and recovering from substance abuse. Journal of Religion and Health, 58(5), 1713–1750. Hausman, B. L. (2016). Commentary: modernity and vaccination: What stories of vaccine injury tell us about vaccines as things. Narrative Inquiry in Bioethics, 6(3), 193–197. https://www.proquest.com/scholarly-journals/commentary-modernity-vaccination-what￾stories/docview/1874321223/se-2?accountid=47253 Karofsky, P. S., Zeng, L., & Kosorok, M. R. (2001). Relationship between adolescent–parental communication and initiation of first intercourse by adolescents. Journal of Adolescent Health, 28(1), 41–45. Kirby, K. C., Marlowe, D. B., Festinger, D. S., Garvey, K. A., & LaMonaca, V. (1999). Community reinforcement training for family and significant others of drug abusers: A unilateral intervention to increase treatment entry of drug users. Drug and Alcohol Dependence, 56(1), 85–96. Lehmann, U., & Sanders, D. (2007). Community health workers: What do we know about them. The state of the evidence on programmes, activities, costs, and impact on health outcomes of using community health workers. World Health Organization. Melgar, J. L. D., Melgar, A. R., Festin, M. P. R., Hoopes, A. J., Chandra-Mouli, V. (2018). Assessment of country policies affecting reproductive health for adolescents in the Philippines. Reproductive Health, 15(1), 205. doi: 10.1186/s12978-018-0638-9 Mercardo, N. (2021). DOH spends only 38 centavos out of every peso in 2021 budget as of August. INQUIRER.net. https://newsinfo.inquirer.net/1494316/doh-spends-only-38-centavos-out-of￾every-peso-in-2021-budget-as-of-august#ixzz7FC9cIdtu USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 56 Monsod, T. M. (2019). Budget credibility in the Philippines’ health sector: Regional comparisons and case studies. https://www.internationalbudget.org/wp-content/uploads/budget-credibility-in-the￾philippines-health-sector-ibp-2019.pdf Panelo, C. I. A., Solon, O. J. C., Ramos, R. M., Herrin, A. N. (2017). The challenge of reaching the poor with a continuum of care: A 25-year assessment of Philippine health sector performance. http://www.upecon.org.ph/wp-content/uploads/2018/03/HSR_061818.pdf Philippine Department of Finance (n.d.). 2018–2020 LGU fiscal data statement of receipts and expenditures. Retrieved December 2021 from https://blgf.gov.ph/lgu-fiscal-data/ Philippine Department of Health. (n.d.). 2014–2021 Statement of appropriations, allotments, obligations, disbursements, and balances. Retrieved November 2021 from https://doh.gov.ph/philippine￾transparency-seal Philippine Department of Health. (n.d.). 2018–2020 FHSIS annual reports. Retrieved December 2021 from https://doh.gov.ph/publications Philippine Department of Health. (n.d.). 2019 Philippine health statistics. Retrieved December 2021 from https://doh.gov.ph/publications Philippine Department of Health. (2017). Initial progress report, January–June 2017 accomplishments of EO No. 12, s. 2017: “Zero unmet need for modern family planning” through the strict implementation of the Responsible Parenthood and Reproductive Health Act. Philippine Department of Health. (2020a). Annual report 2020: Responsible Parenthood and Reproductive Health Act of 2012. Philippine Department of Health. (2020b). No P2.2B expired overstocked medicines and supplies. http://doh.gov.ph/press-release/DOH-NO-P2.2B-EXPIRED-OVERSTOCKED-MEDICINES￾SUPPLIES Philippine Department of Health. (2021). Philippine national tuberculosis report 2021. https://ntp.doh.gov.ph/download/ntp-annual-report-2021/ Philippine Department of Health. (2021a). Department circular No. 2021-0522: FY 2021 3rd quarter local health systems maturity levels progress report for the 58 universal health care integration sites. Philippine Department of Health (2021b). Guidelines on the allocation, utilization, and monitoring of, and accountability for, the specific health fund. Memorandum Circular No. 2021-0003: DOH-DBM￾DOF-DILG-PHIC Joint Memorandum Circular No. 2021-0001. Philippine Department of Health. (n.d.). National Family Planning Program. https://doh.gov.ph/family￾planning Philippine Health Insurance Commission. (2020). PhilHealth Circular No. 2020-0022: Governing Policies of the PhilHealth Konsultasyong Sulit at Tama (PhilHealth KonSulTa) Package: Expansion of the Primary Care Benefit to Cover All Filipinos. https://www.philhealth.gov.ph/circulars/2020/circ2020-0022.pdf Philippine Statistics Authority and ICF. (2014). Philippines national demographic and health survey 2013. Philippine Statistics Authority and ICF. (2018). Philippines national demographic and health survey 2017. Raftree, L. (2019). Digital and social media for social and behaviour change communication. iMedia Associates and Gates Foundation. https://imediaassociates.org/app/uploads/2019/07/Digital-and-Social￾Media-for-SBCC-March-2019.pdf. Schiff, B. (2012). The function of narrative: Towards a narrative psychology of meaning. Narrative Matters, 2(1), 33–47. https://journals.lib.unb.ca/index.php/NW/article/view/19497/21063 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 57 Schwarz, E. (1994, April). A metamodel to interpret the emergence, evolution and functioning of viable natural systems [Paper presentation]. European Meeting on Cybernetics and Systems Research, Vienna, Austria. Schwarz, M. R., & Wojtczak, A. (2002). Global minimum essential requirements: a road towards competence-oriented medical education. Medical Teacher, 24(2), 125–129. Ulep, V., Uy, J., Siy Van, V., & Bagas, J. (2021). An assessment of national-level governance of the Philippines’ Responsible Parenthood and Reproductive Health Law: Trends and ways forward. Discussion Paper Series No. 2021-09. Weinman, M. L., Small, E., Buzi, R. S., & Smith, P. B. (2008). Risk factors, parental communication, self, and peers’ beliefs as predictors of condom use among female adolescents attending family planning clinics. Child and Adolescent Social Work Journal, 25(3), 157–170. World Health Organization. (n.d.). WHO Global Learning Laboratory for Quality UHC. https://www.who.int/initiatives/who-global-learning-laboratory-for-quality-uhc World Health Organization. (2008). Task shifting: Global recommendations and guidelines. World Health Organization. (2014). Ensuring human rights in the provision of contraceptive information and services: Guidance and recommendations. https://apps.who.int/iris/bitstream/handle/10665/102539/9789241506748_eng.pdf World Health Organization. (2017). Task sharing to improve access to family planning (No. WHO/RHR/17.20). World Health Organization. (October 2021). Global tuberculosis report 2021. https://www.who.int/publications/i/item/9789240037021 Yolles, M., & Di Fatta, D. (2017). Antecedents of cultural agency theory: In the footsteps of Schwarz living systems. Kybernetes, 46(2), 210–222. https://doi.org/10.1108/K-11-2016-0316 Documents related to USAID/Philippines Health Project Carrino, C., Luelmo, F., Cabral, E., De Guzman, E., Agbayani, V., & Laguna, E. (August 2016). USAID/Philippines health portfolio evaluation. USAID Contract No. AID-OAA-C-14-00067. Chemonics International. (2020). Human resource for health in 2030 Philippines annual report (Year 1–Year 3). Cooperative Agreement No. AID-OAA-A-15-00046. FHI 360. (Various years). TB innovations annual report (Year 1–Year 3). Cooperative No. 720492128CA00001. Management Sciences for Health. (Various years). MTaPS annual report (Year 1–Year 3). Cooperative No. 7200AA18C00074. Palladium. (Various years). ProtectHealth annual report (Year 1–Year 3). Cooperative Agreement No. 72049219CA00002. Panagora Group. (Various years). CLAimHealth annual report (Year 1–Year 4). Cooperative Agreement No. 72049218F00001. RTI International. (Various years). ReachHealth annual report (Year 1–Year 3). Cooperative Agreement No. 72049218CA00009. University Research Co., LLC. (Various years). BARMMHealth annual report (Year 1–Year 3). Cooperative Agreement No. 72049219CA00001. University Research Co., LLC. (Various years). RenewHealth annual report (Year 1–Year 3). Cooperative Agreement No. 72049219CA00004. University Research Co., LLC. (Various years). TB platforms annual report (Year 1–Year 3). Cooperative Agreement No. 72049218CA00005. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 58 USAID Collaborating Learning and Adapting for Improved Health Activity. (2019). Models of family planning programs in selected Philippine hospitals. Good Practices and Promising Interventions, Technical Series No. 1. Panagora Group. USAID Collaborating Learning and Adapting for Improved Health Activity. (2019). Improving family planning and maternal, neonatal, and child health access for underserved populations in geographically isolated and disadvantaged areas through service delivery networks in the provinces of Zamboanga del Sur and Sultan Kudarat. Good Practices and Promising Interventions, Technical Series No. 1. Panagora Group. USAID Collaborating, Learning, and Adapting for Improved Health Activity. (2019). Health Project performance data consolidation: FY2019 midyear report. https://pdf.usaid.gov/pdf_docs/PA00WC39.pdf USAID Collaborating, Learning, and Adapting for Improved Health Activity. (2021). Good practices and promising interventions. Technical Series No. 6: Engaging Local Chief Executives to Build Local Capacity and Strengthen Health Systems. https://pdf.usaid.gov/pdf_docs/PA00Z2W7.pdf USAID LuzonHealth. (October 2018). Promoting family planning uptake through enhanced Usapan: Results from USAID’s LuzonHealth Project. https://pdf.usaid.gov/pdf_docs/PA00TJ3B.pdf USAID. (2019). ReachHealth improved health for underserved Filipinos: Family planning and maternal and neonatal health innovations and capacity building platforms (FP/MNH ICP): Annual report, Year 1/Fiscal Year 2019 (Dec 2018–Sept 2019). https://pdf.usaid.gov/pdf_docs/PA00XK6X.pdf USAID. (2021a). Medicines, Technologies, and Pharmaceutical Services (MTaPS) program: Annual report fiscal year 2021 and quarterly report fiscal year 2021 quarter 4 (July–September 2021). Management Sciences for Health, Inc. https://pdf.usaid.gov/pdf_docs/PA00XWTW.pdf USAID. (2021b). ReachHealth improved health for underserved Filipinos: Family planning and maternal and neonatal health innovations and capacity building platforms (FP/MNH ICP): annual report, year 2/fiscal year 2020 (Oct 2019–Sept 2020). https://pdf.usaid.gov/pdf_docs/PA00XQP7.pdf USAID. (2021c). ReachHealth improved health for underserved Filipinos: Family planning and maternal and neonatal health innovations and capacity building platforms (FP/MNH ICP): January 1–March 31, 2021, quarterly report. https://pdf.usaid.gov/pdf_docs/PA00XVW8.pdf USAID Philippines. (n.d.). Country development cooperation strategy. USAID Philippines. (2017). Project appraisal document: USAID/Philippines Health Project 2017–2022. USAID Philippines. (2017). USAID/Philippines Health Project 2017–2021 [Redacted concept paper]. https://fp2030.org/sites/default/files/USAID_HealthConceptPaper_REDACTED_01-24-2017.pdf USAID Philippines. (2020). FY2020 Health Element Key Issues. Zuellig Family Foundation. (Various years). Institutionalization of the Health Leadership and Governance Program (IHLGP) annual report (Year 1–Year 5). Cooperative Agreement No. AID-492-A-17- 00005. Zuellig Family Foundation. (n.d.). Institutionalization of the Health Leadership and Governance Program (IHLGP) terminal report. Philippine Government Laws/Administrative Orders Republic of the Philippines Republic Act No. 11223 or the Universal Health Care Act (2019). Republic of the Philippines Republic Act No. 11469 or the Bayanihan to Heal as One Act (2020). Republic of the Philippines Republic Act No. 11494 or the Bayanihan to Recover as One Act (2020). USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 59 Online Resources CNN Philippines Staff. (2021, February 8). Teenage pregnancies in PH up by 7%. CNN Philippines. https://www.cnnphilippines.com/news/2021/2/8/Teenage-pregnancy-cases-up-in-Philippines.html Peabody, J., Shimkhada, R., Adeyi, O., et al. (2017). Quality of care. In D. T. Jamison, H. Gelband, S. Horton, et al. (Eds.), Disease control priorities: Improving health and reducing poverty (3rd ed., pp. 185–214). The International Bank for Reconstruction and Development/The World Bank. https://www.ncbi.nlm.nih.gov/books/NBK525309/ doi: 10.1596/978-1-4648-0527-1_ch10 Philippine Health Insurance Corporation. (n.d.). 2012–2021 stats and charts. Retrieved November 2021 from https://www.philhealth.gov.ph/about_us/statsncharts/ USAID (n.d.). USAID global accelerator to end TB. https://www.usaid.gov/global-health/health￾areas/tuberculosis/resources/news-and-updates/global-accelerator-end-tb USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 60 ANNEX A. SCOPE OF WORK Scope of Work Collaborating, Learning, and Adapting for Improved Health IDIQ No. AID-OAA-I-15-00025, Task Order No. 72049218F00001 Health Project Multi-Year Whole of Project Evaluation (WOPE) Scope of Work 1. Description of the Evaluation Statement of Work Evaluation Information Item Details I. Title Health Project Multi-Year Whole of Project Evaluation (WOPE) II. Requester/Client USAID Mission/Division: Philippines III. Funding Account Source(s) IV. Cost Estimate V. Performance Period Expected Start Date (on or about) July 19, 2021 Anticipated End Date (on or about) January 31, 2022 VI. Location(s) of Assignment Philippines VII. Type of Analytic Activity/ EVALUATION Multi-year Project Evaluation • Performance Evaluation • Process Evaluation 2. Background 2.1 Background of the USAID/ Philippines Health Project The goal of the Health Project (HP) 2018– 2024 is Improved Health for Underserved Filipinos. “Underserved” in the primary context of the HP refers to people exposed to or with drug-susceptible tuberculosis (DS-TB) or multidrug-resistant TB (MDR-TB), youth and adults at risk for unwanted, early pregnancy and childbirth, people in need of voluntary community-based drug rehabilitation (CBDR) services, and people most at risk for HIV. The HP focuses on geographic areas where the health burden is the greatest, i.e., where the TB disease burden is the highest, where unmet need for family planning (FP) is the highest, where there are high teenage pregnancy rates, and where there is a high, evidence￾based need for drug demand reduction services.21 21 Development Objective Grant Agreement between the United States of America and the Republic of the Philippines for Development Objective “Improved Health for Underserved Filipinos.” USAID Grant Agreement No. 492-DO-IHUF. May 18, 2020. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 61 Health systems strengthening (HSS), which spans across all these four programs, is national and local in scope. At the national level, HSS supports various funding streams to help develop relevant policies and guidelines and assist the Department of Health (DOH) and other government agencies and other partners with systematic implementation of these policies and guidelines at the regional and local government unit (LGU) levels. At the LGU level, the HP is also supporting implementation of the Universal Health Care (UHC) Law in 12 UHC Integration Sites located within HP project sites. HP activities also prioritize work in Cities Development Initiative cities where high health burden overlap with technical areas where USAID works.1 The HP 2018– 2024 of the USAID/Philippines consists of four primary health programs: (1) FP and adolescent reproductive health (FP/ARH); (2) TB control; (3) CBDR; and (4) HSS. Under each program, USAID/Philippines funds one or more mechanisms, also referred to as activities. These activities have varying start-up and close-out dates, with several carried over from the previous HP cycle (2012 – 2018). (Table 1. Details on each implementing mechanisms/activity are provided in Annex A). Table 1. Health Project Activities 2018 – 2024* Implementing mechanism/activity by project Type of mechanism Start date End date Implementi ng partner (IP) Name of AOR/COR Community Maternal, Neonatal, Child Health & Nutrition Scale-up (CMSU2) Cooperative Agreement August 2016 December 2019 IMAP Ma. Teresa Carpio FP/MNH Health Innovations and Capacity Building Platforms (ReachHealth) Cooperative Agreement December 2018 November 2023 RTI Yolanda Oliveros Bangsamoro Autonomous Region in Muslim Mindanao Health Capacity Building (BARMMHealth) Cooperative Agreement February 2019 February 2024 URC David Dereck Golla VI Treat TB: Supporting MDR-TB Activities in the Philippines Cooperative Agreement (through field support) September 2016 March 2019 Vital Strategies Tito Rodrigo TB Innovations and Health Systems Strengthening (TB IHSS) Cooperative Agreement February 2018 February 2023 FHI 360 Tito Rodrigo TB Platforms for Sustainable Detection, Care and Treatment (TB Platforms) Cooperative Agreement April 2018 April 2023 URC Ernesto Bontuyan TB Local Organizations Network (TB LON) Grant October 2020 September 2023 Action for Health Tito Rodrigo USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 62 Table 1. Health Project Activities 2018 – 2024* Implementing mechanism/activity by project Type of mechanism Start date End date Implementi ng partner (IP) Name of AOR/COR Initiatives, Inc. Institutionalization of the Health Leadership and Governance Program (IHLGP)* Cooperative Agreement July 2017 September 2020 Zuellig Family Foundation Ma. Teresa Carpio Health Equity and Financial Protection Platform (ProtectHealth)* Cooperative Agreement March 2019 March 2024 Palladium Joseph Lachica Medicines, Technologies, and Pharmaceutical Services Program (MTaPS)* Contract (through field support) September 2018 September 2023 Management Sciences for Health Helen Hipolito Human Resources for Health 2030/Philippines (HRH2030) Cooperative Agreement (through field support) October 2017 June 2020 Chemonics International, Inc. Yolanda Oliveros Expanding Access to Community-based Drug Rehabilitation Program in the Philippines (RenewHealth) Cooperative Agreement May 2019 May 2024 URC Yolanda Oliveros Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) Contract March 2018 March 2022 Panagora Group, LLC Maria Guadalupe David** Helen Hipolito*** * For the multi-year whole-of-project evaluation, the HIV activity will not be covered as it was only launched in September 2020. ** up to Aug 14, 2021 *** from Aug 15, 2021 to March 29, 2022 The HP midterm evaluation will cover the four health programs being supported by USAID/ Philippines: TB, FP/ARH, CBDR, and HSS. It will evaluate the various activities (mechanisms) under these four programs to determine their contributions in reaching the HP’s goal of ‘Improved health for underserved Filipinos’ under USAID/ Philippines’ Country Development Cooperation Strategy (CDCS) 2019 – 2024, which focuses on good governance and self-reliance to achieve the goal of the Philippines becoming “A Well-Governed and More Self-reliant Indo-Pacific Partner.”22 The three development 22 USAID/Philippines. Country Development Cooperation Strategy 2019 – 2024. USAID, Manila, Philippines. 2019. Available at: https://www.usaid.gov/sites/default/files/documents/1861/Philippines_CDCS_2019-2024.pdf USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 63 objectives (DO) of the CDCS are: DO1: democratic governance strengthened; DO2: inclusive, market￾driven growth expanded; and DO3: environmental and community resilience enhanced. The CDCS also emphasizes three cross-cutting strategies: increased private sector engagement, enhanced gender and social inclusion, and civil society strengthened. Please see the HP’s results framework below (Figure 1). Figure 1. Framework of the USAID/Philippines Health Project, 2019–2024 2.2 Development Context The USAID/Philippines CDCS provides the overall frame in which the HP operates. The focus for the CDCS covering the period 2013–2019 was on Partnership for Growth (PFG) to attain the goal of a More Stable, Prosperous and Well-Governed Nation. While looking back at 2018 and 2019 with the PFG lens, the HP trajectory for the present and future will be guided by the new CDCS (2019–2024), which focuses on good governance and self-reliance to achieve the goal of the Philippines becoming “A Well-Governed and More Self-reliant Indo-Pacific Partner.” The three DOs of the new CDCS are: DO1: democratic governance strengthened; DO2: inclusive, market-driven growth expanded; and DO3: environmental and community resilience enhanced. The new CDCS also emphasizes three cross-cutting strategies: increased private sector engagement, enhanced gender and social inclusion, and civil society strengthened.23 The CDCS goals are consistent with and complement those of the Philippine Health Agenda 2016–2022, which focuses on financial risk protection, better health outcomes, and a responsive health system that provides access to services. The Philippine Health Agenda was framed at a time when the country was making considerable headway in addressing inequities and inefficiencies in service delivery, financing, regulation, and demand generation. The CDCS is also aligned with the medium-term National Objectives for Health (NOH) 2017–2022 of the Department of Health which serves as a roadmap for 23 USAID/Philippines. Country Development Cooperation Strategy (CDCS), November 25, 2019 – November 25, 2024. Available at: https://www.usaid.gov/philippines/cdcs USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 64 the Philippines to achieve universal health care. The NOH is the guiding post for implementing FOURmula One Plus (with the tagline “Boosting Universal Health Care”), which provides the medium￾term strategic framework for revitalizing the health reform agenda. FOURmula One Plus focuses on the four pillars of health reforms (i.e., service delivery, regulation, financing, and governance) with emphasis on accountability. The CDCS and the HP strategy are likewise aligned with the Philippine Development Plan (PDP) 2017 – 2022, which aims to lay the foundation for inclusive growth, a high-trust and resilient society, and a globally competitive knowledge economy. The PDP goal is supported by three pillars: Malasakit (building trust), Pagbabago (reducing inequalities), and Patuloy na Pag-unlad (sustainable growth and development). These pillars are further supported by strategic policies and macroeconomic fundamentals, and built on a solid bedrock of safety, peace and security, infrastructure, and a healthy environment. Years 2018 – 2019. The enactment of the UHC Law (Republic Act [RA] 11223) in July 2018 gave the impetus to the HP IPs to refocus some of their resources to supporting the DOH and PhilHealth, and more critically the local governments in the sites assisted by the United States Government (USG), in putting in place the necessary infrastructure (systems, networks of providers) to improve access of the underserved and the poor to health services. However, the RA 11223 Implementing Rules and Regulations (IRR) were released only in November 2019, postponing the actual implementation of planned activities. Numerous consultations at the national and local levels as well as issues of corruption, claims fraud and leadership changes at the helm of the Philippine Health Insurance Corporation (PhilHealth) during the year caused delays in the development and approval of the IRR. While the annual budgets of the DOH steadily rose from about PhP7 billion in 1991 to PhP93 billion in 2019,24 the delay in the congressional approval of the 2019 government budget curtailed the implementation of activities, impeded the procurement process, and led to stock-outs in needed drugs and medicines. This lag in budget approval exacerbated the persistent problems of low disbursement of funds at the DOH. Even as the obligation rates averaged 88 percent in the last five years (with high rates of 95 percent in 2017 and 94 percent in 2018), the disbursement rates were in the low 60 – 65 percent. Despite efforts to train staff and impose bureaucratic remedies, budget planning and execution, procurement bottlenecks, and supply chain management issues remain. The budget obligation rate of the Family Health and Nutrition and Responsible Parenting (FHNRP) unit under the DOH Disease Prevention and Control Bureau (DPCB) decreased from 62 percent in 2018 to 51 percent in 2019, while the very low disbursement rates improved marginally from 1 percent to 6 percent during the same period. The average obligation rate in the DCPB was 82 percent in 2018 and 80 percent in 2019, while the average disbursement rate was only 12 percent in 2018 but increased to 26 percent in 2019. The National TB Control Program (NTP) had zero disbursement in 2018, despite obligating 61 percent of its budget. The DOH acknowledged that procurement problems – bidding and contracting issues – caused this non-disbursement of obligated funds. However, the NTP performed better in 2019, showing an obligation rate of 99 percent and a disbursement rate of 51 percent. 24 The 2018 budget for DOH was higher at PhP 106 billion. The decrease in the 2019 budget was because the budget subsidy for PhilHealth is no longer included in the 2019 DOH budget. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 65 The late disbursement of funds, compounded by the lack of storage space and interruptions in delivery due to the requirements of the DOH Food and Drug Administration, aggravated the stock-outs of drugs and commodities. In the case of subdermal implants, local stocks expired due to a moratorium on the Responsible Parenthood and Reproductive Health (RPRH) Law implementation (lifted only in late 2018) and a global shortage of implants worsened the resupply problem. The procurement and hiring ban prior to and immediately after the election of local and congressional officials in May 2019 likewise contributed to the lags in purchase and delivery of goods and services. While the IPs were not directly affected by the bans, the DOH and the LGUs were significantly affected. Moreover, the IPs’ priorities and approaches to project implementation were affected by changes in leadership at the LGUs and members of both houses of the Philippine Congress, with whom they had to discuss policy concerns and/or program implementation. The outbreaks of measles and dengue were anticipated and the DOH had the resources and protocols to respond. But the public and congressional uproar over the outbreaks, particularly the re-emergence of polio, distracted the attention of the DOH administration and disrupted the implementation of its program plans, which were already seriously compromised by budget approval delays. However, in response to these outbreaks, the disbursement rates for the National Immunization Program (NIP) and Prevention and Control of Other Infectious Diseases (PCID) Program dramatically increased in 2019. The disbursement rates for the NIP rose from 16 percent in 2018 to 42 percent in 2019; those for the PCID increased from 11 percent in 2018 to 42 percent in 2019. Year 2020 and COVID-19. In 2020, the HP continued to assist PhilHealth and the DOH in the analysis of the financial impact of the UHC Law’s full implementation, and in presenting options within the available public fiscal space given the expanded Sin Tax Law and other excise taxes earmarked for health, and the budgetary burden and future requirements for responding to the COVID-19 pandemic. The National Expenditure Program 2020 set aside PhP172.4 billion for the UHC Law implementation and allocated PhP71.4 billion for PhilHealth premium subsidies. However, funding shortfalls were expected since there was no increase in premium subsidies from 2019 levels and the requirements for the national COVID-19 responses were tremendous. Moreover, there was an expected uptick in utilization arising from a combination of universal enrollment and immediate eligibility to benefits packages. The collection from the formal sector that usually mitigates the shortfall was imperiled, given the loss in premium payments due to job losses and business closures during the community quarantines in response to the COVID-19 pandemic. As of end September 2020, PhilHealth received a total of PhP112.5 billion in premium contributions from all sources, a slight increase (3.7 percent) from 2019, and had paid benefit claims of around PhP110 billion, almost 30 percent more than in 2019, likely due to the costs of treating COVID-19 cases. The total benefit claims include estimated payments for primary care benefits (actuarial estimates as of December 2019 based on the primary care benefits per family payment rate) but not yet released. Primary care benefit payment cheques, still in the possession of PhilHealth, representing claims processed, amounted to approximately PhP132 million as of September 30, 2020, and approximately PhP174 million as of December 31, 2019. Charges of inefficiency and corruption continue to hound USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 66 PhilHealth, jeopardizing the approval of any requests for more subsidies or increase in premium rates to recover and offset reductions in premium contributions due to job losses and business closures. The total budget allocated for the DOH in 2020 was approximately PhP105 billion, higher than the PhP93 billion in 2019. The biggest increases were for hospital operations (a budget of close to PhP40 billion), with a PhP8.7 billion increase for 2020 as hospitals bore the brunt of the COVID-19 response. The Family Health and Nutrition and Responsible Parenting (FHNRP) Unit was appropriated a budget of 18 percent less than the 2019 total (PhP2,033,916,000 vs. PhP2,472,022,000, respectively) and it was almost half (44 percent reduction) of that in 2018 (PhP3,639,663,000). Approximately PhP100 million of its budget (i.e., the budget for Public Health Management that funds soft components such as training and monitoring) was realigned to the COVID-19 response. The 2020 total budget for the National TB Control Program (NTP) increased by 4 percent (PhP949 million) from the 2019 budget (PhP880 million) and increased 16 percent from the 2018 budget (PhP785 million). Available information up to the end of the CY 2020 Q3 showed that the budget obligation rate of the FHNRP Unit was 42.5 percent (51 percent in the entire 2019) but the disbursement rate was only 5 percent. The NTP had a 96 percent obligation rate, but the disbursement was only 10 percent by the end of CY 2020 Q3. This slow utilization of current year budgets is compounded by the need to first spend the previous year’s continuing appropriations. By the end of September 2020, the FHNRP Unit had obligated only 73 percent of its 2019 continuing appropriations, of which only 23 percent was disbursed. The NTP obligated almost all of its 2019 continuing appropriations (98 percent) and disbursed 88 percent of it. While this slow movement of funds in the early part of the year is common in government agencies, it appears that procurement problems such as bidding and contracting issues, which were already recognized in 2019, were not immediately addressed in early 2020.The slow disbursement of funds could have been affected by the enhanced community quarantine, which started in mid-March 2020, when even government offices slowed down operations. Likewise, the Taal volcano eruption in January 2020 and subsequent displacement of the population in the surrounding areas, diverted the attention of the DOH and local health officials to provide emergency response in these areas. The budget for the Prevention and Control of Other Infectious Diseases Unit increased by 38 percent, from PhP738,347,000 in 2019 to PhP1,018,345,000 in 2020. However, this is still less than the Program’s 2018 budget of approximately PhP1.6 billion. By the end of CY 2020 Q3, 86 percent was obligated, but only 4 percent was disbursed. This could have been a case of the unit using the collected funds specifically for the COVID-19 pandemic response, instead of its own 2020 appropriations. The budget of the Epidemiology and Surveillance Unit, which is at the forefront of COVID-19 case monitoring and analysis, was reduced by 56 percent from around PhP266 million in 2019 to PhP115 million in 2020. USAID’s IPs operations were affected by the community quarantines imposed in almost the entire country due to COVID-19. Expanded community quarantine was imposed the longest in the National Capital Region (NCR), Region III, and Region IV-A, and other hotspots like Cebu City and Baguio City. The first of these lockdowns started in mid-March 2020 and various versions were imposed at different localities and at varying times throughout the year depending on COVID-19 case rates. Because most IP activities work with and engage DOH officials, local health professionals, and community health workers, USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 67 they were negatively affected when it became extremely difficult to secure time to discuss policies and programs with health officials who were fully occupied managing the responses to the pandemic. Moreover, the travel restrictions and distancing protocols halted face-to-face community engagements (e.g., conduct of Usapan sessions and trainings) and interactions with health facilities staff on data monitoring and reporting. Likewise, closures of small private health clinics limited collaboration with the private health sector, while the re-deployment of health staff and conversion of health facilities, including laboratories, to COVID-19 case treatment and testing affected data recording and reporting, and more importantly the provision of services other than for COVID-19. Nevertheless, local health offices and providers continued to serve the public and navigate through imposed quarantine protocols and associated limitations to comply with the DOH Circular 2020-0167 dated March 23, 2020 on Continuous Provision of Essential Health Services during the COVID-19 pandemic. The IPs also responded quickly to assess the situation in USG-assisted sites by conducting rapid surveys of local health facilities’ operations during the community quarantine period. These rapid assessments showed that almost all health care providers contacted (more than 90 percent) reported that their facilities remained open and were able to deliver essential services, including for family planning (FP). Health providers employed adaptive measures, such as house-to-house delivery of FP commodities in quantities good for longer periods, and deployment of more barangay volunteers (e.g., Barangay Health Emergency Response Teams) to mitigate the effects of the mobility restrictions, such as reduced number of health staff and transport constraints of clients. In the case of TB services, anti-TB drugs were provided in health facilities and were also delivered to the homes of patients in quantities good for a month. Additional barangay health volunteers were engaged to monitor treatment compliance at home, and TB testing centers were re-zoned to ensure continued access in spite of the conversion of some laboratories to COVID-19 testing laboratories. However, even with these measures, significant adverse impact of the COVID-19 pandemic was expected on the FP/ARH, TB, and CBDR programs, and the country as a whole. While the government, health facilities, and the IPs were able to utilize various digital platforms (e.g., mobile phone apps and online meetings), many locations faced unstable mobile phone and internet connections and remote areas lacked the required digital infrastructure (i.e., smart phones, computers, internet connections) to perform their tasks, participate in meetings and trainings, and report and coordinate with other health facilities and clients. 2.3 Coordination Mechanisms USAID/Philippines has a synchronized and integrated approach among its HP activities in coordinating and collaborating with national agencies, LGUs, and other stakeholders. Through the signed Development Objective Grant Agreement (DOAG), USAID and the Philippine Government convened a DOAG Management Committee and a DOAG Steering Committee composed of representatives from national agencies and USAID, which oversee the implementation of the provisions of the DOAG and conduct program implementation reviews. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 68 The USAID Office of Health (OH) also organizes regular coordination meetings and learning sessions among its implementing partners for TB, FP, CBDR and HSS activities to effectively track, monitor, and adapt key operational activities across the phases of the program cycle. CLAimHealth supports OH in implementing collaborating, learning, and adapting (CLA) approaches to ensure a harmonized, evidence￾based whole-of-project planning and implementation of the HP. 2.4 Theory of Change The underlying Theory of Change (ToC) of the HP 2018–2024 is: If key aspects of the health system are strengthened and institutionalized, then the health of underserved Filipinos will improve, and the overall health profile of the country will improve. By addressing the needs of the individual, the quality and equity of services, and the sustainability of services and systems, underserved Filipinos will be able to develop and maintain healthy behaviors and seek and receive quality health care. In line with the CDCS and based on the 2016 Health Portfolio Evaluation25 and the Project Appraisal Document,26 the appropriate strategy to influence desired change is to bolster and institutionalize a system for the GPH health sector, which reinforces healthy behavior, quality services, and agile system functions. This is a significant shift from a strategy of providing technical assistance to fill service and management gaps, to a GPH partner that strengthens sustainable systems through evidence-based innovative approaches that deeply engage actors at both national and local levels in the context of an increasingly decentralized governance system.27 The assumptions for positive change through the HP are: • Reasonable access to the underserved Filipinos (regions/sites with the highest TB burden and unmet need, and low-income groups in these areas) • Political support to effectively implement the RPRH (Republic Act 10354, also called the RPRH Law), the Comprehensive TB Elimination Plan Act of 2016 (TB Law, Republic Act No. 10767), and the UHC Law (Republic Act 11223) • Public sector funding levels at the central and regional levels will be, at least, maintained at current levels and/or increased • Services will continue or resume during periods of natural disaster or political unrest • USAID funding and staffing levels from USAID will be, at least, maintained at current levels and/or increased 25 USAID Philippines. Health Portfolio Evaluation. August 2016. 26 USAID Philippines. Project Appraisal Document: USAID/Philippines Health Project 2017 – 2022. 27 Ibid. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 69 • For TB, funding from the Global Fund will be, at least, maintained and/or increased until public sector funding can sufficiently fill the funding gap The change expected from implementing the HP activities will be measured through a set of indicators at the level of outcomes and outputs for the three sub-purposes (see Figure 2). USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 70 Figure 2. USAID Health Project Results Framework, 2019–2024 (with updates to performance indicators as of May 2021) USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 71 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 72 2.5 Geographic Coverage The HP has identified project sites based mainly on ‘underserved populations,’ i.e., where the TB disease burden is the highest, where unmet need for FP is the highest, where there are high teenage pregnancy rates, and where there is a high, evidence-based need for drug demand reduction services. Table 2 lists the 14 regions of the Philippines where the HP Activities provide technical assistance for FP/ARH, TB, and CBDR. At the national/central level, the HP provides technical support on health financing for UHC and procurement and supply chain management to DOH, POPCOM, and PHIC. A list of sites covered by specific HP Activities is provided in Annex B. Table 2. Regional Coverage of the Health Project Region FP Activities (ReachHealth & BARMMHealth) TB Activities (TB IHSS & TB Platforms) CBDR (RenewHealth) NCR    Region III    Region IV-A   Region IV-B  Region V  Region VI  Region VII    Region VIII  Region IX  Region X   Region XI  Region XII   Region XIII  BARMM    3. Purpose This multi-year whole-of-project evaluation (WOPE) of USAID/Philippines’ HP for the period FY 2018 to FY 2021 aims to: 1) Inform the future strategic direction and approach of the USAID/Philippines HP. It will identify key aspects of the political, economic, social, and HSS context that have influenced and will influence health programming in the future. It will recommend strategic shifts considering current realities and future anticipated changes, particularly for TB. 2) Validate the HP theory of change and determine, based on available evidence, whether current USAID-supported activities are contributing to output and outcome indicators for these health programs and to the achievement of the HP purpose as a whole. Depending on the findings and results, it will recommend adjustments or modifications in the design and implementation of current approaches and interventions to improve their effectiveness in the remaining three years of the current project cycle as well as key directions and interventions for the next project cycle. 3) At the activity level, determine factors that contribute to or hinder the achievement of FP/ARH, TB, and CBDR outcomes and high-level program indicators as well as HSS for USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 73 UHC. It will identify packages of interventions and proven innovative solutions for these programs that may be recommended for replication and scaling up in the next project cycle, taking into consideration the factors required for the interventions to work optimally. It will also recommend adjustments to the design of current interventions that could be implemented in the remaining years of the HP activities. It will also identify gaps in the current USAID health programming to inform the design of the health strategy for FY 2024−2029. 4. Evaluation Questions The overarching questions of this Multi-year WOPE aim to: (1) inform future strategic directions and approach of the USAID/Philippines HP; (2) recommend adaptations in the design of current interventions that could be implemented in the remaining years of current HP activities; (3) determine whether current USAID-supported activities contribute to the achievement of the HP purpose and sub-purposes; and (4) determine facilitating and hindering factors affecting the achievement of the HP goals for TB, FP/ARH, CBDR, and HSS. Other learning questions for consideration are to examine the validity of the HP ToC, value perceptions of stakeholders on USAID assistance, and/or unintended results that contributed to the HP outcomes. The WOPE will address the primary and secondary learning questions comprising the HP Learning Agenda for 2018 – 2024 provided below. Evaluation/analytic question Secondary questions 1. For the next USAID program cycle (2022 – 2026), what type of support should the HP invest in to achieve desired health outcomes and sustainable systems changes in the Philippines? • Which health system pillar/s should be prioritized? • What type/form/nature of technical assistance should be provided for TB, HIV, FP/ARH, and CBDR? • What mechanisms should be used for the delivery of support? 2. How has the USAID HP contributed to improve health outcomes among the underserved? [Improving Health Outcomes] • To what extent have the HP Activities contributed to achieving the targets (i.e., extent of performance relative to the annual targets and end-of-project targets)? • What were the critical enabling factors associated with achievement of HP targets? • What were the challenges and barriers to achieving HP targets? • What innovations contributed to improving health outcomes? 3. How has the USAID HP interventions improved social norms and behaviors among the underserved seeking treatment and prevention services? [Improving Demand] • What demand generation platform and messaging were most effective for: men, adolescent youth, urban poor, women with unmet need for FP, and persons who use drugs (PWUD)? • What HP interventions have improved health seeking behavior and treatment adherence? 4. How has the USAID HP increased client satisfaction? [Improving Demand and Supply] • How does the HP operationalize people-centered care in delivery of health services? • How does the HP measure client satisfaction? • How have USAID health interventions increased and sustained FP use, increased TB treatment compliance, and reduced CBDR discontinuation rates? 5. How has the USAID HP led to continuous quality improvement in care service delivery? [Improving Supply] • How has HP improved the skills of FP/ARH health providers? • How has HP expanded FP services? USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 74 Evaluation/analytic question Secondary questions 6. How has the USAID HP contributed to improving financial risk protection? [Improving Health Systems] • What HP interventions have helped reduce out- of￾pocket cost for TB and FP/ARH for the underserved? • What HP interventions have improved the social and financial benefit packages related to TB and FP/ARH? • What HP interventions have improved the delivery of social and financial benefit packages related to TB and FP/ARH? • What HP initiatives have helped to increase and/or secure program budgets for TB, FP/ARH, and CBDR at the national and local levels? 7. How has the USAID HP supported the strengthening of health systems and the operationalization of UHC in the Philippines? [Improving Health Systems] • [General] What HSS reforms are in place because of USAID interventions? How are these HSS reforms impacting health outcomes? • [Governance and Policy] What HP interventions have improved effectiveness of DOH (Disease Control and Prevention Bureau, Health Promotions Bureau, Pharmaceutical Division, Health Human Resources and Development Bureau, Dangerous Drugs Abuse & Prevention & Treatment Bureau, its information systems), , Ministry of Health BARMM, PHIC, and Dangerous Drugs Board in delivering their mandates? • [Governance and Policy] What HP interventions have improved local health system capacity for policy formulation, budget planning & execution, inter-local cooperation, program implementation and M&E? • [Governance and Policy] What HP interventions have increased national and local capacity in health systems management in sustainably supporting TB and FP/ARH Programs? • [Human Resources for Health] What HP interventions have helped in expanding and building health provider capacity for quality health care delivery for TB and FP/ARH Programs? • [Human Resources for Health] What HP interventions have helped in improving supportive supervision and mentoring in HRH? • [Logistics] What HP interventions have helped ensure TB and FP/ARH health commodity security at the national and local levels? • [Information System] How has the HP supported improvement of the TB, FP/ARH, and CBDR information systems? 8. How has private sector engagement contributed to achieving better health outcomes? [Improving Private Sector Engagement] • How has the USAID HP contributed to the greater involvement of the private sector? • What limits the private sector engagement in effectively contributing to better health outcomes? • What engagement models increased private sector engagement? 9. How has CSO engagement contributed to achieving better health outcomes? [Improving Civil Society Engagement] • How has the USAID HP contributed to the greater involvement of CSOs? • What limits the civil society engagement in effectively contributing to better health outcomes? USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 75 Evaluation/analytic question Secondary questions • What engagement models increased civic participation? 10. How has the USAID HP contributed to addressing GEWE concerns in accessing health services? [GEWE] • How many people were reached and provided with services related to gender-based violence? • How many people were trained to advance outcomes consistent with GEWE though their roles in public or private sector institutions or organizations? • How can access of males to TB services be increased? • How can access of men, women, and adolescents to FP services be improved? • How can access of male, female, and adolescent PWUD to CBDR services be improved? • How did HP Activities contribute to the above? 11. How has the USAID HP improved the ability of health service delivery points to mitigate environmental risks and withstand climate risks? [CRM] • What interventions have been effective in improving resiliency of service delivery points? 12. How has the USAID HP ensured that interventions for the underserved are sustainable beyond project life? [J2SR] • What interventions have contributed to the resilience of national and local health systems? • What processes have led to reforms or adoption of high impact interventions by the GPH at the national and local levels? 13. To what extent are CLA practices integrated in the USAID HP? [CLA Mainstreaming] • [Collaboration] How have IP collaboration & coordination improved project performance? • [Learning] What learning platforms were most effective in supporting evidence generation and utilization? • [Adaptive Management] What adaptive management platforms were most effective in supporting decision making and adoption? 14. What innovative technologies and adaptations are being introduced, given the current environment and realities? [Innovation] For the consideration and guidance of the evaluation team, supplemental guide questions, as suggested by USAID/Philippines OH, are found in Annex C. 5. Audience The main audience of this multi-year WOPE of the USAID HP include the following: (1) leadership and technical staff of the USAID/Philippines OH and USAID/Philippines Program Resource Management Office, (2) the DOAG Management Committee and Steering Committee, (3) leadership and technical/field staff of the nine HP Activities currently in place, and (4) local stakeholders of the HP, including technical and program staff of the DOH including Center for Health Development (CHDs), the National Economic Development Authority (NEDA), Commission on Population and Development (POPCOM), Philippine Health Insurance Corporation (PhilHealth), local chief executives (LCEs) and provincial health offices, and private sector and civil society organization (CSO) partners. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 76 6. Methods 6.1 General description of methods This portfolio evaluation is a multi-year, cumulative WOP exercise that spans the entire HP cycle and all component activities. This is designed as a mixed-methods evaluation, drawing on both quantitative and qualitative data and triangulating the data gathered. The mixed-method evaluation design will employ a mix of quantitative and qualitative data collection methods and tools. The primary quantitative data sources will be regular reports from IPs on their accomplishments vis-à-vis targets for OH indicators and selected activity-level indicators, as well as secondary data from national, regional, and provincial databases, where available and applicable. The data sources for the results for the OH indicators are multiple: records of DOH (e.g., the NTP’s Integrated TB Information System [ITIS]), LGU-level health facilities such as FHSIS reports where possible, and the PhilHealth; and, where needed, primary data collection through rapid online surveys. Information on the quality of data, derived from data quality assessments and data validation exercises, will also be gathered. Qualitative data collection methods will be used to dive deeper into why and how interventions work or do not work. This will involve key informant interviews and focus group discussions with IPs and their AORs, and selected stakeholders such as DOH, POPCOM, PhilHealth, LGUs, clients and communities served, and other partner agencies. Records of pause-and-reflect sessions and after-action reviews will also be valuable sources of information to understand performance. In addition, results of implementation research and case studies of good practices and promising interventions (GPPI) conducted by the IPs will be reviewed and analyzed to provide supplementary evidence on what works (or does not work). 6.2 Document and Data Review The desk review will be used to provide background information on the HP and will also provide relevant data for this evaluation. Indicative documents and data to be reviewed include: Phase 1 • Existing HP documents, including: o USAID Philippines. Health Portfolio Evaluation. August 2016. o USAID Philippines. Redacted Project Appraisal Document: USAID/Philippines Health Poject 2017 – 2022. o Updated HP MEL Plan, HP Learning Plan, HP Dissemination Plan (as of May 2021) o Annual HP Performance Evaluations (CY 2019 and CY 2020) o HP Performance Evaluation Report (FY 2018) o HP Mid-year Reports (FY 2019 and FY 2020) o HP GPPI Reports ▪ FP in Hospitals ▪ Program for Young Parents ▪ FP Days ▪ Journey to Self-Reliance: Integrated Midwives Association of the Philippines (IMAP) ▪ Service Delivery Network ▪ Engaging Local Chief Executives USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 77 ▪ Midwives Mentoring & Monitoring (3Ms) and Supportive Supervision Visits Plus o HP secondary data analysis ▪ Socio-demographic patterns, knowledge, attitudes, and health-seeking behavior among persons with presumptive pulmonary tuberculosis in selected regions of the Philippines: Secondary analysis from 2016 and 2017 national survey datasets ▪ Secondary data analysis on unmet need for postpartum family planning ▪ Rapid assessment on the effect of the COVID-19 pandemic on TB case finding activities (April 2021) ▪ Rapid assessment of DS-TB and DR-TB situation in the Philippines (May 2021) • Existing Activity/IP documents, including: o RFP/A and corresponding proposals/applications (for each Activity/IP) o Redacted copies of current contracts and cooperative agreements with USAID cooperating agencies, including modification documents if related to substantive programmative changes rather than budgets/incremental funding o HP performance indicators and HP performance tracking table (PITT), including targets o Annual reports o Activity monitoring, evaluation, and learning plans (AMELPs) and activity PITTs o Activity annual implementation plans and similar related documents o Formative research or technical products developed by IPs • National Demographic and Health Survey (NDHS) Reports (2017, 2013) • Responsible Parenthood and Reproductive Health, National Implementation Team (RPRH NIT) Annual Reports (2020, 2019, 2018) • National FP Program Costed Implementation Plan (2018 – 2022) • National TB Prevalence Survey (2016) • Expansion of social protection is necessary towards zero catastrophic costs due to TB: The first national TB patient cost survey in the Philippines (Florentino, Arao, et. al, 2021) • Updated PhilSTEP1 2020-2023 • National TB Adaptive Plan (June 2020) • National TB Health Promotion and Communication Strategy • TB Joint-Program Review (2019) • TB Joint Program Review 2019 Epidemiological Report • TB Preventive Treatment Roadmap 2020–2023 • TB Roadmap Plan and Reports • National Health and Expenditure Survey (ongoing) • Philippine National Health Accounts • Annual Reports of the Philippine Health Insurance Corporation • CBDR background documents (to be specified) • HSS background documents (to be specified) • Relevant policy documents (to be specified) • Other Documents: o Research studies, surveys, evaluation reports and similar documents and literature related to: (i) SBCC and demand generation, (ii) provision of FP/ARH, TB, and CBDR services and (iii) policy, systems, and financing support to FP/ARH, TB, and CBDR o TB modeling studies from AuTuMN, STOP TB, and others o The Global Fund to Fight AIDS, TB and Malaria – strategic initiatives and strategy papers on private sector engagement, human rights, and gender equality o Activity strategy papers, concept paper of Activity approaches o Evaluation reports and other relevant reports of similar nature o FP/ARH, CBDR, TB-related surveys • Databases o NDHS USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 78 ▪ 2017 (https://dhsprogram.com/data/dataset/Philippines_Standard￾DHS_2017.cfm?flag=0&flag=0) ▪ 2013 (https://dhsprogram.com/data/dataset/Philippines_Standard￾DHS_2013.cfm?flag=0&flag=0) ▪ DHS STATcompiler (https://www.statcompiler.com/en/) o DOH NTP ITIS/ Race to End TB Dashboard (http://racetb.doh.gov.ph/#!/layouts/dashboard￾fullview.html) Phase 2 Additional review of Phase 1 documents and data, as needed, plus any additional documents and data not yet reviewed that are identified during Phase 1. 6.3 Secondary Analysis of Existing Data Secondary data analyses will be initiated in Phase 1 and supplemental analyses conducted in Phase 2. For FP, the latest data set available is the NDHS 2017 data, which will not capture the results of HP interventions in 2018 to 2021. However, for TB, data from NTP’s ITIS and the WHO database can be utilized to assess the intermediate results and effect of USAID-assisted TB interventions, i.e., statistically significant increases across time of selected TB indicators. Specifically, the performance in USG-assisted provinces and non-USG￾assisted provinces on the following indicators, among others, can be analyzed in terms of: (i) treatment success rate; (ii) case notification rate; (iii) percent of successfully treated MDR-TB cases, and, to the extent possible, all 10 core TB Accelerator indicators28. To the extent possible, the WOPE will analyze performance in terms of the high-level indicators (e.g., for FP -- new acceptors and current users; for TB – case notification and treatment success rate), taking into consideration baseline data, annual performance, annual targets, and end-of-project targets. To the extent possible, data disaggregation of high-level indicators will be done up to the local level (province/city/municipality) by type of health facility. 6.4 Key informant interviews and FGDs Interviews (via video/teleconference, but preferably face-to-face) will be conducted to identify strengths, best practices, gaps, obstacles to health program and project efficiency, management, coordination, collaboration, learning, adaptation, institutionalization, and sustainability. Key informants will include, but not limited to: • USAID/Philippines OH staff • FP/ARH, TB, HSS, and CBDR IP staff, including subcontractors • DOH (Disease Control & Prevention Bureau, specifically, NTP, Family Health Office, Health Policy Development & Planning Bureau, Health Promotions Bureau, Bureau of International Cooperation, Procurement & Supply Chain Management Team, Pharmaceutical Division, Health Human Resource and Development Bureau, Food and Drugs Administration, Field Implementation & Control Teams, Centers for Health Development, Dangerous Drugs Abuse & Prevention & Treatment Program), POPCOM (central and regional), and PHIC (central and regional). • Other national government agencies (Dangerous Drugs Board, Department of Social Welfare and Development 28 USAID. Global Accelerator to End TB. Available at: https://www.usaid.gov/global-health/health-areas/tuberculosis/resources/news-and￾updates/global-accelerator-end-tb USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 79 • LGU representatives at project sites (local chief executives, provincial/city/municipal health officers, NTP and FP Coordinators) • Local private sector and CSO/NGO partners who have been engaged through the HP activities • Local public and private service health care providers who have been engaged through the HP activities • Development agencies supporting FP/ARH, TB, CBDR programs, and health systems activities Some key informants can be clustered into small groups for interview. The evaluation team will give due consideration to any potential power differentials within a group and to ensure that all participants in a group feel comfortable sharing their opinions. Focus group discussions will be conducted to gain further insight into the relevance and context of the HP activities. Participants in these discussions can include, but are not limited to: • Adolescents • Women of reproductive age (to include groups of young mothers: 15-19 and 20-24 years old) • Males • Other community members who are the target of SBCC interventions (for CBDR, TB, FP, ARH) and beneficiaries of selected high-impact interventions • Health care providers (public and private) at different levels of care • Community health workers 6.5 Remote surveys and/or site visits For selected activities and specific interventions, the evaluation team will do field visits and observations (to the extent possible) and/or rapid online surveys. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 80 ANNEX B. EVALUATION FRAME, ANALYTICAL PROTOCOL, DATA COLLECTION METHODOLOGY AND TOOLS USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 81 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 82 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 83 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 84 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 85 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 86 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 87 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 88 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 89 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 90 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 91 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 92 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 93 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 94 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 95 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 96 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 97 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 98 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 99 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 100 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 101 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 102 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 103 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 104 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 105 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 106 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 107 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 108 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 109 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 110 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 111 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 112 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 113 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 114 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 115 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 116 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 117 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 118 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 119 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 120 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 121 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 122 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 123 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 124 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 125 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 126 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 127 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 128 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 129 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 130 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 131 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 132 ANNEX C. ALL SOURCES OF INFORMATION Persons Interviewed USAID/Philippines Director, Office of Health Deputy Director, Office of Health Project Development Specialists (8), Office of Health USAID/Washington Senior TB Technical Advisor Technical Assistance Support to Country STAR Advisor USAID Implementing Partners Family Planning and Maternal and Neonatal Health Innovations and Capacity-Building Platforms (ReachHealth) Chief of Party Deputy Chief of Party MERLA Director Behavior Change Communication Specialist Health Capacity-Building in BARMM (BARMMHealth) Chief of Party Deputy Chief of Party M&E Specialist Gender Specialist TB Innovations and Health Systems Strengthening (TBIHSS) Chief of Party Deputy Chief of Party and Private Sector Engagement Advisor Demand Generation Advisor DR-TB Technical Advisor Laboratory and Diagnostic Advisor Strategic Information Lead Operations Research Specialist M&E Specialist Operations and Compliance Manager and Gender Specialist Senior Health Systems Specialist iNTP Specialist KM and Communication Specialist Health Systems Specialist TB Platforms for Sustainable Detection, Care, and Treatment (TB Platforms) Activity Chief of Party M&E Advisor Technical Team Lead MEL Officer and Gender Specialist USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 133 Tuberculosis Local Organizations Network: Strengthening Civil Society Movement Towards a People￾Centered TB Response (TB LON) Chief of Party Health Equity and Financial Protection Platform (ProtectHealth) Chief of Party Deputy Chief of Party Senior Advisor for HSS and Health Financing M&E Officer KM Officer Gender Specialist USAID Medicines, Technologies, and Pharmaceutical Services (MTaPS) Program Chief of Party M&E Officer Gender Specialist Expanding Access to Community-Based Drug Rehabilitation (CBDR) Project (RenewHealth) Chief of Party Deputy Chief of Party M&E Officer Gender Specialist SBCC Specialist Human Resources for Health in 2030 Philippines’ Activity (HRH2030/Philippines) M&E Associate Collaborating, Learning, and Adapting for Improved Health (CLAimHealth) Chief of Party Deputy Chief of Party Senior M&E Specialist Senior Research and Learning Specialist Senior Engagement, Learning, and Adapting Specialist Data Management and Analytics Specialist Communications Specialist Learning and Capacity Building Specialist M&E Associate Monitoring, Evaluation, and Learning Coordinator Commission on Population and Development (POPCOM) Executive Director Executive Assistant / Planning Officer III Chief Administrative Officer, Administrative Division Program Assistant, POPCOM NCR Department of Health Undersecretary of Health, Public Health Services Team Undersecretary of Health, Field Implementation and Coordination Team (FICT) – NCR and North Luzon Undersecretary of Health, FICT – Visayas and Mindanao Director IV, Disease Control & Prevention Bureau and Health Promotions Bureau Officer-in-charge, Director IV, Health Policy Development and Planning Bureau Officer-in-charge, Director IV, Health Human Resource Development Bureau USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 134 Program Manager, Pharmaceutical Division Program Manager, Family Health Office Program Manager, National TB Control Program Deputy Head, National TB Reference Laboratory, Research Institute for Tropical Medicine Program Manager, Dangerous Drug Abuse Prevention and Treatment Program (DDAPTP) Director IV, Knowledge Management and Information Technology Service Director General, Food and Drug Administration Director IV, Bureau of International Health Cooperation (BIHC) Senior Health Program Officer, BIHC Department of Health – Regional Health Office NCR Chief, Local Health Support Division TB Program Coordinator FP/ARH Program Coordinator FP Program Coordinator DDAPTP Coordinator Department of Health – Regional Health Office 3 Chief, Local Health Support Division Medical Officer, Non-Communicable Disease TB Program Coordinator FP Program Coordinator HSS Coordinator Department of Health – Regional Health Office 4A Director IV Department of Health – Regional Health Office 7 Director IV Chief of Hospital III Ministry of Health, BARMM Minister of Health Planning Officer IV Medical Officer Philippine Health Insurance Corporation Acting Senior Manager, Benefits Development and Research Department Local Government Unit and Public Health Facility Contacts Province of Cavite Infectious and Non-Communicable Disease Cluster Head Family Health Cluster Head NTP Coordinator Maternal Health and FP and GAD coordinator Chief of Technical Services Office Municipality of Tanza, Cavite Municipal Health Officer USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 135 Province of Rizal FP/MNCHN Coordinator, Rizal Provincial Health Office Quezon City City Health Officer Data Management and Field Operations Officer Medical Officer, FP Division Mental Health, and Adolescent Health Program TB Coordinator Caloocan City Assistant City Health Officer Manila City Division Chief and MNCHN Coordinator Medical Officer, Doña Aurora Quezon Health Center Two (2) other staff of Doña Aurora Quezon Health Center Cebu City City Health Officer and TB Coordinator, Cebu City Health Office Lapu-Lapu City Assistant CHO and TB Coordinator, Lapu-Lapu City Health Office Mandaue City City Health Officer, Mandaue City Health Office Province of Maguindanao Provincial Health Officer, IPHO Maguindanao Technical Division Chief, IPHO Maguindanao Lamitan City Nurse, Lamitan RHU West Sultan Kudarat CBDR focal person, Sultan Kudarat Balay Silangan Reformation Center Private Health Facilities Vice-Chancellor for Research, De La Salle Medical Center and Health Science Institute Executive Director, Mary Johnston Hospital Faculty member, University of Cebu Medical Center Other Partners/International Health Agencies Country Medical Officer, WHO Tuberculosis Technical Officer, WHO Program Manager, Philippine Business for Social Progress Executive Director, Philippine Coalition Against Tuberculosis Founder, Lunas Collective USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 136 ANNEX D. CONSOLIDATED REPORT ON KEY INFORMANT INTERVIEWS General Findings The COVID-19 pandemic was a particularly significant event that triggered necessary adaptations. As a result, projects have fallen behind targets. Restrictions imposed because of COVID-19 increased the difficulty in performing tasks required by projects. Some people did not anticipate restrictions to last as long as expected, and this resulted in further difficulty implementing programs. New needs related to COVID-19 also posed opportunity costs for the projects, such as laboratories that had to cope with the new burden of testing for COVID-19. News on COVID-19 has also overshadowed most other health concerns, policies, and messages, hindering information dissemination and awareness. USAID/Philippines continues to support the projects despite pandemic-related disruptions. The participants in key informant interviews cited similar experiences in the Asia-Pacific region. Tapping into lessons in similar settings like other Asian countries is helpful. The respondents also appreciated the supplementation in commodities due to government limitations in procuring essential items for various health services. Respondents cited the UHC pilot as a positive example. The pilot’s initial plan was to focus on the central levels, but the implementers realized that health programs needed more localized support. 1. On Partnerships with Stakeholders Respondents have identified UNICEF, UNFPA, EU, World Bank, and private industries as potential partners. They also recognized gaps in mechanisms to engage the private sector in health program partnerships effectively. These opportunities were seen especially in the BARMM region. 2. ReachHealth Partnership with ReachHealth appears to be working, as respondents cited this was a good collaboration for FP. Respondents expressed favorable views on establishing a separate COVID-19 team while maintaining their family planning programs. Their positive performance has garnered continuous support for their activities and has encouraged funding for new activities. 3. TB Innovations Respondents described FHI as a good collaborating partner for TB. FHI has abundant experience with TB laboratories and was able to apply it to COVID-19. However, there is a need to re-examine the capacities of laboratories regarding COVID-19 and TB testing to decongest them and figure out a more efficient system. 4. DOH There is a shift in government focus to more integrated approaches to assistance. This change emphasizes cross-cutting projects, primary care, UHC, and capacity-building. This contrasts with the usual vertical approach, which focuses on particular diseases or health care aspects. The respondents cited the DOH as a good partner for profiling adolescents in their various health programs. There is a need for higher-level training and capacity-building to develop skills and expertise within the DOH for emerging health leaders. A specific example provided was an opportunity for the Health Promotions Bureau to send a staff to the Johns Hopkins Center for Communication for training. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 137 5. LGUs There is a need for more transparency and traceability of funding given to LGUs, as funding earmarked for health can get lost. There is also a plan by the Health Project’s health systems strengthening activities, such as ProtectHealth, for expanded collaboration at the LGU level in the future. On facilitators and enabling factors identified by key informants USAID can establish regular partnerships with other organizations, such as UNICEF, UNFPA, EU, World Bank, and the DOH, which already have a presence in the selected project sites. These partnerships can enable a deeper understanding of the context of the areas. Respondents cited a clear geographical scope of work to facilitate better work processes. During the COVID-19 pandemic, USAID-supported comprehensive information campaigns allayed fears and promoted proven measures that improved the public’s confidence in going out of their homes while still practicing COVID-19 prevention measures. On challenges and hindering factors 6. Monitoring Monitoring was challenging, specifically in BARMM. There is also difficulty relaying changes to health program indicators and their definitions to local health systems. 7. Availability of Data Regularly updated data should be available. Health programs use real-time data to respond faster to a dynamic health environment. The data provided by the government are usually released two to three years after data gathering. Respondents cited a need for localized data that are integrated into nearby hospitals. Respondents suggested expanded surveillance at hospitals as a possible solution. The data are vital to provide better perspectives on quality, insurance, and services. 8. Issues with Partners Respondents observed that partners might not always see themselves as part of the larger team. There is a need for improved collaboration among organizations. Specifically, respondents identified gaps in DOH in private-sector engagement mechanisms such as bureaucratic bottlenecks in bidding and partnerships. There were also problems coordinating between partners. DOH program leaders initially preferred top-down coordination processes and felt bypassed when partners directly engaged LGUs. However, the DOH has recently seen the value of coordinating with LGUs. There is also a shift in leadership, staff position, and DOH priorities. The shifting priorities make it challenging to capitalize on the progress of previous projects. The DOH has cited a lack of a central liaison or coordinating officers for all projects. Although all official DOH communications are addressed to the USAID/Philippines Office of Health Director, coordination has involved separate discussions with various USAID staff and implementing partners. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 138 Program-specific findings 9. FP Program Some stakeholders gave a favorable assessment of the project with reasonable satisfaction for coordination and support by USAID. Cited was an ability to adapt and incorporate new techniques such as facilitating deployment of FP commodities, tele medicine outreach, and use of social media, especially in the setting of COVID-19, which positively affected meeting performance targets and expectations. However, the use of RPRH indicators has been paused due to COVID-19 system disruptions. USAID accomplished assistance and activities for health promotion, with some still ongoing. USAID and Agreement Officer’s Representative (AORs) were described as generally supportive. There was involvement and support from a communication team to develop plans. USAID also provided technical support for partners and assistance in service delivery and developing policies such as IEC materials for FP and training for implant insertion and bilateral tubal ligation. The FP programs have adapted to the COVID￾19 pandemic by facilitating deployment of FP commodities, telemedicine, outreach, and the use of social media. Respondents have suggested expanding implementation to take advantage of opportunities such as a partnership with POPCOM and the DOH. The program has provided a strategic partnership for POPCOM and the DOH. The partnerships could provide opportunities to ramp up activities. BARMM is still experiencing transition; some areas are devolved while others are still centralized. Respondents noted opportunities for connecting with other organizations in the region. However, the Universal Health Initiative and other initiatives encountered problems due to the region’s uncertain status (e.g., no planned interventions in Maguindanao and Cotabato City because the status of Cotabato is still in limbo). This also presents difficulties in access and monitoring. The principles of Islam guide the BARMM government officials, and organizations such as the Darul Ifta are authorized to represent the religious sector and speak on behalf of the BARMM government. These organizations are de facto parts of the government itself. Activities for improving health-seeking behaviors of the various partners for FP have varied approaches. Some are focused on behavior change, but others are not. Counseling customized to the needs of specific geographic areas (e.g., BARMM, rural areas) or groups (e.g., adolescents) is provided. Radio stations have continually aired messages and programming that discuss FP. The informants, midwives, nurses, and religious leaders are invited to radio programs to discuss FP and respond to questions. An essential intervention in the past has been interpersonal approaches through Usapan sessions, wherein the implementers adopt a more personalized, community-engaged approach. The project team in BARMM is focused on meetings with women and community members to discuss specific FP issues. Muslim religious leaders have received training to discuss FP in a proper Islamic context to alleviate the concerns of community members. This is considered a key initiative. Preventing early childhood marriages needs a different approach. It’s not about making sure they have FP commodities, but about increasing the conversation between parents and adolescents, enforcing policies and laws against childhood marriages, and getting communities involved in the discussions. The use of a USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 139 gender lens in providing health services has also been instrumental. In the BARMM region, integration, and acceptance of health topics in the Islam sociocultural context have been important in their activities. Gender transformative processes are part of interventions to ensure that women participate in decision making, have control over decisions, and gain the support of men and their partners, and that the community respects the role of women as leaders. Some partners conduct regular data quality checks. The number of patients educated through health programs is used as an indicator of program effectiveness (e.g., number of participants who adopt a modern FP method). Some partners also researched to supplement data from the NDHS and FHSIS. This is done through key informant interviews and focus group discussions. Instability in the BARMM region, particularly in conflict-affected areas, has hindered the region’s activities and discouraged potential project staff. However, working conditions and security have been improved in the area to attract potential project staff. Improving the perspective of coming to frontline work and supporting the transition for more lasting peace in BARMM have also attracted more potential staff for the Health Project staff based in BARMM. There has also been increasing acceptance of USAID projects that have facilitated activities. Respondents identified a need to develop health promotion materials appropriate to different audiences. This includes different versions for different demographic segments (e.g., age, occupation) of the same area. They have also stated that quality improvement efforts continue as they provide services at their facilities. Respondents also described a need for assistance for organizational development such as team-building activities. To sustain the projects, continued technical assistance and support in terms of training and updates are needed. Respondents have stated the need for continued provision of commodities by the BARMM Ministry of Health due to partners’ procurement limitations. While assistance for postpartum FP is being provided, respondents suggested continuing this as a possible point of intervention in the future. Integration of FP services along the continuum of care is needed. Respondents have also described plans for research on demand generation for FP. 10.TB Program The partnership with CiTEC started through a collaborative meeting between the DOH, Cebu City government, PhilHealth, and USAID representatives. A plan to implement a pilot of UHC was envisioned, with the hope that strategies for implementation, strengths, and weaknesses could be gathered. CiTEC sites were selected because of a high incidence rate of TB. The selected barangays also had a high proportion of informal settlers. CiTEC was envisioned to be a screening program. The program would screen residents of Pasil and Mambaling barangays through chest X-ray using an active case-finding strategy. Suspicious densities and findings on chest X-rays warranted sputum microscopy exams. The previous USAID-supported TB project, IMPACT, also had a “screen-all” approach to TB and provided chest X-ray vouchers to encourage TB screening. They also conducted training on policy development for LGUs as well as on the TB Manual of Procedures, which TB Platforms has continued in its project sites. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 140 The ProtectHealth activity of the Health Project planned to create new criteria for barangays to expand CiTEC coverage to 10 other barangays. TBIHSS, on the other hand, spearheaded the coordination and close monitoring of the program, including coordination with health center staff such as nurses, midwives, and CHWs of Pasil and Mambaling. They provided daily counts of residents or households profiled and refusals for screening and monitoring of problems that occurred. There was also a collaboration with the DOH National TB Program. At the LGU level, when the number of individuals who underwent chest X-ray screening was low, the Health Project met with barangay officials to plan for increased demand for services. Project funding is supposedly sourced from multiple public funds such as PAGCOR and the Cebu City government, but the projects encountered problems with city ordinance legislation. The COVID-19 pandemic disrupted program implementation through limited participation because of restrictions on travel and misconceptions. In Mambaling, there was hesitation due to a misconception that their saliva would be used for COVID purposes, and they were afraid to be quarantined. Respondents described a lot of passion and good effort around the TB program. Barangay officials influenced health-seeking behaviors by telling residents about the advantages of X-ray and discussing TB screening. IPs also engaged barangay health workers, senior citizens, purok leaders, and other NGOs because residents respected these groups. Barangay counselors also went to houses in their sitios to tell constituents about the benefits of TB screening. IPs also used online group chats, senior groups, open group activities, and virtual meetings. Respondents cited COVID-19 as one of the factors that hampered execution of activities. An adaptation plan was developed in time for the pandemic. Areas of improvement cited include establishing laboratories with COVID-19 testing that is balanced with testing for other diseases, including TB. Activities were focused on TB screening. There have been calls to ramp up the project. There is also room to grow on the ground by improving laboratories and hospitals. Learnings from other Asian countries can be tapped, especially during the COVID-19 pandemic. The project was hindered by inadequate staffing and pushed to do more beyond the current team’s capacity. Respondents cited a need to focus on the primary task of finding cases. There were also sociocultural challenges that made implementation difficult. Additionally, local officials started to engage with different priorities, especially during the COVID-19 pandemic and the upcoming election in 2022. There were also concerns about the confidentiality and anonymity of participants. For the project’s sustainability, a stable source of funding is needed. Co-funding from the city government through the ordinance was expected but did not push through because the proposal did not pass into a city ordinance. There are also plans to scale up to include more barangays, but this will have to go through LGU approval. 11.CBDR Respondents reported good coordination with USAID. Parties are well-informed of plans and activities, and regular meetings are held. Respondents rated support and coordination of USAID as moderate. Key activities USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 141 frequently highlighted by key informants included the translation of the SBIRT toolkit into the local language and the development of an online resource within the DOH Academy, with rollout in 2022. The project team showed adaptation. Although they started slow, they found their footing and figured out how to move forward in the pandemic. Part of their work was to train more trainers. Respondents cited a need for the project team to rethink its approach, such as working at the regional level. The project was hindered by challenges in coordination in the beginning and challenges in reporting. There was also a need to harmonize functions with different agencies to develop the program. Support for disseminating IEC materials on drugs between partners is needed to sustain future activities. Drug abuse treatment using comprehensive approaches through the entire continuum of care can also be explored in the future. Additionally, USAID can expand the reach of drug rehabilitation services through developing trainers, training programs, and supporting drug abuse councils that incorporate monitoring and evaluation aspects. 12.Health Systems Strengthening USAID supported the development of a digital platform and e-modules, aligning efforts of USAID with the needs of PhilHealth. However, there was a varied rating of the support and coordination of USAID. Respondents noted assistance on costing models and development of benefit packages. There was also coordination regarding the pilot of UHC and healthcare provider networks (HCPNs), particularly for Konsulta, a takeoff point for the model. Different strategies such as pamphlets and live streaming on Facebook were used for orientation on PhilHealth benefits for members, including some information on. FP, maternal care, and TB. However, in the information dissemination to paying members, there was not much focus on specific benefits and the value of their membership. USAID also used DOH’s regional offices, through their public affairs unit, for information dissemination. However, lack of expertise about UHC at certain levels of the health system was also viewed as a hindering factor to UHC implementation, especially when the private institutions are the primary decision makers. Key informants suggested continuing assistance for policies on the Konsulta package. They also preferred hiring local digital education providers rather than international companies. Suggested activities or areas to explore in the future include social marketing of PhilHealth, assistance in developing policies regarding HCPN, provider payment, and management of the global budget. With respect to human resources for health as a pillar of the health system, key informants noted that the HRH2030 project abruptly ended even though their deliverables were not yet completely accomplished. Respondents also stated that there was no satisfactory or helpful output from MTaPS and no progress on the eLMIS project. The human resources management tool that was being promoted by HRH2030, the Workload Indicators for Staffing Needs (WISN), was also interrupted and not sustained during the pandemic because of COVID-19 among the DOH staff. The brain drain of health care workers also hindered the Health Project. There is still a mindset that CHWs leave the country because they lack nationalism and not because of socioeconomic factors. Although the DOH and the Health Project promoted many e-courses during the COVID-19 pandemic, informants still expressed a preference for live and in-person interactions for capacity building. Informants observed that the e-modules being developed for clinical practice guidelines are already organized according to a primary health care framework and life stages and not by disease. Barriers USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 142 identified by the informants against the effective dissemination of online learning materials were: inadequate telecommunications facilities and equipment; need for IT and graphic artists to scale up the online courses; lack of engagement of private institutions; and insufficient training of trainers. 13.Gender Equality and Women’s Empowerment Activities include hospitals’ assistance in establishing referral systems between women and children protection units, gender sensitivity training, and help with gender balance on boards. LGUs are also assisted in establishing referral systems, particularly in nine provinces without women and child protection units. The Gender Responsive Family Planning Service Delivery Point is a checklist that is also being developed to ensure responsiveness in FP services. In response to the issue of disrespect and abuse of women clients in health facilities, a pilot for continuous quality improvement in Batangas integrated client feedback related to disrespect and abuse. A men’s health program manual is being developed, with the policy signed in January 2021. The project can draw on partnerships with other organizations such as the Women and Child Protection Network, which has great expertise in women and child protection. Integrated into the other FP projects, using a gender lens has been instrumental, but progress has been slow. Gender transformative processes are part of interventions to ensure that women participate in decision making, have control over decisions, and gain the support of men and their partners, and that the community respects the role of women as leaders. There is also an active call for a holistic approach to TB management where gender-specific issues must be addressed to find and manage more cases within the community. Health-seeking behaviors between males and females differ in such a way that males usually seek medical help at a much later time, when the disease has progressed (TB in males has an increased mortality rate compared to females). In contrast, females would delay care and prioritize other family members’ medical conditions. Activities such as targeted active case finding were done in predominantly male occupations to maximize hours of productivity. Health indicators should have an end-user impact and not just count initiatives’ inputs. The training of reproductive health counselors is an identified bottleneck in scaling the impact of reproductive health services. Only physical and sexual abuse cases against women and children are being considered. Verbal abuse is only noted and is reported to barangays. Most IEC materials on reproductive health are also written for women and seldom for men. GBV is highly targeted among LGBTQ individuals, subject to psychosocial counseling. Reorganization of the DOH, the COVID-19 pandemic, and the push for integration left the status of the men’s health program remains unclear, as no one was sure who would take over the program. USAID programs during the COVID-19 pandemic have managed to boost GBV awareness and have reached over three million people through online engagements, 68 percent of whom are women aged 18 to 24. Partnerships facilitated this, with celebrities aligned to help increase awareness of GBV. There were initial commitments to initiate a fatwa against GBV, although it has not yet materialized. BARMM can be a priority site for GEWE-related activities in the future. It is highly suggested that stakeholders and partners use a gender-sensitive lens in formulating health programs to address specific health concerns related to being male, female, a child, or LGBTQ. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 143 Synthesis of Key Insights 14.Social Behavior Change Partners should consider monitoring behavior change indicators of target populations during project evaluation. Ultimately, the activities aim to change attitudes through the provision of services. Platforms have been developed to try to get away from number counting. USAID is assessing the impact along with the changes in numbers and movement regarding targets. USAID is using information gathered during consultations with partners and discussing work plans. Key informants cited work with religious leaders in the BARMM area as promising. There is room for progression, especially in adapting to the COVID-19 pandemic. Major work plan strategies should be evaluated to determine how best to improve and institutionalize activities to achieve particular targets. Projects should continue participating in deep-dive analysis and impact evaluation of the results and impact of active case finding strategies (e.g., increasing chest X-ray screening for TB should increase notifications). Lessons gained during the COVID-19 pandemic regarding behavior change include consistent iteration of public health interventions and feedback from communities in crafting messaging; importance of private-sector engagement in propagating health messages; and support from medical experts to reinforce the credibility of health messages. 15.Effect of Gender in Supply Chain Management The HP included gender analysis to deepen the understanding of access to health services. There has been progress in increasing a gender lens approach in health programs. Recently, MTaPS examined supply chain management of and access to FP and TB commodities through a gender lens. A respondent said that this kind of analysis has not been done widely and that it is fascinating to understand gender issues in relation to supply chain management. 16.Monitoring of Funds at the LGU Level LGUs are given financing but tracking and accountability for the use of the funds may not always be as transparent. There is difficulty in tracing the utilization of funds earmarked for health. The monitoring of funds is important. 17.Approaches Used The Pareto principle [which states that 80 percent of results come from 20 percent of actions] is a useful concept in prioritizing high-burden areas, i.e., address the problems where there is the highest burden of disease, for example, the “big three” regions with the highest burden of TB. On-the-ground approaches were effective, and USAID should consider possible adoption in other communities. There was also a shift to regional CHDs for the training of local personnel, with plans to expand the capability of regional CHDs in training staff in their areas. 18.Sustainability and Future Activities and Projects Key informants suggested more focused and direct communication with LGUs for future activities and projects. They also projected an increased need for high-level training and specialization, specifically for middle-level employees. USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 144 ANNEX E. DISCLOSURE OF ANY CONFLICT OF INTEREST Certification No. 1 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 145 Certification No. 2 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 146 Certification No. 3 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 147 Certification No. 4 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 148 Certification No. 5 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 149 Certification No. 6 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 150 Certification No. 7 USAID/Philippines Health Project Whole-of-Project Evaluation (2018–2021) | 151 Certification No. 8