USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) JUNE 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Meghan Holohan, Fabio Luelmo, Ronnie Morfe, Timothy Ryan, and Michael Voniatis through through the Global Health Technical Assistance Bridge Project. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 12-01-041) is available in printed or online versions. Online documents can be located in the GH Tech website at www.ghtechproject.com. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com This document was submitted by Development and Training Services, Inc., with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. AID-OAA-C-12-00004. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) i ACKNOWLEDGEMENTS The project team for the evaluation of the tuberculosis (TB) portfolio of the United States Agency for International Development (USAID) Mission in the Philippines (USAID/Philippines) would like to thank many people for their contributions, including: Staff from USAID/Philippines and USAID implementing partners, for sharing information about TB control issues associated with their various sectors and projects; The many individuals who generously provided time for important interviews, including representatives of the Government of Philippines, the National TB Control Program, other government institutions (such as PhilHealth and the National TB Reference Laboratory), public-private enterprises (such as PhilCAT and PTSI) and the donors, non-governmental organizations, and research institutions collaborating with them; and Staff from the Global Health Technical Assistance Project for their superior administrative and logistical support. The team is especially grateful for the assistance of staff working in hospitals and primary/rural care facilities, whose willingness to openly discuss their experiences and views helped ground this assessment in local realities. A complete list of individuals interviewed during the evaluation is presented in Annex B. ii USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) iii CONTENTS ACRONYMS ...................................................................................................................................... vii EXECUTIVE SUMMARY .................................................................................................................. xi Lessons Learned ........................................................................................................................ xiii Key Recommendations ............................................................................................................ xiv I. INTRODUCTION ....................................................................................................................... 1 Evaluation Purpose ....................................................................................................................... 1 Evaluation Method ....................................................................................................................... 2 About this Report ........................................................................................................................ 3 II. BACKGROUND .......................................................................................................................... 5 About the Philippines .................................................................................................................. 5 Tuberculosis in the Philippines .................................................................................................. 7 USAID Support for TB Control .............................................................................................12 III. EVALUATION FINDINGS ......................................................................................................17 Case Detection and Notification ............................................................................................17 Treatment Quality and Outcomes .........................................................................................23 Access to Diagnosis and Treatment ......................................................................................26 Gender-related Barriers/Enhancers .......................................................................................28 Geographical Access .................................................................................................................30 Stigma and Discrimination ........................................................................................................31 Vulnerable Populations .............................................................................................................32 Health-seeking Behavior ...........................................................................................................33 Policy and Financing ...................................................................................................................34 Private Sector Engagement ......................................................................................................37 Drug-Resistant TB .....................................................................................................................40 Infection Control........................................................................................................................42 Waste Management ...................................................................................................................42 IV. ADVOCACY, COMMUNICATION AND SOCIAL MOBILIZATION ........................45 Laboratory Strengthening .........................................................................................................47 Anti-TB Drug Supply .................................................................................................................49 Human Resources and Capacity Building .............................................................................51 Sustainability ................................................................................................................................58 iv USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Project Management ..................................................................................................................60 Monitoring and Evaluation .......................................................................................................61 V. LESSONS LEARNED .................................................................................................................65 VI. CONCLUSION ..........................................................................................................................67 Overarching Recommendations .............................................................................................67 Overarching Observations for the NTP ...............................................................................68 ANNEXES ANNEX A: SCOPE OF WORK ....................................................................................................69 ANNEX B: KEY CONTACTS .......................................................................................................91 ANNEX C: REFERENCES ........................................................................................................... 101 ANNEX D: DATA COLLECTION TOOL .............................................................................. 103 ANNEX E: TABLES OF MORBIDITY AND MORTALITY IN THE PHILIPPINES ......... 123 ANNEX F: POLICIES DEVELOPED WITH TB LINC SUPPORT....................................... 125 TABLES Table 1. Overview of USAID/Philippines TB Projects and TB Targets (2006–2011) ......................................................................................................................... xi Table 2. Epidemiological Indicators of TB from National Surveys: 1983, 1997 and 2007 ...................................................................................................................... 7 Table 3. Prevalence of Bacteriologically Confirmed TB by Age and Sex: 1997 and 2007 ...................................................................................................................... 8 Table 4. Examples of TB Case Detection by Microscopy in Sites Visited ...........................18 Table 5. Examples of TB Case Detection in Provinces Visited ..............................................19 Table 6. Detection and Treatment of MDR-TB.........................................................................24 Table 7. Costs of TB Products and Services in the Private Sector, 2012 ............................27 Table 8. Human Resources for TB Control in the Davao Region (Region 11), 2008–2011...................................................................................................52 Table 9. Case Notification Rate, Davao Region (Region 11), 2006–2011 ...........................54 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) v GRAPHS Graph 1: Population Pyramid of the Philippines (2010) ............................................................. 5 Graph 2: HIV/AIDS Cases Reported, 2000-March 2012 ........................................................... 6 Graph 3: Organization and Structure of the Philippine TB Program ....................................10 Graph 4: Case Detection Activities in USAID-supported and Non-supported Provinces/Cities, 2008-2010 ..........................................................................................20 Graph 5: Reported Incidence of SS+ TB in USAID-Supported* and Non-Supported* Provinces, Region 11, 2006–2011 ................................................................................21 Graph 6: Prevalence of Smear Positive TB by Age, Gender (2007) ......................................28 Graph 7: Comparison of Health Seeking Behavior in the Philippines, 1997 and 2007 .....34 vi USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) vii ACRONYMS ACSM Advocacy, communication and social mobilization ARMM Autonomous Region in Muslim Mindanao ART Anti-Retroviral Therapy BCG Bacillus Calmette-Guérin BHW Barangay health worker BSL Biosafety level CA Cooperative agreement CBO Community based organizations CDC U.S. Centers for Disease Control CDR Case detection rate CHAT Community Health Action Team CHC City Health Center CHD Centers for Health Development CHT Community health team CHV Community health volunteers CNR Case notification rate CUP Comprehensive and Unified Policy for Tuberculosis Control DALY Disability-adjusted life years DOH Department of Health DOT Direct observation of drug intake DOTS Directly observed treatment short-course DRS Drug resistance survey DSSM Direct sputum smear microscopy DST Drug susceptibility testing EQA External quality assurance FDA Food and Drug Administration FDC Fixed-dose combination FLD First-line anti-Tuberculosis drug FP Family Planning GH Tech Bridge Global Health Technical Assistance Bridge Project GHI U.S. Government Global Health Initiative GIDA Geographically inaccessible depressed areas Global Fund Global Fund to Fight AIDS, Tuberculosis, and Malaria viii USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) HealthGOV Strengthening Local Governance for Health HealthPro Health Promotion and Communication HIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome HMO Health maintenance organization HPDP Health Policy Development Program IEC Information, education, and communication INH Isoniazid IPT Isoniazid preventative therapy ISTC International Standards for Tuberculosis Care LCE Local chief executive LEAD Local Enhancement for Health and Development (LEAD) LGU Local government unit M&E Monitoring & Evaluation MCH Maternal and child health MDG Millennium Development Goal MDR-TB Multidrug-resistant Tuberculosis MHC Municipal Health Center MHO Municipal health officer MSH Management Sciences for Health MTCP Maguindanao Tuberculosis Control Project NCDPC National Center for Disease Prevention and Control NGO Non-governmental organization NTP National TB Control Program NTRL National Tuberculosis Reference Laboratory PBSP Philippine Business for Social Progress PDI Pharmacy DOTS Initiative PhP Philippine Peso PHC Primary health care PhilCAT Philippine Coalition Against Tuberculosis PhilHealth Philippine Health Insurance Corporation PhilPACT Philippine Plan of Action to Control TB PhilTIPS Philippine Tuberculosis Initiative for Private Sector PHN Public health nurse PHO Provincial health office PMDT Programmatic management of drug-resistant Tuberculosis USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) ix PPD Purified protein derivative PPM Public-private mix PTSI Philippine Tuberculosis Society, Inc. QA Quality assurance RCBI Regional Capacity Building Initiative RHU Rural Health Unit RNHEALs Registered Nurses for Health Enhancement and Local Services SHIELD Sustainable Health Improvement through Empowerment and Local Development SLD Second-line anti-Tuberculosis drug SPS Strengthening Pharmaceutical Systems SS- Sputum-smear negative SS+ Sputum-smear positive TA Technical Assistance TB Tuberculosis TB LINC Linking Initiatives and Networking to Control Tuberculosis Project TBDC Tuberculosis Diagnostic Committees USAID United States Agency for International Development USAID/Philippines United States Agency for International Development Mission in the Philippines USG United States Government USP PQM United States Pharmacopeia Promoting the Quality of Medicine WHO World Health Organization x USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) xi EXECUTIVE SUMMARY Tuberculosis (TB) is a major public health concern in the Philippines. Drug-susceptible TB continues to spread and kill, with an estimated more than 260,000 new cases and 30,000 people dying from the disease in 2010. Multidrug-resistant Tuberculosis (MDR-TB), which can take years to treat and is far more expensive than drug-susceptible TB to cure, is an emerging, urgent threat: there were an estimated 8,800 prevalent cases in 2010, raising the Philippines to sixth on the list of 27 countries with the highest burden of MDR-TB. Over the past five years (2006–2011), the United States Agency for International Development (USAID) Mission in the Philippines (USAID/Philippines) has invested in a portfolio of eight TB￾related projects aimed to strengthen and enhance the work of the Philippines’ National TB Control Program (NTP). To assess the performance, quality and impact of these projects and to inform future support, USAID/Philippines initiated an independent, external evaluation of its TB portfolio. This report presents the results of that evaluation. USAID-supported TB projects (2006–2011) enhanced national TB control efforts and improved the quality of care for TB patients. All eight projects achieved their high-level objectives, with both immediate benefits for policy and service delivery and, as a result of extensive capacity building efforts, anticipated long-term benefits. They reached an important quality of care target (i.e., cure rate), however they did not achieve the targets set for some important indicators, such as case detection rates (CDR) and case notification rates (CNR). Among projects that had TB-specific targets, approximately half of the project-specific targets were achieved (primarily process and output targets). The lack of TB-specific targets for five projects did not allow for an overall quantitative judgment (in terms of targets met) of the entire USAID portfolio (see Table 1). TABLE 1. Overview of USAID/Philippines TB Projects and TB Targets (2006–2011) Project TB focus only? TB-specific targets? TB targets met? SPS Yes Yes Partially TBLINC Yes Yes Partially USP-PQM Yes Yes Fully WHO Country Office Yes No n/a HealthGov No No n/a HealthPro No No n/a HPDP No No n/a SHIELD No No n/a The primary contributions of USAID-supported TB projects were in the areas of policy, service regulation and financing development and implementation, in particular at the local government unit level; capacity building in TB care and control through trainings; TB laboratory strengthening (including to the National TB Reference Laboratory [NTRL]); anti-TB drug monitoring; and (to a lesser extent) on information management and communication. xii USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) The number of TB cases reported to the NTP is increasing slowly, driven by small increases in detection of suspects and involvement of private providers and improved quality of reporting. The USAID projects evaluated supported the basic elements of TB case detection and diagnosis, with case notification increasing more in USAID-supported areas than in non-supported areas— but case detection remained insufficient. Cases are diagnosed late, in part because the Direct Observation Treatment Short-course (DOTS), which is the internationally recommended strategy for TB control coverage, does not include all public hospitals and the involvement of private providers is still quite limited. Political commitment to DOTS is strong, with free TB services and anti-TB drugs widely available and delivered according to international recommendations, but many people remain unaware that TB can be cured and that these free services are available. While treatment outcomes are very good, strict adherence to directly observed treatment (DOT) varies, so these outcomes should be validated. Outcomes did appear to be marginally better in USAID￾supported sites Access to TB services in the Philippines is generally good, despite barriers such as cost, gender, stigma and discrimination. Vulnerable populations can access services, but may have a more difficult time than the general population. Health-seeking behaviors also can affect access to DOTS services as many Filipinos choose to receive care in the private sector. Although USAID projects have contributed to the development of a significant number of national and local guidelines and policies, many of these guidelines/policies are either not completed or not fully deployed in the health system. National financing for TB has substantially increased, however, the Philippines' devolved health system requires more (and more consistent) financing by local government units (LGU). This will become a greater priority with the end of support from the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) in 2014, which will substantially reduce financing for MDR-TB diagnosis and treatment. Supported by donors, including USAID, the country has begun to engage the private sector and there is strong cooperation between public and private providers in several municipalities. Coverage is still quite limited, however; efforts to engage the private sector have not been undertaken in all provinces. Furthermore, disrupted efforts by USAID-supported projects in some provinces (there was a delay between the end of the PhilTIPS project in 2006 and TBLINC efforts to engage the private sector in 2010) led to a halt in DOTS services by some providers; and, subsequently, a lack of interest in future involvement. USAID projects have helped introduce standardized regimens for MDR-TB treatment and have strengthened laboratory diagnosis of MDR-TB, including the introduction of policies for rapid MDR-TB diagnosis. Regional MDR-TB diagnosis and treatment centers are operating effectively and new satellite treatment facilities are being introduced to improve access to treatment. There are reasons to be concerned about the emergence of new cases of MDR-TB, such as gaps in DOTS coverage, stockouts of first-line anti-TB drugs (FLD) in some parts of the country, and the availability of both first- and second-line drugs in the private sector without prescription (which enables self-treatment according to inappropriate regimens that can contribute to resistance). Ensuring the sustainability of the MDR-TB program is a priority, as it is largely dependent upon funding from the Global Fund and there does not appear to be a comprehensive plan to fully finance these activities with national resources in the near future. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) xiii Advocacy for new policies and increased support for TB control activities was conducted by USAID projects with good effect. Other communication efforts, while valuable in the locations they reached, were too limited in coverage and there does not appear to have been a significant effort to provide education about TB to the general public. Communication activities and the various messages developed by the USAID-supported TB projects were consistent with NTP￾defined messages. However, it appears that each project pursued their communication activities independent from each other (i.e., different focus/thrust/theme, different format, different execution) instead of taking advantage of materials/strategies developed by other projects. The TB laboratory network has good coverage by smear microscopy and is generally of high quality. USAID-supported projects provided technical assistance (TA) that improved the network, with particular benefits for the NTRL. There are sufficient microscopy units and the workload is acceptable, except in areas where there are an insufficient number of microscopists. The standards for ensuring quality microscopy are rather low, and should be more ambitious. The scale-up of tools to diagnose drug-resistant TB is following a rational strategy but patient coverage is still limited. Existing human resources delivering TB services appear highly competent and well-trained. In some parts of the country, however, there are shortages in manpower. The impact of this is magnified by serious gaps in supervision in parts of the country, at all levels of the TB program. Barangay health workers and other volunteers have helped increase the scope and quality of health services available for the community, especially in remote areas. USAID projects have attempted several models to engage them, some more successful than others. Training provided by USAID projects leveraged national experts and was considered to be of high quality at all levels, but did not adequately assess in advance the needs of beneficiaries and often employed classroom-type seminars, where greater emphasis on supervision and mentoring during actual delivery of services may have been more effective. The overall design and execution of the TB portfolio focused on important national priorities and was carried out in close consultation (in most cases) with national stakeholders, but it appears that they were not implemented as part of an overarching strategy for USAID support. It also appears that there were insufficient resources for the coordination of the portfolio as a whole, leading to some areas of overlapping responsibility and geographical coverage by projects and some fragmentation of efforts. While the designation of one project (the Linking Initiatives and Networking to Control Tuberculosis program [TB LINC]) in 2009 to coordinate the efforts of other projects was effective, doing so from the beginning of the project would have been more effective. Likewise, the late introduction of TB activities into some projects led to some communication and management challenges. LESSONS LEARNED Many lessons were learned from the evaluation. Some key ones include the need to balance public private mix (PPM) initiatives with initiatives that destigmatize public services and increase awareness about the availability of free/discounted services; the effectiveness (when properly managed/supervised) of innovative approaches to filling human resource gaps; the vital importance of forward planning when introducing new TB services; the need to balance resource-intensive new technologies for TB case detection with low-cost ones; the importance of considering the usability/value of indicators at the local level when planning monitoring and evaluation (M&E) systems for TB projects; the value lost when projects are implemented xiv USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) independently, rather than as part of a strategic whole; and effective approaches to advocating for sustainable increases in funding for health, including TB, at local levels. KEY RECOMMENDATIONS The full evaluation report outlines detailed recommendations in eleven technical/programmatic areas. Four key recommendations, however, are: 1. USAID should continue its support for TB prevention and control in the Philippines at approximately the same level of funding. 2. As a precursor to developing separate plans of activities for future projects, USAID should develop an overarching strategic plan for how to best invest its resources in TB activities in the Philippines. Greater efforts should be made to continue activities that had the highest impact in the period evaluated (as outlined in this report). 3. USAID should strategically focus on ensuring that national policies and tools are directed and expanded at lower levels of the health system. The priority for USAID should be to ensure that its TA and capacity building are closely aligned with national strategies and plans and support existing and emerging national approaches to delivering care. 4. USAID must remain closely in touch with other donors and institutions to ensure its activities are complementary and not duplicative. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 1 I. INTRODUCTION TB is a leading cause of mortality in the Philippines, with approximately 75 deaths a day. There were an estimated 260,000 new cases of TB and 32,000 deaths from TB in 2010, making it the sixth among the 22 global high-TB burden countries. The Philippines also ranks sixth on the list of 27 countries with the highest burden of MDR-TB. For more than a decade, USAID has been supporting efforts by the Philippines’ NTP to prevent and control TB. Between 2006 and 2011 alone, USAID/Philippines invested approximately USD $32 million in a portfolio of eight TB-related projects to strengthen and enhance the work of the NTP. These TB-related projects are now concluding, and USAID is considering its future support for TB prevention and control in the country. To assess the performance, quality and impact of recent investments and to inform future support, USAID/Philippines initiated an independent, external evaluation of its TB portfolio through the GH Tech Bridge project. Global Health funds earmarked for TB activities of USAID/Philippines were used to fund this assessment. This report presents the results of that evaluation. EVALUATION PURPOSE The purpose of the evaluation was to determine whether or not USAID-supported interventions contributed to increased TB case notification and successful treatment outcomes in the Philippines. This was to be done by assessing the recent performance and lessons learned by USAID TB programs and implementing partners, as guided by the USAID Evaluation Policy.1 Specifically, the evaluation assessed whether or not the package of interventions provided by recipients under cooperative agreement (CAs) and support provided by other USAID health programs improved TB case notifications and treatment success to effectively reduce TB prevalence and mortality in the Philippines. The evaluation aimed to assess outcomes in specific areas of TB control, such as MDR-TB, PPM, and advocacy, communication and social mobilization (ACSM), and analyze the common factors or patterns that contribute to success and identify areas for improvement. With the projects’ focus on capacity building of the health providers, support for accreditation of DOTS facilities, information campaign, community involvement, laboratory strengthening and private sector engagement, it is expected that these initiatives translate into improved access to quality services that result in increased treatment success rate for patients. The evaluation was expected to answer the following high-level questions: Have USAID-supported interventions contributed to increased TB case notifications? Have USAID-supported interventions contributed to successful treatment outcomes? Overall, what are the key results and outcomes of the USAID TB program? Did the program accomplish its objectives and achieve its targets? 1 See http://www.usaid.gov/evaluation. 2 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) How have USAID TB projects coordinated activities with the NTP? What are the perceived benefits/shortcomings of USAID programs to the NTP? What program needs are not being met? What USAID-supported interventions really made an impact on the NTP goals? What interventions are working/not working? What policies, financing and other enabling environments have been introduced by the projects? Have any new policies regarding TB control been developed, rolled-out and implemented with the assistance of the TB projects? What is the uptake at national and local levels? How did TB projects improve the capacity of the private sector to work in TB control? How and to what extent have partners’ capacity and engagement been strengthened? Are more private sector providers (practitioners, pharmacies, work places, etc.) notifying TB patients? Referring to public DOTS facilities? To what extent have USAID TB projects contributed to high quality TA at the national, regional and local levels? What policy instruments and programmatic tools have been developed and can be attributed to the USAID TB projects? To what extent have USAID TB projects contributed to capacity building of community based organizations (CBO), NGOs, and civil societies in participating in local TB control initiatives? Have the projects developed a process to ensure sustainability? What are the plans of the Department of Health (DOH) and local governments for sustaining systems and interventions developed under the projects? How have gender considerations been integrated into USAID’s TB programs and activities? What are the differential effects of the project on male and female beneficiaries? What are the overall lessons learned from the implementation and evaluation of TB LINC and other USAID TB and TB-related projects? What are the best practices from the TB LINC and other TB programs that could be adopted and replicated by follow-on activities? The complete scope of work for this evaluation can be found in Annex A. EVALUATION METHOD The evaluation took place from 2 April to 16 June, 2012. It was conducted through the GH Tech Bridge Project by a five-person team of experienced international consultants, a local independent TB control expert and a USAID/Washington health staff member. Based on its terms of reference, the evaluation team developed a detailed data collection tool to ensure the information needed to answer evaluation questions could be answered. With this, the evaluation team took a three-pronged approach to the evaluation. First, by carrying out a desk review of USAID TB project plans and the progress reports of agencies implementing USAID TB activities, compared with published data and information about national TB epidemiology and TB control activities. Second, by interviewing management and technical representatives of implementing agencies and national stakeholders (e.g., from the national TB program, laboratory network and financing and research institutions). Lastly, and of critical importance, the evaluation team members divided into sub-teams to visit and interview a wide USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 3 variety of service providers at all levels of the health system throughout the country, both public and private, including those who were (and those who were not) supported by USAID activities, in order to personally assess the conditions and needs of providers, suspects, patients and the general population, and to validate the information reported by USAID implementing partners and national-level stakeholders. Team members documented the results of these formal interviews, informal discussions and observations into a shared repository structured by review area (e.g., laboratory strengthening). As a team, results were compared within each review area to arrive at consensually agreed upon observations and conclusions for each area. The team developed recommendations for USAID and key observations for the national TB program based on these conclusions and field experience. Preliminary outputs of the review were shared with USAID and the national program for feedback and discussion, and these were then refined into the final evaluation results that appears in this report. There were three phases to the TB portfolio evaluation: Phase 1 (April 2–13): Research and evaluation planning. Phase II (April 14–30): In-country interviews, data collection and team analysis. Team members traveled to several areas of the Philippines (Albay, Baguio, Bulacan, Capiz, Cotabato, Davao, Iloilo, Metropolitan Manila, and Sorsogon) to interview stakeholders at high/low performing sites delivering USAID-supported (and non-USAID supported) TB services. Phase III (May 1–June 16): Report writing, review and finalization. ABOUT THIS REPORT This report begins with background information about the Philippines and the epidemiology of TB in the country before focusing on specific areas of USAID focus from 2006–2011: TB case detection and notification Treatment quality and outcomes Access to diagnosis and treatment Policy and financing for TB control Private-sector engagement Drug-resistant TB Advocacy, communications and social mobilization Laboratory strengthening Anti-TB drug supply Human resources and capacity building Sustainability Project management and M&E 4 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) For each of those areas, the report describes the findings and conclusions of the evaluation team; presents recommendations to USAID; and notes key observations for the NTP of the Philippines. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 5 II. BACKGROUND This section of the report provides contextual information about the Philippines and its people, as well as about its public health (and TB) challenges, systems and financing. ABOUT THE PHILIPPINES The Republic of the Philippines is spread over 7,107 islands in the western Pacific Ocean. Most of the islands are covered in tropical rainforest. Located near the "ring of fire," the country experiences frequent, varied and severe natural disasters. There are up to 20 earthquakes a day, periodic volcanic eruptions, and seasonal typhoons and monsoon rains that bring floods and landslides. The country is administratively divided into 17 regions, covering three big island groups: Luzon, Mindanao and Visayas. It is estimated that half of the population resides on the island of Luzon. Although the regions do not possess independent governments, the Autonomous Region in Muslim Mindanao (ARMM) was created in 1989 with an elected regional assembly and governor. The capital is the city of Manila, with a total population of more than 1.6 million people in 2010. Although it is only the second most populous city in the Philippines, its small geographical area (38.55 square kilometers) makes it the most densely populated city in the world. The Metro Manila, however, comprises 17 cities (including the City of Manila), has a population of 12.3 million and is the biggest urban site in the country. Population The Philippines has a population growth rate of 2%, one of the highest in Asia. According to its National Statistics Office, the population has grown from 88.55 million in 2007 to an estimated 92.34 million in 2010.2 The population’s median age is 22.9 years with 61.1% aged 15 to 64 years old. Life expectancy at birth is 71.94 years (75 years for females and 69 years for males).3 Graph 1: Population Pyramid of the Philippines (2010) 2 2010 Census of Population and Housing, Philippines National Statistics Office. 3 USA CIA World Fact Book, accessed 3 May 2012. 6 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Notes: Data reported through March 2012. Source: Philippines Department of Health, National Epidemiology Center Economy Historically an agricultural-based economy, the Philippines is in the process of transition toward one based on services and manufacturing. The national economy is the 45th largest in the world, with an estimated 2011 gross domestic product of $216 billion. Primary exports include high￾tech manufacturing and products made from raw materials, as well as energy: the country is the world's second-biggest geothermal energy producer. Although the 1997 Asian financial crisis led to a decline in the value of the peso (the currency of the Philippines), comparatively conservative government spending on economic development prior to the crisis shielded it from the severe impact felt by other Asian countries. The economy continues to grow more slowly than many others in the region. Public Health Public health in the Philippines is the responsibility of the DOH. Within the DOH, the National Center for Disease Prevention and Control (NCDPC) provides national guidance and policies, as well as assistance to the country’s regions. At the provincial and city levels, public health guidance and assistance is provided through the Provincial City Health Offices (PHO) and City Health Offices, respectively, and by the support of the Regional Centers for Health Development (CHDs). Public health services are delivered in an integrated fashion at the municipal level through the primary health care (PHC) units, (i.e., Municipal Health Centers [MHC], Rural Health Units [RHUs] and Barangay Health Stations). Over the last two decades, communicable diseases have continued to hound the country with pocket of outbreaks, while noncommunicable diseases, mostly lifestyle-related, are noted to be rising since the 1990s—significantly contributing to the country’s top causes of mortality (details are shown in Annex D). Seven of the Top 10 causes of morbidity are infectious in nature. Malaria is no longer a leading cause of death, but continues to be a major threat in parts of the country, together with other vector-borne diseases like dengue and filariasis. Graph 2: HIV/AIDS Cases Reported, 2000-March 2012 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 7 HIV/AIDS (human immunodeficiency virus/acquired immunodeficiency syndrome), although still less than 0.1% prevalence, has risen rapidly over the last five years—with 313 new cases reported for the month of March 2012 alone (averaging now at 10 per day, up from less than one case per day in 2006 and 82% higher compared to the same period last year), bringing the total number of HIV positives registered since 1984 to 9,163. This number mostly reflects men who have sex with men in recent years (about 80% in 2010, a shift from the previously predominant heterosexual mode). More than half of recently reported cases were found in the age group of 20–29 years; nearly half of whom are from the National Capital Region and 12% from overseas Filipino workers. Other reported modes of transmission in the registry were mother-to-child transmission, blood transfusion, IV drug use (increased incidence particularly in Cebu) and accidental needle pricks. Of the 83% TB patients in Metro Manila screened for HIV in 2011, 0.1% were found to be positive for HIV; while 50% of HIV patients on antiretroviral therapy (ART) are also on TB treatment and 6% on isoniazid (INH) preventative therapy (IPT). TUBERCULOSIS IN THE PHILIPPINES Respiratory TB is the seventh leading cause of morbidity and the sixth leading cause of mortality in the Philippines. There were an estimated 470,000 prevalent TB cases and 260,000 new cases of TB, corresponding to rates of 502 and 275 per 100,000 population. This prevalence and the 32,000 deaths from TB in 2010 make the country sixth among the 22 global high-TB burden countries. The country has data from three national TB surveys (1981–1983, 1997 and 2007). Between the last two surveys there was a trend of gradual decline in terms of incidence, prevalence (annual reduction of 2% of sputum-smear positive [SS+] and 4.7% of culture positive), and annual risk of infection. This is a relatively modest decrease. The TB mortality in 2010 was 33 per 100,000 (31,000 TB deaths per year), an annual reduction of 3.5% per year since 2005. TB incidence estimates have been maintained since 2005 at 160,000 TB cases per year, but the estimated rate has decreased from 301 to 275 per 100,000 due to the rapid growth of the population. The current estimate is probably too high, because it assumes that the average duration of disease is two years. With the current case detection of sources of infection and high cure rates achieved, the duration of active disease is probably much lower. The World Health Organization (WHO) and the Philippines regularly revise these estimates. TABLE 2. Epidemiological Indicators of TB from National Surveys: 1983, 1997 and 2007 Indicator 1983 1997 2007 TB infection (% population) 54.5 63.4 N/A Annual risk of infection (%) 2.5 2.3 2.1 Prevalence of smear positive TB (per 1000) 6.6 3.1 2.0 Prevalence of positive culture (per 1000) 8.6 8.1 4.7 Prevalence of suspect X-ray (per 1000) 4.2 4.2 6.3 MDRTB among new TB cases N/A 1.5 2.1 MDRTB among re-treatment TB cases N/A 14.5 13.0 Sources: 1983, 1997, and 2007 National TB Prevalence Surveys 8 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Among TB risk factors, there is a substantial impact of smoking (relative risk ~2.5, vulnerable to interventions) and diabetes (not vulnerable). The impact of HIV (relative risk >5) is minimal because the prevalence is still very low—about 0.1% in TB cases—although the HIV prevalence trend in the population has increased rapidly over the last five years. In 2010, the country reported 163,248 new and relapse TB cases (65% total CDR) of which 89,198 were new pulmonary SS+ TB (55% of the new cases). The cases notified and those found in the prevalence surveys were relatively older in age, while the population is quite young (35% under 15 years). This suggests that control interventions to reduce TB transmission will have a high impact in the future, because the older population will die and be rapidly replaced by newborns that are uninfected and partially protected by the Bacillus Calmette-Guérin (BCG) vaccine. It also means that little can be done to prevent most infectious cases, because IPT has more risk of death through liver toxicity than the risk of death due to TB for older persons. TABLE 3. Prevalence of Bacteriologically Confirmed TB by Age and Sex: 1997 and 2007 Age Direct microscopy smear positive pulmonary TB, per 1000 Culture positive pulmonary TB, per 1000 Females Males Females Males 1997 2007 1997 2007 1997 2007 1997 2007 10-29 0.6 0.8 2.2 1.2 2.7 2.6 5.3 3.1 30-49 2.6 2.8 8.1 4.0 8.7 4.3 23.1 12.4 >49 3.9 3.0 11.1 9.3 9.2 4.3 25.2 22.6 Total 1.9 1.9 5.4 3.5 5.8 3.5 13.9 9.3 Sources: 2007 National TB Prevalence Survey In the Philippines, there was an estimated prevalence of 8800 cases of MDR-TB in 2010, making the Philippines 6th on the list of 27 countries with the highest burden of MDR-TB. A 2007 survey found 2.1% MDR-TB in new cases (compared with 1.5% in the 1997 prevalence survey), while for previously treated cases these were 13% and 14.5% respectively. A 2004 drug resistance survey (DRS) showed that 4.1% of untreated cases and 21% in previously treated cases in the Philippines is MDR-TB and partial data on processed samples in a second DRS (currently being conducted) is finding 3% MDR-TB positive, suggesting that there is a relatively stable situation, in spite of the still very low coverage of MDR-TB treatment. This can be attributed to consistently high levels of cure of new TB cases. However, the burden of MDR-TB is still very high; the Philippines has one of the highest rates of INH resistance among new TB patients (14%) in the world. Overview of the History of TB Control Formal national efforts to control TB have been ongoing for more than a century, beginning with the creation of a Philippines Islands Anti-Tuberculosis Department in 1910. The TB Commission of the Philippine Health Service in 1932 paved the way for the Tuberculosis Law of 1954 and a nationwide NTP in 1978. Ten years later, a strengthened NTP, incorporating short-course chemotherapy, was adopted. More recent changes in health care delivery in the Philippines have had serious implications on TB control. The Local Government Code of 1991 devolved the USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 9 health service delivery system, transferring the responsibility for managing and delivering health programs to 1,600 local government units (LGU). The ARMM, on the other hand, has its own Secretary of Health and maintains a sub-centralized approach in relation to the national government. In this devolved health system, the DOH remains responsible for policy development, regulation and provision of technical and financial assistance, while LGUs (such as municipal governments) are responsible for managing and delivering TB services. The Health Sector Reform Agenda of 1999 of the Philippines attempted to strengthen this devolved health system by assigning supervisory and regulatory powers to the DOH, particularly in providing health standards, policies and technical guidelines, and assistance through planning, evaluation and inspection to sub-national public health care providers. The Philippines adopted the DOTS Strategy in 1996 and achieved 100% DOTS population coverage by the public sector in 2002. In 2003, the DOH and the Philippine Coalition Against Tuberculosis (PhilCAT) developed the Comprehensive and Unified Policy (CUP) for TB Control in the Philippines in order to harmonize TB control and care in the country for both the wider public (beyond the DOH) and the private sectors. The main thrust of CUP is the adoption of the DOTS strategy by all those providing TB care. Structure of the Current TB Control Program As described above, the DOH maintains overall responsibility for public health in the Philippines. The NTP, under the Infectious Disease Office within the National Center for Disease Prevention and Control (NCDPC), is responsible for policy development, national planning, training and provision of TA at the regional level. In addition, it estimates, forecasts and procures FLD for public and private DOTS facilities. The NTP monitors and supervises the overall performance of the TB control program in the country, collecting data and information for each region (by province). The allocation of funding, human resources, drugs and other commodities (except FLD) is the responsibility of each municipality, and neither PHOs nor regional offices have any authority on these. CHDs oversee the implementation of TB control activities in the provinces. They monitor and supervise these activities, providing training and TA at the provincial level. The CHDs also collect and compile annual reports on the performance of the provinces for submission to the NTP. There are provincial and regional hospitals/medical centers that treat TB, however, most TB services are provided by City Health Centers (CHCs), MHCs and RHUs as part of a package of PHC services (e.g., immunization and maternal and child health [MCH]). Numerous Barangay health workers (BHWs), usually volunteers, further increase the scope of services and geographical area covered. Each PHO monitors and reports on the TB control activities of those CHCs, MHCs and RHUs. The PHOs typically also maintain a quality assurance center for the external quality assurance (EQA) of direct sputum smear microscopy (DSSM) facilities in the province. Overall, the structure of the TB control delivery system in the Philippines is well devolved and decentralized. The devolution has led to some significant gaps and challenges in responding to the health needs of the population across the country, such as the reduction of the number of TB control staff at the central level. 10 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Graph 3: Organization and Structure of the Philippine TB Program The priority for TB control (and the staffing and funding for local TB services) varies significantly from LGU to LGU, and the election of a new Local Chief Executive (LCE) can lead to a decrease in resources for a previously high performing program. Furthermore, the varying engagement of public and private facilities to deliver DOTS services means that a significant number of TB patients are diagnosed and treated but not reported. As a result, the performance of the program shows significant variations among provinces. Private Sector Involvement in TB Control The Philippines has a vast network of private physicians, pharmacies, clinics and hospitals. In 2003, the Philippine NTP, recognizing that the 70% case detection was not achieved, formally adopted the PPM DOTS (PPMD) as a national strategy to increase TB case detection and further improve treatment success, in particular among private providers. Given the health-seeking behavior of Filipino TB symptomatics, who prefer to consult private health care providers, it is logical to assume that controlling TB in the country would need the support of this sector. This became the guiding spirit behind the founding of the PhilCAT in 1994—a body that would serve to coordinate and unify TB control initiatives among various USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 11 government and non-government agencies; professional medical societies; private groups, including from academic institutions. With a grant from the Global Fund in 2003, PhilCAT (as sub-recipient) began to roll out the establishment of PPMD clinics across the country. The Philippine Tuberculosis Initiative for Private Sector (PhilTIPS), a USAID-funded project, also supported the replication and expansion of PPMD clinics across 22 key cities of the country through a grant mechanism. The Philippine Business for Social Progress (PBSP), a sub-implementer of PhilTIPS, developed three models for DOTS in the workplace: a full-service delivery model and workplace models that referred to public or private DOTS facilities; and by 2006, reported 23 companies linked to the NTP through such workplace models. In many areas, this private sector has supplemented public initiatives to fill gaps in public services, particularly in urban centers where private medical institutions thrive. In far-flung areas, private providers are scarcer and health needs are mainly provided by the public sector. The PPMD units are supported by provincial and city health offices, to which the PPMD units report their results (performance) and from whom they receive free FLDs. PhilHealth (the Philippine Health Insurance Corporation) manages the national health insurance program of the Philippines. With PhilHealth’s TB/DOTS outpatient benefit package as a possible funding source for sustainability, both Global Fund/PhilCAT and PhilTIPS PPMDs established networks of DOTS referring private physicians through certification trainings. Though over 4,000 DOTS referring private physicians were trained, a significant number chose not to pay the PhP 500 certification fee that would include them in PhilCAT/PhilHealth’s official roster of physician beneficiaries for qualified cases claimed by accredited centers, as they were yet to be convinced of the benefits of such status. Still, the majority of these certified and trained DOTS physicians (67% of 704) were identified by PhilTIPS to have actually referred patients and accounted for nearly half of the total number of patients seen by the USAID-supported clinics at that time. TB Laboratory Network The Philippines TB laboratory network is based on the DSSM units that are functioning at the RHU/MHC level. Currently, approximately 2,000 of the 2,500 RHU/MHC in the country have a microscopy center. Almost all of the microscopy centers participate in the revised EQA system (based on sampling and blind re-checking of results), supported by provincial and city quality assurance centers, that was established in 2007. There are TB culture centers in seven regions, and the NTRL and Cebu Reference Laboratory are proficient in drug susceptibility testing (DST). In the past year, 15 GeneXpert machines were deployed in Manila and most regions, but coverage remains limited. Financing for TB Control In the past decade, financing for health in the Philippines (3.3% of its Gross Domestic Product [DDP] in 2005) is below internationally recommended levels (5%) and far below the investments of industrial countries (between 8% and 12%). The Philippines has one of the lowest government spending for health among its Asian neighbors and among the highest for private and out-of￾pocket spending, a trend that continues to increase. It appears that the impact of government investments in health have been hampered by the devolution of health services, which, according to the DOH, “brought planning and management 12 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) of public health services closer to the people, [but] has also unintentionally splintered health services and financing. Thus, the country’s health financing system is extremely fragmented, leading to major coordination problems, aggravated by inadequate regulation.”4 According to WHO, the total budget for the NTP of the Philippines was USD $81 million in 2011 and USD $79 million in 2012, but available funding has been significantly lower than that (USD $55 million in 2011 and USD $48 million in 2012). Slightly more than half of the available resources have been from domestic resources, with most of the remainder (42% in 2011 and 49% in 2012) provided through grants by The Global Fund. The available funding is equivalent to USD $200 to 300 per patient, sufficient for the needs of a basic integrated TB program but not enough to deal with the more expensive treatment of MDR-TB. The Government of the Philippines is increasingly relying on socialized health insurance to support the costs of TB diagnosis and treatment. The National Health Insurance Act of 1995 or Republic Act 7875 led to the creation of PhilHealth. PhilHealth is a tax-exempt government￾owned and controlled corporation, attached to the DOH, providing social health insurance coverage for Filipinos. It is mandated to provide universal coverage to all Filipinos in 15 years’ time from its creation. As a mechanism for addressing the needs of the poorest populations, however, PhilHealth has had mixed progress. According to the DOH, “PhilHealth enrollment of the poorest households has not been sustained during the period of 2005–2010,”5 and a new national strategy for health care financing (for 2010–2020) has been recently instituted to facilitate improved financing for services. USAID SUPPORT FOR TB CONTROL Since 2000, USAID has intensified its support for TB prevention and control in the Philippines, such as by assisting NTP efforts to increase and improve public services for TB and build TB program capacity, with a focus on strengthening the capacity of the private sector to implement DOTS. Early support from USAID/Washington (2000–2003), implemented by PhilCAT and the U.S. Centers for Disease Control (CDC), focused on the development of five models for DOTS in various private health sector settings through pilot projects. The models enrolled TB patients and provided DOTS services to those referred by private medical practitioners and walk-in patients from the catchment areas of the respective models. The three-year project was evaluated in 2004 to determine the contribution of the pilot projects to TB control in select parts of the country, and concluded that significant steps had been taken to establish PPMD projects. The models identified potential sites for PPMD expansion. 4 Toward Financial Risk Protection: Health Care Financing Strategy of the Philippines 2010-2020, Health Sector Reform Agenda Monograph No. 10, 2010, Health Policy Development and Planning Bureau, Republic of the Philippines. 5 ibid USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 13 Continuing its support for PPMD, USAID/Philippines implemented PhilTIPS with Chemonics International, Inc., from 2002–2006. PhilTIPS aimed to enhance coordination with the NTP; create an enabling environment for the private sector to fully engage in TB control; strengthen TB diagnosis and treatment by private providers in over 25 sites throughout the country; build the capacity of private providers to deliver quality DOTS services; and motivate behavior change (suspects and providers). PhilTIPS provided TA to two DOTS clinics that had been supported under the CDC/PhilCAT project, and developed a TB DOTS curriculum in partnership with the Association of Philippines Medical Colleges in 10 of the country’s top medical colleges. In parallel with PhilTIPS, USAID/Philippines initiated a project to build the capacity of LGUs for health. The Local Enhancement for Health and Development (LEAD) project was awarded to Management Sciences for Health (MSH) in October 2003 and operated until September 2006. Although TB was a component of the project, it main objectives were family planning (FP) and contraceptive self-reliance. Through LEAD’s activities, the number of LEAD LGUs meeting the 70% TB case detection target increased from 135 to 194 from 2003–2005. The project provided the majority of its TA for TB during its last year, so it is difficult to measure impact since the project ended. However, the project did achieve success in gaining more support for health, including TB, at the LGU level. USAID Washington, in coordination with the USAID Philippines Mission, also provided approximately $1.5 million to Catholic Relief Services from 2005–2009 for the Maguindinao TB Control Program (MTCP), located in the ARMM. This was funded through the Child Survival Health Grants Program, which provides funding for small-scale community-based TB projects. CRS worked closely with the Integrated Provincial Health Office (IPHO) in Maguindinao to institutionalize the DOTS strategy, and engaged BHWs through a “Microscopists on Wheels” program and TB clubs, which are peer support groups of volunteers who ensure treatment compliance and work to reduce stigma in their communities. The project was very successful in engaging community volunteers in TB control and in working in close coordination with the provincial health office. USAID TB Projects (2006–2011) Over the past decade, USAID has supported the Philippine NTP at the national and sub-national levels through several implementing partners. Focused on capacity building through service delivery and targeted TA, USAID invested approximately $32 million in eight TB-related projects in the Philippines from 2006–2011. The Linking Initiatives and Networking to Control Tuberculosis Project (TB LINC), a six-year cooperative agreement (2006–2012) initially focused on three components in supporting the NTP: enhancing the health policy, financing and regulatory environment for DOTS; building systems capacity to strengthen quality DOTS; and improving service utilization through behavior change. In 2009, a fourth component was added: strengthening private sector participation in TB control. There are 32 TB LINC sites nationwide, including five new project sites and several municipalities in the ARMM in its 14 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) sixth year. PBSP, a local non-governmental organization was the main implementer. USAID invested approximately $20 million in TB LINC-supported activities over the first five years. The Strengthening Local Governance for Health (HealthGov) Project complements TB control initiatives in areas outside the TB LINC sites (25 provinces by increasing the capacity of LGUs—over 550 in 25 provinces—to plan, provide, manage and finance quality health services sustainably. Implemented by the Research Triangle Institute, USAID had invested approximately USD $3.0 million of the TB fund for three years. The Health Policy Development Program (HPDP) CA supports the DOH and TB LINC in developing national policies and guidelines that will strengthen the implementation of the TB control initiatives in the country. Implemented by the University of the Philippines Economics Foundation, Inc., about $323,617 has been invested over the past five years. The Health Promotion and Communication (HealthPRO) project supports health￾related behavior change communication (BCC) activities in the country by providing TA to DOH and LGUs, particularly in developing information, communication and education materials for TB care providers, patients and policymakers. Approximately USD $2.4 million of the TB fund has been invested over the past five years and is being implemented by the University Research Co. The Sustainable Health Improvement through Empowerment and Local Development (SHIELD) is a six-year project focused in providing assistance in the ARMM. Helen Keller International Inc. received approximately USD $2.4 million of TB funds for SHIELD’s integrated technical support for the TB control program in these areas. The Strengthening Pharmaceutical Systems (SPS) CA promotes TB pharmaceutical and laboratory best practices, including pharmaceutical management training of DOH pharmacists, support to forecasting and quantification of MDR-TB medicines, development of standardized pharmaceutical management policies, TB laboratory systems management, and management information systems (including e-TB Manager). With MSH as implementer, SPS works with the DOH, the Lung Center of the Philippines and other partners in NTP’s Programmatic Management of Drug-resistant TB (PMDT) and about USD $1.54 million has been invested over the past four years. The United States Pharmacopeia Promoting the Quality of Medicine (USP PQM) project monitors the quality of anti-TB drugs in the country in close collaboration with Food and Drug Administration (FDA) and the DOH/NCDPC. Working in six sentinel sites in the country’s three major island groups, approximately USD $530,000 has been invested to strengthen the national TB drug quality assurance system since its inception in 2009. Since 2009, the WHO Country Office for the Philippines (WHO/Philippines) has received an annual grant of approximately USD $442,500 from USAID to support a TB Medical Officer who provides TA to the NTP. This includes assistance to help strengthen strategic planning, monitoring and implementation of the NTP. Four of those projects (SPS, TB LINC, USP PQM and WHO) were exclusively focused on TB activities. The four other projects, while not limited to TB, included TB-specific activities. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 15 Areas of USAID Focus USAID Philippines contributes to the U.S. Government’s Global Health Initiative (GHI) targets, which include contributing to the treatment of a minimum of 2.6 million new sputum SS+ TB cases and 57,200 cases of MDR-TB. While earlier support mainly focused on engaging the private sector and building the capacity of LGUs, USAID’s support for TB in the Philippines from 2006–2011 focused on two overarching objectives: 1)Strengthening the national TB program and 2) TB program capacity building through service delivery and targeted TA at the regional and local levels. With an understanding that changes in policy are needed to strengthen political support and financing for TB, USAID placed significant emphasis on these areas in a few of its projects, and also focused on improving the regulatory environment for drugs through a close collaboration with the Philippine FDA. In response to the fact that many people in the Philippines seek care for TB in the private sector and following on its work in the period before 2006, USAID has continued its work in PPMD. Other focus areas, from 2006–2011, included PMDT; laboratories, including strengthening the laboratory network; anti-TB drug management, including training on forecasting and drug quantification; capacity building at all levels of the health system; systems strengthening, including improvements in service delivery; ACSM; and targeted TA for vulnerable populations, with a specific focus in ARMM. USAID Methods/Approaches All USAID activities are implemented through a variety of acquisition (contracts) and assistance (grants and CAs) mechanisms. The USAID TB program in the Philippines works through a mix of mechanisms, both for those based in Washington and those awarded by USAID/Philippines itself to different partners in the country. All of the agreements in the Philippines’s TB portfolio were awarded through competitive processes, and require regular reporting as outlined in the individual agreements. Reporting is usually done on a quarterly basis, and there has been a shift in policy that promotes a strong emphasis on impact evaluation to ensure that the best investments are made. USAID Philippines has placed a strong emphasis on country ownership and building the capacity of local organizations, which is in line with USAID Forward’s Procurement Reform. To promote country ownership, the Mission has worked closely with the National TB Program and other sections of the Government of the Philippines to build the country’s capacity in planning, managing, and financing its TB program. To build local capacity, the Mission awards a certain percentage of its funding to local organizations and works closely with various different local partners. The main partner for the flagship TB project, TB LINC, was a local partner that has significantly increased its capacity over the past five years. In addition, USAID Philippines has taken the Agency’s new policy on M&E very seriously, and has started the process of measuring the impact of its programs through this evaluation of the TB program. Lessons learned from this report will inform future USAID TB projects so that they build on the successes of past projects and make improvements going forward. 16 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 17 III. EVALUATION FINDINGS The evaluation findings are presented per outcome of the TB interventions (Case Detection and Notification, Treatment Quality and Outcomes, Access to Diagnosis and Treatment, Policy and Financing, Private Sector Engagement, Drug-resistant TB), per component (ACSM, Laboratory Strengthening, Anti-TB Drug Supply, Human Resources and Capacity Building) and other vital aspects (Sustainability, Project Management). At the end of each section are relevant conclusions, observations for the NTP and recommendations. CASE DETECTION AND NOTIFICATION The main strategy to reduce TB mortality, morbidity and transmission is the identification and treatment of persons with active TB, particularly those with pulmonary TB that are most infectious and present cough and sputum positive to direct microscopy. The Philippines achieved the global target of 70% CDR and since then has sustained and increased that performance. Case Detection The evaluation team found that identification of TB cases in the Philippines is mainly done through the detection of persons who visit health facilities and are coughing and/or have additional TB symptoms, including fever, night sweats and weight loss. The request for smear examination for diagnosis (three samples) is done mainly by the physician consulted (or a nurse in the absence of a physician). Thus, persons with TB symptoms who visit the facilities for other reasons are not always examined by sputum microscopy and may pass through the facility without diagnosis and treatment. There is very little or no information provided to general outpatients in waiting rooms (posters on cough) and with few exceptions, no routine system for detecting cough in general outpatients who consult public or private facilities for symptoms other than respiratory disease. The evaluation team found a very high positive rate in TB microscopy, reflecting high prevalence in the community and the selection of those persons that report symptoms spontaneously. In the community, voluntary health workers (often BHWs) ask about cough of over two weeks and refer suspects or collect sputum and fix slides to take or send to the laboratory for staining and reading. In USAID-supported areas (e.g., Albay), a small increase of persons examined by microscopy between 2009 and 2011 was seen in visits to facilities, with decreasing rate of About Case Detection Pulmonary TB positive only by culture or culture negative is five times less infectious and (if the patient is not infected with HIV) presents half the risk of dying from TB even if untreated. Extra￾pulmonary TB is not infectious so it has a low priority in public health. All diagnosed TB cases should be treated to prevent suffering and death, but the most infectious are a public health priority and require case detection activities. Case detection starts in the community with recognition of respiratory symptoms and the decision to consult a health provider; it continues through the identification of suspects (persons with cough or abnormal X-rays) and sputum examination by direct microscopy; and it ends with the enrollment in treatment, registration and reporting. Diagnosis of SS- or extra-pulmonary TB requires additional tools such as X-rays and culture or new rapid diagnostics, pathology and biochemical tests. Children are usually SS-; and HIV infected persons are more often SS- even with large bacillary loads and high risk of death— so diagnosis without these tools is difficult. Active case detection is recommended for specific groups such as contacts of TB patients, HIV infected persons and populations with higher prevalence such as prison inmates. 18 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) positivity (14% to 11%). The same was seen in ARMM (e.g., Awang), with identification of one person with symptoms every two days in 2010, and one per day in 2011 and 2012. However, the proportion of positive suspects among the examined is still very high (10% to 30% in the very small sample observed). The numbers in sites visited are too small to draw conclusions on trends for USAID-supported or non-supported areas. In addition, USAID activities in support of the service level started in the last three years, do not have high coverage of health facilities and it is too soon to measure impact (national data for 2011 is not yet available). TABLE 4. Examples of TB Case Detection by Microscopy in Sites Visited Site Visited Indicator 2006 2007 2008 2009 2010 2011 Pulilan, Bulican USAID supported area Pop: ~100,000 TB symptomatics examined 259 229 350 134 200 217 SS+ TB cases detected 58 42 89 35 53 43 Positivity (%) 22% 18% 25% 26% 26% 20% San Rafael, Bulacan USAID supported area Pop: ~30,000 TB symptomatics examined 174 162 177 143 167 186 SS+ TB cases detected 34 14 30 15 24 21 Positivity (%) 19% 9% 17% 11% 14% 11% Roxas City, Capiz USAID supported area Pop: ~160,000 TB symptomatics examined N/A N/A 524 553 559 N/A SS+ TB cases detected N/A N/A 136 152 119 N/A Positivity (%) N/A N/A 26% 27% 21% N/A Pototan, Iloilo USAID supported area Pop: ~70,000 TB symptomatics examined N/A N/A 175 270 253 328 SS+ TB cases detected N/A N/A 134 125 90 105 Positivity (%) N/A N/A 77% 46% 36% 32% Oton, Iloilo Not supported by USAID Pop: ~77,000 TB symptomatics examined N/A N/A 340 554 656 931 SS+ TB cases detected N/A N/A 77 55 79 140 Positivity (%) N/A N/A 15% 10% 12% 15% La Trinidad, Benguet Not supported by USAID Pop: ~105,000 TB symptomatics examined N/A N/A N/A 274 347 353 SS+ TB cases detected N/A N/A N/A 14 11 13 Positivity (%) N/A N/A N/A 5% 3% 4% Banaybanay, Davao Oriental Not supported by USAID Pop: ~35,000 TB symptomatics examined N/A N/A N/A N/A N/A N/A SS+ TB cases detected 58 30 54 58 64 56 Positivity (%) 14.5% 9.5% 9.5% 12% 11% 11% During field visits, however, it was noted that the positivity is very high in most sites (the expected proportion would be 10% to 15%); that the highest positivity corresponds to a site with a very high default rate; and that the recording and reporting is generally good and better USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 19 in USAID-supported sites. One health facility with excellent organization and conditions, in a non-supported municipality, indicated that they had implemented the observations made in a neighbor municipality that had received input from TB LINC. At provincial level or higher, the number examined can be compared with the population. In 2009, the country reported 0.34% of the population examined by microscopy with a positivity of 19%, with 97% of the detected cases enrolled on treatment. National data by province (excluding cities) provided by the NTP showed an increase of suspects examined from 390,000 to 411,000 between 2008 and 2010; however this only accompanied the increase of population from 70.5 to 73.4 million in the period. The proportion of persons examined was constant at 0.5% and the positivity remained high at 16% in 2008 and 17% in 2010. Data from some provinces visited showed that about 1% of the population was examined by microscopy and the positivity is quite high. All provinces visited show some decrease in smear positivity and consistent proportion of cases enrolled, although it is evident that more should be done to follow the infectious cases detected. TABLE 5. Examples of TB Case Detection in Provinces Visited Site Visited Indicator 2006 2007 2008 2009 2010 2011 Bulacan Province USAID supported area Pop: ~1,360,000 TB symptomatics examined 14576 13439 13599 11522 14290 7831* Population examined (%) 1.1% 1.0% 1.0% 0.8% 1.1% 0.6% SS+ TB cases detected 2540 2419 2421 2162 2537 1264* Positivity (%) 17% 18% 18% 19% 18% 16% SS+ enrolled on treatment 1359 2323 2120 2171 2282 2450 New/re-treatment enrolled (%) 56% 96% 88% 100% 90% N/A Albay Province USAID supported area Pop: ~820,000 TB symptomatics examined N/A N/A 9098 9267 9986 10840 Population examined (%) N/A N/A 1.1% 1.1% 1.2% 1.3% SS+ TB cases detected N/A N/A 1805 1617 1552 1627 Positivity (%) N/A N/A 20% 17% 16% 15.0% SS+ enrolled on treatment N/A N/A 1565 1533 1494 1507 New/re-treatment enrolled (%) N/A N/A 87% 95% 96% 93% Sorsogon Province Not USAID supported Pop: ~590,000 TB symptomatics examined N/A N/A 5658 5961 6077 6713 Population examined (%) N/A N/A 1.0% 1.0% 1.0% 1.1% SS+ TB cases detected N/A N/A 922 935 925 929 Positivity (%) N/A N/A 16% 16% 15% 14% SS+ enrolled on treatment N/A N/A 870 931 958 892 New/re-treatment enrolled (%) N/A N/A 94% 99% 104% 96% *Partial information The information reflects only the reporting of the facilities where patients attend and have also implemented DOTS. According to the 2007 National TB Prevalence Survey, a quarter of the persons with TB symptoms take no action and 43% self-medicate (with a risk of partial treatment and developing resistance to TB drugs). Of the persons that take action (35%), one 20 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) quarter attend DOTS centers, one quarter attend clinics or public hospitals, and nearly 40% visit private physicians or private hospitals. The coverage of public hospitals with organized TB control (the NTP strategy) is not complete, particularly with reference to outpatients. Only a very small proportion of private hospitals and practitioners are involved in the DOTS strategy. In addition, private hospitals charge for smears (up to USD $10 per smear; USD $30 for the three diagnostic smears) and for X-rays (up to USD $13 per film). In some cases, but not all, private hospitals offer a discount for suspects passing through their TB DOTS program. After diagnosis of TB, microscopy and anti-TB drugs are free of charge in private facilities provided the patient is a qualified PhilHealth beneficiary or dependent of a beneficiary. Graph 4: Case Detection Activities in USAID-supported and Non-supported Provinces/Cities, 2008-2010 Notes: Differences between the performances of USAID- and non-USAID supported sites may exist for a variety of reasons, including site selection. Source of data: NTP, Philippines Case Notifications National case notification increased rapidly between 2005 (137,100 cases; CNR of 160 per 100,000) and 2010 (166,300 cases, CNR of 178 per 100,000), reaching a CDR of 65%. The proportion of SS+ in new pulmonary cases was 55% in 2010; there were 1,610 extra-pulmonary cases (10% of the new and relapses reported). These numbers do not reflect what was observed in rural facilities and DOTS centers where diagnosis is mainly restricted to pulmonary SS+ cases and the availability of other diagnostic tools is limited. The capacity for diagnosing non-infectious TB is higher in cities. The detection and treatment of infectious cases has higher impact and is more cost-effective. A good case detection and treatment intervention should result in an initial increase of persons examined and TB cases found. As a result, the prevalence of TB in the community will diminish and increased activities to find suspects will result in fewer TB cases found (reduction of microscopy positivity and reported incidence). The USAID-supported project showed an increase in TB cases reported as the geographical area expanded. The CDR is based on national USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 21 estimates but applied to the (partially) covered populations, which might not have the same TB epidemiology as the national average. Therefore, at local (provincial and municipal) level it becomes inaccurate and cannot be used effectively to monitor progress. At the service level, it is inappropriate to target a constant increase in TB cases and to use the CDR as measure of impact. A comparison of USAID-supported and non-supported provinces in Region 11 (Davao del Sur and Compostela versus Davao del Norte, Davao Oriental and Davao City) shows that SS+ TB incidence initially increased more rapidly in supported areas and then decreased earlier, suggesting impact on real prevalence. This trend of increase of cases reported with initial expansion and then decrease due to epidemiological impact should be expected in all provinces. The more appropriate indicators at local level to measure success are the increase of suspects examined and the reduction of positivity rate. Graph 5: Reported Incidence of SS+ TB in USAID-Supported* and Non-Supported* Provinces, Region 11, 2006–2011 *Davao del Sur and Compostela; ** Davao del Norte, Davao Oriental and Davao City. USAID projects in the Philippines established a baseline of 68% CDR of new SS+ cases and a target of 74% for all areas; the achievement was 73%—below expectations but over the international and national targets. Eight of the 17 regions did not achieve the CDR target while performance of provinces and cities showed wide variations. This continues to be observed across all sites whether USAID-supported or not. Diagnosis and reporting of sputum smear-negative (SS-) TB in the Philippines must in general be reconfirmed by a provincial TB diagnostic committee (that does not see the patient) with a delay of two weeks or more. Doctors are authorized to start treatment if the delay is longer. The committee can be a useful method to increase the quality of diagnosis and reduce over￾diagnosis, but may be a barrier to involve physicians in private hospitals. Large variations in the proportion of SS- new cases were observed in one province (from 30% of total pulmonary TB to 60% in three years), without a clear reason. Proportions of SS- cases vary for different reasons. For example, facilities may approve treatment for symptomatic cases from far-flung areas as it would be difficult to comply with a second set of sputum smears for doubtful cases (i.e., single SS+ out of three smears) or repeat smears for those done in non-NTP-trained/linked 22 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) laboratories. The proportion of cases diagnosed without laboratory confirmation is a good indicator of quality of diagnosis: at least 50% of all cases and two thirds of pulmonary cases should be confirmed by smear or culture if appropriate criteria are followed. An important contribution of TB LINC was assistance to PhilPACT in the development of guidelines for pediatric TB and to the DOH on childhood diagnosis and treatment. However, there is very limited capacity to diagnose TB in children and extra-pulmonary TB; as a result, few cases of extra-pulmonary TB and TB in children are notified. Training to manage TB in children has been initiated recently, and this activity varies drastically: in some cases, facilities are “overwhelmed” with pediatric TB—not always confirmed as disease—and others have not yet begun diagnosing and treating children. Because of insufficient access to X-rays due to limited availability and cost; and frequent absence of purified protein derivative (PPD), children may be under-diagnosed in some areas and over-diagnosed in others. Conclusions TB notifications are increasing slowly, driven by small increases in detection of suspects and involvement of private providers and improved quality of reporting. The gradual reduction of real prevalence and incidence will result in stabilization of the number of cases diagnosed. The rapid growth of the population resulted in decrease of the notification rates and increase in the CDR. Case detection is still insufficient: cases are diagnosed late, the proportion of SS+ among suspects examined is very high, DOTS coverage does not include all public facilities (mainly outpatient departments of city hospital) and the involvement of private providers to diagnose and report TB cases is still minimal. USAID projects strongly supported the basic elements for case detection and diagnosis (guidelines, health staff capacity building, coverage/quality of detection of suspects and microscopy, community involvement to detect suspects and involvement of private providers). Case notification increased more in USAID-supported areas than in non-supported areas, but the time from implementation at service level is too short to detect significant differences. Service level support expanded in the last two years and has still partial coverage of facilities, and data after 2010 is not available at national level. In some cases the improved selected municipalities’ experience expanded to neighbor areas, reducing differences between supported and non-supported. The indicator used by the DOH and USAID TB projects to evaluate case detection performance (CDR or proportion of the expected CDR) should only be used at national level; it is inappropriate for municipal and provincial levels because there are large regional variations in real incidence and prevalence. Increased notification can be used as target only during expansion. Areas where the program has impact will reduce TB prevalence and the TB cases reported. All LGUs and provinces have data on the number of persons examined by microscopy and the positivity rate. This information is not widely used as the main criteria to evaluate case detection, and was rarely seen in graphs on the walls of the DOTS facilities or microscopy units. The number of persons examined is slowly increasing and the positivity is decreasing USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 23 in supported areas, but slowly. These should be the main indicators for service level in the future. Observations for the NTP Give priority to the expansion of DOTS in public hospitals and the transition from the 3- smear diagnostic algorithm to a 2-smear one. Expand the use of the persons examined for diagnosis by microscopy and the positivity rate as indicators of case detection at municipal and provincial levels. Use CNR, especially at the provincial level (and below) to monitor performance in case detection. Recommendations to USAID Prioritize efforts to increase/accelerate identification of suspects to be examined by smear microscopy, by providing simple messages to the population and developing non-medical systems to detect cough among outpatients and request sputum smears. Focus on the outpatient departments of public and private hospitals, and expand involvement of private practitioners in the referral of suspects for microscopy. At municipal and provincial levels, use the number of persons examined for diagnosis by microscopy and the positivity rate as indicators; and train staff to self-evaluate and visibly display the line trends in the facility. TREATMENT QUALITY AND OUTCOMES Treatment must be implemented for all TB cases diagnosed, infectious or not. Treatment must follow currently recommended regimens—with drugs of good quality and ensured regular intake for the complete duration. Because TB control is of public health interest, drugs should be provided free of charge and treatment should be easily accessible to patients (patient-centered approach, International Standards for Tuberculosis Care [ISTC], 2009). Treatment outcomes should be monitored with microscopy examination in SS+ patients. Political commitment to DOTS (funding, management and technical guidelines) is strong and appears to be increasing in the country. The Philippines follows international recommendations on treatment regimens; drugs are free of charge to the patient in public facilities; and community/family DOT with treatment partner support is the method of choice. Treatment outcomes in patient cohorts are reported regularly, with a target of 85% cure in new SS+ patients. Treatment in public facilities is according to national regimens, using fixed-dose combination (FDCs) and blister packs. However, strict adherence to DOT varies. Treatment is provided by the partner or treatment supporter in the community. Direct observation of therapy is rarely done in the facility, in all sites visited. In theory, DOT is observed by a health provider, treatment partner or family member. Patients are typically (but not always) provided with drugs for a week in the initial month and for two weeks afterwards. In some cases, in both USAID and non-USAID sites, patients are provided with drugs for over a month, sufficient until their next sputum smear examination. In a few facilities, the dose is administered under direct observation in the day that the patient/partner attends for collection. In practice there is limited supervision to see if the 24 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) drug intake is observed. In both USAID and non-USAID facilities visited, rarely have patients attend regularly for drug intake. The quality of recording and reporting is generally high in both public and private sector DOTS facilities. The registers observed in USAID-supported areas were usually filled out properly. In non-supported areas this was less so, and in some units there were major misclassifications. There appears to be good understanding of the methodology and the priorities among the staff. Treatment outcomes reported are excellent, with success exceeding WHO targets at 94% in 2010 (note that the country uses 85% cure as a target, while WHO normally refers to 85% success rate). In the national reports to WHO since 2007, cohorts of new SS+ cases have had consistently around 1% failure, 2% deaths and 4% default; cures have increased from 79% to 82% and the success rate has been maintained at 89%. On average, USAID-supported sites improved cure slightly from 83% in 2006 to 84% in 2010, due to reduction of the proportion of treatment completed without laboratory confirmation and of transferred out; while the success rate reached 92%, somewhat higher than the national rate. There are large variations in cure rates among provinces and this depends on the accessibility of microscopy services. At this very high level of treatment success, it is not feasible to achieve major increases. Nationally, re-treatment with Category II has much poorer results, with analysis of less than half the notified cases in 2008 and 2009, cure rates of 56% and 48% and success rates of 71% and 61%, respectively, and large proportion of cases not evaluated (14% and 26%). Outcomes of the few cohorts of patient treated with second line regimens (Category IV) with results already available are as expected rather poor. The number of patients diagnosed as MDR-TB was small compared with the estimated prevalence; few of them were enrolled on treatment with Category IV regimens from 2007 to 2009. TABLE 6. Detection and Treatment of MDR-TB Site Visited 2007 2008 2009 2010 2011 No of MDR-TB cases diagnosed 591 738 980 619 503** No enrolled in Category IV* 315 530 569 869 N/A Interim outcome (%) 80% 78% 73% 71% N/A Treatment success rate (%) 64 64 55 N/A N/A Death rate (%) 11 10 11 N/A N/A Default rate (%) 21 25 34 N/A N/A * Enrolled includes MDR-TB, SR-A, SR-B, XDR-TB, DR-TB, and empiric treatments. ** Cases are pending DST from the NRL. Source: Lung Center of the Philippines, 2012. TB patients have limited access to treatment outside of DOTS centers; treatment by private providers and even in some public hospitals is not free of charge; while drugs for those who intend to self-medicate are available and sold over the counter with or without prescription. In general physicians prescribe according to national guidelines, but patients may or may not be able to afford all drugs for the required time and there is no system to ensure intake. The gradual expansion of coverage of PhilHealth insurance will facilitate access to treatment in the private sector, but will not ensure that the drugs are taken regularly. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 25 It was evident in interviews with community members in multiple sites visited that the population in general is still unaware that treatment for TB can be obtained from public facilities free of charge. Conclusions Treatment outcomes for new SS+ cases are very good, both nationally and in the USAID￾supported areas. The outcomes refer only to the cases reported to the NTP; the enrollment on treatment and outcomes of TB patients not reported by public hospitals and private providers is uncertain. Field observation suggests that treatment outcomes are marginally better in USAID￾supported sites, and the reported average was slightly better than the national data. However there are large variations among provinces and it is too soon to observe results of recent activities as the outcome data is available only for 2009. At the current level of cure/success in treatment of new SS+ cases it is not feasible to use treatment outcome to measure the impact of additional interventions, or to compare areas with and without additional support. The indicator can however be used if applied to re￾treatments, that have unsatisfactory results and for whom the information is of less quality (partial cohorts, high proportion of outcomes not evaluated). For effective reduction of prevalence and mortality, the most important priority is maintaining the levels of cure/success for many years and to expand treatment to patients not yet diagnosed or diagnosed but not accessing the DOTS facilities. The treatment outcomes need validation, particularly as they are the result of a national strategy (community and family DOT), there is little information regarding the actual drug intake by patients and the quality of drugs utilized (not accredited by WHO and may be maintained in inadequate storage, see below). Validation should include operational studies on real intake (though interviews with patients with completed treatment), and treatment practices at facilities and among CHWs, treatment partners and patients. The availability of first- and second-line drugs over the counter and the limited access of patients to free of charge treatment—in the private system and some hospitals of the public system—is a major cause for concern as it may result in drug resistance. Re-treatment of patients with Category II requires special attention, as the coverage seems inadequate and the results poor. Improving regular treatment with Category II may reduce the load of the much more expensive Category IV regimens. Guidelines for treatment of TB in children were recently developed, with support of USAID, and some staff have been trained. However, drug supplies in the periphery did not take into account the limited capacity for diagnosis and the drug stocks are sometimes not utilized. The insufficient knowledge of the population regarding free of charge TB treatment in public facilities is still a major obstacle to treatment and DOT coverage; USAID-supported sites, as well as non-supported sites, were found not to have provided sufficient information to the public. Communication to the public should be a priority in the future. 26 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) The impact of USAID activities on high-quality DOTS was mixed. In some areas, impact was high (e.g., political commitment, and less so through training in EQA and data quality). Less impact has been seen in other areas, such as the continuous supply of anti-TB medicines and, to some extent, diagnosis of non-infectious patients. Observations for the NTP With support from USAID, validate the DOT treatment model and outcomes, and publish the results for international experience. Increase attention to the use and cohort evaluation of treatment with Category II. DOH/NTP should ensure that the “no prescription, no TB drugs” policy is implemented widely and strictly followed. Consider strengthening routine monitoring and supervision at all levels of the NTP. Recommendations to USAID Continue to support efforts to maintain high political commitment at the national level, and implement strategies to scale-up political commitment at sub-national levels, especially among municipalities. For example, demonstrate the benefits of stricter rules on the allocation of LGU budgets for health, including TB, to ensure continuous, sufficient access to medicines and diagnostic tools. Support activities to expand access to treatment while maintaining the current level of cure rates. In particular, inform the community of the availability of free treatment and advocate to restrict over-the-counter sale of drugs without prescription. Working through the NTP, consider validating the effectiveness of the system to provide DOT and the systems for recording and reporting. Explore inclusion in work program/deliverables of the upcoming “Tuberculosis Prevention and Control Project” operational studies on real intake and treatment practices at facilities and among CHWs, treatment partners and patients. ACCESS TO DIAGNOSIS AND TREATMENT Access to TB services in the Philippines is generally good, despite barriers such as cost, gender, stigma and discrimination, which can often be significant challenges in other countries. Vulnerable populations, including children, prisoners, the poor, and those living in remote areas, can access services, but may have a more difficult time than the general population. Health￾seeking behaviors also can affect access to services as many Filipinos choose to receive care in the private sector, and often do not realize that they can get free TB diagnosis through microscopy and free treatment through the DOH. These barriers are outlined in greater detail below. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 27 Costs of Services Cost does not appear to be a barrier in accessing TB services in the Philippines. Every public facility that the evaluation team visited offered free DSSM, tuberculin skin tests for children and anti-TB medicines. TB culture and DST for re-treatment cases are provided free of charge at PMDT centers, and Gene Xpert tests are also free. Even in the private sector, many people who visit facilities have access to DOTS services due to PhilHealth benefits, which are outlined in another section of this document. When people seek care for TB in the private sector and do not have access to DOTS PhilHealth benefits, they often have to pay for consultation fees, DSSM, chest X-rays, TB culture and drugs, if the facility or hospital has not received free anti-TB drugs through the DOH. The costs of services and drugs varies and estimated costs are outlined in Table 7. TABLE 7. Costs of TB Products and Services in the Private Sector, 2012 Service Estimated range of cost in the private sector Outpatient consultation fees PhP 150–1000 = USD $3.50-$24 Chest X-ray PhP 100–750 = USD $2.30-$17.70 TB culture (outside PMDT) PhP 800–1800 = USD $18.90-42.25 DST PhP 400–3,700 = USD $9.45-$87 TBDC processing (charged by some private providers) PhP 50–100 = USD $1.20-$2.30 ATB Drugs: Branded FDC A PhP 10 per tablet = USD $0.24 ATB Drugs: Branded FDC B PhP 8.50 per tablet = USD $0.20 6-month Category 1 ATB drugs PhP 4,500–5,600 = USD $106-$132 Because some of these costs are relatively high and add up over time, it was reported that some people will get diagnosed in the private sector, start treatment, and then move to the public sector to access free services when they can no longer afford private services. This makes it challenging for public providers, who sometimes cannot access the patient’s treatment history. Through its initiative to engage the private sector, USAID’s PhilTIPS project recommended that private providers participating in PPMD charge PhP 75 for each smear, which is the equivalent of less than USD $2. This, however, has not been followed up or monitored by the TB LINC initiative for private-sector engagement. Conclusions Cost is not a barrier for access to TB services in the Philippines. Free diagnosis and treatment for patients with SS+ TB can be accessed with relative ease by everyone through public DOTS facilities. Accessing free diagnosis and treatment of MDR-TB can sometimes be challenging since coverage throughout the country is still rather limited. Many Filipinos prefer to go to the private sector for TB diagnosis and treatment; however, private DOTS facilities typically charge a fee for laboratory-confirmed diagnosis through microscopy, and diagnosis of other forms of TB (e.g., extra pulmonary and SS- pulmonary TB) usually requires X-rays that must be paid for by the patient. USAID has actively engaged 28 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) the private sector through PPMD, and this engagement of private facilities as DOTS providers has enabled a greater number of patients to access free TB treatment. Observations for the NTP The NTP should continue to strengthen the referral system from private health facilities, and should aim to expand DOTS coverage to all public facilities, especially hospitals. Recommendations to USAID To address the fact that many Filipinos continue to seek care for TB in the private sector, USAID should increase support to scale-up the engagement of private providers. USAID should also develop strategies to ensure that private DOTS facilities provide free diagnosis of all forms of TB. This could possibly be done through engagement with PhilHealth. In addition, USAID should also consider supporting efforts to expand the network of PMDT diagnosis and treatment centers, and investigate strategies, such as the use of enablers (such as transport costs) to reduce the barriers to free access. GENDER-RELATED BARRIERS/ENHANCERS The consideration of gender issues in TB control projects is a common international recommendation. In most countries, males represent two thirds of the cases diagnosed; this may be attributed to biological reasons or differential access to health services. If access plays a major role in the lower proportion of females diagnosed, actions to improve access would have a high priority for TB programs. In general, there are more females among outpatients attending PHC facilities for any reason (for their own sake or for childhood immunization or care), while the proportion by sex is similar in hospital outpatient attendance. Graph 6: Prevalence of Smear Positive TB by Age, Gender (2007) Source of data: Philippines TB Prevalence Survey, 2007 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 29 Access and use of health facilities may be curtailed for women because of economic reasons or social/religious factors. Males may consult later because they often prioritize earning an income, they fail to recognize their symptoms, or may not be able to access public facilities because the hours of these facilities coincide with the work day. Studies of prevalence in the general population, in which access has less influence, show that TB is slightly more frequent in females during adolescence—while after men and women reach their mid-twenties, the risk of developing TB grows with age in males and remains more stable in women. In 2007, the TB Prevalence Survey in the Philippines showed similar prevalence of symptoms in adult males and females (14% and 12.8%, respectively); double frequency in males with TB treatment (20% versus 11% in persons with symptoms); and similar duration of treatment for both sexes. The proportion of abnormal radiographies compatible with pulmonary TB was 7.9% in males and 5.1% in females. The prevalence of SS+ TB increased with age in females from 0.8 per 1000 in 10–29 years old to 3.0 in women over 50, and from 1.2 to 9.3 per 1000 in males. Males with symptoms compatible with TB consulted health facilities less frequently than females (27.9% versus 35.8%) and more frequently took no action (31% versus 19.5%); while females self￾medicated more frequently than males (45.9% versus 40.9%). National reporting data indicates that men are more often identified as TB suspects and examined by sputum microscopy, and they have TB much more often. In 2009, in 10 regions reporting,6 the 174,000 male TB suspects examined were 55% of the total and produced 64% of the SS+ TB cases. During field visits, it was noted that in public health facilities, the general outpatient attendance has a higher proportion of females, and that the proportion by sex of adults with respiratory symptoms tested by microscopy was about the same. The TB cases diagnosed were more frequently males; confirming the findings of the survey. Interviews suggested that gender is not considered a major priority by the NTP or by other organizations involved because it is not considered a barrier for access, although gender consideration is mentioned in the planning documents. The perception was that there is no major restriction of access for females; if at all there might be a restriction for males in urban areas due to the opening hours of the facilities. Conclusions No gender-related barriers were observed, with the possible exception of full-time employed individuals (many of them men) unable to access public facilities during office hours. In conclusion, the gender difference in TB notification seems mainly due to biological factors, although it is probably also influenced by higher tobacco consumption in males. As the country has several TB surveys and KAP studies, plus abundant data on case notification and good laboratory registration by age and sex, simple operational studies could analyze the available data to justify the level of priority or develop interventions, if necessary. This analysis should be done by region, as parts of the country may have different attitudes towards women, which could constitute barriers to TB care. It should also include treatment outcomes, per gender, as an indicator of access and compliance with drug intake. Observations for the NTP The NTP should develop policies and guidelines to ensure access to TB services for individuals who cannot reach TB facilities during working hours. 6 Field Health Services Information System Annual Report 2008 30 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Recommendations to USAID Carry out analysis of the available data on gender to justify the current level of priority and if necessary support operational studies in regions of the country that may represent barriers to care. GEOGRAPHICAL ACCESS The Philippines is geographically diverse, and geographical barriers are a significant issue in some areas of the country, such as in ARMM and in other areas where transportation is poor. Some MHCs cover large farming areas, requiring that suspects and patients travel long distances to reach national facilities. Many municipalities are on separate islands and require boat or airplane access, and therefore, the costs of supervision and monitoring of TB programs and the delivery of drugs and transport of samples for diagnosis can be high. Most of the areas visited had identified geographically inaccessible depressed areas (GIDA), but it was generally assumed that the LGU facilities cater to the GIDA in their units. Decentralization of PMDT, which was implemented outside of USAID support, exists in certain cities and provinces visited, benefitting patients so they can receive treatment in locations closer to their homes and families. Some provincial hospitals will admit MDR-TB patients who are seriously ill. In other areas, like Baguio City, there is a PMDT center in the city, but there are no halfway houses where patients can stay. If they have to travel long distances to access this center and do not have family in the area, they may choose not to access services since it would be very costly to stay in the area for their treatment. Despite geographical challenges, the evaluation team’s impression was that access to diagnosis and treatment was generally good. In remote areas, people could access services through intermediaries, such as BHWs, who were supported by USAID projects in some areas through trainings and capacity building. Conclusions Overall, access to services is good in spite of many geographical challenges, including having a country made up of multiple islands and the fact that some people live in remote rural areas. There are frequent natural disasters in the country, including typhoons, which interfere with the provision of health services. Access to MDR-TB services remains a challenge for many people since services are still limited and people often have to travel long distances to receive these services. The ARMM is a special case—the geographical barriers and peace and order situation continue to limit access to services for many. USAID projects have addressed these geographical barriers by training CHWs and staff in rural health facilities so that more people in remote areas can access services. Observations for the NTP There are still major gaps in services in ARMM, and the NTP should consider greater coordination with this region, especially with regard to supervision of services. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 31 Recommendations to USAID Because ARMM has significant challenges, both politically and geographically and has extremely low health indicators, USAID should prioritize support to ARMM through a targeted TB project, with a focus on drug supply management and supervision. STIGMA AND DISCRIMINATION The stigma around TB continues to prevent many Filipinos from seeking care; however, public health officials, including community volunteers, reported that there has been an overall reduction in stigma over the past five years. The decrease was reported in areas covered by USAID, as well as those not directly supported by USAID. Reports of stigma were noted in some of the sites visited. In Sorsogon, at a private hospital, it was reported that “people are failing in general awareness” about TB, and that “we have not diminished the stigma of the disease, so people do not want to be identified.” As a result, hospital staff stated that people often travel outside their community to pay for care at more discrete private facilities, instead of going to the free TB facilities that are located near their homes. In another site, health center staff reported that TB patients would rather go to the facility for DOT than have a treatment supporter come to their home, for fear that someone in the community might know that they have TB. In ARMM, however, the TB coordinator reported that “there is not much stigma in ARMM.” There were no reports of stigma against people with TB leading to discrimination against them. However, there were reports of some community health volunteers (CHVs), such as members of Community Health Action Teams (CHATs), fearing TB and thus being very reluctant or unwilling to provide TB services, such as collecting sputum. This was handled by assigning team members who were more open to filling this role for the team. Based on this, one could assume that such fears have led to fewer services and less support for TB patients, but this has not been proven and does not seem to have impacted patients since other volunteers were willing to take on the role. In all sites visited, including USAID-supported and non-supported sites, there is still a high level of stigma and insufficient knowledge about the possibility of cure and access to free of charge TB drugs among the general population. Conclusions Although decreasing, there is still a high level of stigma in the country toward people who have TB, and there is insufficient knowledge about the possibility of cure and access to free anti-TB drugs in the population. Information, education and communication (IEC) materials have been developed, through USAID projects and by non-USAID sites, but are not readily available to health facilities staff and general public. This makes it difficult to increase TB awareness in the general population. Where USAID-developed IEC materials are available, they are of high quality, are appreciated and well-used. USAID’s training of community volunteers has led to increased walk-in consultations, implying that denial of the disease decreased. 32 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Observations for the NTP The NTP in coordination with the National Center for Health Promotion should consider approaches to take ownership of IEC materials for reproduction and replication by other donors. This will ensure that TB messaging throughout the country is consistent. Recommendations to USAID Because stigma is still such a significant issue in the Philippines, USAID’s activities need to focus on drastically increasing awareness about TB in the general population. These activities should focus on consistent, essential messages around TB symptoms, access to free care, and the fact that TB is curable. Monitoring the impact of these materials and awareness activities will enable USAID to measure the impact of the interventions. VULNERABLE POPULATIONS There are several vulnerable populations living in the Philippines, including indigenous people, children, prisoners, the poor, and people living in remote locations, among others. There does not appear to be formal strategies or policies at any level for tailoring TB case finding and treatment to the needs of vulnerable populations, with two exceptions: piloting workplace strategies (such as mines, factories and shopping malls) and strategies for prisons. TB LINC supported the Bureau of Jail Management and Penology to develop a proposal to gain access for funding from the DOH to address TB in prisons. The most important emerging strategy in targeting vulnerable populations is the PhilHealth effort to provide services to the poorest Filipinos (indigents). In recent years, there has been an initiative to address the needs of children with TB as it can be very difficult to diagnose. However, while national guidelines have been prepared with the assistance of USAID-supported projects (e.g., TB LINC and WHO), actual pediatric TB diagnosis and treatment coverage is limited and guidelines have only reached some areas of the country. Some USAID and non-USAID supported sites visited were actively diagnosing and treating children, but there were many places where it is not possible to implement the strategy due to a lack or shortage of diagnostic tools, including PPD and X-ray. Sites visited did have the drugs needed and most were found to have received training, but could not implement the strategy. Some health workers perceived diagnosing and treating TB in children as an additional load: in one facility, a nurse commented that the demand for pediatric treatment was so great that they required an additional nurse just for those cases. Other health workers found it too burdensome to carry out the PPD follow-up and the daily administration of the medications, when it can be administered on an intermittent basis as suggested by some practitioners. The TB in the Workplace initiative, a contribution of TB LINC, operates in 11 workplaces. At these sites, a TB DOTS clinic was established in the compound and staff visited the company, talked to the supervisors in charge, and convinced them to establish a clinic with a doctor and a nurse at their own cost. Staff at these workplace clinics have been trained in TB case identification, and refer patients and link them to the RHU. They also do active case finding. In addition, TB LINC assisted in the establishment of 210 workplaces referring units to DOTS as a result of its assistance. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 33 Conclusions USAID-supported projects have contributed to the development of essential national policies to support access by vulnerable populations, namely for TB in prisons and in the workplace. However, there does not appear to be formal strategies at any level for tailoring TB case finding and treatment to the needs of other vulnerable populations. USAID has not yet been successful in expanding PhilHealth benefits to children, TB re￾treatment cases and MDR-TB patients. PhilHealth’s effort to provide services to the poorest Filipinos is a key strategy and needs to be expanded. Some progress has been made in developing a strategy for pediatric TB, but implementation is absent in some areas and hindered by lack of diagnostic tools. Observations for the NTP The NTP urgently needs to identify the bottlenecks to delivering pediatric TB services in facilities and share this information with partners, so as to quickly address these bottlenecks. The NTP should also implement strategies to overcome barriers to services for vulnerable populations. Recommendations to USAID Building on previous experiences, USAID should continue its support for strategies targeting vulnerable populations, particularly indigents and children. USAID should also capitalize on the potential of PhilHealth in providing greater access to many Filipinos, and should do this by providing greater technical support to PhilHealth to quickly introduce benefits for children, re-treatment TB cases and MDR-TB patients. HEALTH-SEEKING BEHAVIOR There have been few significant changes in health-seeking behaviors for TB in the Philippines in the past decade. A comparison of NTPS data from 1997 and 2007 on health-seeking behaviors among TB symptomatics, as displayed in the chart below, shows a decrease in those who took no action for their symptoms, from 49.1% in 1997 to 25.1% in 2007. However, there was a significant increase, 24.3% to 43.4%, among those who self-medicated. The proportion of those who actually took anti-TB medications changed minimally, at 16% in 1997 and 15.5% in 2007. Of those who consulted for their TB symptoms, over a third went to private providers—36.2% in 1997 (private doctors) and 37.7% (private doctors and hospitals) in 2007. Most of these were non-DOTS providers. During field visit discussions, reasons reported for seeking care in the private sector included confidentiality, and the perception of higher-quality and more rapid services. In one public facility in Oton, the number of referrals from private physicians who were part of the PhilCAT project decreased significantly over the past two years, purportedly because TB symptomatics increasingly prefer to go to the private facility directly for services. Public health facility staff reported that a significant number of their TB patients first sought care in the private sector, and then after a short period, around two months, switched to a public facility. This could be due to the costs of continuously going to the private facility. It was noted by health staff in one facility in Sorsogon that patients who drop out of treatment before cure often do not return when they begin feeling ill, and instead purchase the medicines from the private sector and attempt to self-treat. 34 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Graph 7: Comparison of Health Seeking Behavior in the Philippines, 1997 and 2007 Notes: Source of data: NTP Conclusions A large part of the population continues to seek care in the private sector and for TB, most people first seek TB diagnosis and care outside of DOTS facilities. USAID support has helped to increase consultation for symptoms in public facilities and referrals from private practitioners. Observations to the NTP The NTP should consider improving the referral system from the private sector. It should also consider broader communication efforts, targeting the general public, to increase awareness about the availability and social acceptability of free public TB services. Recommendations for USAID To address the fact that people continue to first seek care in the private sector, USAID’s project should focus on increasing awareness about the availability of free services in the general population. USAID should also explore strategies to reduce self-medication with TB drugs. This could include working closely with pharmaceutical companies to ensure that TB messaging is included when drugs are sold in private pharmacies. POLICY AND FINANCING Although TB LINC has contributed to the development of a significant number of national and local guidelines and policies (see Annex E), it was observed in the field that several of the guidelines/policies, that USAID-supported TB projects contributed to, were either not completed or deployed in the health system. For example, at the time of the evaluation, the following policies and guidelines were still in the development stage: NTRL Strategic Plan; USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 35 Laboratory Network Strategic Plan; Infection Control Guidelines; Pediatric Guidelines; The MDR-TB PhilHealth Benefits Package; and the Manual of Procedures of the NTP. Still, TA from TB LINC and HPDP helped with the development of the 2010–2016 Philippine Plan of Action to Control TB, which is now the overarching framework of the NTP. The task was spurred by the desire to achieve the Millennium Development Goals (MDGs) for TB control in 2015. With technical and logistical support—from the Global Fund, WHO and USAID through TB LINC and the HDPD—the NCDPC of the DOH systematically assessed the TB burden and control efforts in the Philippines and became the basis for the 2010–2016 Philippine Plan of Action to Control TB (PhilPACT). This was defined by multisectoral, broad based collective and technical inputs from various national and local agencies and partners, underlines the key strategic approaches towards achieving these targets at the national level. It had the following objectives: Align the TB control strategic direction with the sector-wide approach of the Health Sector Reform Agenda and incorporate the TB control plan within ARMM/provincial/city investment plans for health. Define the long-term actions to address key issues and constraints identified by various program evaluation and monitoring teams. Generate better estimates of TB epidemiology and set realistic programmatic targets. Strategize how substantial resources from the government and other sources could be effectively and efficiently utilized. Identify how to maximize recently developed technologies and global guidelines to achieve the MDGs for TB control. At the local level at USAID-supported sites, TB LINC has helped in the development and passing of ordinances, such as “No Prescription, No TB Drugs.” (Ironically, this does not appear to be followed in private pharmacies situated next door to USAID sites). USP, in their mystery shopper activity, reported that they were able to buy any anti-TB drugs without prescription in some locations (0 in Iloilo and Zamboanga, but in two out of five sites in Cebu). There has also been assistance by TB LINC to the NTRL in developing (since 2008) their strategic plan and for the development of its TB Laboratory Network Plan, New Diagnostics Guidelines, and Remote Smearing Stations Plan. In one province that received assistance from HealthGov and prepared its Provincial Investment Health Plan, it included TB. Also in the same province, all municipalities through this initiative were convinced to include TB allocations in their budget. There has been no direct TB LINC support to the regions, except for the development of their Regional TB Plans recently (April 2012) but HPDP has been leading the inter-cooperating agencies (CAs) in assisting the CHDs in their Regional Capacity Building Initiative (RCBI). According to a study on the economic burden of TB for the Philippines in 2003, about 514,300 disability-adjusted life years (DALY) are lost annually and cost the country a total of 7.9 billion pesos in lost wages annually. A recent systematic review (in 2012) found that the cost per 36 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) patient for MDR-TB treatment in the Philippines was USD $3,613.7 Available funding in 2011 was USD $55 million. Fifty five percent (55%) of this was from domestic sources and 42% from the Global Fund. This does not include USAID external support. This equates to TB spending of USD $0.6 per capita and USD $300 per patient notified. Various studies attest to the financing gap for the TB control program. Although the national government has substantially increased its budgetary support to NTP, LGU support is variable and often not sustained. PhilHealth provides some funding for the TB control through the outpatient benefit package and payment for inpatient services of its members with TB. Resources are also provided by the foreign assisted projects. Still, the TB program is characterized by funding inadequacy and inefficiency of fund utilization, such that out-of-pocket expenditures remain substantial and serve as an access barrier to DOTS. Government funding for first-line TB control is primarily through the DOH. This covers all of the country’s needs for FL and nearly all staff providing TB services at all levels. However, much of the funding for second-line TB control (including drugs, medical officers, diagnostic facilities, and so on) is through the Global Fund. Ninety of 112 NTRL staff are funded by the Global Fund. Financing for infrastructure is the responsibility (outside of ARMM) of the LGUs. In ARMM, which is not devolved, financing for staff, activities and infrastructure is downloaded from the regional level to the provinces, and then to the municipalities. The evaluation team found that some primary care facilities were funded adequately, while others claimed to require additional resources, and that the level of support was dependent on the interests/priorities of the local chief executives. According to PhilPACT, “strong political commitment exists at the national level but local government unit (LGU) support and ownership of the TB control program varies.” With the health program decentralized to be mainly under the control of the LGUs while maintaining lines of coordination at all levels, no specific budget allocations for TB have been observed in most of the sites visited. TB seemingly appears not to be a priority of LGUs and assumed to be sufficiently addressed by the overall budget for health. HealthGov, in partnership with TB LINC, aimed to increase funding at local levels, assisting local governments in health. They worked in 23 provinces; 12 of them were TB LINC provinces. For 11 areas that were not TB LINC, HealthGov did assessments to collect information on various aspects of the TB program to get a complete profile of the local government. In one province, the provincial health officer stated that HealthGov provided TA for the preparation of the Provincial Investment Health Plan that included TB. Also through this initiative, all municipalities in that province (21) were convinced to include TB allocations in their budget. HPDP has significantly contributed to the development of the PhilPACT 2010–2016, in particular on the financing section. Some LGUs provide enablers (e.g., in Region III, LGUs provide transportation and meal allowance for MDR-TB patients and patients to be presented in TB diagnostic committees [TBDC]). Transportation allowances for MDR-TB patients were also seen in several provinces. In terms of the PhilHealth TB DOTS Outpatient Package implementation, there are mixed results among the sites visited. In one province, PhilHealth accreditation of public DOTS 7 C. Fitzpatrick and K. Floyd. A Systematic Review of the Cost and Cost Effectiveness of Treatment for Multidrug￾Resistant Tuberculosis, 2012, PharmacoEconomics, Volume 30, Number 1, 1 January 2012 , pp. 63-80(18). USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 37 facilities was much higher before, whereas, the current trend is a reduction, since the health center staff cannot receive their legitimate share from the Outpatient Package. On the other hand, in all of Region VI, the Outpatient Package is available in 98% of the PhilHealth accredited facilities (only two health facilities are not PhilHealth accredited); its full implementation with the funding going to the health facilities (public) reaches 70%. These facilities have been supported by the LGU with specific ordinances so they have a Capitation Fund were the PhilHealth funds are deposited. None of these findings were related to the USAID-supported visited sites; it is difficult to measure the USAID impact on its contribution for more PhilHealth accredited health facilities, since the indicator used by TB LINC did not measure the total number of PhilHealth accredited facilities (to show increase or decrease), but instead TB LINC used one PhilHealth accredited facility per intervention area. Conclusions Policy: Several national/subnational policies related to TB control have been initiated and developed in recent years. However, a number of them were not completed or widely implemented in the health system. Financing: The national government has substantially increased its financial support to the NTP; however, LGU support is variable and often not sustained. The end of Global Fund support in 2014 will substantially reduce financial and human resources, particularly for MDR-TB and laboratories. The field visits did not reveal any difference in the number of PhilHealth accredited facilities between USG-supported and non-USG sites. Impact of USAID TB projects: USAID projects supported the development of national/subnational TB policies and have led to an increase in political commitment by local LGUs to provide additional support for TB. Observations for the NTP Continue monitoring the increasing incidence of HIV and consider the need for TB/HIV policies in the future. Continue to fast track certification of DOTS facilities in order to accelerate PhilHealth accreditation. Recommendations to USAID Prioritize the completion and roll out of policies in progress. Work with the NTP to develop a strong post-Global Fund sustainability plan and ensure smooth transition of funding to national resources. PRIVATE SECTOR ENGAGEMENT Though it succeeded PhilTIPS in 2006, TB LINC pursued private sector engagement only towards the end of its project lifetime. The Philippine Tuberculosis Society, Inc. (PTSI) became a sub-implementer for this purpose in January 2010 and had only a little over a year for this major endeavor. It later received an extension and a second grant during TB LINC’s extension year. PTSI, through the cooperation of the various professional medical societies—particularly the 38 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) PCCP’s Council on Tuberculosis, rolled out the second edition of the ISTC during the scientific meetings and conventions. Module development for this was funded by a grant from the American College of Chest Physicians, which has a local chapter. The Philippine Medical Association was also tapped as a sub-implementer in the roll out among its members. A total of 286 DOTS referring physicians were engaged and signed Letters of Intent to refer to DOTS clinics. Sixteen new PPMD facilities were reportedly established by TB LINC in seven implementation sites—installing the first private DOTS facilities in Bulacan, Bohol, Aklan and Marawi City (also a first in ARMM); and the rest in Albay, Pangasinan and Negros Occidental. Ten were later DOH￾certified and two PhilHealth-accredited in 2011. With 16% of Filipino TB symptomatics seeking care in private hospitals and 21.7% consulting private physicians, PTSI established the System for an Enhanced, Comprehensive, Unified Referral, Recording and Reporting for TB (SECURE TB) mechanisms—an expanded version of WHO/NTP’s Collaboration for Additional TB cases through Contacts and Hospitals (CATCH TB)—designed to track and refer TB patients who are initially managed in hospitals and clinics. Forty private hospitals were linked to the NTP through this initiative; 18 became DOTS providers and 22 as DOTS referring facilities. The HMO Medicard, was engaged at the national level and saw 442 TB suspects in their three clinics—of which 226 received treatment, including three new SS+ cases. Twenty-five private physicians (pulmonologists, internists, family medicine practitioners) in Quezon City were also engaged to pilot the Stand Alone Practice (SAP) DOTS Model, exclusively managing TB patients who refuse to be referred to DOTS clinics. Usual reasons why private patients prefer to be managed by their personal physicians despite free TB services from the NTP include the following: familiarity, trust and confidence in the health care provider; perception of premium services in the private sector – including “personal touch,” confidentiality, ease and continuity of services; the inconvenience of daily reporting to the facility, not favorable for those gainfully employed; low regard for government-provided anti￾TB medications; and disdain for long lines commonly encountered in public health centers. Other TB LINC initiatives directed for the private sector included the following: a revival of the PDI, which was revised and simplified for implementation in 165 pharmacies this time; the establishment of five new TB Diagnostic Committees (i.e., in Marawi City, Compostela Valley, Aklan, Bohol and Negros Occidental) to ease certification of disease activity among SS- cases as recommended by the NTP manual of procedures; and the collaboration with Global Fund for the training of 92 MTs representing 55 private laboratories, including Medicard, PASMETH and PTSI, on TB microscopy laboratories and included in the EQA system. This activity was aimed in further expanding the EQA network to cover the private sector and envisioned possibility of having a private provider deputized by the NTRL for QA services—a load likely to burden the public sector and still awaiting fruition to date. In addition, a revised TB core curriculum was launched and adopted by APMC, the Philippine Association of Schools of Medical Technology, and the Philippine Association of Colleges of Pharmacy; and training on TB counseling was likewise reported to have been provided by PTSI for private providers. Despite being PhilTIPS’ sub-implementer for the DOTS in the Workplace, PBSP’s own TB LINC project only reported TB in the Workplace initiatives after initial testing of policies in Zamboanga City in December 2010. This was later expanded to cover the following provinces: Albay (Camalig, Legazpi City, Tiwi, Malinao); Bulacan (Sta. Maria, Marilao, Pulilan, Calumpit); USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 39 Pangasinan (Sison, Calasiao, Villasis, Sual); Aklan (Kalibo, New Washington); Bohol (Tagbilaran, Panglao); Negros Occidental (Bacolod, Manapla, La Carlota, Kabankalan); Negros Oriental (Bais, Manjuyod); Basilan (Isabela, Lamitan); Bukidnon (Malaybalay, Maramag, Impasug-ong); Compostela Valley (Compostela, Maragusan, Nabunturan, Mabini, Monkayo, Pantukan); Lanao del Sur (Wao, Bumbaran); Sarangani/General Santos City (Maasim); and, here in the national capital region, Paranaque City and Quezon City (Districts III, IV), including the SM Group of Companies’ network of shopping malls nationwide. In summary, TB LINC reported 4,511 DOTS referrals from the private sector, 1,439 of which were diagnosed to have TB—401 NSP cases by private physicians captured through SECURE TB; 104 NSP cases by private physicians through direct referrals to DOTS facilities outside the SECURE TB framework; 29 NSPs detected by Stand Alone DOTS Practice (SAP) physicians; and 45 NSP cases by pharmacies—contributing at least 3% to the CDR in the 12 sites. A total of 4,964 referrals from private physicians (including Global Fund-trained) were received by the private TB microscopy labs which were also accessed by government facilities specifically in Aklan, Negros Occidental, Bulacan—of which 605 (12%) were NSP. And a total of 443 TB symptomatics were referred to the RHU by the TB in the Workplace activity, of which 185 cases were enrolled in the DOTS Program. Despite all these activities aimed at the private sector, TB LINC’s target for TB symptomatics who consult DOTS clinics was not achieved. Of the targeted 6,596 TB cases to be referred to the NTP by non-DOH sources, only 4,348 were reached by end-of-project—the annual target surpassed only during the last year upon PTSI’s engagement. Not surprisingly, targets for CDR and CNR among USAID-supported sites were not achieved. These can be largely attributed to the long interval between interventions targeting the private sector which led to some private providers discontinuing their engagement and becoming jaded about re-starting their involvement. It is unfortunate that gains and confidences earned by PhilTIPS in the private sector was not sustained and belatedly tapped by the current project—despite the numerous research supporting the need for the NTP to work with the private sector (i.e., 2005 Private Provider Study by UPEcon, 2007 National TB Prevalence Survey, 2009-2010 PhilCAT survey of non-NTP providers) to work for the attainment of MDG goals for TB. It is a glaring missed opportunity to take advantage of lessons learned from every past enterprise. In general, though, there is strong political commitment from the NTP and among institutional stakeholders for PPM initiatives—enough to contribute 6% to the national CDR by 2010. Strong cooperation was observed by the evaluation between public and private providers in several municipalities—especially in Region 6, where high referrals from private physicians and volunteerism from non-government sector contribute to high CNRs. In 2008, all 20 health facilities providing TB services in the region (16 RHUs, one CHC, one private hospital, one military hospital and one PTSI Unit) became PhilHealth-accredited for the TB/DOTS outpatient benefit package and has been annually renewed thereafter, which is a problem in other regions. St. Anthony’s Hospital in Roxas City, a recipient of USAID support during PhilTIPS, continues to operate with the support of the hospital administration and sustained by PhilHealth claims; in 2011, 75 cases were referred by the private sector, including 30 documented as SS+. In Ivizan City, Capiz, three private physicians received ISTC training by TB LINC and have increased their referrals through the years with five in 2010, 18 the following year and 12 in the first quarter of 2012. This is a significant improvement from having just one DOTS referring physician in 2009. At the Western Visayas Medical Center, 15 referring private physicians were noted—including 40 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) two newly trained in ISTC by TB LINC. In Bulacan (Region 3), two of three private hospitals receive anti-TB drugs from the RHU as DOTS providers. However, in other provinces, efforts to engage the private sector through PPMD workshops and trainings (including ISTC) did not lead to increased referral and notifications. In some of these provinces, little (or none) of the private sector even accepted the invitation to participate. Even in facilities supposedly engaged in PPMD (i.e., RHU in Bulacan), it was noted that many physicians working in the facilities do not themselves adhere to DOTS. In other areas, like far￾flung municipalities in Davao Oriental, private practice and PPMD are virtually non-existent. Conclusions Significant efforts have been made to engage the private sector in the last decade, even with earlier USAID TB projects. There is, in fact, strong cooperation between public and private providers in several municipalities. However, the coverage is still very limited. Interruption in USAID support for PPMD initiatives, particularly between the PhilTIPS and TB LINC projects, led to a halt in DOTS services by some providers; and, subsequently, a lack of interest in future involvement. Given the significant number of Filipino TB symptomatics who consult private providers, the PhilHealth TB/DOTS outpatient benefit package has the potential of increasing access to DOTS through its accredited private DOTS facilities, somewhat underutilized to this day. In general, USAID-supported efforts to engage private providers led to some increase in referral in DOTS facilities but greater and continued effort is still needed to sustain and improve this. Observations for the NTP There is still a need to increase public awareness of DOTS services in the private sector, particularly targeting those who prefer consulting private providers. A review and, as necessary, update of PPM guidelines may be considered to respond to current realities; while continuing to partner with PhilHealth for universal access to TB services. Recommendations to USAID Learning from the negative effect of past interruption of USAID support, while noting the continuing significance of the private sector in TB control in the country, it would be highly recommended that efforts to engage private providers be continued with greater resolve while ensuring the continuity of prior successful initiatives. And being the identified major sustainability measure for most PPMDs, it would also be worthwhile to support the PhilHealth in their development and update of pertinent TB policies, including a review of certification standards. DRUG-RESISTANT TB Available data shows that the vast majority of new SS+ TB cases managed in public and private facilities implementing the DOTS strategy are being cured (89% treatment success in 2009, WHO). This suggests that there is little emergence of new drug resistance by patients being treated by DOTS facilities. However, the actual prevalence of MDR-TB has been confirmed through rigorous drug resistance surveillance since 2004, when MDR-TB was found in 4.1% of USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 41 new TB cases and 21% of previously treated cases. The Philippines has one of the highest rates of INH resistance (14%) among new TB patients in the world. Based on the evaluation team’s experience in the field, it is clear that there is an increase in the reporting of MDR-TB, but that this represents a small fraction of the real prevalence. Some facilities visited during the evaluation reported high numbers of MDR-TB in a short span of time and unusually high default rates. Other challenges were observed in the field: In some areas visited, monotherapy and other forms of non-adherence to NTP-approved regimens continues in public hospitals and in private hospitals that have been engaged in PPMD, including public/private facilities that have been accredited by PhilHealth. MDR-TB treatment is being scaled up, in some cases, at a faster pace than implementation of DOTS. For example, in one public hospital (in AARM) that had not implemented DOTS strategy yet was developing a BSL-3 facility and preparing to become a satellite MDR-TB facility. In some regions, patients must travel great distances to receive MDR-TB treatment. For example, any MDR-TB patient in the Albay region has to travel to a single facility in Sorsagon for diagnosis and care. Although transportation in such cases is often supported by the local government, in other cases patients have refused to move because they prefer to remain with their families, or must remain with them because they are the primary providers. It appears that much of the new financial investment in technological infrastructure and staffing appear to be focused on MDR-TB, while new investment in preventive infrastructure (such as LED/FM microscopy) and staffing (such as nurses, microscopists and laboratory technicians) is sometimes insufficient. The Government of the Philippines, working with donors, is making great efforts to introduce and scale-up diagnosis and treatment of MDR-TB. The roll out of the PMDT program in the provinces began in 2010; prior to that, MDR-TB cases were rarely diagnosed and treated. The Government and non-USAID donors (principally the Global Fund) are providing support for MDR-TB, such as diagnostic tools, drugs, human resources and training in PMDT. Therefore, USAID support from 2006–2011 focused primarily on 1) ancillary services, such as systems for drug supply and tools for monitoring/reporting on MDR-TB cases; and, more often, 2) the prevention of further resistance through improved, earlier first-line diagnosis and treatment or through policy development, such as national infection control guidelines. Some USAID projects did specifically target MDR-TB, such as the MSH effort to introduce e-TB manager through the SPS project. This included the development of software, user training materials and the launch through Consultative Planning Workshops and User’s Training. It was later concluded, however, that the system was not appropriate for the national DOH information systems architecture, so this system is currently on hold. A new system, the Integrated TB Information System (ITIS) is being developed to address both first- and second￾line TB case information. USAID-funded support through the WHO has actively provided technical support for both national strategies and input on local problems presented by national level staff. For example, the WHO supported the adaptation of existing national policies on MDR-TB diagnosis and treatment to support the introduction of rapid diagnosis and standardized regimens. 42 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Although implementation of MDR-TB diagnosis and treatment strategies at the provincial and rural levels is occurring, it does not appear that WHO played a hands-on role in the evaluation/supportive supervision of this work. Likewise, in areas with serious impediments to the effective delivery of first-line treatment (and thus the prevention of further resistance), such as Region 12 and the ARMM, WHO has been absent. The IPHO staff in both Region 12 and ARMM claimed to have never been visited by WHO TB staff, while they did report meeting WHO staff from other program areas (such as malaria) as well as TB staff from other organizations (such as JICA). INFECTION CONTROL USAID-supported TB projects, including TB LINC and WHO, provided TA that contributed to the development of guidelines for infection control, but it does not appear that these have been finalized or rolled out: Provincial health officers interviewed at USAID- and non-USAID supported sites were not aware of any recent initiatives to implement or enhance supervision of infection control in their facilities. The lack of urgency to complete/implement these guidelines at the national level may be because infection control does not seem to be a priority issue in health facilities; all primary care facilities visited had satisfactory natural ventilation, and only one site (a major public hospital) reported staff contracting TB disease in recent years. Private and public hospitals visited have satisfactory infection control protocols in place, although the emphasis is on infection control in laboratories rather than on waiting areas. However, the most risky procedure, sputum collection is happening outside the building (as recommended). Of note, however, is the potential role of PhilHealth accreditation on the adoption of beneficial policies for TB control. One facility (Tarragona, Davao Oriental) was not accredited by PhilHealth because their laboratory did not comply with its standards; in response, the local chief executive built a new TB clinic to address the non-compliance. With the expansion of MDR-TB diagnosis and treatment facilities, greater rigor in ensuring adequate infection control will be a priority. WASTE MANAGEMENT TB LINC assisted in the development of national guidelines for waste management. Also like infection control, these guidelines do not appear to have been implemented in hospitals and primary care facilities. It was noted that in a few facilities supported by USAID, waste management was well organized, with protocols for disposal and designated waste disposal areas, however this was not the case in most facilities (USAID and otherwise). As would be expected, private and public facilities serving larger populations and covering a broader range of needs had more established and well functioning waste management. Conclusions Support from USAID (through WHO/Philippines) has contributed to the introduction of the revised PMDT policy for rapid diagnosis and standardized regimens (2010) and laboratory strengthening for MDR-TB. Its support for improved data collection/management for MDR￾TB (through the MSH-developed e-TB Manager) has helped define and refine the requirements for a national electronic MDR-TB treatment information system. Regional MDR-TB diagnosis and treatment centers are operating effectively and new satellite treatment facilities are being introduced to improve access to high-quality treatment for ambulatory patients. The revised national policy for PMDT is being followed consistently at USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 43 these centers. Partly due to availability and scale-up of centers following appropriate treatment guidelines, current MDR-TB prevalence appears stable nationwide and cure rates for new patients are very high. Still, there are reasons to be concerned about the future impact of MDR-TB: DOTS coverage is incomplete; there continues to be stockouts of FLD in some parts of the country (despite USAID support improved drug quantification); and first- and second-line drugs are available in the private sector. Furthermore, while PMDT services are expected to increase with support from the Global Fund in coming years, there are remain gaps in access to treatment even as systems are being put in place to diagnose greater numbers of cases. The most serious concern, however, is the sustainability of the PMDT program. It is largely dependent upon funding from the Global Fund that will end in 2014; while the Government has a plan in place to increase its funding for MDR-TB by 2014, this is only expected to cover 50% of the budget needed. Political commitment for PMDT is strong, but national financing for MDR-TB (or identification of another source of financing for MDR-TB) should be considered a priority in the immediate future. Observations for the NTP Consider ways to intensify prevention of MDR-TB. Initiate and promote the implementation of the National Infection Control and Waste Management Guidelines and ensure mechanism to monitor compliance. Analyze the costs of supporting MDR-TB services post-Global Fund (including funding for second-line drugs) and develop a strategy for ensuring sustainability through national resources. Recommendations to USAID Work with the NTP to develop a strong post-Global Fund sustainability plan and ensure smooth transition of funding to national resources. Provide TA and support to ensure sufficient infection control in the increasing number of PMDT diagnostic and treatment centers. Provide refresher training for existing PMDT facilities and expansions sites. 44 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 45 IV. ADVOCACY, COMMUNICATION AND SOCIAL MOBILIZATION Advocacy, communication and social mobilization are essential components of TB prevention and control. At the country level, advocacy includes ensuring that there is political commitment for TB at all levels of the government, and communication involves changing the knowledge, attitudes, and behaviors of people, which can be very helpful in reducing stigma. Social mobilization brings together members of communities, including community volunteers and TB patients, to engage them in TB prevention and control through several different activities, which can include assemblies, lectures, drama, radio shows and theatre. According to the WHO Handbook on ACSM for TB Control, “although distinct from one another, advocacy, communication, and social mobilization are most effective when used together.”8 USAID’s projects have worked on ACSM activities throughout the Philippines. TB LINC and HealthGov have worked closely with the NTP, PHOs and LGUs on policies and advocacy to increase funding for TB and increase awareness among policymakers and politicians. In some provinces, like Bulacan, this has resulted in increased funding for health, including TB. This is described in more detail in the policy section of this report. Through formal evaluation interviews and site visits, and informal conversations with taxi drivers, hotel staff, waiters and others, the evaluators observed that there is still insufficient knowledge in the general population about TB. Many people do not know about the availability of free services, and do not know that TB is curable. The communication aspects of the USAID TB projects, TB LINC and HealthPro, were somewhat limited. Of the 24 health facilities that the evaluation team visited, public information materials about TB in waiting areas were only seen in one of them, which was a high performing non-USAID site. The simple message of “if you have a cough, inform a nurse/doctor” was not observed anywhere. In the San Rafael Health Center in Bulacan Province, a TB LINC site, there was a poster about TB, but it was inside the microscopy center, out of view of the people coming in. In Benguet, a site that was supported by the Global Fund as part of PhilCAT, several posters were hanging in the DOT center. Several sites had flipcharts used to educate people about TB. Some of these were developed by TB LINC, while others were developed by earlier projects, such as World Vision through the Global Fund projects and other USAID projects like PhilTIPS. These were reproduced by the DOH, and were observed in treatment rooms at both USG and non-USG sites. They appear to be a useful resource for health center staff. TB LINC also designed a “treatment card” that looks like a laminated placemat, and includes instructions about DOT and informational messages about TB in English and Filipino (each side of the placemat is in a different language). TB staff at some of the health centers, TB LINC sites, Pulilan and San Rafael, have them taped to their desks and reported that they regularly use them to review information about TB with their patients during DOT. Although these treatment cards seem to be a very useful resource, they were not uniformly observed. Brochures with TB messages were observed in several sites, but their quality was inconsistent and health center staff noted that their stocks of both brochures and flipcharts ran out and were 8 “Advocacy, Communication, and Social Mobilization (ACSM): A Handbook for Country Programmes,” the World Health Organization, accessed 3 May 2012. 46 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) not replaced. Some materials developed under the USAID projects were available in non-USAID sites, and had been photocopied or reprinted in other formats by their health units on their own initiative and with their own funding. Other facilities without support from USAID designed and printed their own brochures. In Pulilan and San Rafael in Bulacan, health center staff reported that they often don’t distribute brochures about TB since people tend to throw them away. They prefer to discuss TB with the patients and then the patient can either take a brochure or leave it at the health center. In Baguio City, a non-USAID site, the population is mainly indigenous. Most people speak English, and do not speak Filipino. Most of the brochures they have received from the DOH were in Filipino, making it difficult to reach their target population with these materials. The brochures that were in English were of poor quality and had been photocopied. Based on information gathered during the evaluation, it appears that there were very few social mobilization activities supported by the USAID projects. Many sites participate in World TB Day each year. Activities have focused on general public audiences, primarily aimed at raising awareness about TB, the signs and symptoms of TB, methods of preventing transmission (such as cough), and sources of TB services. Orientations on TB in high schools on World TB Day occurred in Cotabato City in 2011 and 2012. One high-performing municipality in another area commented that she no longer believes in holding such activities, and that resources are better spent in other TB control activities. Since World TB Day activities only happen once a year and messaging about TB often does not happen regularly during other parts of the year, it is unclear whether this information does more harm than good in terms of decreasing fear of the disease and stigma toward people who have TB. Innovative social mobilization activities were discussed in some of the non-USAID sites. In Baguio City, the City Health Department staff members have “Flavor of the Month” health activities where they focus on a specific health area each month. March (World TB Day) and August (Lung Month in the Philippines) are focus months for TB. In March 2012, health staff gave lectures on TB to taxi drivers, but thus far, no taxi driver TB symptomatics have come in for TB screening. Other sites discussed the use of radio programs to communicate TB messages. In Albay and Sorsogon, there are “LGU hours” where the LGU can communicate to populations about services, but these programs completely rely on executive support in the LGU. Conclusions The strongest component of ACSM in the USAID TB projects was advocacy as they accelerated the introduction of policies and increased support for TB control activities. Communication materials that were developed by USAID were of good quality, but they were not distributed widely and there does not appear to have been a significant effort by the projects to provide education about TB to the general public through IEC materials. Instead, they focused primarily on education to private practitioners, health care workers, government officials and people visiting health facilities. Other than the mobilization of volunteers, social mobilization efforts do not appear to have made a significant impact. As mentioned in the beginning of this section, ACSM activities are most effective when they are implemented together. One of the weaknesses of ACSM in USAID’s TB portfolio is that the activities were implemented independently of one another, and do not appear to have been part of an overarching strategy. There are multiple innovative approaches to ACSM USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 47 being implemented throughout the country that future USAID projects can learn from. USAID can also learn from successful ACSM projects in other countries in the region. Recommendations to USAID USAID/Philippines would benefit from a clear ACSM strategy that could drive the selection and implementation of activities. This would be extremely useful in situations where more than one project is implementing ACSM activities, and it would help the Mission guide future ACSM activities. In addition, activities should be focused at the community level, where there is potential for large impact. Explore the inclusion of all ACSM recommendations to USAID into the work program of the TB component of the upcoming “New Health Communication Project.” LABORATORY STRENGTHENING The Philippines has an extensive network of DSSM laboratories for diagnosing drug-susceptible TB that covers all provinces of the country. The NTRL, established in 2002, helps coordinate routine EQA of that network. However, according to PhilPACT, high-quality DSSM is available in most municipalities, but not universally accessible. Only “around 60% of the microscopy centers provide DSSM within the [EQA] standards.” Nonetheless, efforts are being made to both improve the quality of public and private DSSM services, as well as to expand the laboratory network to address the need to screen for (and diagnose) drug-resistance. The DOH, as advised by the WHO/Western Pacific Regional Office, promoted the establishment of TBDCs to curb the over diagnosis of pulmonary TB disease activity by chest X￾ray alone. The main function of TBDC is to review symptomatic SS- cases with X-ray findings suggestive of pulmonary TB or referred cases with such radiologic features that may warrant anti-TB treatment. USAID support, from 2006–2011, for laboratory and diagnostic network strengthening primarily centered around the expansion of microscopy services through training for microscopists and the introduction of volunteer health care workers into the diagnostic network. In addition, MSH helped strengthen the management capacity of the NTRL, including human resources as well as training in planning, leadership and financing through workshops and mentoring. They also helped develop a laboratory information system and assisted with lab supply management in the NTRL. TB LINC has been working with the NTRL to develop the NTRL Strategic Plan and the TB Lab Network Strategic Plan; both were started in 2008 and remain unfinished. The delay, according to NTRL, is due to their limited human resources to attend the meetings with TB LINC. The Strategic Plan will be important to help calculate among other things the cost of human resource gaps upon Global Fund completion. As a result of USAID support, WHO was able to help finalize the design of the biosafety level (BSL) 3 laboratory, review the standard operating procedure and verify the whole validation procedure and certification— all of which led to the BSL-3 lab being certified internationally in the Philippines. It also helped to organize the first training of trainers for implementation of LED/FM in Palawan for the first time in the Philippines. Accessibility of rapid diagnosis is available in 15 sites through GeneXpert, and this enhances the enrollment of treatment for MDR-TB. Twenty machines are available, but five have not been 48 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) deployed at the time of this report. Although the network of MDR-TB diagnostic facilities are being scaled up (with Global Fund support), most provinces still do not have their own dedicated MDR-TB diagnosis/treatment facility, so they often have to refer patients to facilities that are a great distance, requiring patients to leave their families behind. Many patients prefer to suffer with the disease—staying with their families and continuing to work, rather than leave their families for extended periods of time. TB LINC provided DSSM training in several provincial facilities. Staff who attended the trainings reported them to be of high quality, although it should be noted that that no pre- and post￾training assessments have been carried out. In one province, officers commented that these trainings were implemented without consideration for the availability of tools (such as microscopes) to be used by people trained; in one province, there were 25 microscopes to cover the needs of 36 municipalities. Microscopy labs were found to be in good condition in both USAID and non-USAID facilities, and in public and private facilities visited. Microscopists were well trained and appeared to execute their responsibilities effectively. In most facilities, the number of slides read per day were typically within the range suggested by WHO (20–25 slides a day). There were exceptions, however, where shortages of medical technicians required facilities to share technicians. It was noted that patients are frequently diagnosed by the private sector, placed on treatment, but then—when they can no longer afford treatment—they arrive at a public facility for treatment. Such cases may not arrive with referral information or documentation of earlier treatment. In one province, covering 36 municipalities, there were six medical technologists providing DSSM. Three of the six technicians were trained to carry out quality assurance for the province, but because of the overwhelming shortage of medical technologists, are no longer conducting EQA and instead providing routine DSSM. The six technicians travel from municipality to municipality (each municipality is visited approximately once a week) and do all of the slides for the municipality in one day, leading to very large workloads (as many as 80 slides in a day). According to one person interviewed, “These medical technologists spend all day reading slides, then have to go home and cook for their families, and so do not have time for preparing reports and paperwork.” Non-engaged private practitioners were reported, in some areas, to be using symptom-based diagnosis for adults rather than X-ray or DSSM, even in cases where a public DOTS/DSSM facility existed nearby. In those facilities diagnosing and treating children, the use of a symptom￾based method of diagnosing children appears to be implemented and functioning effectively. In facilities using PPD for screening of TB in children, when they have a supply of PPD, it is being effectively implemented. In a significant number of facilities, there were stockouts of PPD and cases detected plummeted to zero. That stockout of PPD had a long-lasting effect on some of the program’s motivation to revive the initiative. PPD is normally procured by the central office of NTP and distributed to the provinces through the regional drug distribution system. The EQA of laboratories appears strong in some areas and weak in others. In well functioning areas, few errors were reported by facilities and those reported were primarily minor. Training by USAID on laboratory data quality and laboratory management appears to have been effective. In one province, the manager of a non-supported facility reported having visited a USAID￾supported facility to learn their practices for lab/records management and applied them in his USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 49 own facility. The non-USAID facility lab was outstanding. In other non-USAID facilities in the same province, however, major errors were found in laboratory record-keeping/management. The TB LINC is promoting two indicatives in areas where national resources for DSSM are weak (or remote); they have promise and should be investigated further. The first is the use of BHWs to collect and fix smear samples, for staining and reading at a public facility (i.e., Remote Smearing Stations). The evaluation team observed the fixed samples, which had been delivered quickly to the facility and were in good condition. This may be an effective stop-gap solution. A second solution implemented is the training of midwives to conduct smear microscopy. While there is reason to believe this could also be an effective stop-gap, the lack of supervision/EQA combined with the need for midwives to cover multiple public health functions besides TB and the need for follow-up training may lead to dangerous weaknesses in perhaps the most important functional area of the rural TB control system. Conclusions Smear microscopy has good coverage within public facilities and is generally of high quality. There are sufficient microscopy units and the workload is acceptable except in areas where there are an insufficient number of microscopists. The standards for ensuring quality microscopy should be more ambitious: 95% of slides without major errors is too low. The laboratory information system for microscopy is strong and the data collected is useful for monitoring case detection, but it is not sufficiently utilized. There is a rational strategy for scaling-up access to culture/DST. Tools for rapid diagnosis have been deployed in 15 centers, but patient coverage is still insufficient. Impact of USAID TB projects: USAID-supported projects provided TA that improved the laboratory network in the Philippines, with particular benefits for the NTRL. Observations for the NTP Consider raising the standard for quality microscopy to 99%. Consider assessing the infrastructure and human resource capacity of rural smear microscopy laboratories to identify areas for improvement. Recommendations to USAID Concentrate future laboratory support on further improving coverage and quality of smear microscopy at service delivery levels. Consider providing support for LED/FM microscopy in high throughput facilities. Continue providing management and TA to help maintain and expand the laboratory network, such as through support to the NTRL and for engagement of private laboratories. ANTI-TB DRUG SUPPLY Drugs for the entire duration of treatment are provided free of charge to the patients in DOTS public facilities. Drug supplies for Category I and II cases are procured and financed by the national government from regular funds, while drugs for Category III cases are expected to be 50 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) sourced from LGUs. In 2008 there was widespread shortage of FLD in the DOTS facilities due to problems with central procurement, and stockouts continue to occur with relative frequency in other regions (e.g., ARRM). Shortages were and continue to be addressed by the procurement of TB drugs by many LGUs as observed in the sites visited. Currently, Category II kits are still out of stock in some of the sites visited (i.e., Region 11) and, consequently, purchases for streptomycin and ethambutol are made by the some LGUs to supplement the available Category I kits for such needs. As of 2011, procurement of all TB drugs, regardless of category, will be done by the Government. Production is national and presentation is in patient boxes with blister packs. Quality has been tested and approved. The MSH collaborated with capacity building in drug management. In theory there should not be stockouts (except in 2008, due to supply delays at global level). In practice there are occasional stockouts in some LGUs, mainly due to insufficient reserve stocks. This was particularly evident as observed in ARMM, where stockouts were frequent and the procedures have changed several times from distribution from central level first to regions, then to provinces and lately directly to municipalities. All facilities visited were stocked with FLDs from a supplier (Natrapharm, Inc.) that is not prequalified by WHO. Second-line drugs (SLDs) are currently available through the public system, mainly through support of the GF grant. However there is rapid increase of cases enrolled (from 315 in 2007 to 869 in 2010, cumulative 2,558) and expansion of the PMDT centers (from three in 2006 to 25 in 2011) and insecurity of future GF support; this may result in risk of interruption of treatment for MDR-TB in the future. The success rate for MDR is 55%, with deaths 11% and default 34% in the 2009 cohort. TB drugs are sold over the counter with or without prescription. Public and private hospitals may send the patient to buy TB drugs, with the risk of interruption or under-dosage when funds are exhausted. In one municipality, SLDs were purchased by the evaluation team without prescription at a private pharmacy, within 20 meters of a public DOTS center. Upon being informed, rural and provincial health officers were not surprised and claimed this was commonplace. TB drugs are usually provided to the patient or treatment partner for a week initially and two weeks or more afterwards. All facilities interviewed about treatment procedures reported that they usually did not conduct DOT in the facility. In facilities in Albay and Sorsagon, it was observed that the conditions of storage of the TB drugs were sub-optimal (mainly regarding temperature) and that some FDC pills were melting although well within the time of validity. At the provincial and rural levels, FLDs were found in poor condition (with signs of bleeding/melting) in both USAID- and non-USAID supported sites. At one provincial drug store, medicines were stored in an open hallway in the same area being used for archiving old paper registries. In some provinces visited, patients who had completed their first phase of MDR-TB treatment were sent home with a full supply of SLDs for continuation of treatment at a rural health facility. Staff at these facilities received no training in PMDT, nor did they have ancillary drugs available for addressing adverse drug reactions. In two of these sites, the SLDs (cycloserine) that require storage at cool temperatures (under 25 degrees celsius) were kept in the same storage facilities as FLDs, well above the maximum temperature. Staff at the facilities claimed to be unaware of the requirement for cold storage. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 51 The USP PQM selected three LGUs and three CHDs as sentinel sites, and they have a GPHF mini lab at each site. They have trained FDA staff at the sites in performing basic screening tests on FLDs, including FDCs and single drugs. By December 2011, PQM had collected over 461 TB medicines, and only 1.53% did not meet the qualifications. The samples that do not qualify are then sent to the legal department for further investigation, and there was a recall for a specific drug that was completely pulled from the market. In fiscal year 2012, PQM’s plan is to expand testing to MDR drugs and other antibiotics. They are also adding two additional sentinel sites. Conclusions Stockouts of FLDs continue, particularly in geographically remote areas where transportation is challenging. FLDs and some SLDs are available in the private sector without prescription; surveillance found some of poor quality. FLDs distributed in public facilities are not pre-qualified by WHO, storage conditions sometimes poor with some drugs deteriorating, raising doubts about their quality at intake. USAID support for surveillance of drug quality in the private sector has provided some evidence of poor drug quality that led to removal of low quality anti-TB drugs from the private market. Observations for the NTP Implement monitoring of anti-TB drug storage and quality at service delivery levels. Consider analyzing the costs of supporting SLD supply post Global Fund funding and develop a strategy to ensure sustainability through national resources. Recommendations to USAID Support surveillance of drug quality and advocacy for control of drugs in the private sector. Intensify efforts to improve drug management at service delivery levels (particularly for drug forecasting and storage), including mentoring. Support national anti-TB drug manufacturers to become prequalified by WHO. Seek ways to enforce existing policies for “No Prescription, No TB Drugs.” HUMAN RESOURCES AND CAPACITY BUILDING USAID support for TB from 2006–2011 did not finance TB control human resources, but funds were used for 1) capacity building of existing staff; 2) the engagement of volunteers to provide support for TB program operations in communities; and 3) advocacy to LGUs for increased funding for the staff. According to the situational assessment for the 2010–2016 PhilPACT, there are 3,047 MHOs, 4,577 public health nurses (PHNs), 16,821 rural health midwives (RHMs), and 1,717 medical technologists (MTs) and 199,546 active BHWs available for the implementation of the TB control initiatives under the LGUs. An estimated 15,000 private medical practitioners, mostly concentrated in urban centers, also cater to the health needs of the Filipinos. 52 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) In general, there were not many gaps noted for NTP human resources. Most sites visited had the full complement of professionals needed to operate a functional TB program. Though the population served by a physician can often be more than the national standard of 1 MHO per 20,000, it would appear to be adequate considering the good treatment outcomes of the TB program. Shortages in health staff were noted though in some municipalities (i.e., ARMM, Regions 11 and 12). TABLE 8. Human Resources for TB Control in the Davao Region (Region 11), 2008–2011 Davao Oriental 11 municipalities; ~183 barangays 2008 2009 2010 2011 MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW Banaybanay 1 1 1 12 156 1 1 1 12 156 1 1 1 17 156 1 1 1 17 156 Mati 1 4 1 31 476 1 4 1 33 484 2 4 2 34 478 - - - - - Caraga - 1 1 9 135 1 1 -10 96 1 2 1 10 135 - - - - - Tarragona 1 1 1 11 79 1 1 1 10 79 1 1 1 10 182 1 1 1 10 164 Lupon 1 2 1 24 363 1 2 1 23 373 1 2 1 23 376 - - - - - Cateel 1 1 1 14 138 1 1 -14 139 1 1 1 14 138 - - - - - Boston - 1 1 9 94 1 1 1 9 94 1 1 1 9 94 - - - - - Baganga 1 2 1 18 282 1 2 -18 291 1 2 1 34 281 - - - - - Manay 1 2 1 16 269 1 2 -11 266 1 1 1 16 269 - - - - - San Isidro 1 1 1 16 167 1 1 1 17 239 1 1 1 16 167 - - - - - Gov. Generoso 1 1 1 23 286 1 1 1 23 268 1 1 1 22 296 - - - - - Total for Province 9 17 11 183 2,445 11 17 7 180 2,485 12 17 12 205 2,572 N/A N/A N/A N/A N/A Other Provinces of Region 11 MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW Compostella Valley 11 municipalities; ~237 barangays - - - - - - - - - - - - - - -18 24 17 173 1,809 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 53 TABLE 8. Human Resources for TB Control in the Davao Region (Region 11), 2008–2011 Davao Oriental 11 municipalities; ~183 barangays 2008 2009 2010 2011 MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW MHO PHN MT RHM BHW Davao del Norte 11 municipalities; ~223 barangays - - - - -18 34 27 168 -18 32 18 166 - - - - - - Davao del Sur 14 municipalities; ~337 barangays - - - - -17 39 11 197 -18 36 15 195 - - - - - - Davao City 182 barangays - - - - -22 46 23 91 -22 46 23 91 - - - - - - The primary gap, particularly because of the role of quality microscopy in improving TB case finding, was for MTs. The Global Fund is filling those gaps related to MDR-TB (such as in the NTRL) by hiring additional MTs and committing to support them until 2014. Several provinces, including Albay, Bulacan, Aklan, Bohol, Negros Occidental, Negros Oriental, Bukidnon, Sarangani, Maguindanao, Marawi City, Zamboanga del Norte, Zamboanga del Sur, received assistance in having all newly hired MTs trained on DSSM for their microscopy units —this was verified in Aklan by a member of the evaluation team. In many cases, one or more “floating” MTs would have to cover multiple primary care units (i.e., ARMM and Region 12). The result is a higher number of slides needing to be processed and read than recommended and—although not observed in the sites visited—thus posing the risk of poor quality smears and doubtful interpretations. However, this arrangement appeared workable, particularly where there were few TB suspects and documented cases, as MTs schedule their visits only in certain days of a week for each health unit served. The establishment of remote smearing stations for sputum collection and slide preparations by trained RHMs and BHWs in some areas (i.e., Compostela Valley, Sarangani, Zamboanga City, Zamboanga del Norte, Zamboanga del Sur, Zamboanga Sibugay, Marawi City, Sulu) lightened the load of MTs and allowed them to focus solely on staining and interpretation. Where a medical officer was found wanting, a nurse trained in TB was present and leading its implementation. However, with numerous other health programs also under their wings, some PHNs at the RHUs felt burdened, consequently affecting the quality of their services (i.e., this was one of the cited reasons for the poor performance of the TB in children program). The Doctors to the Barrios and Registered Nurses for Health Enhancement and Local Services (RNHeals) program of the DOH temporarily addressed these gaps by supplying underserved LGUs with newly licensed physicians and unemployed registered nurses under two- and one￾year contracts, respectively. In Capiz and Iloilo, the RNHeals project allowed PHNs to be relieved of other clinical duties in order to focus on TB activities. The need to regularly train replacements to ensure continuity of services, however, is a limitation to this approach. The DOH relies heavily on CHVs to expand the reach of its programs. BHWs are trained in basic health care services (such as MCH, FP and TB) and cater to communities or neighborhoods (barangays or purok). Nationally, it is estimated that there is one BHW per 443 population or 74 families; and their numbers from 2006 to 2011 seem fairly stable among the 54 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) sites visited. Their contributions to the implementation of TB control activities depend on multiple factors—including the provision of enablers or incentives by some LGUs; leadership and support from the MHOs or PHNs; availability of IEC materials; awareness of health issues; geographic accessibility; and personal commitment and initiative. No direct associations could be established between their numbers and TB control performance in the areas they serve. USAID￾supported provinces in Region 11, however, seem to fare slightly better in terms of case notification than non-supported sites. Although the norm for primary care facilities was typically a passive case finding approach, it was observed that BHWs and midwives routinely carried out contact tracing and identification of suspects for diagnosis in many areas. Where patients lack other treatment supporters, BHWs often fill this role, sometimes providing DOT at patients’ houses. In Banaybanay, Davao Oriental—a high performing municipality with no USAID support— barangay captains (the LCEs of the community) provide funding that allowed the provision of one BHW per purok. In addition, five RHMs dedicated for birthing at the RHU’s lying-in facility were also funded by the LCEs, enough to allow the other RHMs to conduct mothers’ classes in the barangays and purok with TB mandated as part of the curriculum. They participate in active case finding even without adequate IEC materials and, through the initiative of the MHO (the sister of the municipal mayor), receive a small share in the RHU’s PhilHealth claims as additional incentive. The RHU’s good performance in the past provides additional impetus for the staff to do well in order to maintain their record and reputation (i.e., when case detection plummeted in 2007 and after being reminded in training of NTP’s passive case finding policy, the staff went back to active case finding in the succeeding years). In Tarragona, Davao Oriental—a municipality with no USAID support, but with a comparable BHW to population ratio to that of Banaybanay’s—NTP performance was poor even after more than 100 BHWs were added in 2010, particularly for case notification. Program targets can definitely influence performance as it sets goals for indicators pursued by the health staff. However, when set too low and not periodically reviewed for resetting, it can promote lackluster implementation. At the City Camp health center in Baguio City, each BHW has a target of identifying one TB symptomatic per quarter. Based on a review of the quarterly reports from the BHWs, it appears that each is actually referring just one TB symptomatic per TABLE 9. Case Notification Rate, Davao Region (Region 11), 2006–2011 Province Case Notification Rate (per 100,000 population) 2006 2007 2008 2009 2010 2011 Did not receive USAID support Davao del Norte 91 108 104 113 148 112 Davao Oriental 91 91 100 113 115 106 Davao City 105 110 112 113 121 100 Received USAID support Compostella Valley 118 108 121 133 123 115 Davao Del Sur 102 96 111 125 109 114 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 55 quarter; and not surprisingly, their referrals identified only a total of one SS+ TB patient last year. Several models developed for leveraging community volunteers to support national health program were implemented by USAID-supported agencies from 2006–2011 in the Philippines. In 2008, TB LINC lobbied and prepared sample ordinances for municipalities to create local TB Task Forces (made available on the website of the League of Municipalities of the Philippines) in compliance with the Department of Interior and Local Government Memorandum Circular no. 98-155, ordering the creation of an Anti-TB Task Force as part of the Local Government TB Control Strategy. The body would be composed by one Barangay council member, one RHM and selected BHWs from the community. They would also later be known in some areas as the Barangay TB Patrol (a moniker based on a popular news program, TV Patrol). In Bulacan and Capiz, both TB LINC sites, the TB Patrol program was noted as a successful way to identify additional suspects. In one municipality, BHWs were able to identify 33 SS+ TB cases in just one month of mobilization—a drastic jump from having no TB suspects identified prior to this intervention. Unfortunately, once financial support from TB LINC ended, the municipality was unable to sustain the program due to lack of effective supervision and monitoring. In a health center of one of the provinces, the staff attributed the general population’s increased understanding of TB and DOTS to home visits by BHWs through this activity. It also became an opportunity for BHWs to become treatment partners for patients unable to go to the facility. SHIELD implemented a project to establish CHATs in the ARMM, typically made up of BHWs, to assist with MCH/FP and other health priority areas, including TB, for early detection and referrals. Though patterned after community mobilization initiatives of the previous Enhanced and Rapid Improvement of Community Health (ENRICH) project in ARMM—of which Hellen Keller International was also one of the implementers—the role of CHATs in TB appears to have been only introduced in recent years after the local USAID Mission added TB to its project deliverables. Their primary contribution appears to have been in mapping information about families and, to a lesser extent, collecting data about TB in the ARMM. Whether the identified cases of TB suspects actually complied with recommended interventions for diagnosis and treatment is unknown to implementers. HPDP also has its own version of community mobilization for integrated MCH/FP and TB services by enlisting CHVs as “navigators” in addressing health needs of families with the aid of a user-friendly book, the Family Health Book initiative, done as an operational research study in Compostella Valley. This may very well be the precursor for the DOH’s own integrated strategy for MCH/FP and TB, which also uses a guidebook. Last year, the DOH launched and began scaling-up CHTs to ensure universal access to health care through home visits by BHWs. Families get evaluated for their health needs in a holistic approach through its modules on newborn/infant/child/maternal health, FP and chronic cough management. “Government is allotting PhP 1.25 billion for this campaign,” according to Health Secretary Enrique Ona. A guidebook on the program has been published by the DOH to guide CHVs. For TB, they provide information about the disease, how to identify suspects and how to monitor the treatment of patients in communities. Unlike CHATs, CHTs work closely with PhilHealth to ensure that the teams have information for vulnerable populations on how to access and make use of PhilHealth packages. Although mentioned by provincial and regional management as “the way forward” for engagement in communities, and by PhilHealth as a key 56 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) mechanism for improving access to services, no CHTs were observed or reported yet at the service delivery level. Given the many acronyms or names associated with BHWs mobilized for the same TB control initiatives and/or related health goals, it would be prudent to institutionalize the best model (i.e., CHT, which is officially endorsed by DOH and recognized by other government agencies like PhilHealth) and expand its coverage across the country rather than confusing the general public of project-based initiatives with questionable sustainability once funding ends. Technical assistance in TB control initiatives, including training activities, tapped Filipino experts and were considered of high quality at all levels. The NTP program manager, Dr. Rosalind Vianzon, expressed that there has been too much focus of TA at the national and regional levels and that the NTP would prefer new equipment or TA targeting service delivery levels in succeeding USAID TB projects. CAs were compared in terms of how they approached the community in providing TA. Those aware of the Maguindanao TB Control Project (MTCP) implemented by Catholic Relief Services appreciated how they were consulted on what assistance they needed before the actual proposal writing, rather than just being informed of planned TA during implementation. A pre-assessment of TA needs would logically be more responsive and highly appreciated by its targeted beneficiaries. With training identified as the main strategy for capacity building, classroom-type seminars were usually employed in their methodology. Though this may be cost-effective, particularly when training large number of participants, it can fail in preparing trainees for real-world situations upon their return to their health care facilities. According to the Final Report of TB LINC, about 4,818 RHU managers received training on strategic information management and 127 NTP core team members—representing 12 CHDs, 28 PHOs, and 10 CHOs—were trained on monitoring, supervision and evaluation, with particular focus on using data for program implementation review, data quality assessment as well as mentoring on recording and reporting. However, serious gaps in supervision were still noted, both in frequency and quality. All sites visited, including USAID-supported sites, expressed a need for additional monitoring and mentoring. One province reported that they had not been visited by regional TB staff for several years. This was not the case for microscopy services as regular EQA reports were observed across the regions, whether USAID-supported or not. And basing it on the less than 5% major error rates observed in the sites visited, the EQA system appears to be functioning well at all levels. TB LINC provided formal quality assurance (QA) system training for assigned QA controllers— supplemented by on-site coaching; monitoring, supervision, and evaluation training; and DSSM training. In 2010, MSH recommended that “the TA focus of (TB LINC) over the next year should be on strengthening the management, leadership and organizational capacity at central and intermediate levels. Immediately, attention should be given to the most critical areas affecting the optimal functioning and performance of the NTRL and the TB laboratory services.” This led to TB LINC and MSH to collaboratively implement the TB Laboratory Services Leadership and Management Development Program, developed by the latter, to improve laboratory coordinators’ skills and enable them to effectively manage and strengthen laboratory performance in support of the TB control program. Other TB training initiatives reported by TB LINC were basic DOTS trainings for RHMs, BHWs, and CHVs (e.g., nine of the 34 barangays in San Rafael, Bulacan were trained by TB LINC USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 57 in July 2011 as part of the Barangay TB Patrol program; DOTS-provider training for doctors and nurses; and TB counseling training for CHVs. The training for BHWs and other community volunteers as treatment partners was designed to improve case holding. This training was complemented by a module on RHMs as supervisors of treatment partners. However, with only 586 community volunteers agreeing to become treatment partners out of 5,699 trained, a stronger focus on behavior change among providers should be considered in the training’s design to maximize its potential. Because of these training activities, TB LINC reported that household education on TB, including in Muslim communities, reached 151,640 families. This contributed to 8,380 suspects identified and referred, 4,824 examined with 3 sputum, and 794 new SS+ TB cases diagnosed. Whether this was actually better than in areas with no USAID support cannot be determined accurately, except probably through an operational research study. In general, the impact of the said trainings on quality DOTS services, particularly on case holding and treatment outcomes, is still too early to be seen by this evaluation. However, it is noteworthy that the evaluation team encountered a trained PHN who still felt not qualified to answer queries on TB during roll outs of TB Patrol trainings to BHWs. She bewailed having received no further supervisory support after her own one-time training. This underscores the need for continuing supervision at the service delivery level or on-the-job mentoring. In Balay balay, Davao Oriental—a non-USAID supported site—Barangay leaders (i.e., LCEs and council members) participated in two sessions of the Barangay Leaders Forum last year where TB was addressed. This was said to have also been attended by BHWs and is attributed to have contributed to the strong political commitment and awareness on issues regarding TB within the communities. A new ISTC-based TB core curriculum, promoting DOTS to students of medical and allied health professions—potential TB health care providers in the future—was developed and later rolled out to 38 medical schools through the Association of Philippine Medical Colleges, 63 member schools of the Philippine Association of Schools of Medical Technology, and 45 members schools of the Philippine Association of Colleges of Pharmacy. This was a revision of an earlier initiative developed during the PhilTIPS project. Conclusions Overall, existing human resources delivering TB services appear highly competent and well￾trained. In some parts of the country, however, there are shortages in manpower—such as doctors, nurses and especially laboratory technicians (i.e., ARMM, Regions 11 and 12). Serious gaps in supervision, both in frequency and quality, were noted—except for microscopy services where the EQA system seem effectively functional. BHWs and other community volunteers are able to increase the scope and quality of health services available for the community, especially in remote areas. Multiple models for utilizing community volunteers in TB control have been developed by USAID projects and their contributions varied depending on the original intent at their inception. Valuable lessons can be learned from these initiatives to guide future engagement of volunteers. Technical assistance, including training, leveraged national experts and was considered to be of high quality at all levels. Though the trainings provided by USAID projects generally introduced new skills to both public and private providers, pre-assessment of technical 58 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) needs did not always occur prior to the implementation of training activities. With trainings as the primary capacity building strategy of most TB projects, classroom-type seminars were often employed and did not put emphasis on supervision and mentoring during actual delivery of services. Observations for the NTP It would be worthwhile to consider mapping areas of the country that lack sufficient supervision and strategies to address gaps in TB service delivery. Given that the DOH relies so much on BHWs for the expanded scope and coverage of their health programs, including TB, it is prudent to assess and monitor the quality of services BHWs and community volunteers deliver. A careful assessment of capacity building needs, transport subsidies and other enablers to identify gaps in TB service delivery can help direct future support from donors, including USAID. Recommendations to USAID It is recommended that existing community-based models of engagement from past and current projects, and even those developed by the community themselves, be evaluated to learn valuable lessons in their potential to contribute to TB control initiatives. The evaluation team recommends that the DOH’s CHT approach be the focus of future support as this has already been officially institutionalized and likely the most sustainable. It also recommended that support be provided for greater and improved supervision at service delivery levels, through enablers (e.g., transport costs and per diems for the health staff and volunteers) and through TA or mentoring at the health facility itself. Continuing support for capacity building through training activities is still recommended, but with improved focus for on-the-job training and follow-up mentoring. It would be best that initiatives of this nature be closely coordinated with the NTP and be designed to minimize the burden on staff. At the service delivery level, USAID ought to consider focusing future capacity building on pediatric TB diagnosis and treatment, analysis of data quality and utilization, and delivery of ambulatory treatment for MDR-TB. SUSTAINABILITY There have been some significant sustainability steps taken through the TB LINC activities that have included the inclusion of TB DOTS in the curriculum and education for medical students, pharmacy students, nursing students and medical technology students. At NTRL there are 60 medical technologists employed through the Global Fund grant to support PMDT. The ending of Global Fund support by 2014 raises serious concerns as to how these human resources will be sustained. At the USAID-supported sites, there is no great concern about the sustainability of the staff at the municipal level, since this has always been from LGU support. There are, however, some staff (additional nurses) at PMDT treatment sites who are currently supported by GF grant, and this again will be a challenge to be sustained by LGUs. At the central office of the NTP there is also a significant number of GF supported staff. The USAID-supported projects led to leverage national expertise. PBSP, in particular after its engagement with the USAID TB project, was able to become principal recipient for the Global Fund. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 59 Financing Financing of TB Control in the Philippines will be in question by 2014 when the Global Fund grant ends; it currently accounts for 42% of the total TB Control funding in the Philippines and covers almost 100% of the needs for PMDT operations. Despite the fact that the DOH has the intention to cover the 50% of PMDT needs by 2014, a significant gap for SLD and PMDT human resources will remain. While PhilHealth contributes to the financial sustainability of the TB control at LGU level, there is the risk that health facilities maximizing PhilHealth reimbursements for TB patients will lead to LGUs decreasing their budgets for health, such as TB, rather than seeing it as additional funding that could be used for maintenance and enhancement of infrastructure and staffing. Technological The NTP has expressed the need for assistance in procuring GeneXpert cartridges as well as LED/FM microscopes. If the NTP chooses to expand GeneXpert for diagnosis of extra pulmonary TB, TB in children, and smear-negative TB as a replacement for culture, this will have significant requirements for sustainable investments. Currently, the Global Fund is the primary source of revenue for DST/rapid diagnostic tools for MDR-TB. The country does not currently have the resources/technical capacity to carry out the annual certification of the BSL-3 laboratory, which was recently opened at NTRL, calibration and maintenance of new diagnostic tools being put in place—especially beyond 2014. There is under-budgeting for supplies of PMDT at the NTRL. The NTP is moving forward to take over the procurement of SLDs. It is expected that by the end of the Global Fund grant, DOH will cover 50% of the financing required for SLDs. It is unclear, however, where the remaining 50% will come from. Conclusions The fact that the government is paying for its first-line TB Control program is a good sign for future sustainability. However, there are concerns about the sustainability of second-line diagnosis and treatment. In some places, PhilHealth support for TB is helping sustain TB services; there is great potential for PhilHealth to play a larger role in TB financing in the future (particularly for those in greatest need). Impact of USAID TB projects: Advocacy to local chief executives by USAID TB projects has helped increase government financial support in selected sites. Observations for the NTP Promote greater allocation of resources for health among LGUs. Closely monitor TB program needs among LGUs and coordinate with their network of donors to provide tailored, targeted assistance. 60 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Recommendations to USAID Continue supporting TA for the development of PhilHealth MDR-TB packages. Ensure that USAID activities continue to leverage national expertise wherever possible. Where support to local chief executives has led to increased commitment to TB, continue advocacy to sustain that progress. PROJECT MANAGEMENT The evaluation team met with management and technical staff operating within the USAID/Philippines and within the cooperating agencies responsible for USAID/Philippines projects operating from 2006–2011. In general, project planning and management of the USAID/Philippines TB portfolio appears to have been focused on important national priorities, but implemented without sufficient consideration for the coordination/management of the portfolio as a whole. The hard work of highly skilled and well-meaning staff employed by implementing agencies minimized the negative impact of this project-specific (versus portfolio-wide) planning. As described above, of the eight projects considered in the evaluation, only four were exclusively focused on TB, while the other projects included TB among other public health issues (such as support for MCH). Three of those other projects (HealthGov, HealthPro and SHIELD) introduced TB only after their initial constitution and activities had begun. It appears that USAID/Philippines designed and implemented those projects to target specific needs of the NTP and only after consultation with the NTP and partners. However, it was not clear that there was an overarching USAID strategy for its investments in TB to guide the selection of projects/activities. As a result, although the original projects appear to have been complementary, the late introduction of TB into other projects and the lack of an overarching strategy may have contributed to some overlapping areas of responsibility (e.g., WHO and LINC) and in terms of geographical areas (e.g., LINC and SHIELD in ARMM). While there was no evidence that this detracted from the success of those initiatives, it was clear from discussions with agencies that additional coordination was required to address these areas of overlap and avoid duplication of efforts or harmful fragmentation of approaches. The routine coordination of USAID-supported projects appears to have been initially assigned to USAID/Philippines staff responsible for the TB portfolio, with each implementing agency focused on managing its own activities. Management staff within multiple cooperative agencies reported, however, that the lack of continuity (four lead staff for TB in five years) led to less effective coordination, as did the inadequate technical knowledge of the program by some staff taking that function within USAID. In 2008/2009, the role of TB LINC appears to have been expanded to include a more coordinating role. However, it does not appear that TB LINC was initially staffed or organized for this management function, but was instead designed as a technical project. As a result, while all parties agreed that providing this function to an agency improved overall coordination, it remained sub-optimal. Monthly meetings by the chief of parties of implementing agencies, as a vehicle for coordination, was also an important, valuable exercise and appreciated. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 61 It is noteworthy that, with few exceptions, projects intended to build the capacity of provincial and municipal health services did not appear to directly engage staff from those areas in the development of their project plans, priorities and deliverables. It was reported several times that the MTCP project led to great benefits specifically because, as reported by one facility, they “met with us when they were planning their project, included us in the development of their proposals, so when they began delivering support we got what we needed and knew what we were getting.” In contrast, the perception at facilities and among government officials interviewed was that TB LINC came with a pre-defined menu of support options, from which they were allowed to choose. The head of one IPHO claimed to have accepted assistance that was not strictly a priority because they were being polite. In general, the perception of national representatives and other agencies/donors operating in the Philippines was that the USAID-supported projects were focusing on important needs of the country and generally worked closely with partners and the NTP to plan and coordinate their work. There were reports that USAID projects occasionally operated “under the radar” in some cases, which led to duplication of work and some communication challenges. For example, TB LINC started a data quality assessment exercise only to learn later that the Global Fund (cooperating with the NTP) had already begun such an initiative. This, and other examples of independent work by other agencies implementing USAID projects. Communication between USAID, implementing partners and the NTP does not appear to have always been optimal. One implementing agency reported serious miscommunications about their responsibilities vis-à-vis TB after the introduction of that work into their portfolio, claiming that they had been asked to report on TB, without an expansion of their scope of work, but later were told by USAID that they were required to also provide TA for TB control. Two other implementing agencies separately reported that they were being micromanaged. And while the coordination between the NTP and USAID was remarkably strong, the physical distance of the TB LINC secretariat/office from the NTP, WHO and head office for PBSP, as was clearly stated by the NTP Manager, was felt to hinder ideal coordination. It is noteworthy, however, that the reliance of USAID projects on national organizations (such as PBSP) has greatly advanced their experience implementing large-scale development projects and helped equip them for even greater, future responsibilities. The perception of the evaluation team, for example, is that the TB LINC experience better prepared PBSP to later become PR for the Global Fund grant. MONITORING AND EVALUATION The approaches for M&E varied between implementation partners. Most projects reported quarterly through narrative descriptions of their work and with different packages of indicators, appropriate to their activities and responsibilities. WHO provided a short report on their activities at the end of each fiscal year. Only TB LINC had an M&E person fully dedicated to collecting data and reporting to USAID. Monitoring and evaluation of the TB LINC project was based on a set of indicators, which included process, outcome and impact indicators. The 19 final indicators were chosen in consultation with the USAID country mission from an original set of 42 indicators. USAID process indicators were appropriate to reflect performance. The impact indicators, however, were less useful at sub-national levels. As a result, it is difficult to measure the impact 62 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) of USAID support at the service delivery level. More useful data that is readily available was not taken advantage of. While the CNR is useful for monitoring, the target assumes that incidence is homogeneous in all regions (it is not). In addition, areas that achieve impact on TB prevalence and transmission will have more difficulty to achieve the target, as one cannot find cases that are not there. At the same time, CDR cannot provide reliable and useful results at the local and provincial levels where it was used. The denominator of expected incidence is based on national estimates and the real incidence can have very large variations from one area to another. Regarding treatment, the potential margin of improvement of the cure rate is marginal because the country has reached and maintains very high levels of cure/completion (over the WHO targets). The data on case detection and treatment was sourced from the NTP. Each of the implementing partners had its own system for collecting other data relevant to their activities (for reporting to USAID). The 19 TB LINC indicators were as follows. Comments and reported outcome on each indicator used are shown below: Case notification rate in new sputum smear (+) pulmonary TB cases per 100,000 population in USG-supported sites. The target was not achieved, but there was an increase of 11% in SS+ cases notified. In comparison, the country increased 9% on average from 2005 to 2010. Percent of the estimated number of new SS+ pulmonary TB cases detected under DOTS (CDR). Not achieved. As described above, this is a weak indicator unsuitable to be used at local level. Existence of a policy updated in the last five years supporting MDRTB diagnosis and treatment. This was dropped because the NTP independently developed a new policy for PMDT. Number of people trained in DOTS with USG funding. Achieved. This is a process indicator and has no outcome to be measured. Percent of USG-supported laboratories performing TB microscopy with over 95% correct microscopy results. Based on the reported data, this target was not achieved since only 91% (and not 95%) of the microscopy laboratories (TB LINC original sites) that submitted slides for validation to the provincial quality assurance center (PQAC) performed within the standard 95% correct microscopy results. The indicator should have been: Percent of USG-supported laboratories performing TB microscopy with over 95% of them without major errors, since this was the actual measurement. Number of TB policies of national significance approved and implemented with USG support. (achieved). This is good outcome indicator: include data. Their goal was to implement 7 policies of national significance, which was achieved. Percent of municipalities/cities with at least one DOTS facility accredited by PHIC (PhilHealth). Achieved. However, research in the field found that in some provinces the total number of PHIC accredited facilities was declining. In this sense, the indicator was not sensitive enough to capture these changes. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 63 National implementation of ISTC. This indicator is unclear, making it difficult to determine if it was achieved. Implementation by the TB LINC was partial (in certain regions and provinces), but not nationwide. Number of people trained in strategic information management with USG assistance. Achieved. This is a process indicator and has no outcome to be measured. Number of TB cases reported to NTP by USG-assisted non-DOH sector. Not achieved. A good outcome indicator. Amount of PHIC reimbursement under the TB DOTS package for USG-assisted LGUs. Achieved. What was the target? The goal was 5.7 million and 9.7 million was reached. Issues with this indicator, since it’s not clear whether the reimbursements were received and used. Number of provincial LGUs implementing an inter-agency TB referral system. Achieved. Percent of LGUs with policy issuances that allocate financial resources for LGU health service improvement that include TB. Not achieved. Very important output indicator. Percent of TB microscopy laboratories that are submitting slides for external quality assessment. Achieved. Important output indicator. Percent of DOTS facilities with no stockouts. Achieved. Number of TB symptomatics in target areas seeking consultation in a DOTS facility. Almost achieved. Very important outcome indicator, should be expressed as number of TB suspects examined by microscopy, no seeking consultation, and should be complemented by the smear positivity rate. Number of LGUs implementing a strategic communication plan and undertaking various activities to implement it. Achieved. However, the goal (21 provinces) appears too low since it does not apply to the total number of municipalities. Number of provinces with community partners conducting activities to improve DOTS utilization. Achieved. Cure rate. Achieved. This is an important indicator. However, the margin of improvement at national level was rather limited from 83% to 84%. The main issue is not how many indicators were achieved, but which indicators were achieved. It is obvious that the important outcome indicators that could measure quality and impact, like case notification and case detection, were not really achieved and therefore one expects that impact on disease at national level to be minimal if any. On the other hand, the quality and impact at local level of the USAID-supported sites was much easier to identify and measure. The quality of monitoring and supervision was quite satisfactory during initial implementation but there was weak follow up once the project ended (sometimes implemented only 11 months). 64 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Conclusions USAID projects appear to have been planned and implemented to fill gaps and address priority needs in the country. However, there does not appear to have been an overarching strategy into which these projects were planned. As a result, there were some areas of overlapping responsibility (e.g., WHO and LINC) and geographical coverage (e.g., LINC and SHIELD in ARMM) and fragmentation (e.g., different approaches to community involvement). The late introduction of TB activities into some projects led to management challenges and sub-optimal performance because they were not staffed appropriately. The lack of continuity (four lead staff for TB inside USAID in five years) led to less effective coordination, as did the perceived inexperience of some of the staff taking that function. The addition of management responsibilities to TB LINC improved coordination of USAID TB activities; however, this coordination would have been even better if TB LINC had been designed for that function at the onset. There was a perception by some donors and government institutions that USAID-supported projects did not always include others in their planning, and that duplication of efforts sometimes resulted (e.g., DQA). Impact of USAID: USAID engagement of local NGOs in the management of its flagship TB project promoted country capacity and their ability to execute future internationally funded projects. It is our perception that PBSP would have found it more challenging to be selected as PR for the Global Fund, if they had not already demonstrated their ability to manage the TB LINC project. Observations for the NTP In addition to a TB strategy to guide the activities of different implementing agencies, ensure an overarching monitoring and evaluation plan is in place and used. In future projects, select indicators that can show impact at subnational levels. Recommendations to USAID As a precursor to developing separate plans for future activities, USAID should develop a focused strategic plan, targeted more toward local activities and interventions, for how to best invest its resources in TB activities in the Philippines. Ensure there is an effective mechanism in place for coordinating the different TB projects in the USAID/Philippines portfolio and their integrated M&E. Consider adding these responsibilities to a lead agency or establishing a dedicated and qualified national who is focused solely on TB within USAID to closely coordinate these different projects. Ensure that projects that will implement TB activities are properly constituted to do so from the very beginning. Based on the findings, every intervention should have a dedicated M&E specialist to collect data and provide regular reports to USAID. Indicators should be finalized in tandem with the USAID Mission and the relevant counterpart personnel from DOH, GOP. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 65 V. LESSONS LEARNED It is well documented that many people seek care for TB in the private sector, despite the fact that free and effective TB services are available and generally easy-to-access in the public sector. This is in part because of a perception that private providers are superior and more convenient than public ones. USAID-supported projects have therefore engaged some private providers in order to ensure their TB services are of high quality and to minimize the cost of their services to patients. However, some patients are still paying for care at non-PPMD private facilities because they do not know that free services are available. Therefore, while PPM initiatives are necessary and should continue, they need to be complemented by BCC campaigns that de￾stigmatize public services and increase awareness about the availability of free and discounted services in the public and private sectors. Given the private sector’s influence as a key opinion leader, greater patronage of the PPMD network across the country can be realized. Due to resource constraints, geographical challenges and/or limited demand for TB services, there are areas of the Philippines that do not have local, formally trained TB medical/laboratory staff. The NTP and USAID-supported projects have tried different approaches to fill these gaps. Some of these approaches (such as the use of remote smearing stations and community volunteers to perform basic TB program functions) have been effective where adequate supervision and quality assurance exists. Likewise, USAID projects have helped introduce new TB services (such as pediatric TB care) with good effect in parts of the country. In some cases, however, interruption of supplies halted service delivery. Scaling-up these new approaches and services should be considered—but only after their on-going training needs are accounted for and only in locations where systems for routine supervision, quality assurance, and the uninterrupted supply of essential commodities (such as pediatric TB drugs and TST reagents) are functioning well. USAID projects have helped expand national systems for TB (and MDR-TB) case detection, such as by engaging private providers, training and organizing cadres of CHWs to actively seek out TB suspects, and preparing for the system-wide introduction of new diagnostic tools. These complex and resource-intensive infrastructural investments have begun to show positive results that (with continued investment and careful management) are likely to continue and increase in the future. However, some discrete, low-cost, high-impact enhancements to existing systems (e.g., the systematic screening of adults with long-term cough attending outpatient facilities for any reason) have not figured as prominently in USAID projects. Higher cost should not preclude USAID or the NTP from investing in long-term improvements, but low-cost opportunities exist that should be pursued. Monitoring TB case detection performance and the impact of TA for TB control at sub-national level is more sensitive and accurate if it is based on the number of persons examined and the proportion of cases confirmed, rather than on national estimates of incidence. Investments to improve upon current cure and success rates in new TB cases should not be expected to have a high impact because those levels appear satisfactory, although investing in efforts to maintain those rates is important. At the same time, however, efforts should be made to validate the quality of community-based DOT approaches and reported treatment outcomes. 66 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) Planning USAID TB projects separately from one another can lead to some duplicate efforts and lack of linkages, and reduces opportunities to share resources effectively and use synergies to amplify results. Instead, USAID TB projects should be based on an overarching strategy for improving TB prevention and control, and designed as parts of that strategy based on the diagnosis of the situation (e.g., the results of this evaluation), discussion with the NTP and previous USAID TB experience in the Philippines and abroad. Likewise, adding TB activities to ongoing, broader health projects has less impact on TB case detection and treatment than projects that were initially designed and constituted to implement TB activities (such TB LINC). USAID TB projects that had clear and measurable targets established at project inception appeared to deliver more satisfactory results. Future projects should have appropriate indicators that can 1) be analyzed quarterly or annually during the life of the project; and 2) that are based on concrete measures, not estimates. Targets should be feasible and easily measurable. Baseline data for all monitoring indicators is essential for evaluating project performance and the collection of this data should be considered a mandatory part of the start￾up phase of projects, if not before. Annual desk reviews by the USAID focal point and an external consultant (working virtually via Internet) and brief summaries by each project would facilitate monitoring and reprogramming. Information sharing and close cooperation between USAID and the NTP, and alignment of USAID activities with national strategies, were effective methods for facilitating the success and effective implementation of USAID TB activities. In isolated cases, communication and alignment of plans by USAID CAs with the NTP was not as strong and led to less impactful, less sustainable interventions. In light of this, USAID should continue to work closely with the NTP to plan its objectives, activities and financing. Staffing and current technology needs in the NTP (at all levels) should be re-evaluated, as new technologies being introduced should allow previous methods/staffing/systems to be reduced or re-assigned. In addition, however, USAID should strengthen its oversight of CA planning and communication with the NTP, to ensure the greatest possible cooperation between CA implementers and government agencies. When structures and resources for oversight of the USAID TB Portfolio were bolstered (by introducing oversight of the portfolio into the terms of reference of TBLINC), organization of CA work appears to have improved. Day-to-day oversight of the TB Portfolio within USAID/Philippines should be properly resourced and, if not, delegated to an external entity that is properly resourced and experienced to do so. Oversight by local organizations may benefit from TA from a non-national (or international) agency to advise on the most effective systems/approaches for ensuring effective reporting, management and implementation by a network of partners. There have been successes in advocating for increases in LGU funding for health, including TB, as a result of USAID projects that have focused on policies and financing. Much of this success has appeared to result from the development of national level policies, manifesting as greater support for TB by local chief executives. While this support is vital, it is not sustainable; changes in leadership at the LGU level require renewed efforts to convince new leaders. Future efforts to improve financing should consider focusing on more permanent and sustainable changes (such as to legal structures) at the LGU level, rather than on advocating to individuals. At the national level, it will be vital that advocacy for increased, sustained financing be made given the approaching end of the Global Fund grant. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 67 VI. CONCLUSION Although there is room for improvement, USAID support in the period evaluated has played a significant role in improving TB control in the country. The USAID-supported TB projects made an impact on the TB control efforts and improved the quality of TB care mainly at the local level, but did not manage to achieve important indicators like CDRs and CNRs, which were set originally as targets. On the other hand they achieved an important quality of care target (i.e., cure rate), although the increment set was marginal. In general, approximately half of the set targets were achieved and these were mainly on process or output indicators. The main contributions of the USAID-supported projects were in the areas of policy, regulation and financing development and implementation, in particular at the LGU level; capacity building in TB care and control, through trainings, TB Laboratory strengthening (including NTRL); anti￾TB drug quality and management; and to a lesser extent, information and communication. USAID-supported TB projects’ advocacy and support to some LGUs in earmarking and increasing their resources for TB and the mobilization and support given to CHVs should be commended. The engagement of the private sector in TB control came late in the project, making the assessment of the results and impact of this strategy difficult to evaluate. Inter-project coordination was addressed, but not in an optimal way and not well thought from the start of the projects. The approach of the eight different projects in planning and implementing TB activities was fragmented and that led to some inefficiencies and sometimes duplication. The monitoring indicators were not always the best to measure impact and even quality, and in this respect it is difficult to evaluate the final outcome of the USAID-supported TB projects. However, TB continues to be a major problem and is diminishing very slowly in the Philippines. Likewise, MDR-TB is a growing concern and as capacity for diagnosis increases it will represent a larger share of the overall cost of TB control. OVERARCHING RECOMMENDATIONS USAID should continue its support for TB prevention and control in the Philippines at approximately the same level of funding. As a precursor to developing separate plans of activities, USAID should develop a focused strategic plan, addressing more local than national needs, for how to best invest its resources in TB activities in the Philippines. Greater efforts should be made to identify activities that had the highest impact in the period evaluated, and to ensure they are continued and expanded in future support. USAID should strategically focus on ensuring that national policies and tools are implemented at lower levels of the health system. The priority for USAID should be to ensure that its activities are closely aligned with national strategies and plans (e.g., PhilPACT) and support existing and emerging national approaches to delivering care (e.g., PhilHealth and CHTs). USAID must remain in close touch with other donors (e.g., Global Fund) and institutions (e.g., NTRL) to ensure its activities are complementary and not duplicative. 68 USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) OVERARCHING OBSERVATIONS FOR THE NTP It is of the highest importance that the NTP begin preparing for Global Fund departure in 2014. Inter-agency coordination should be further enhanced in this transition period and beyond, in order to optimize the needed support, while monitoring and supervision should be strengthened for more efficiency in performance. In parallel to efforts to scale-up MDR-TB diagnosis and treatment, sufficient investments in first-line TB detection and treatment should continue to be prioritized and ensured at the lowest levels of the health system. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 69 ANNEX A: SCOPE OF WORK Global Health Technical Assistance Bridge Project GH Tech Contract No. AID-OAA-C-12-00004 Scope of Work (FINAL 05-04-12) I. TITLE: USAID/PHILIPPINES EVALUATION OF TUBERCULOSIS PORTFOLIO II. PERIOD OF PERFORMANCE The assignment will be conducted from on/around March 31, 2012 through June 22, 2012. A six￾day work week in-country is approved for this assignment. III. FUNDING Global Health funds earmarked for tuberculosis activities of USAID/Philippines will fund this assessment. IV. IDENTIFICATION OF THE TECHNICAL ASSISTANCE The U.S. Agency for International Development/Philippines (USAID/Philippines) seeks the services of GH Tech to conduct a comprehensive evaluation of the performance and lessons learned by USAID tuberculosis programs and implementing partners for the past five years (2006–2011). This evaluation will determine whether or not USAID-supported interventions have contributed to increased TB case notification and successful treatment outcomes in the Philippines. The assignment will be conducted from on/around late March 2012 through May 15, 2012 or with the level of effort of 37 days. A six-day workweek is approved for this assignment. V. BACKGROUND TB in the Philippines Tuberculosis is a leading cause of mortality in the Philippines today, with approximately 75 deaths a day or about three Filipinos dying from TB every hour. Despite the continuing challenge, TB control in the Philippines has consistently improved over the past decade. TB prevalence has decreased by more than 48% since 1990 and the country is on track to meet the Millennium Development Goals (MDG) target of reducing prevalence by 50% by 2015. However, while the estimated number of deaths from TB has decreased over the past 10 years, it is unlikely that the Philippines will meet the MDG target of reducing deaths from TB by 50% by 2015, compared to the 1990 data. The proportion of estimated TB cases detected was 67% for smear-positive cases (in 2008) and 57% for all forms of TB (in 2009)9. The proportion of cases successfully treated has consistently been above the global target of 85% since 2000. 9 WHO Global TB Control Reports, 2009 and 2010. Note: As of 2010, WHO no longer reports on smear-positive TB case detection rate. 70 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) However, even with these high average rates of detection and cure, there were still an estimated 260,000 new cases of TB and 32,000 deaths in 2009. The Philippines ranks ninth on the list of 27 countries with the highest burden of multidrug￾resistant (MDR) TB cases: in 2008, the World Health Organization (WHO) estimated that there were 13,000 cases of MDR-TB. The 2003–2004 National Drug Resistance Survey revealed that the prevalence of MDR-TB among new cases was 3.8 percent; among previously treated cases, it was 20.9 percent. In 2007, an estimated 7,500 adults were living with HIV and WHO estimates the prevalence of HIV among TB patients in the Philippines as less than 1%. USAID/Philippines TB Portfolio USAID/Philippines, through a number of implementing partners and health programs, has supported the Government of the Philippines (GPH) to prevent and control TB. USAID￾supported TB activities have focused on national TB program strengthening and regional and local TB program capacity building through service delivery and targeted technical assistance. Over the past five years, USAID has invested approximately USD $32 million in TB-related activities in the Philippines that are listed below: TB LINC: The Linking Initiatives and Networking to Control Tuberculosis Project (TB LINC), is a six￾year cooperative agreement (2006–2012) that supports the National Tuberculosis Program in achieving its Millennium Development goals and objectives by focusing on four components: enhancing the health policy, financing and regulatory environment for Directly Observed Treatment, Short course (DOTS); building systems capacity to strengthen quality DOTS; improving service utilization through behavior change; and strengthening private sector participation in TB control. TB LINC works in 32 sites nationwide, including five new project sites and several municipalities in the Autonomous Region in Muslim Mindanao (ARMM) in the sixth year. The main implementing partner is the Philippines Business for Social Progress (PBSP), a local non-governmental organization. Over the past five years, USAID has invested approximately $20 million in TB LINC-supported activities. SPS: The Strengthening Pharmaceutical Systems (SPS) cooperative agreement worked with the Department of Health (DOH), Lung Center of the Philippines and partners to promote TB pharmaceutical and laboratory best practices, including pharmaceutical management training of DOH pharmacists, support to forecasting and quantification of Multi-Drug Resistant (MDR)-TB medicines, development of standardized pharmaceutical management policies, TB laboratory systems management, and management information systems (including e-TB Manager). SPS began in 2007 and approximately USD $1.54 million have been invested over the past four years. USP PQM: The United States Pharmacopeia Promoting the Quality of Medicine (USP PQM) project works in close collaboration with the DOH and Food and Drug Administration (FDA) to strengthen the national TB drug quality assurance system. USP PQM began in 2009 and approximately $530,000 has been invested over the past two years. WHO Country Office: USAID supports TB-related activities carried out by the WHO country office in Manila through the TB Medical Officer who provides technical support directly to the National TB Control Program (NTP) manager of the DOH. Support includes strengthening managerial, planning, monitoring and operational capacities of the national TB USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 71 program. Since 2009, USAID has been supporting the WHO TB Medical Officer in the Philippines with a grant of approximately USD $442,500. HPDP: The Health Policy Development Program (HPDP) cooperative agreement implemented by the University of the Philippines Economics Foundation supported the DOH and the TB LINC in developing national policies and guidelines that will strengthen the implementation of the TB control program in the country. Over the past five years, USAID has invested approximately USD $323,617.00 of TB fund into this Program. HealthGov: The Strengthening Local Governance for Health (HealthGov) Project implemented by the Research Triangle Institute complemented the assistance for TB control in areas (over 550 Local Government Units (LGUs) in 25 provinces) outside the TB LINC sites. USAID had invested approximately $3.0 million of TB fund for three years. SHIELD: The Sustainable Health Improvement through Empowerment and Local Development (SHIELD) is a six-year project being implemented by the Helen Keller International Inc. that focuses its assistance on ARMM. SHIELD integrated technical support for the TB control program in these areas with funding of approximately USD $2.4 million of TB funds. HealthPRO: The Health Promotion and Communication (HealthPRO) project being implemented by the University Research Co., supports health related behavior change communication activities in the country. HealthPRO is the lead vehicle of USAID in providing technical assistance to DOH and LGUs on health promotion and communication, and in developing information, communication and education materials for TB care providers, patients, and policy-makers. USAID has invested approximately USD $2.4 million of TB funds over the past five years. VI. PURPOSE The purpose of the evaluation is to determine whether or not USAID-supported interventions have contributed to increased TB case notification and successful treatment outcomes in the Philippines. This will be done through the conduct of a comprehensive evaluation of the performance and lessons learned by USAID TB programs and implementing partners listed above for the past five years (2006–2011), as guided by the USAID Evaluation Policy. The evaluation will specifically assess whether or not the package of interventions provided by TB LINC, SPS, USP PQM and WHO and other recipients under Cooperative Agreement (CAs) and support provided by other USAID health programs, such as HPDP, HealthGov, SHIELD and HealthPRO has improved TB case notifications and treatment success to effectively reduce TB prevalence and mortality in the Philippines. The evaluation will also assess outcome in specific areas of TB control, such as MDR-TB, public-private mix (PPM), advocacy, communication, social mobilization (ACSM), and analyze the common factors or patterns that contribute to success and identify areas for improvement. With the projects’ focus on capacity building of the health providers, support for accreditation of the DOTS facilities, information campaign, community involvement, laboratory strengthening and private sector engagement, it is expected that these initiatives are being translated to improved access of TB patients to quality services that results to increased treatment success rate. Results of this evaluation will be disseminated and discussed with the Department of Health, implementing partners, other development partners and other organizations/agencies working for TB control in the country. Recommendations will be considered to enhance the operational 72 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) plan of the National TB Control program as well as the future assistance of the USAID. Good practices that can be identified will be considered for replication and scaling-up. To reduce bias, service of a third party evaluation team is being sought with participation of local evaluation specialist, who has not in any way been involved in the implementation of the TB projects to be evaluated. At the highest level, USAID’s TB strategy strives to assist countries reach the Millennium Development Goals of halving TB prevalence and mortality compared to 1990 levels. At this level, the evaluation will focus on outcomes and results as measured by the following illustrative indicators at the national level using the National TB Prevalence Survey (2007) and DOH-NTP Annual report. The Evaluation Team may suggest other higher-level indicators as appropriate: TB case detection and notification rates. TB treatment outcomes, such as treatment success, mortality, defaulters, failures, etc. Number of successfully treated TB cases (M/F). Number of MDR-TB patients diagnosed and initiated on treatment (M/F). Private sector contribution to TB case notification. In the absence of a rigorous impact evaluation design (i.e., through the use of a pre-defined counterfactual comparison group) this evaluation will examine whether or not USAID programs had any effect on these indicators by looking at data before and after USAID-specific interventions were implemented and by comparing geographical areas where USAID was present compared to areas without USAID projects. Within USAID assisted provinces, analysis on the factors affecting variations in the performance of the different LGUs will also be done. VII. SPECIFIC OBJECTIVES AND ILLUSTRATIVE EVALUATION QUESTIONS In addition to a defined list of outcome level indicators (illustrative list described above), listed below are some illustrative questions in evaluating USAID interventions on TB control in the Philippines. Quantitative indicators should be provided to the extent possible (i.e., before and after project intervention implementation output and outcome). For many of the evaluation questions, each TB project should be evaluated alone. The evaluation is interested in assessing individual TB project contributions along with the total collective contributions of USAID TB projects’ “package” of interventions to the overall NTP goals and objectives. The list of questions below and detailed questions in Appendix B is illustrative and will be reviewed and revised by the evaluation team to inform a reasonable data collection tool. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 73 Main Evaluation Questions Expected Follow On Actions Programmatic Improvement Operational Enhancement of Future Projects Performance and Outcome: Overall, what are the key results and outcomes of the USAID TB program? Did the program accomplish its objectives and achieve its targets? Overarching Issues: How have projects coordinated their activities with the NTP? What are the perceived benefits/shortcomings of USAID programs to the NTP? What program needs are not being met? Describe any synergies of the TB projects with other USAID health programs (HPDP, HealthPro, HealthGov, SHIELD), with other development partners and with Global Fund. General TB DOTS Interventions: What USAID-supported interventions really made an impact on the NTP goals? What policies, financing and other enabling environment have been introduced by the USAID projects? What interventions are working/not working? Have any new policies regarding TB control been developed, rolled-out and implemented with the assistance of the TB projects? What is the uptake at national and local levels? Private Sector Involvement: How did TB projects improve the capacity of private sector to work in TB control? How and to what extent have partners’ capacity and engagement been strengthened? Are more private sector providers (practitioners, pharmacies, work places, etc.) notifying TB patients? Referring to public DOTS facilities? Main Evaluation Questions Expected Follow On Actions Programmatic Improvement Operational Enhancement of Future Projects National and Local TB Control Program Capacity Building/Strengthening To what extent have USAID TB projects contributed to high quality technical assistance at the national, regional and local levels? What policy instruments and programmatic tools have been developed and can be attributed to the USAID TB projects? To what extent have USAID TB projects contributed to capacity building of community based organizations (CBOs), non-government organizations (NGOs), and civil societies in participating in local TB control initiatives? Sustainability Mechanism: Have the projects developed a process to ensure sustainability of the systems and interventions after the project life? What are the plans of the DOH and local governments for sustaining systems and interventions developed under the projects? 74 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Main Evaluation Questions Expected Follow On Actions Gender: How have gender considerations been integrated in USAID’s TB programs and TB-related activities? What are the differential effects of the project on male and female beneficiaries? Lessons Learned: What are the overall lessons learned from the implementation and evaluation of TB LINC and the other USAID TB and TB￾related projects? What are the best practices from the TB LINC and other TB programs that could be adopted and replicated by follow-on activities? VIII. METHODOLOGY The evaluation will gather information through reports and data review, and in-country interviews with (but not limited to): government staff (NTP, National TB Reference Laboratory (NTRL), Lung Center of the Philippines (LCP), national government agencies (DOLE, Bureau of Jail Management, DILG, etc), regional and local TB control officers, etc.); local service providers (Public and private), professional societies, NGOs, CBOs, treatment partners, support groups, patients; USAID implementing partners (TB LINC, SPS, USP PQM, WHO); USAID/Philippines staff; other USAID health projects; and other TB control partners and funders. This evaluation also acknowledges some limitations in terms of the selection of few representative sites to be visited and partners and stakeholders that have to be engaged out of so many sites the current projects operate. It is expected that the Evaluation team can accomplish the task in the prescribed Level of Effort of up to 37 days. The team is to develop gender-sensitive data collection and evaluation tools (using the outcome/output indicators and list of questions suggested above as a guide) for ensuring consistency of information for different partners and areas of TB control. The team will use the following methods to generate data and information for this evaluation: Desk review - The team will also collect data before and after project implementation at sites. Baseline data from TB LINC project and data collected at the national level such as the National Prevalence survey and NTP data from 2005 to 2011 will be available for review. Conduct site visits and interviews to selected USG-assisted sites and non-project sites to compare the outcomes and performances between the two groups. Focus Group Discussion for data validation. Descriptions of sources of information and methods used are to be included in the final report. Desk Review and development of data collection tool The evaluation team will carry out a desk review of materials generated by USAID/Philippines and each project (TB LINC, SPS, USP PQM, WHO, other CAs) as preparation for team background reading and prior to the country visit. This task will include review of various USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 75 sources of information such as quarterly reports, annual reports, and other relevant documents. This desk review will help to organize the materials for external evaluation analysis and review of progress. It will allow the team to quickly digest the wealth of information and maximize their time while in country. USAID/Philippines will provide the team with historical program documents including USAID health strategy information, “request for assistance” documents or other scopes of work, and information about other USAID health projects (such as HealthPro, HealthGov and SHIELD). The implementing partners will be responsible for sharing all projects’ reports and work plans, as requested. Both USAID/Philippines and the partners will assist in getting necessary government and NTP documents and reports, such as the Philippines National TB Strategic Plan (PhilPACT and its predecessor), annual reports and data from regions and local units, etc. During the desk review and in advance of the country visit, the evaluation team will be provided with project work plans, reports, and data, and data from the National TB program. The team will be expected to extract relevant components of the reports for the purposes of the evaluation. The evaluation team will do analysis and confirmation of project-specific data either during the desk review or during the country visit. The evaluation team can request additional reports and data from USAID Mission and partners, and the Mission will make every effort to ensure that the requested information is provided to the team before the country visit. Partners will be instructed to provide the team with all reasonable information requested. To ensure gender responsiveness, the evaluation team should be guided by the new USAID’s Gender Policy10. During the desk review time, the evaluation team will be expected to participate in several Team Planning conference calls with USAID/Philippines to: Review the goals and objectives of the assignment. Develop a workplan, evaluation design with timeline. Review the background material. Clarify team members’ roles and responsibilities. Develop gender-sensitive data collection methods and instruments. Develop a country itinerary and schedule of interviews. Based on an agreed list of outcome/output indicators and interview questions, the evaluation team will develop a data collection instrument. This instrument will serve as the major tool by which the team will collect data (before and after USAID intervention; USAID-site vs. non￾USAID-site), both quantitative and qualitative. The final data collection instrument and proposed work plan will be discussed with USAID and other stakeholders and finalized before the country visit. For all people-level indicators, collection of data will be sex-disaggregated. Country Visit 10 USAID Gender Equality and Female Empowerment Policy. Accessed at http://energy.gov/sites/prod/files/USAID%20GenderEqualityPolicy%20March%202012.pdf . 76 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) The evaluation team will spend approximately 2.5 weeks in the Philippines interviewing key stakeholders and partners and visiting project sites and non-project sites. The country visit will start with a Team Planning Meeting held at USAID/Philippines on the first day to: Review the goals and objectives of the assignment. Review the status of the workplan. Clarify any issues on the background material. Reiterate team members’ roles and responsibilities. Review and make last revisions to the data collection methods and instruments. Review and finalize the country itinerary and schedule of interviews. Interviews/visits will be made to (but not limited to): USAID Mission. Other USG agencies working in health.. Senior Philippine Government Department of Health Officials. Other Government agencies officials and staff engage in TB Control (CUP). National TB Program manager and staff. Food and Drugs Administration. National TB Reference Laboratory manager and staff. Other key TB institutions like the Lung Center, TB Treatment Centers, DST and Culture centers, etc. International and local institutions implementing TB activities. Other funders supporting TB activities like the Global Fund. TB program staff in regions and local levels (M/F). Health care workers and managers (M/F). Clients, treatment partners (M/F). Organization of people living with the disease. Health facilities and laboratories. Local chief executives, local leaders (M/F). Set of questions appropriate to specific interviewees will be used. The different USAID TB projects cater to different clients. (For example, the SPS project assists the Lung Center of the Philippines in SLD logistic management, NTRL in laboratory strengthening and DOH in eTB Manager application; WHO assist the national NTP manager). Interviews should be designed to make sure that useful information can be generated to determine if desired outcome has been USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 77 achieved. A quasi experiment can be done in TB LINC assisted sites with non-assisted sites of comparable characteristics. Interviewers are free to ask normative questions, which are relevant to determine facilitating factors or barriers affecting implementation of the projects. Debrief Report At the end of the country visit, the team will provide an oral PowerPoint presentation on the major findings and analysis to USAID/Philippines, DOH and other relevant partners for validation prior to the completion of final report. Analysis should be based on the result of the data review and interviews vis-a-vis expert’s judgment and logical analysis. Participants’ bias should be determined in order to come up with more logical and objective analysis. The team will provide a draft written report within an agreed upon length of time (approximately 3 weeks) to USAID/Philippines. After Mission staff review of the initial draft report, the final report will be submitted. A dissemination of the final report will be conducted with DOH-NTP and relevant stakeholders. IX. TEAM COMPOSITION AND QUALIFICATIONS Senior TB Expert (Team Leader): Expert in the field of international TB control with excellent understanding of the global strategy and its implementation. S/he should have at least 5 years of experience monitoring and evaluating various TB programs, especially in Asia. Previous experience in working with USAID is desirable, but not necessary. S/he should have expertise in the overall Stop TB Strategy (i.e., DOTS generalist), but may have expertise in one of the following specialties: PMDT, private sector engagement, ACSM, laboratory and diagnosis, policy, planning and health system. TB Specialists (4): The team should include an additional four other TB specialists with expertise in areas listed above. There should be no duplication in the possessed expertise in order to cover all areas/components needed. Experience in TB control programs in Asia is desired. There should at least one person on the team experienced with collecting and analyzing TB data. The team should have at least one local consultant and one TB technical expert from USAID/W out of the four TB specialists. The following general development skills should be covered by one of the 5 experts including policy, health system strengthening, program quality assurance, advocacy, and gender. USAID/Philippines, with support from the USAID/Washington TB Team, will provide suggestions on possible team members and will provide the final concurrence for all proposed team members. A local administrative coordinator will be hired to provide support to the Team while in country and until such time that his/her services are necessary but not to exceed the official period of engagement. X. PERIOD OF PERFORMANCE The assignment will be conducted from on/around March 31, 2012 through on/around June 22, 2012. A six-day work week in-country is approved. 78 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Illustrative Table of Level of Effort (LOE) ACTIVITIES/TASK Approximate Dates Level of Effort Background Preparation Development of detailed evaluation plan, methodology, and timeline for the evaluation. This will include, but will not be limited to: initial schedule of interviews, interview guides, site visit, and list of critical documents to be reviewed. Desk review of relevant materials Development of data/information collection instruments and methods for analysis March 31–April 11 10 days Travel day(s) April 12–13 1-2 days In-country work April 14–30 April 14: Team Planning Meeting April 16: Initial meeting/briefing with Mission April 17-29: Field visits/ interviews/information collection, focus group discussion, report drafting April 30: De-briefing activity with USAID and NTP/departure 14 days Travel day May 1 1 day Report writing o/a May 2–10 5 days USAID reviews draft report and submits comments to GH Tech Bridge (6 business days) May 11–18 Final report writing and submission May 19–25 5 days Total: Up to 37 days XI. DELIVERABLES 1. Minutes/summary from all team planning calls. 2. Detailed evaluation plan, methodology, and timeline for the evaluation. Final data collection instruments and method of analysis. 3. Interview and site visit summary. 4. Detailed evaluation report with summary of findings, including results and indicators for the project, feedback on performance and implementation, and recommendations for improved implementation and results. a. A debriefing will be held with initial findings on the last day of the country visit. The debriefing will be accompanied by an oral PowerPoint presentation. b. Draft Report: A draft report will be provided to USAID/Philippines no later than May 10, 2012. The report should not exceed 50 pages with an executive summary of no more than 5 pages. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 79 c. USAID/Philippines Feedback on Draft Report: USAID will have 6 business days to review the draft report and provide feedback on Friday, May 18. d. Revised Report: The revised executive summary and report will be submitted by the team after they receive comments from USAID/Philippines on the first draft by Friday, May 25th. The revised report must meet the applicable standards provided in the “Checklist for Assessing USAID Evaluation Reports” in Appendix A. e. Final Content Approval: USAID/Philippines will have 3 business days to review the revised report and provide their approval of the final content by May 31, 2012. USAID/Philippines will highlight if there is any procurement sensitive information in the report so that information can later be removed from the final publishable report. f. Final Publishable Report: GH Tech Bridge will provide the edited and formatted final document on/about June 16, 2012 after USAID provides final approval of the content. Procurement sensitive information will be removed from the final report and incorporated into an internal USAID Memo. The remaining report will then be released as a public document on the USAID Development Experience Clearinghouse (DEC) (http://dec.usaid.gov) and the GH Tech project web site (www.ghtechproject.com). The contractor shall submit 5 hard copies and one electronic copy of the final report to the USAID/Philippines. XII. RELATIONSHIPS AND RESPONSIBILITIES GH Tech will coordinate and manage the evaluation team and will undertake the following specific responsibilities throughout the assignment: Recruit and hire the evaluation team. Make logistical arrangements for the consultants, including travel and transportation, country travel clearance, lodging, and communications. USAID/Philippines will provide overall technical leadership and direction for the evaluation team throughout the assignment and will provide assistance with the following tasks: Before In-Country Work SOW. Respond to queries about the SOW and/or the assignment at large. Consultant Conflict of Interest (COI). To avoid conflicts of interest or the appearance of a COI, review previous employers listed on the CVs for proposed consultants and provide additional information regarding potential COI with the project contractors evaluated/assessed and information regarding their affiliates. Documents. Identify and prioritize background materials for the consultants and provide them to GH Tech, preferably in electronic form, at least one week prior to the inception of the assignment. Local Consultants. Assist with identification of potential local consultants, including contact information. Site Visit Preparations. Provide a list of site visit locations, key contacts, and suggested length of visit for use in planning in-country travel and accurate estimation of country travel line items costs. 80 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) – Lodgings and Travel. Provide guidance on recommended secure hotels and methods of in-country travel (i.e., car rental companies and other means of transportation) and if necessary, identify a person to assist with logistics (i.e., visa letters of invitation etc.). USAID/Philippines will make hotel reservations in Manila for all consultants, including the USAID/Washington Team Member, Meghan Holohan. GH Tech Bridge and Meghan Holohan will provide flight itineraries to USAID/Philippines as soon as they are available. All consultants will reside in the same hotel, the Pan Pacific, which is close to the Mission. The hotel will provide transport to/from the airport and should provide transport to/from the Mission. – Meghan Holohan will not require any support from GH Tech Bridge. During In-Country Work Mission Point of Contact. Throughout the in-country work, ensure constant availability of the Point of Contact person and provide technical leadership and direction for the team’s work. Meeting Space. Provide guidance on the team’s selection of a meeting space for interviews and/or focus group discussions (i.e., USAID space if available, or other known office/hotel meeting space). Meeting Arrangements. Assist the team in arranging and coordinating meetings with stakeholders. Facilitate Contact with Implementing Partners. Introduce the evaluation team to implementing partners and other stakeholders, and where applicable and appropriate prepare and send out an introduction letter for team’s arrival and/or anticipated meetings. After In-Country Work Timely Reviews. Provide timely review of draft/final reports and approval of deliverables. Mission Contact Person Dr. Yolanda E. Oliveros, MPH MHSA Development Assistance Specialist USAID-OH, Philippines Email: yoliveros@usaid.gov Phone Number: +632-552-9869 XIII. COST ESTIMATE GH Tech will provide a cost estimate for this activity. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 81 SOW APPENDIX 1: CHECKLIST FOR ASSESSING USAID EVALUATION REPORTS Good Practice Elements of an Evaluation Report EVALUATION REVIEW FACTOR 1. Does the evaluation report have a cover sheet attached indicating the type of evaluation conducted (e.g., performance evaluation or impact evaluation) and general design? 2. If a performance evaluation, does the evaluation report focus on descriptive and normative evaluation questions? 3. If the evaluation report uses the term “impact evaluation,” is it defined as measuring the change in a development outcome that is attributable to a defined intervention (i.e. impact evaluations are based on models of cause and effect and require a credible and rigorously defined counterfactual)? 4. Regardless of the type of evaluation, does the evaluation report reflect use of sound social science methods? 5. Does the report have a Table of Contents (TOC)? 6. Do Lists of Figures and Tables follow the TOC? 7. Does the report have a Glossary of Terms – Are abbreviations limited to the essential? 8. Is the date of the report given? 9. Does the body of the report adhere to the 20 page guide? 10. Is the report well-organized (each topic is clearly delineated, subheadings used for easy reading)? 11. Does the report’s presentation highlight important information in ways that capture the reader’s attention? 12. Is the report well written (clear sentences, reasonable length paragraphs, no typos, acceptable for dissemination to potential users)? 13. Does the evaluation report focus on the essential issues concerning the key questions, and eliminate the “nice to know”, but not essential information? 14. Does the evaluation report discuss any issues of conflict of interest, including the lack thereof? 15. As applicable, does the evaluation report include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team? 16. Does the evaluation report begin with a 3- to 5-page stand-alone summary of the purpose, background of the project, main evaluation questions, methods, findings, conclusions, recommendations and lessons learned (if applicable) of the evaluation? 17. Does the Executive Summary concisely state the main points of the evaluation? 18. Does the Executive Summary follow the rule of only saying what the evaluation itself says and not introducing new material? 19. Does the report introduction adequately describe the project? – Does the introduction explain the problem/opportunity the project was trying to address? 82 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) EVALUATION REVIEW FACTOR – Does the introduction show where the project was implemented (physical location) through a map? – Does the introduction explain when the project was implemented? – Are the “theory of change” or development hypotheses that underlie the project explained? (Does the report specify the project’s inputs, direct results (outputs), and higher level outcomes and impacts, so that the reader understands the logical structure of the project and what it was supposed to accomplish?) – Does the report identify assumptions underlying the project? – Does the report include sufficient local and global contextual information so that the external validity and relevance of the evaluation can be assessed? – Does the evaluation report identify and describe any critical competitors to the project that functioned at the same time and in the project’s environment? – Is USAID’s level of investment in the project stated? – Does the evaluation report describe the project components funded by implementing partners and the amount of funding? 20. Is the purpose of the evaluation clearly stated? 21. Is the amount of USAID funding for the evaluation indicated? 22. Are all other sources of funding for the evaluation indicated as well as the amounts? 23. Does the report identify the evaluation team members and any partners in the evaluation? 24. Is there a clear statement of how the evaluation will be used and who the intended users are? 25. Are the priority evaluation questions presented in the introduction? 26. Does the evaluation address all evaluation questions included in the Statement of Work (SOW)? – Are any modifications to the SOW, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline indicated in the report? – Is the SOW presented as an annex? – If so, does the annex include the rationale for any change with the written sign-offs on the changes by the technical officer? 27. Does the report provide a clear description of the evaluation’s design? – Is a design matrix or similar written tool presented in an annex that shows for each question/sub-question the measure(s) or indicator(s) used to address it, the source(s) of the information, the type of evaluation design, type of sampling if used, data collection instrument(s) used, and the data analysis plan? 28. Does the report state the period over which the evaluation was conducted? 29. Does the report state the project time span (reference period) covered by the evaluation? 30. Does the evaluation report indicate the nature and extent of consultation on the evaluation design with in-country partners and beneficiaries? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 83 EVALUATION REVIEW FACTOR 31. Does the evaluation report indicate the nature and extent of participation by national counterparts and evaluators in the design and conduct of the evaluation? 32. Does the report address each key question around which the evaluation was designed? 33. Is at least one of the evaluation questions directly related to gender analysis of outcomes and impacts? – Are data sex-disaggregated? 34. In answering the questions, does the report appropriately use comparisons made against baseline data? 35. If the evaluation is expected to influence resource allocation, does it include information on the cost structure and scalability of the intervention, as well as its effectiveness? – As appropriate, does the report include financial data that permits computation of unit costs and analysis of cost structure? 36. Is there a clear description of the evaluation’s data collection methods (summarized in the text with the full description presented in an annex)? – Are all tools (questionnaires, checklists, discussion guides, and other data collection instruments) used in the evaluation provided in an annex? – Does the evaluation report include information, as appropriate, on the pilot testing of data collection instruments? – Does the evaluation report include information, as appropriate, on the training of data collectors? 37. Are all sources of information properly identified and listed in an annex? 38. Does the evaluation report contain a section describing the limitations associated with the evaluation methodology (e.g. selection bias, recall bias, unobservable differences between comparator groups, small samples, only went to villages near the road, implementer insisted on picking who the team met with, etc)? 39. Does the evaluation report indicate the evaluation methodology took into account the time, budget, and other practical considerations for the evaluation such as minimizing disruption and data burden? 40. Does the report have sufficient information to determine if the evaluation team had the appropriate methodological and subject matter expertise to conduct the evaluation as designed? 41. If an impact evaluation was designed and conducted, does the evaluation report indicate that experimental methods were used to generate the strongest evidence? Or does the report indicate that alternative methods for assessing impact were utilized and present the reasons why random assignment strategies were not feasible? 42. Does the evaluation report reflect the application and use to the maximum extent possible of social science methods and tools that reduce the need for evaluator-specific judgments? 43. Does the evaluation scope and methodology section address generalizability of the findings? 44. Are percentages, ratios, cross-tabulations, rather than raw data presented, as appropriate? 45. When percentages are given, does the report always indicate the number of cases used to calculate the percentage? 84 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) EVALUATION REVIEW FACTOR – Is use of percentages avoided when the number of cases is small (<10)? 46. Are whole numbers used or rounding-off numbers to 1 or 2 digits? 47. Are pictures used to good effect? – Relevant to the content – Called out in the text and placed near the call-out 48. Are charts and graphs used to present or summarize data, where relevant? – Are the graphics easy to read and simple enough to communicate the message without much text? – Are they consistently numbered and titled? – Are they clearly labeled (axis, legend, etc?) – Is the source of the data identified? – Are they called out in the text and correctly placed near the call-out? – Are the scales honest (proportional and not misleading by virtue of being “blown-up”)? 49. Are FINDINGS specific, concise and supported by strong quantitative and qualitative evidence? – As appropriate, does the report indicate confirmatory evidence for FINDINGS from multiple sources, data collection methods, and analytic procedures? 50. Are adequate data provided to address the validity of the “theory of change” or development hypothesis underlying the project, i.e., cause and effect relationships? 51. Are alternative explanations of any observed results discussed, if found? 52. Are unplanned results the team discovered adequately described? 53. Are opinions, conclusions, and recommendations kept out of the description of FINDINGS? 54. Is there a clear distinction between CONCLUSIONS and FINDINGS? 55. Is every CONCLUSION in the report supported by a specific or clearly defined set of FINDINGS? 56. Are the CONCLUSIONS credible, given the FINDINGS the report presents? 57. Can the reader tell what CONCLUSIONS the evaluation team reached on each evaluation question? 58. Are RECOMMENDATIONS separated from CONCLUSIONS? (Are they highlighted, presented in a separate section or otherwise marked so that the reader sees them as being distinct?) 59. Are all RECOMMENDATIONS supported by a specific or clearly defined set of FINDINGS and CONCLUSIONS? (Clearly derived from what the evaluation team learned?) 60. Are the RECOMMENDATIONS practical and specific? 61. Are the RECOMMENDATIONS responsive to the purpose of the evaluation? 62. Are the RECOMMENDATIONS action-oriented? 63. Is it clear who is responsible for each action? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 85 EVALUATION REVIEW FACTOR 64. Are the RECOMMENDATIONS limited/grouped into a reasonable number? 65. Did this evaluation include lessons that would be useful for future projects or programs, on the same thematic or in the same country, etc.? 66. Are the LESSONS LEARNED highlighted and presented in a clear way? 67. Does the report indicate who the lessons are for? (e.g., project implementation team, future project, USAID and implementing partners, etc.) 68. Does the evaluation report give the appearance of a thoughtful, evidence-based, and well organized effort to objectively evaluate what worked in the project, what did not and why? 69. As applicable, does the evaluation report include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team? 70. Is the evaluation report structured in a way that will promote its utilization? 71. Does the evaluation report explicitly link the evaluation questions to specific future decisions to be made by USAID leadership, partner governments and/or other key stakeholders? 72. Does the evaluation report convey the sense that the evaluation was undertaken in a manner to ensure credibility, objectivity, transparency, and the generation of high quality information and knowledge? REPORT DISSEMINATION 73. Have all evaluation team members signed a statement attesting to a lack of conflict of interest, or describing and existing conflict of interest relative to the project being evaluated? 74. Was the Report Submitted to the Development Experience Clearing House (DEC)? 75. Has a dissemination plan been developed for this report? 76. Is the report widely shared to interested stakeholders? DEFINITIONS: Performance evaluation: focuses on descriptive and normative questions: what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management and operational decision making. Performance evaluations often incorporate before-after comparisons, but generally lack a rigorously defined counterfactual. Impact evaluation: measures the change in a development outcome that is attributable to a defined intervention; impact evaluations are based on models of cause and effect and require a credible and rigorously defined counterfactual to control for factors other than the intervention that might account for the observed change. Impact evaluations in which comparisons are made between beneficiaries that are randomly assigned to either a―treatment or a―control group provide the strongest evidence of a relationship between the intervention under study and the outcome measured. Theory of change: A tool to design and evaluate social change initiatives. It is a blueprint of the building blocks needed to achieve long-term goals of a social change initiative. 86 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Development Hypothesis: Identifies causal linkages between USAID actions and the intended Strategic Objective (highest level result). External Validity: The degree to which findings, conclusions, and recommendations produced by an evaluation are applicable to other settings and contexts. Findings: Empirical facts collected during the evaluation Conclusions: Interpretations and judgments based on the findings Recommendations: Proposed actions for management. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 87 SOW APPENDIX 2: ILLUSTRATIVE EVALUATION QUESTIONS Performance and Impact: What has been the overall impact of USAID TB programs on TB case notification and treatment outcomes in the Philippines? Is TB prevention and control in the Philippines better now than it was 5 years ago? Compare areas with and with no USAID TB programs. How has each of the TB programs and TB-related activities contributed to meeting the goals and objectives of the National TB Program? Overarching Issues: How have projects coordinated with other partners (USAID/non-USAID, TB, other health and non -health partners)? Describe the synergies of the TB projects with other development partners working for TB in terms of activities, support, sites, etc. Have the TB projects been able to leverage with The Global Fund for Health (TGF) activities? Have the TB projects provided any interventions to ease bottlenecks of GF activities? How and to what extent have partners’ capacity been strengthened? General TB DOTS Interventions: Have TB case detection increased in project sites? Nationally? Provide baseline, target and actual data sets. Have TB treatment outcomes improved in project sites? Nationally? Provide baseline, target and actual data sets. What attention has been paid to vulnerable populations, like indigenous groups, people with disabilities, plantation and mining populations, geographically isolated and depressed areas, and congregate settings of the urban poor? Have TB case notifications or treatment outcomes in these groups improved? Provide sex-disaggregated data for all people level indicators. Have TB projects been able to improve the capacity of the Department of Labor and Employment (DOLE) to advocate, enforce and monitor DOTS in the workplace? Are there adequate and skilled government health staff managing TB patients in USAID assisted sites? Have TB projects been able to enhance information system at national and local levels? Policy, Financing and Regulatory Environment: Were the TB project-supported LGUs able to develop and implement local policies to support the program? 88 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Do TB project-supported LGUs have increased financing for TB? Were the TB projects able to leverage resources provided by the national and local governments? Are there more PhilHealth accredited facilities (public vs. private)? Are there more TB patients availing of PhilHealth DOTS benefits in the project sites? (M/F) Have TB projects strengthened the capacity of DOH, Food and Drugs Administration (FDA), PhilHealth in their regulatory functions? How? Private Sector Involvement: Are more private sector providers (practitioners, pharmacies, work places, etc.) notifying TB patients? Referring to public DOTS facilities? Are more private sector physicians adhering to the International Standard on TB Care? Programmatic Management of Drug Resistant TB (PMDT): Have USAID TB projects helped to increase the number of MDR-TB patients diagnosed? Have USAID TB projects helped to increase the number of MDR-TB patients initiating and successfully completing treatment? (M/F) Have USAID TB projects assisted in the expansion of treatment centers and in decentralization of MDRTB management? What social support interventions for PMDT, if any, have USAID TB projects supported? What has been the impact? Advocacy, Communications, Social Mobilization: Are DOTS facilities better utilized? Have misperceptions/misconception and stigma on TB among the general population decreased? Are more persons with TB symptoms seeking treatment at public facilities? (M/F) Is communication material available at DOTS facilities (public and private)? If so, has it made an impact? Are participating providers able to provide TB related information and counseling? Are communities in USAID assisted sites more involved in TB control and prevention activities? TB Laboratory Strengthening and Diagnostic Network: How have USAID TB projects supported TB laboratory (both public and private) strengthening nationally? Locally? Has external quality assurance of smear microscopy indicators improved over the past five years? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 89 Do TB suspects have adequate access to TB diagnosis (labs, x-rays, etc.)? Are there quality TB smear microscopy labs in local settings? Are they staffed by trained microscopists? Have the projects supported rational lab diagnostic strategies at national level? At regional levels? Anti-TB Drug Supply and Network: Have there been any TB drug stockouts during the five-year life of the USAID-supported TB projects? What USAID-supported interventions have been implemented to support the TB drug management and distribution system? Are these interventions sustainable? Have the projects supported information and TB drug management systems (forecasting, procurement, distribution, utilization) strengthening to improve the capacity of the national and local staff to monitor the availability and quality of anti-TB drugs? National TB Control Program Capacity Building/Strengthening: What has been the support to the NTP in terms of capacity building and technical assistance (guideline, policy development, training, etc.)? What are the needs that have not been met? What policy instruments and programmatic tools have been developed and can be attributed to the USAID TB projects? What is the relationship between the NTP manager and staff and the various USAID TB projects? To what extent have USAID TB projects contributed to high quality technical assistance at the national level? Regional TB Control Program Capacity Building/Strengthening: What has been the support to the Regional TB program in terms of capacity building and technical assistance (guideline, policy development, training, supervision, etc.)? What is the relationship between the Regional staff and the various USAID TB projects? To what extent have USAID TB projects contributed to high quality technical assistance at the regional level? Local TB Control Program Capacity Building/Strengthening: What has been the support to the local TB programs in terms of capacity building and technical assistance (guideline, policy development, training, etc.)? What are the needs that have not been met? What local policy instruments and operational improvement tools can be attributed to the USAID TB projects? What is the relationship between the local health facility staff and the various USAID TB projects? 90 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) To what extent have USAID TB projects contributed to high quality technical assistance at the local level? To what extent have USAID TB projects contributed to capacity building of community based organizations (CBOs), non-government organizations (NGOs), and civil societies in participating in local TB control initiatives? Sustainability Mechanism: Have the projects developed a process to ensure sustainability of the systems and interventions after the project life? What are the plans of the DOH and local governments for sustaining systems and interventions developed under the projects? What are the plans for sustaining and/or increasing the locally generated funding for TB control? Gender: How have gender considerations been integrated in USAID’s TB programs and TB-related activities? What are the differential effects of the project on male and female beneficiaries? What gender issues were identified in the USAID TB programs and how were they addressed? To what extent has the TB LINC Gender Action Plan been implemented? What was lacking in the project design and/or implementation that would have improved gender considerations? Lessons Learned: What are the overall lessons learned from the implementation and evaluation of TB LINC and the other USAID TB and TB-related projects? What are the best practices from the TB LINC and other TB programs that could be adopted and replicated by follow-on activities? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 91 ANNEX B: KEY CONTACTS Name Title Awang Barangay Health Station Chan, Rosana F. Medical Technologist de Cruz, Michelle V. Provincial Cold Chain Manager Sinsuat, Minette Public Health Nurse Baguio City Health Department Allaga, Judith N. Director Guanzon, Rebecca B. Nurse IV Lopez, Valeriano V. Assistant Regional Director, Cordillera Administrative Region Magsino, Ruby Marie Medical Technologist Maquiling, Randel A. Volunteer Nurse Rajes, Florence G. City Health Officer Tiwakew, Helen A. Nurse Tubera, Donnabel L. Medical Officer Tiwing, Gaudenicio T. Engineer Banaybanay Rural Health Unit Del Campo, Rhodara A. Nurse III Francisco, Diego S. Jr. Municipal Medical Technologist Benguet General Hospital Basinga, Marieta O. Nurse Naguen, Purita G. NTP Nurse Coordinator Tagudar, Daisy Mae C. Medical Specialist, Head PPMD Unit Bureau of Jail Management and Penology Aranas, Emilie P. Director, Operations Castelo, Marlon B. Deputy, DIWD Domgo, Doris R. Deputy Chief Maderazao, Elna Deputy, Health Service Mamaril, Diony OIC-Chief, BSMP 92 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Name Title Bureau of Working Condition, Department of Labor and Employment Valeros, Marc Medical Officer Capiz Provincial Health Office Biclar, Leo C. Provincial Medical Technologist Bolida, Evelyn Medical Specialist Delfin, Rose Marie DOH Representative Delfin, Samuel Provincial Health Officer Quimpo, Charmaine G. Nurse TB Coordinator Center for Health Development (CHD) Region VI Gimotea, Edith Regional TB Coordinator Madarieta, Susana K. Regional Director Monicimpo, Emilia P. OIC, Assistant Regional Director CHD III (Central Luzon) Guinto, Virginia C. Nurse II, Regional NTP Nurse Coordinator Pangan, Renato S. MS III, Regional Medical TB Coordinator CHD XI (Davao) Del Rosario, Jose Regional Medical Laboratory Technician Uy-Gelito, Evelyn Nurse III, Regional NTP Nurse Coordinator Yumang, Annabelle P. Regional Medical TB Coordinator City Camp Health Center Johnson Alad, Anges Nurse II Posadas, Catherine Medical Officer Cotabato Regional Medical Centre de Paralta-Yambao, Helen Chief of Hospital Kalim, Sherjan Ass. Pathologist, Head Blood Bank Tesoro, Belma M. Medical Technologist Department of Health, Autonomous Region in Muslim Mindanao Kamid, Noraina NTP Nurse Coordinator Raki-in, Sadaila K. NTP Medical Coordinator Sinolinding, Kadil Jojo M. Regional Secretary of Health USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 93 Name Title Department of Health, Philippines Bobier, Ed Systems Analyst, Information Management Service Esteban, Cherrie ISA, Information Management Service Gutierrez, Andro Systems Analyst, ITIS Project Sosito, Jocelyn T. Senior Health Program Officer, Bureau of International Health Cooperation Turbolencia, Virna IT Officer, Information Management Service Vianzon, Rosalind National TB Program Manager Estevez Memorial Hospital, Legaspi City Camba, Phoebes R. Family Physician Dinela, Khristian Andrew Staff Nurse, TB Moran, Alfredo M. Chief of Clinics Fernando B. Duran Sr. Memorial Hospital Babasa, Gloria F. Nurse Bolo, Renato B. Medical Officer Dorion, Maria Liduvina F. Provincial Health Officer Eco, Rica B. Nurse Esponilid, Cyril Medical Officer Estavillo, Josephine Nurse Labayo, Lourdes C. Medical Technologist Navion, Erwin Medical Officer Food and Drug Administration (FDA) of the Philippines Lazo, Suzette H. Director General Health Policy Development Program (HPDP) Mabang, Hexsan O. HPDP Fellow Panelo, Carlo Deputy Chief of Party Solon, Orivlle Jose C. Chief of Party Health Promotion and Communication Project (HealthPRO) Alba, Rhea M. Community Mobilization Specialist Manuel, Cecilia L. Field Operations Director 94 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Name Title Sacci, Inna Chief of Party Iloilo Provincial Health Office Quiñon, Maria Socorro C. Provincial Health Officer Trabado, Patricia Grace S. Provincial Health Officer Villar, Myrna Provincial Nurse Coordinator for TB Ivisan Municipal Health Center Andara, Mignon S. Municipal Health Officer Andrada, Gloria Public Health Nurse Qpmebo, Roselyn Medical Technologist Linking Initiatives and Networking to Control Tuberculosis (TB LINC) Alcantara, Grace Private Sector Specialist Amiana, Maru Area Coordinator, Laguna Balacar, Alicia B. Communications Specialist Batangan, Dennis Health Policy and Finance Specialist Bautista, Fidel C. Monitoring and Evaluation Specialist Bucad, Averdin T. Field Coordinator, Calaca Caccam, Eugene Advocacy, Communications, and Social Mobilization Specialist Castillo, Dolores C. Chief of Party Delosa, Sylvia Area Coordinator, Maguindanao Escaño, Emily O. TB LINC Office Manager Feliscuzo, Michelle Medical Technologist., Assisstant Area Coordinator, Maguindanao Jalandra, Sharlence April G. TB LINC Field Assistant Mabasa, Pilar F. Quality Assurance Specialist Masulit, Aniette P. Monitoring and Evaluation Officer Morales, Mona Lisa TB LINC Office Manager Villanueva, Andre Deputy Chief of Party Yu, Teodoro Jr. Field Coordinator, Mindanao Lucban Health Center Lopaz, Evangelina D. Nurse II USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 95 Name Title Tabo-oy, Edna A. Medical Officer Lung Centre of the Philippines Balanagva, Vincent Administrative Manager Basilio, Ramon P. Facility Operation Coordinator Lofranco, Vivian Operations Manager, PMDT Maguindanao Integrated Provincial Health Office and CH Team Angias, Amelia Barangay Health Volunteer Balayman, Alejandra Barangay Health Volunteer Kadatuan, Karen Heath Education and Promotion Macapeges, Geraldine Chief, Technical Services Division Mangelen, Marissa Public Health Nurse Olivo, Gloria P. H. Nurse, Information System Señase, Jean Gisela Provincial NTP Coordinator Tenorio, Rebecca Public Health Nurse Mayor’s Office, Ligao City Gonzalez, Linda P. Mayor Municipal Health Office and Health Centre, Guinobatan Dayandanc, Ditas N. Medical Technologist de los Reyes, Tirso Rural Health Physician Limos, Joana M. Municipal Health Officer Sevillano, Marylyn N. Nurse Municipal Health Office and Health Centre, Juban, Sorsogon Apin, James V. Medical Health Officer Rebusino Runnez, John RHP Registered Health Municipal Health Office and Health Centre, Ligao City Atutubo, Catalina Nurse Coordinator TB-DOTS Hao, Vilma A. City Health Officer Municipal Health Office and Rural Health Centre, Casiguran, Sorsogon Desdir, Gerald D. Nurse 96 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Name Title Hallig, Epifania P. Municipal Health Officer Lorilla, Rosarie C. Medical technologist Municipal Office and Health Centre, Legaspi City Evasco, Sarah E. Nurse Gillego, Fulbert Alec R. City Health Officer Miranda, Maria Fe Liza Medical Technologist Nabunturan, Compostella Valley Rubillos, Daniello O. Municipal Health Officer National TB Reference Lab Ama, Maria Cecilia Medical Specialist Burgomio, Kathrine Mae PMDT Galit, Marienella P. Senior Research Specialist Narisco, Cristino R. Science Research Specialist Tan, Lriso A. Administrative Officer Villarico, Cristina A. Bacteriologist Oton Municipal Health Center Camus, Rafael S. Contractual Nurse Escrupulo, Marilou T. Public Health Nurse Jocsing, Maria Nonita F. Public Health Nurse Nismal, Mary Jane Municipal Health Officer (MO V) Octavio, Maria Jolly E. Public Health Nurse Philippine Business for Social Progress (PBSP) Nalda, Reno Carter Global Fund Project Operations Manager Philippine Coalition Against Tuberculosis (PhilCAT) Sarmiento, Amelia Executive Director Philippine Health Insurance Corporation (PhilHealth) Banzon, Eduardo P. PhilHealth President and Chief Executive Officer Diaz, Gilda Salvacion Senior Manager, International and Local Cooperation Nieva, Maria Lourdes Project Development Officer, International and Local Cooperation USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 97 Name Title Reganit, Teresa Project Development Officer, International and Local Cooperation Roan, Giovanni M. Medical Officer, Benefits Development Department Saugay, Marisa J. Manager, Accreditation Department Philippine Tuberculosis Society Inc. (PTSI) Cadena, Elizabeth Executive Director Palasi, Winston A. Director, Field Operations Division Pototan Municipal Health Center Monnagon, Rodina Municipal Health Officer Porchia, Lourdes P. Public Health Nurse Pototan Municipality Perez, Pablo L. Mayor Provincial Health Office, Albay Province Bracia, Gay Gloria B. NTP Coordinator Mendoza, Luis Domingo NTP Medical Coordinator Moyo, Gilda NTP Coordinator Provincial Health Office, Compostella Valley Ablin, Arvin A. Health Education Promotion Officer Rasonabe, Desiree M. Provincial NTP Nurse Coordinator Provincial Health Office, Davao Oriental Caadiang, Mariele Joyce M. Provincial NTP Nurse Coordinator Sanico, Joy S. Provincial Health Officer; Head, Technical Division Provincial Health Office, Malolos, Bulacan Esquerra-Gomez, Jocelyn Provincial Health Officer Viray, Elizabeth NTP Coordinator, PHO Pulilan Rural Health Unit Cruz, Cecilia Medical Technologist Mercado, Elizabeth PHN Region 12 Centre for Health Development Ang, Venancio Medical Specialist (finance, PPM) 98 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Name Title Condez, Virgie R. Medical Coordinator Convocar, Marlyn W. OIC Director Taruc, Amebella G. Regional NTP Coordinator Roxas City Health Office Aymonsurin, Bonificia Nurse TB Coordinator Funa, Ranier B. PMDT Nurse Gepilano, Maria Esperanza Medical TB Coordinator Lut, Ruby C. Medical Technologist Robles, Amelita R. City Health Officer Sigma Municipal Health Center Abellavito, Melya Midwife Arbis, Elvie M. Administrative Assistant Bocala, Jasmin Marie L. Midwife Capundan, Kristel Jul F. RN Heals Conlu, Susy Grace B. Public Health Nurse Curatcho, Maricar B. RN Heals Degaja, Arnold B. RSI 1 Dejeran, Conney D. RN Heals Delos Santos, Maria Fatima RN Heals Garcia, Katrina RN Heals Peralda, Cheryl M. Midwife Protasio, Lorna Carmelia A. Municipal Health Officer Quirao, Emma Q. Midwife Seraspe, Lucille Midwife Wallero, Anna Marie D. RHMPP Sigma Municipal Office Andaya, Mary T. Mayor St. Rafael Rural Health Unit de Dios, Cristina Vergel Medical Health Officer Pecson, Julie Ann Medical Technologist Villaraga, Suzie PHN USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 99 Name Title Strengthening Pharmaceutical Systems (SPSP)/Management Sciences for Health (MSH) Gabra, Michael Senior Technical Advisor Lagos, Arthur B. Sustainable Health Improvements through Empowerment and Local Development (SHIELD) Barquilla, Emerita G. Monitoring and Evaluation Manager Evidente, Angelita Regional Technical Advisor Gonzaga, Raymundo Deputy Chief of Party Tandigan, Edward B. Quality Assurance Analyst Tarragona Rural Health Unit Andrada, Maria Aliza May B. Municipal Medical Technologist Daruca, Jessica E. Public Health Nurse Silveron, Prosperidad P. Municipal Health Officer United States Pharmacopeia (USP) / Promoting the Quality of Medicines (PQM) Olivarez, Maria Katrina Consultant for Philippine Activities USAID/Philippines Aeschliman, Reed Deputy Mission Director Chen, Judy Deputy Chief, Office of Health Hirschey, Ann G. Chief, Office of Health Oliveros, Yolanda Development Assistance Specialist Steele, Gloria Mission Director World Health Organization–Office of the Representative Lew, Woo-Jin Medical Officer, Stop TB and Leprosy Elimination Nyunt-U, Soe WHO Representative to the Philippines World Health Organization Regional Office for the Western Pacific Hiatt, Thomas Technical Officer Osuga, Katsunori Medical Officer, Stop TB and Leprosy Elimination Quelapio, Imelda D. MDR-TB Focal Point van Weezenbeek, Catharina TB Team Leader 100 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 101 ANNEX C: REFERENCES Alternative Budget Initiative. Proposal for Health Budget for FY 2012. Retrieved from: http://www.socialwatchphilippines.org/images2/abi2012_health.pdf, accessed May 3, 2012. Catholic Relief Services. Maguindanao Project, supported by USAID 2005–9: Final Report. Catholic Relief Services. (2008.) Maguindanao TB Control Project (2005-2010) Mid-term Evaluation Report. Catholic Relief Services. (2006.) TB KAP Study, Maguindanao. Report #2, 2006 and Summary of Final Report. Central Intelligence Agency. (2012.) World Factbook. Updated July 5, 2012, Retrieved May 3, 2012, from https://www.cia.gov/library/publications/the-world-factbook/geos/rp.html. Department of Health. Administrative Order No. 2010-0037, Diagnosis and Treatment Guidelines for Paragonimiasis. Retrieved from: http://home.DOH.gov.ph/ao/ao2010-0037.pdf, accessed May 4, 2012. Department of Health. DOH Launches Door-to-Door Community Health Teams. http://www.gov.ph/2011/10/11/doh-launches-door-to-door-community-health-teams/, accessed May 5, 2012 Department of Health. DOH Launches ‘RN Heals’ Projects to Deploy Nurses to Poor Communities. Retrieved from: http://www.doh.gov.ph/doh_launches_rn_heals, accessed May 5, 2012. Department of Health. (2010.) Health Sector Reform Agenda—Monographs. Manila, Republic of the Philippines, (DOH HSRA Monograph No. 11). Department of Health. Leading Causes of Mortality. Retrieved from: http://www.DOH.gov.ph/node/198, accessed May 3, 2012. Department of Health. (2005.) Manual of Procedures for the National Tuberculosis Control Program. (4th ed.). Manila, Philippines. Department of Health, Government of the Philippines and Philippine Coalition Against Tuberculosis. (2004.) Comprehensive and Unified Policy for TB Control in the Philippines (CUP 2004). Department of Health, National Epidemiology Center. (2012.) Philippine HIV and AIDS Registry, March 2012. Retrieved from: http://www.DOH.gov.ph/sites/default/files/NEC_HIV_March-AIDSreg2012.pdf, accessed May 3, 2012. Division of Public Health Surveillance and Informatics Division, National Epidemiology Center, Department of Health. (2008.) Field Health Services Information System Annual Report 2008. Manila, Philippines. Garfin, A. “NTP 2012 Update.” Presentation to the 19th Annual Scientific Meeting of the American College of Chest Physicians, Philippine Chapter, April 12, 2012. 102 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) Fitzpatrick, C. and K. Floyd. A Systematic Review of the Cost and Cost Effectiveness of Treatment for Multidrug-Resistant Tuberculosis. PharmacoEconomics, 30(1):63-80(18). Health Policy Development and Planning Bureau, Republic of Philippines. “Toward Financial Risk Protection: Health Care Financing Strategy of the Philippines 2010-2020” Health Sector Reform Agenda Monograph No. 10, 2010. Philippine Coalition Against Tuberculosis. (2008.) Our Journey Together: The Public-Private Mix DOTS Report 2008. Manila, Philippines. Philippines—Country Health Information Profile 2011. Retrieved from: http://www.wpro.who.int/countries/phl/26PHLpro2011_finaldraft.pdf, accessed May 3, 2012. Philippines Global Health Initiative. (2012). Philippines Global Health Initiative (GHI) Strategy 2012– 2016, Draft. Philippines Global Health Initiative. (2012.) Strategy Document, 2012–2116. Philippines National Statistics Office. 2010 Census of Population and Housing. Philippine Tuberculosis Initiatives for the Private Sector (TIPS). (2006.) Harnessing Public-Private Partnerships in TB Control: Final Report. Manila, Philippines. Philippine Tuberculosis Society Incorporated. Retrieved from: http://www.ptsi.org.ph/, accessed May 3, 2012. TB LINC. (2011.) End-of-Project Report (Oct. 1, 2006–Sept. 30, 2011). Manila, Philippines. TB LINC. (2011.) Final Report. Manila, Philippines.Tropical Disease Foundation Inc. (2007.) National Tuberculosis Prevalence Survey (NTPS). Republic of the Philippines. Strengthening Pharmaceutical Systems (SPS) Program, The Philippines. Work Plan 2010. Vianzon, R. Presentation at the Seventh Meeting of the Subgroup on Public-Private Mix for TB Care and Control, in Lille, France. October 23 and 24, 2011. Retrieved from http://www.who.int/tb/careproviders/ppm/L_vianzon.pdf, accessed May 8, 2012. World Health Organization. (2011.) Global Tuberculosis Control Report. “Advocacy, Communication, and Social Mobilization (ACSM): A Handbook for Country Programmes,” the World Health Organization. accessed 3 May 2012. USAID. USAID Evaluation Policy: Evaluation Learning From Experience. (2011) http://www.usaid.gov/evaluation USAID. USAID Policy: Gender Equality and Female Empowerment Policy. http://transition.usaid.gov/our_work/policy_planning_and_learning/documents/GenderEq ualityPolicy.pdf USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 103 ANNEX D: DATA COLLECTION TOOL USAID/PHILIPPINES TB PORTFOLIO EVALUATION DATA COLLECTION TOOL 12 APRIL 2012 Background Tuberculosis (TB) is a leading cause of mortality in the Philippines today, with approximately 75 deaths a day or about three Filipinos dying from TB every hour. There was an estimated 260,000 new cases of TB and 32,000 deaths from TB in 2009. Furthermore, the Philippines ranks ninth on the list of 27 countries with the highest burden of MDR-TB. Despite the continuing challenges, however, TB control has consistently improved over the past decade. The United States Agency for International Development (USAID), through a number of implementing partners and health programs, has supported efforts by the Government of the Philippines and its National TB Control Program (NTP) to prevent and control TB. Over the past five years (2006–2011), USAID has invested approximately USD $32 million in TB-related activities in the Philippines. USAID has initiated an independent, external evaluation of the performance and lessons learned from those investments. The evaluation will determine whether or not USAID-supported interventions have contributed to increased TB case notifications and successful treatment outcomes in the Philippines during the past five years. This document summarizes the plan for that evaluation. Overview of Project Schedule There will be three phases to the TB portfolio evaluation: Phase 1 (April 2-13): Research and evaluation planning. Team members will review information and data related to TB prevention and control activities, challenges, and achievements in the Philippines, with an emphasis on USAID-funded efforts. Phase II (April 14-30): In-country interviews, data collection, and team analysis. Team members will travel to several areas of the Philippines (e.g., Albay, Capiz, Cotabato, Davao, Metropolitan Manila, and Sorsogon) to interview stakeholders at sites delivering USG￾supported (and non-USG supported) TB services. Phase III (May 1-16): Report writing, review, and finalization. Working together in the Philippines and upon return to their home bases, team members will prepare and submit their evaluation of USAID support for TB control in the Philippines. Evaluation Framework During Phase I and Phase II of the TB portfolio evaluation, the team will collect qualitative/quantitative information and data to inform its analysis and the conclusions that it reports on during Phase III. USAID has suggested several areas of inquiry (and provided examples of specific questions) as a guide for Phase III. The team will elaborate on these areas/questions during the project as its understanding deepens; nonetheless, a basic framework 104 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) for data and information collection has been developed and will be used by the team during its national-level and site-specific investigations. The following framework will be completed by the team in preparation for Phase III. National Program Performance 1.1 Is TB prevention and control in the Philippines better now than 5 years ago? 1.1.1 National-level data for the following key TB impact and outcome indicators: Indicator 2006 2007 2008 2009 2010 2011 Case Notification Rate /100,000 pop TB Prevalence Rate TB Mortality Rate Cure Rate TB Treatment Success Rate Treatment Failure Default rate 1.1.2 National-level data on the programmatic management of drug-resistant TB (PMDT): Indicator 2006 2007 2008 2009 2010 2011 Number of PMDT Centers CHDs with PMDT Centers 1.1.3 Number of MDR-TB patients and treatment outcomes Indicator 2006 2007 2008 2009 2010 2011 MDR-TB prevalence (est.) MDR-TB cases diagnosed New Re-treatment All MDR-TB patients enrolled (#) Treatment success rate (%) Death rate (%) Default rate (%) 1.1.4 Data on public-private/public-public mix (PPM) DOTS Indicator 2006 2007 2008 2009 2010 2011 PPM units reporting to the NTP Private Public Total SS+ patients reported from all PPM units Number of TB patients reported from all PPM units USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 105 % contribution of PPM SS+ to national case notification % contribution of PPM all TB patients to national case notification Public hospital contribution to case notification (in number of SS+ cases) 1.1.5 Data on TB control in prison settings Indicator 2006 2007 2008 2009 2010 2011 Number of TB SS+ cases notified (from prisons) Number of MDR-TB cases notified (from prisons) Number of TB SS+ cases enrolled for treatment (in prisons) Number of MDR-TB enrolled for treatment (in prisons) Treatment success rate (in prisons) 1.1.6 Data on the incidence and treatment of TB-infected children Indicator 2006 2007 2008 2009 2010 2011 Number of TB cases in children notified Number of TB cases in children enrolled for treatment 1.2 Are USAID projects effectively coordinated with the NTP and other TB initiatives? 1.2.1 How have USAID-supported projects coordinated their activities with the NTP? What have been the perceived benefits/shortcomings of USAID programs to the NTP? 1.2.2 Is there an official (DOH-initiated) coordinating mechanism? What is the role of this mechanism in USAID support? 1.2.3 What is the role of the Global Fund Country Coordination Mechanism (CCM) in the coordination of the USAID – supported TB projects? 106 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 1.2.4 How have USAID-supported projects coordinated with other partners (both USAID- and non-USAID supported partners working in TB and other areas of health/development)? How do NTP partners (international, national, and local) perceive these efforts to coordinate? 1.3 Have USAID projects leveraged synergies with other initiatives? 1.3.1 Did interviews or background information reveal any synergies between USAID-supported TB projects and other USAID programs (e.g., HPDP, HealthPro, HealthGov, and SHIELD)? 1.3.2 Have USAID-supported TB projects improved the capacity of the Department of Labor and Employment (DOLE) to advocate, enforce, and monitor DOTS in the workplace? 1.3.3 Have USAID-supported projects been able to leveraged Global Fund activities? Have the TB projects provided any interventions to ease bottlenecks to Global Fund implementation? 1.4 What has been the impact of USAID on TB control in the Philippines? 1.4.1 Has the USAID TB “package” of interventions made an impact on increasing TB case notifications across the Philippines? Indicator 2006 2007 2008 2009 2010 2011 TB Symptomatics examined SS+ TB Cases detected Positivity % Number of SS+ enrolled on treatment USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 107 1.4.2 How and to what extent have partners’ capacity been strengthened through USAID support? 1.4.3 What USAID-supported interventions were the most helpful in enabling the NTP to achieve its goals? What interventions were the least helpful? 1.4.4 What are the overall lessons learned and best practices from the USAID TB projects? Site-specific (Provincial/Municipal) Program Performance 2.1 Has TB case detection increased in USAID-supported project sites? 2.1.1 Indicate the number of SS+ TB cases notified for each site visited. Site Population USAID￾supported? SS+ cases notified 2006 2007 2008 2009 2010 2011 2.1.1 Indicate the Case Notification Rate (CNR) for each site visited. Site Population USAID￾supported? Case notification rate 2006 2007 2008 2009 2010 2011 108 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 2.2 Have access and treatment outcomes improved for vulnerable populations? 2.2.1 What vulnerable populations (e.g., indigenous groups, plantation and mining populations, geographically isolated and depressed areas) exist in your site area? 2.2.2 What USAID activities have been implemented to improve access and treatment outcomes for these populations? 2.2.3 What have been the outcomes? Have TB case notifications or treatment outcomes in these groups improved? 2.2.4 Have the number of SS+ cases notified changed for vulnerable populations? (Complete for each site visited). Site name: …………………….. USAID supported: Yes/No SS+ cases notified 2006 2007 2008 2009 2010 2011 Vulnerable populations (define……………………………….) Indigenous populations Plantation populations Mining populations Geographically isolated populations Depressed populations Other populations (define ……………………………………………) USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 109 2.4 Have treatment outcomes improved for vulnerable populations? (Complete for each site visited). Site name: …………………….. USAID supported: Yes/No Treatment success rate 2006 2007 2008 2009 2010 2011 Vulnerable populations (define ……………………………….) Indigenous populations Plantation populations Mining populations Geographically isolated populations Depressed populations Other populations (define ……………………………………………) 2.3 Service delivery 2.3.1 Are there adequate and skilled government health staff delivering services to TB patients in sites visited? Is there any difference between USAID-supported sites and non-supported ones? 2.3.2 Complete the following table for each site visited. Site name: …………………….. USAID supported: Yes/No Number of trained staff providing TB services 2006 2007 2008 2009 2010 2011 Medical Officer Public Health Nurse Laboratory Technologist (Microscopist) Midwife Drug Dispenser Barangay Health Volunteers 110 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) National and Site-specific (Provincial/Municipal) Performance 3.1 Has USAID support enhanced TB information systems? 3.1.1 Has the NTP made any improvements to its TB recording and reporting system during the period being evaluated? If so, did USAID facilitate or support these improvements? 3.1.2 Are sites using electronic or paper recording and reporting registers for TB? Are these registers being used properly and completed accurately, particularly for case notification and treatment data? 3.1.3 Have other information systems to support NTP activities (such as procurement and supply chain management) been introduced or expanded during the period being evaluated? To what extent did USAID support these efforts? Are these systems being used effectively? 3.2 Has USAID support improved the policy environment for TB? 3.2.1 Have new policies related to TB control (e.g., TB in prisons or among children) been developed, rolled-out and implemented with the assistance of the TB projects? What has been the uptake at local/national levels? 3.2.2 Specifically, what is the current status of development/implementation of each of the following TB LINC-supported policies? What were the experiences drawn during their development? Have any impacts been seen to date as a result? Revision of the NTP Manual of Operations Revised NTP guidelines for treatment and drug distribution to children with TB and adults with MDR-TB that ensure appropriate treatment and drug supply for TB in children and MDR-TB among adults. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 111 Estimation and adoption of TB health accounts for planning, resource allocation purposes, policy formulation and review of service delivery strategies. Improved DOH-NTP budget structure, allocation and execution that reflect priorities for budget line items supporting agreements with LGU concerning performance levels. Greater budget coordination for TB control between DOH and attached agencies. Issuance of Executive Orders, DOH Administrative Orders and/or Memorandum of Agreement between DOH, the Department of Budget Management (DBM) and the Department of Finance (DOF) adopting a multi-year budgeting framework for TB. Number of LGUs with increased financing for TB Number of LGUs with full or partial manpower complement. New payment scheme for the PhilHealth TB DOTS OP package with attract greater private sector participants. PhilHealth attains an improved regulatory capacity and procedures for its TB DOTS OP package. Memorandum of Agreement among stakeholders for greater coordination and collaboration in TB control especially at the regional and local levels. 3.2.3 At sites visited by the evaluation team, were USAID-supported LGUs able to develop and implement local policies to support the program during the evaluation period? 3.3 Has USAID support improved financing for TB? 3.3.1 Do LGUs that were supported by USAID TB projects have increased financing for TB? 112 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.3.2 Complete the following table for each LGU visited. LGU name: …………………….. USAID supported: Yes / No PHP 2006 2007 2008 2009 2010 2011 Total LGU Budget* Budget for Health Budget for TB * Exclude funding from other donors, such as the Global Fund. 3.3.3 Were USAID-supported TB projects able to leverage resources provided by the national and local governments? 3.3.4 Are there more PhilHealth accredited facilities (public vs. private)? 3.3.5 Complete the following for each site visited. Site name: …………………….. USAID supported: Yes / No Total # of facilities / # accredited by PhilHealth 2006 2007 2008 2009 2010 2011 Public DOTS facilities / / / / / / Private DOTS facilities / / / / / / 3.3.6 Are there more TB patients availing of PhilHealth benefits in the project sites? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 113 3.3.7 Complete the following for each site visited. Site name: …………………….. USAID supported: Yes / No Total # of TB patients / # PhilHealth beneficiaries 2006 2007 2008 2009 2010 2011 Public DOTS facilities / / / / / / Private DOTS facilities / / / / / / 3.4 Has USAID support improved the regulatory environment for TB? 3.4.1 Have USAID-supported TB projects strengthened the capacity of regulatory bodies (e.g., DOH, Food and Drug Administration/FDA, PhilHealth) in their regulatory functions? How? 3.5 Has USAID support increased private sector involvement in TB control? 3.5.1 Are more private sector providers (practitioners, pharmacies, work places, etc) notifying TB patients? Referring to public DOTS facilities? 3.5.2 Complete the following for each site visited. Site name: …………………….. USAID supported: Yes / No Number of facilities/practitioners 2006 2007 2008 2009 2010 2011 PPM DOTS Public facilities PPM DOTS Private facilities DOTS Referring Private Practitioners Single Practice DOTS Practitioners 114 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.5.3 Have efforts been made to promote the International Standards on TB Care (ISTC) among private practitioners? Has USAID support helped these efforts? 3.5.4 Complete the following for each site visited. Site name: …………………….. USAID supported: Yes/No 2006 2007 2008 2009 2010 2011 Private practitioners trained on ISTC 3.5.5 Has USAID support otherwise improved the capacity of the private sector to work in TB control? 3.6 Has USAID support improved/enabled PMDT? 3.6.1 Have USAID-supported TB projects helped increase the number of MDR-TB patients diagnosed? 3.6.2 Have USAID TB projects helped to increase the number of MDR-TB patients initiating and successfully completing treatment? 3.6.3 Have the USAID TB projects assisted in the expansion of treatment centers and in the decentralization of MDR-TB management? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 115 3.6.4 What patient support interventions for PMDT (e.g. psychosocial, nutritional, transport, ancillary drugs, etc) if any, have USAID TB projects supported? What has been the impact? 3.7 Has USAID supported TB advocacy, communication, and social mobilization (ACSM)? 3.7.1 What USAID-supported activities were conducted to promote community participation in TB? Who conducted them and who supported them? 3.7.2 What areas in TB control the communities participated? Was their participation sustainable? In what way? 3.7.3 Do you think these activities made a difference? 3.7.4 Have misperceptions/misconception and stigma on TB among the general population decreased? Is there data (such as the results of KAP studies) that support these conclusions? 3.7.5 Are more persons with TB symptoms seeking diagnosis and treatment at DOTS facilities? 116 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.7.6 Are communication materials available at DOTS facilities (public and private)? If so, has it made an impact? For example, have the number of TB symptomatic seeking diagnosis increased at facilities, or the number of TB patients enrolled on or completing treatment? 3.7.7 Are communication materials available at DOTS facilities (public and private)? If so, has it made an impact? For example, have the number of TB symptomatic seeking diagnosis increased at facilities, or the number of TB patients enrolled on or completing treatment? 3.7.7 At each site, what communications were available? Please describe the type, variety, quality, and quantity of each. 3.7.8 Have other advocacy campaigns (e.g., public awareness campaigns) been conducted at local or national awareness levels? Have they made a difference? 3.7.9 Are participating providers able to provide TB related information and counseling? (Document and witness this during the site visits; look for relevant materials – e.g. flip charts) 3.7.10 Are communities in USAID-supported sites more involved in TB control and prevention activities? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 117 3.7.11 Complete the following table for each site visited. Site name: …………………….. USAID supported: Yes / No 2006 2007 2008 2009 2010 2011 Number of CBOs involved in TB care Number of Barangay Health Workers attached to the site (health facility) 3.8 Has USAID support improved TB diagnosis? 3.8.1 Have USAID-supported TB projects helped local/national TB laboratories (public and private) and the national TB laboratory network? Has USAID support enhanced the National TB Reference Laboratory (NTRL)? Has it benefitted provincial/municipal laboratory technicians? 3.8.2 Has external quality assurance for sputum smear microscopy improved as a result? 3.8.3 Complete the following table for each site visited. Site name: …………………….. USAID supported: Yes / No Number of major errors 2006 2007 2008 2009 2010 2011 Microscopy laboratory name ……………………………………….. Microscopy laboratory name ……………………………………….. Microscopy laboratory name ……………………………………….. 3.8.4 Do TB suspects have adequate access to TB diagnosis (Labs, x-rays, etc)? In each site visited, describe the local TB laboratory network (including availability of x-ray, microscopy, culture or molecular rapid diagnostics) as well as any available sputum transportation system. 118 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.8.5 Are there quality TB sputum smear microscopy labs in local settings? Are they staffed by trained microscopists? 3.8.6 Complete the following table for each site visited. Site name: …………………….. USAID supported: Yes / No # of microscopists / # of trained microscopists 2006 2007 2008 2009 2010 2011 Microscopy laboratory name ……………………………………….. / / / / / / Microscopy laboratory name ……………………………………….. / / / / / / Microscopy laboratory name ……………………………………….. / / / / / / 3.8.7 Have the projects supported lab diagnostic strategies at national level? At regional level? 3.9 Has USAID support improved anti-TB drug supply and management? 3.9.1 Have there been any TB drug stockouts during the evaluation period? 3.9.2 Complete the following table for each site visited. Site name: …………………….. USAID supported: Yes / No # of major stockouts / # of minor stockouts 2006 2007 2008 2009 2010 2011 First-line Anti-TB Drugs / / / / / / Second-line Anti-TB Drugs / / / / / / Pediatric Anti-TB Drugs / / / / / / USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 119 3.9.3 What USAID-supported interventions have been implemented to support TB drug management and distribution system? Are these interventions sustainable? National level Provincial/municipality level 3.9.4 Have the projects supported information and TB drug management systems (forecasting, procurement, distribution, utilization) strengthening to improve the availability and quality of TB drugs? National level Provincial/municipality level 3.10 Has USAID support created or strengthened NTP capacity? 3.10.1 At the national level, what has been the USAID support at the national level in terms of capacity building and technical assistance (e.g., guidelines, policy development, and training)? 3.10.2 What policy instruments and programmatic tools have been developed and can be attributed to USAID TB projects? 120 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.10.3 Was this technical assistance at the national level of high quality and useful? 3.10.4 What support has USAID provided to the Regional TB program in terms of capacity building and technical assistance (guidelines, policy development, training, supervision, etc)? 3.10.5 Was technical assistance to the Regional TB program of high quality and useful? 3.10.6 What support has USAID provided to specific, local sites in terms of capacity building and technical assistance (guidelines, policy development, training, supervision, etc)? 3.10.7 Was technical assistance to local sites supported by USAID of high quality and useful? 3.10.8 To what extent have USAID TB projects contributed to capacity building of CBOs, non￾governmental organizations and other community based initiatives? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 121 3.11 Are USAID-supported initiatives sustainable? 3.11.1 At the national level, what are the plans of the DOH and local governments for sustaining systems and interventions developed under the USAID TB projects? 3.11.2 At the regional level, what evidence is there (such as financial or political commitments or plans) that the USAID-supported interventions will be sustained? At the municipal/local level? 3.11.3 At the national level, are there plans for sustaining and/or increasing locally generated funding for TB control? At the local level? 3.12 Has USAID supported efforts to improve infection control? 3.12.1 Does the country have a national policies and/or guidelines for infection control? If yes, to what extent have USAID-supported TB projects contributed to their development, implementation? 3.12.2 Have these policies/guidelines been adopted and implemented at local sites? 3.12.3 Based on your observations at local sites, are they being implemented and is there implementation being supervised/routinely assessed? (E.g., Is there adequate ventilation? Are N95 Respirators available for health personnel? Are administrative arrangements being followed? Have environmental measures been taken? 122 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) 3.12.4 Is there any monitoring of health center staff infection with TB? USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 123 ANNEX E: TABLES OF MORBIDITY AND MORTALITY IN THE PHILIPPINES TABLE 1. Top 10 Causes of Morbidity in the Philippines, by Gender, 2008 (Number of Cases and Morbidity Rate/100,000 Population) Cause of Morbidity (and Ranking) Male Female Both sexes Number Rate Number Rate Number Rate 1 Acute respiratory infection 780,598 1,716.2 866,580 1926.8 1,647,17 8 1,840.6 2 ALTRI and pneumonia 393,996 866.2 386,110 858.5 780,199 871.8 3 Bronchitis/bronchi olitis* 255,267 561.2 263,686 586.3 519,821 580.8 4 Hypertension** 210,058 461.8 287,181 638.5 499,184 557.8 5 Acute watery diarrhea*** 221,004 485.9 208,407 463.4 434,445 485.4 6 Influenza 182,207 400.6 180,248 400.8 362,304 404.8 7 TB respiratory 58,584 128.8 37,605 83.6 96,497 107.8 8 Acute febrile illness 17,734 39.0 18,135 40.3 35,381 39.5 9 Diseases of the heart 14,406 31.7 17,647 39.2 32,541 36.4 10 Chicken pox 12,959 28.5 12,586 28.0 25,677 28.7 Notes: * 4th among females; ** 5th among males; ** 4th among males. Source: 2008 FHSIS Annual Report. 124 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) TABLE 2. Top 10 Causes of Mortality in the Philippines (2001-2006) (Number of Deaths and Mortality Rate/100,000 Population) Cause of Mortality (and Ranking) 5-Year Average (2001-2005) 2006* Number Rate Number Rate 1 Diseases of the heart 69,741 85.5 83,081 95.5 2 Diseases of the vascular system 52,106 64.0 55,466 63.8 3 Malignant neoplasms 39,634 48.6 43,043 49.5 4 Accidents** 33,650 41.4 36,162 41.6 5 Pneumonia 33,764 41.5 34,958 40.2 6 Tuberculosis, all forms 27,017 33.2 25,860 29.7 7 Chronic lower respiratory diseases 19,024 23.3 21,216 24.4 8 Diabetes mellitus 15,123 18.5 20,239 23.3 9 Certain conditions originating in the perinatal period 13,931 17.2 12,334 14.2 10 Nephritis, nephrotic syndrome, and nephrosis 9,785 12.0 11,981 13.8 Notes: * Reference year; ** External causes of mortality. Source: Philippines Department of Health website. USAID/PHILIPPINES: EXTERNAL EVALUATION OF THE TUBERCULOSIS PORTFOLIO (2006–2011) 125 ANNEX F: POLICIES DEVELOPED WITH TB LINC SUPPORT Revision of the NTP Manual of Operations (ongoing). Revised NTP guidelines for treatment and drug distribution to children with TB and adults with MDR TB that ensure appropriate treatment and drug supply for TB in children and MDR TB among adults. Estimation and adoption of TB health accounts for planning, resource allocation purposes, policy formulation and review of service delivery strategies. Improved DOH-NTP budget structure, allocation and execution that reflect priorities for budget line items supporting agreements with LGU concerning performance levels. Greater budget coordination for TB control between DOH and attached agencies. Issuance of Executive Orders, DOH Administrative Orders and/or Memorandum of Agreement between DOH, the Department of Budget Management (DBM) and the Department of Finance (DOF) adopting a multi-year budgeting framework for TB. Number of LGUs with increased financing for TB. Number of LGUs with full or partial manpower complement. New payment scheme for the PhilHealth TB DOTS OP package with attract greater private sector participants. PhilHealth attains an improved regulatory capacity and procedures for its TB DOTS OP package. Memorandum of Agreement among stakeholders for greater coordination and collaboration in TB control especially at the regional and local levels. 126 EXTERNAL EVALUATION OF THE USAID/PHILIPPINES TUBERCLOSIS PORTFOLIO (2006–2011) For more information, please visit http://www.ghtechproject.com/resources GH Tech Bridge Project 1725 Eye Street NW, Suite 300 Washington, DC 20006 Phone: (202) 349-3900 Fax: (202) 349-3915 www.ghtechproject.com