EVALUATION OF TUBERCULOSIS PROGRAM IN INDIA PATH REPORT MARCH, 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Aime De Muynck, David Berger, Tim Clary, Selva Kumar, Gani Perla, and Rajeswari Ramachandran, of Social Impact, Inc. ii EVALUATION OF TUBERCULOSIS PROGRAM IN INDIA PATH REPORT DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document is available in printed or online versions. Documents are also made available through the Development Experience Clearing House (www.dec.org). Additional information can be obtained from: Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com This document was submitted by Social Impact, Inc., to the United States Agency for International Development under USAID Evaluation Services, Contract No. RAN-I-00-09-00019. ACKNOWLEDGEMENTS The evaluation team would like to thank USAID/India, PATH and the various partner organizations that shared their time, experiences and insights that were critical for the team to understand the scope and achievements of PATH’s TB activities in India, the important challenges and constraints faced by PATH, and the efforts undertaken to overcome them. ACRONYMS ACSM Advocacy, Communication, and Social Mobilization AFB Acid Fast Bacillus AIC Airborne Infection Control AIDS Acquired Immune Deficiency Syndrome AID/W Agency for International Development/Washington Office AP Andhra Pradesh (India) ART Anti-Retroviral Therapy ASM American Society of Microbiology BPHRC Blue Peter Public Health and research Centre BSL3 Biosafety Level 3 (India) CBO Community-Based Organization CDC Centers for Disease Control C-DOTS Community-Based DOTS C&DST Culture and Drug Sensitivity Testing CTD Central TB Division (India) DCP Drug Control Officer DMC Designated Microscopy Centre DOTS Directly Observed Therapy, Short Course DQA Data Quality Assurance DST Drug Susceptibility Testing DTC District Tuberculosis Center DTO District Tuberculosis Officer EQA External Quality Assurance FIND Foundation for Innovative New Diagnostics FY Fiscal Year GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GLC Green Light Committee GoAP Government of the State of Andhra Pradesh GoI Government of India HIV Human Immunodeficiency Virus HCF Health Care Facilities IC Infection Control IEC Information, Education, and Communication IMA Indian Medical Association IQC Indefinite Quantity Contract IRL Intermediate Reference Laboratory JMM Joint Monitoring Mission KAP Knowledge, Attitude and Practice LED Light Emitting Diode LPA Line Probe Assay LRS Institute of TB and Respiratory Diseases (in Delhi) M&E Monitoring and Evaluation MDR-TB Multidrug-Resistant TB MGIT Mycobacteria Growth Indicator Tube MOH Ministry of Health MSH Management Sciences for Health NGO Nongovernmental Organization NRL National Reference Laboratory vii NTC NGO TB Consortium NTI National Tuberculosis Institute, Bangalore ODCA Ongole Drug and Chemist Association OPD Outpatient Department OR Operational Research PATH Program for Appropriate Technology in Health PIH Partners in Health PMDT Programmatic Management of Drug-Resistant TB PMP Performance Monitoring Plan PP Private Practitioner PPM Public-Private Mix PMDT Programmatic Management of Drug Resistant TB PR Principal Recipient QA Quality Assurance QC Quality Control RNTCP Revised National Tuberculosis Control Programme (India) SEARO WHO Regional Office for South-East Asia SLD Second-Line Drug SOP Standard operating procedures SOW Statement Of Work SR Sub-Recipient SSR Sub-Sub-Recipient STDC State Training and Demonstration Centre STO State Tuberculosis Officer TA Technical Assistance TASC2 Technical Assistance and Support Contract II TB TEAM TB Technical Assistance Mechanism TB Tuberculosis TO 2 Task Order 2 TO 2015 Task Order 2015 TOT Training of Trainers TPM Team Planning Meeting TRC Tuberculosis Research Centre, Chennai TU Tuberculosis Unit TWG Technical Working Group UNITAID International facility for the purchase of Drugs Against HIV/AIDS, Malaria and TB UP Uttar Pradesh USAID United States Agency for International Development WHO SEARO World Health Organization Regional Office for South-East Asia WHO World Health Organization WV-I World Vision-India XDR-TB Extremely Drug-Resistant TB 1 TABLE OF CONTENTS ACKNOWLEDGEMENTS........................................................................................................................................... iv ACRONYMS...................................................................................................................................................................... vi TABLE OF CONTENTS ................................................................................................................................................ 1 EXECUTIVE SUMMARY............................................................................................................................ 4 I. INTRODUCTION................................................................................................................................................... 7 THE DEVELOPMENT PROBLEM ....................................................................................................................... 8 The Problem Statement............................................................................................................................................ 8 Intermediate Laboratory diagnostic capacity......................................................................................................... 8 Multi-Drug Resistant TB (MDR-TB)..................................................................................................................... 9 Airborne Infection Control...................................................................................................................................... 9 Engage all providers.................................................................................................................................................. 9 Advocacy, communications and social mobilization (ACSM) ........................................................................... 9 Intervention Theory.....................................................................................................................................................10 Purpose of the Evaluation ..........................................................................................................................................10 II. RESEARCH DESIGN AND EVALUATION METHODOLOGY ..........................................................11 Methods.........................................................................................................................................................................11 Team Planning Meeting (TPM)............................................................................................................................. 11 Site Visits and Interviews........................................................................................................................................12 The Evaluation Framework........................................................................................................................................12 III. FINDINGS..........................................................................................................................................................12 PROCESS FINDINGS ABOUT THE MANAGEMENT AND IMPLEMENTATION OF THE PROGRAM...................................................................................................................................................................12 Implementation Arrangements, Management, and Program Monitoring......................................................12 Core Recommendations.........................................................................................................................................14 Additional Recommendations............................................................................................................................... 15 MONITORING AND EVALUATION (TECHNICAL ASSISTANCE TO GLOBAL FUND ROUND 9) .....................................................................................................................................................................................16 Core Recommendations.........................................................................................................................................16 Additional Recommendations............................................................................................................................... 17 OUTCOME AND RESULTS FINDINGS ABOUT THE PROJECT’S ACHIEVEMENTS AND CONSEQUENCES, CONCLUSIONS AND ACTIVITY SPECIFIC RECOMMENDATIONS............17 PATH Airborne Infection Control (AIC)...........................................................................................................17 Strengthen Intermediate Reference Laboratory Capacity .................................................................................21 ACSM Activities.......................................................................................................................................................26 Engage Other Providers.........................................................................................................................................32 PATH’S Objectives.................................................................................................................................................33 Instruments for partnership develop by PATH ................................................................................................. 33 Expansion of MDR-TB control............................................................................................................................ 37 Test Innovative approaches...................................................................................................................................40 IV. GENERAL CONCLUSIONS and RECOMMENDATIONS ................................................................. 40 FIGURES: Figure 1. Project Management cycle of PATH............................................................................................13 Figure 2: Number of Pharmacy referrals of TB suspects in Ongole TU, Andhra Pradesh 2010-11..........35 TABLES: Table 1: Implementation of IRL strengthening activities ............................................................................23 Table 2. Government Roles for Planning and Conducting ACSM .............................................................27 2 APPENDICES Appendix 2: FRAMEWORK of REVIEW QUESTIONS .............................................................................51 Appendix 3: PERSONS CONTACTED......................................................................................................61 Appendix 4: REFERENCES .........................................................................................................................68 Appendix 5: LIST of the SENIOR STAFF of PATH as of MARCH 2011 ...................................................70 Appendix 6: INTERVENTION MODEL DEVELOPED by PATH...............................................................77 3 PROJECT SUMMARY • Project name: India TB Program • Strategic Objectives (SO): To strengthen the Intermediate Reference Laboratory (IRL) network of the Revised National Tuberculosis Control Program (RNTCP) at the state level to increase capacity to diagnose multidrug resistant TB (MDR-TB). Other activities include pilot testing infection control guidelines and preparing recommendations for scale up and supporting the full engagement of key stakeholders in TB control through advocacy, communication and social mobilization interventions (ACSM). • Life of the project (LOP): October 2008 to September 2009 • Implementing Partners: Programs for Appropriate Technologies in Health (PATH) Project Name and Strategic Objectives: 1. Strengthen the intermediate reference laboratory network through strategic provision of technical assistance, training, equipment, and upgrading facilities etc., to attain and maintain accreditation 2. Accelerate accreditation of Intermediate Reference Laboratory network, and ensure the maintenance of accreditation through periodic site visits and mentoring.3. Establish pilot studies to test improved infection control practices, participating in the National Airborne Infection control committee meetings, and pilot test the National Airborne Infection Control Guidelines in Andhra Pradesh. 4. Support RNTCP health communication efforts by translating existing comprehensive ACSM strategy into results-oriented field activities; develop the capacity of the different stakeholders to design, implement and monitor needs based ACSM activities. 5. Design and implement community level activities to effectively engage private sector providers, both formal and informal, and other segments of society in TB control activities to support RNTCP goals and objectives 6. Support effective expansion of MDR-TB control activities by identifying and addressing gaps in the DOTS-Plus program. 7. Test innovative approaches to improving TB case detection, treatment success and preventing MDR-TB Life of Project: October 1, 2008 – September 30, 2011 Implementing Partners and contract numbers: PATH, TASC2 TB, Task Order 02 GHS I-02-03-00034-00 GHN-I-00-09-00006-01, Task Order 01 (or TB IQC Task Order 2015) Project Funding: USAID committed US$ 2 million in TB Task Order 2 and US$ 4.64 million in TB Task Order 2015 Evaluation of Tuberculosis Program in India: PATH Report 4 EXECUTIVE SUMMARY India has the highest tuberculosis (TB) burden in the world, amounting to twenty percent of the global burden of TB. Although the operational targets of one hundred percent coverage, seventy percent case detection and eighty-five percent cure rate of New Smear Positive (NSP) cases were met years ago, there is as yet no proof of an epidemiological impact. The Revised National Tuberculosis Control Program (RNTCP) can have an epidemiological impact only if the private sector is fully involved in TB diagnosis, treatment and the follow-up of the TB patients. To face the growing problem of multi-drug resistant TB (MDR-TB), India launched the program “Programmatic Management of Drug Resistant TB” (PMDT), which will expand the network of accredited laboratories capable of processing culture and drug-sensitivity testing (C&DST). Effective infection control measures are necessary in high-risk settings such as hospitals and outpatient facilities. Education of patients and vulnerable communities is needed to improve early diagnosis and treatment adherence. It is within this challenging context that USAID/India sought out the services of the Program for Appropriate Technology in Health (PATH) to strengthen the laboratory network’s capacity to diagnose TB and to identify drug-resistant strains of TB; facilitate the introduction of improved infection control practices; assist the RNTCP in strengthening its approaches and methodologies related to advocacy, communication, and social mobilization (ACSM); and enhance public-private partnerships. USAID/India contracted Social Impact, Inc. to carry out an in-depth evaluation of the tuberculosis prevention and control activities implemented by PATH. The objectives of the review were to determine the impact of the PATH projects relative to stated objectives and achievements, and to make suitable recommendations for the future direction and priorities of the projects. The evaluation team collected and analyzed the information necessary to assess the evaluation objectives, conducted an initial planning meeting with USAID/India, prepared an overall framework, which was reviewed and approved by USAID/India, and carried out a series of field visits in Andhra Pradesh, Uttar Pradesh, Maharashtra, Gujarat and Rajasthan. The following findings, conclusions and recommendations were made: 1. Implementation arrangements for PATH’s TB Project in India The yearly task orders issued under the USAID/W Indefinite Quantity Contract (IQC) result in PATH being unable to develop long-term work plans or a performance monitoring plan (PMP), attract sufficient senior-level staff, or show attribution beyond the output level. Modifications to the contract can take from four to six months, thus slowing implementation. This situation needs to be remedied by USAID/India, who should consider moving toward a bi-lateral cooperative agreement and, perhaps, nesting the TB project within a larger, health-systems-strengthening activity. The project has grown tremendously during the last few years yet its management structure remains highly centralized. Finally, the PATH’s TB Project’s lack of a Delhi presence has reduced PATH’s ability to interact with its counterparts and has probably affected key relationships. 2. Monitoring and Evaluation—M&E PATH’s current internal capacity for M&E is sufficient. However, if PATH will be required in the future to perform in-depth evaluations, then it will need to hire a senior-level M&E officer in addition to developing an M&E work plan. Furthermore, PATH’s activity matrix needs to be improved, so that its key achievements can be more readily understood. The intention to have Evaluation of Tuberculosis Program in India: PATH Report 5 PATH provide M&E technical assistance to other organizations should be reconsidered, though past participants in PATH’s M&E trainings have expressed satisfaction with the information they received. 3. Airborne Infection Control (AIC) Untreated TB patients, and especially MDR-TB patients, are a source for noso-comial (patient-to￾patient) and occupational (patient-to-health care worker) M. tuberculosis transmission. PATH developed the innovative Airborne Infection Control (AIC) Program in collaboration with the government of India (GoI); it developed guidelines, established AIC checklists, provided technical assistance trainings, and facilitated the formation of the National Airborne Infection Control Committee; developed the action plan on AIC for Andhra Pradesh; and conducted AIC trainings in Kolkata (West Bengal), Ahmadabad (Gujarat) and Hyderabad (Andhra Pradesh). PATH, along with the World Health Organization (WHO), was instrumental in conducting a national-level workshop in New Delhi. PATH, along with the Central Tuberculosis Division (CTD) and WHO support, conducted baseline assessments of AIC measures and practices at thirty-four selected health care facilities in three states, built capacity for the district officials, healthcare facility administrators and infection control focal points, and recommended AIC implementation. PATH provided technical support on AIC piloting, provided AIC assessment kits, and facilitated several healthcare facility risk assessments. USAID/India should continue to fund PATH’s AIC activities. PATH should develop training material on infection control within the current context of community-based MDR-TB treatment. 4. Intermediate Reference Laboratory (IRL) PATHS objectives are to assist in installing the equipment for solid C& DST, develop infrastructure for molecular biology laboratory up-gradation, and to train the new laboratory technicians to maintain the quality expected by the RNTCP. PATH, together with its partners, established and/or upgraded existing laboratories into molecular biology laboratories; built the clean air rooms; and deputed an experienced laboratory technologist from NRL for onsite implementation of standard operating procedures in IRLs. Thanks to PATH’s involvement, more than 375 MDR-TB cases were diagnosed in one IRL and 175 cases in another laboratory in Hyderabad initiated treatment. However, the delay to upgrade a site in Tamil Nadu for line probe assay (LPA) and BSL-3 capability has hampered the process of establishing the laboratory; laboratory committee meetings were not conducted by CTD since February 2010, delaying the execution of work plans. USAID should continue to provide support to PATH to strengthen the IRLs, hire technical experts, and develop a troubleshooting document for the LPA and MGIT 960. 5. Advocacy, Communication and Social Mobilization (ACSM) The overall objective is to develop government and non-governmental staff capacity at all levels to effectively plan, implement and evaluate ACSM activities. PATH will achieve this through trainings and technical assistance, facilitating experience-sharing workshops, and providing evidence that ACSM can contribute to improving TB control performance and outcomes. The most successful component to date has been the development and delivery of a series of ACSM workshops. However, many other ACSM activities have been delayed or not fully implemented. Despite not completing any assessment of its effectiveness, or fully identifying operational and design issues with its pilot project, PATH prematurely proposed scaling-up its technical assistance project to six other states. Evaluation of Tuberculosis Program in India: PATH Report 6 USAID/India should work with PATH to address the following recommendations: (1) continue trainings only after addressing coordination and duplication issues; (2) improve overall partner coordination by formalizing agreements with partners; (3) delay scale-up activities until PATH addresses management and design issues, and; (4) ensure sufficient senior technical staff capacity in￾country. 6. Public-Private Mix (PPM) In India, the private sector is the dominant provider of health services. Collaboration with private providers is important if RNTCP wants to reach a wider audience. PATH developed a pharmacy initiative to improve the referral system from private chemists to designated microscopy centers; increase TB case detection and reduce diagnostic delay for TB treatment; improve access to quality DOTS services; and discourage the sale of TB drugs without prescription. PATH piloted this project in Ongole Tuberculosis Unit (TU), Prakasam district, Andhra Pradesh. But a series of deficiencies hampered this project in its development, such as lack of baseline data and choice of inappropriate attendees for the training. To make the pharmacy project successful, private providers (PPs) should become an integral part of the PPM initiative; a baseline study should be carried out; staff core skills should be strengthened; effective supervision and monitoring should be an integral aspect of the pilot; and an incentive system should be developed. As long as these conditions are not fulfilled, PATH will have difficulty proving its comparative advantage in PPM. GENERAL CONCLUSIONS AND RECOMMENDATIONS The main evaluation approach towards the PATH support was of a qualitative and descriptive nature, focusing on the following key questions: (1) quality of technical expert support to RNTCP; (2) the level of achievement of the specific objectives of the various support activities, such as PPM, capacity building, lab strengthening, airborne infection control, MDR/XDR, and ACSM; and (3) its ability to coordinate and the coordination practices. The evaluation has identified three program areas of recognized expertise: AIC, lab strengthening, and training. PATH has been an innovator in the domain of AIC in India and continues to be actively involved in it: PATH is recognized as providing high-quality support to lab strengthening and has developed an original methodology for training that is recognized as very effective, and its handouts are being used by other institutions today. Little evidence was found for a specific niche that PATH could occupy in the other program areas such as ACSM, PPM (Pharmacy approach), and MDR, but PATH may not have had enough time to show its potential in these areas. PATH needs to demonstrate that it is capable of designing and conducting sound methodological interventions, of recruiting and retaining top level staff, and provide evidence that its interventions are both innovative and effective. Some parsimony in the design of the project activities is desirable, as this could lead to a more focused implementation and better outputs/outcomes. If the operational challenges of intense supportive supervision and monitoring, and if improving the program and HR management could be faced adequately, then the cost-effectiveness of the projects in the areas of ACSM, PPM and MDR could certainly increase. Therefore, further financial support to these three areas should be made conditional upon obtaining extra guarantees for improvement in project planning, project and HR management, levels of expertise of the senior staff, and coordination with the partners. The question of the “value for money invested by USAID/India in PATH” can be answered positively for the program areas of AIC, lab strengthening, and training. The evaluation team Evaluation of Tuberculosis Program in India: PATH Report 7 recommends continuing—and, if possible, increasing—the level of funding for the activities and encourages PATH to take more initiatives in these areas; however, the administrative set-up of the funding should be looked upon critically by USAID and improved. INTRODUCTION USAID/India has supported TB control activities in India for more than ten years. These efforts have focused on enhancing DOTS services, improving lab capacity to diagnose drug resistant TB, operations research, TB-HIV collaboration, and health systems strengthening. Since implementing its Revised National Tuberculosis Control Programme (RNTCP) in the nineties, the government of India (GoI) has achieved significant progress towards control of TB disease, achieving global TB control objectives of detecting seventy percent of all TB sputum smear positive TB cases and curing eighty-five percent of new smear positive cases by 2009. 1 Given these achievements, India is now ready to embark on a major new initiative to provide universal access to TB care, decrease treatment delay, increase treatment outcomes, address on a large scale the problem of multi-drug resistance, and involve the private sector in a full-fledged and comprehensive manner. This ambitious plan, referred to as RNTCP3, will require early and complete detection of TB, increased human resources, expansion of laboratory and treatment centers, involvement of the private providers, and innovative strategies to deliver care to vulnerable populations.2 India faces enormous and unique challenges on a scale unlike any other country. This country has the highest TB burden in the world, amounting to twenty percent of the global burden of TB.3 An estimated 1.9 million new cases of TB occur in India alone, of which about 0.8 million are diagnosed with infectious new smear positive pulmonary TB cases. Every day, more than 5,000 people develop TB disease and over 750 people die from it. This translates to one death every two minutes. 4 Deaths due to TB exceed the combined deaths from all other communicable diseases and account for twenty-six percent of all avoidable adult deaths. TB kills more women, and results in more orphans, than is produced by all other causes of maternal mortality combined.5 The risk of developing MDR-TB amounts to about three percent among new cases, and twelve to seventeen percent among re-treatment cases; 6 recently, this risk shows an increasing trend.7 The current rates of MDR-TB translate into an estimated annual incidence of 110,000 cases. 8 1 “TB India 2010, RNTCP Status Report,” Central TB Division, Ministry of Health and Family Welfare, March 2010. The GoI plans to address MDR through its “Programmatic Management of Drug Resistant TB” (PMDT) program. Adequately coping with MDR-TB will require expanding both diagnostic and treatment capacity. There is an urgent need to develop a network of accredited laboratories capable of processing C&DST to accurately diagnose TB disease and identify drug resistance and 2 “TB India 2010, RNTCP Status Report,” Central TB Division, Ministry of Health and Family Welfare, March 2010. 3 “Tuberculosis Control in the SEA Region,” the Regional Report, World Health Organization, South-East Asia Regional Office , New Delhi, 2010 4 USAID/India. “5-Year Tuberculosis Strategy 2010-2014,” Draft. N.D. 5 “TB India 2009 – RNTCP Status Report,” Central TB Division, Government of India. 6 Paramasivan, C. N. “Anti-Tuberculosis Drug Resistance Surveillance In Tuberculosis,” Editors S. K. Sharma and A. Mohan, Jaypee Medical Publishers Pvt Ltd, New Delhi, 2001, p463–476. 7 Rawat, J., G. Sindhwani, R. Juyal, R. Dua. “Five-Year Trend of Acquired Anti-Tubercular Drug Resistance In Patients Attending a Tertiary Care Hospital at Dehradun (Uttarakhand),” Lung India, 2009; 26: 106–108 doi: <10.4103/0970- 2113.56342> 8 Joint Monitoring Review Mission May 17–28, 2010, DRAFT AIDE MEMOIRE Evaluation of Tuberculosis Program in India: PATH Report 8 susceptibilities. In addition, the GoI will need to scale up treatment services for MDR-TB patients, as they require more complex and longer treatment regimens. It is also necessary to decrease the noso-comial infection threat to care seekers and medical staff. Furthermore, education of patients and vulnerable communities is needed to improve early diagnosis and treatment adherence. Given that a significant fraction of the Indian population consults private practitioners, the Revised National Tuberculosis Control Programme (RNTCP) can reach an epidemiological impact only if the private sector is fully involved in TB diagnosis and treatment as a full partner. It is in this challenging context that USAID/India sought out the services of PATH to provide services to the GoI and with the private and non-governmental sectors. The main objectives of the partnership are to provide technical assistance for TB control efforts in India by strengthening the laboratory network’s capacity to diagnose TB and to identify drug-resistant strains of TB; facilitating the introduction of improved infection control practices (focused on reducing noso-comial infections in health care settings, especially with regards MDR-TB and TB-HIV); and assisting the RNTCP in strengthening its approaches and methodologies related to advocacy, communication, and social mobilization (ACSM) and enhancing public-private partnerships. THE DEVELOPMENT PROBLEM The Problem Statement India has the highest tuberculosis (TB) burden in the world, amounting to twenty percent of the global burden of TB. Although the operational targets of one hundred percent coverage, seventy percent case detection and eighty-five percent cure rate of New Smear Positive (NSP) cases were met years ago, there is as yet no proof of an epidemiological impact. The RNTCP can have an epidemiological impact only if the private sector is fully involved in TB diagnosis, treatment and the follow-up of TB patients. To face the growing problem of multi-drug resistant TB (MDR-TB) India launched the program “Programmatic Management of Drug Resistant TB” (PMDT), which will expand the network of accredited laboratories capable of processing culture and drug sensitivity testing (C&DST). Effective infection control measures are necessary in high-risk settings such as hospitals and outpatient facilities. Education of patients and vulnerable communities is needed to improve early diagnosis and treatment adherence. Intermediate Laboratory diagnostic capacity A well-functioning laboratory network has been established across the country to carry out sputum smear microscopy and plans are underway to develop culture and drug susceptibility testing. This network also assures External Quality Assurance (EQA) for smear microscopy, culture and DST laboratories. The diagnostic and follow-up requirements of the Category IV (CATIV) services in the country for culture and DST have to be met through the existing twenty-seven Intermediate Reference Laboratories (IRLs). However, this diagnostic network requires infrastructure strengthening and upgrading lab capacity. This needs to occur in the context of a comprehensive Evaluation of Tuberculosis Program in India: PATH Report 9 laboratory design network that allows for the flexibility to absorb new technologies and includes accredited laboratory capacity in the private sector and in the medical schools. 9 Multi-Drug Resistant TB (MDR-TB) Central TB Division- India (CTD) estimates that about 50,000 MDR-TB (detectable) cases occur in India annually. This huge problem has implications on TB control, not only in India but also globally10. Early reports on treatment outcomes from the PMDT sites are not very encouraging, as a relatively high proportion of cases do not start treatment, die before or shortly after initiating treatment, default or remain bacteriologically positive at 6 – 12 months. The higher than expected default rate makes it important to look at effective means of linking MDR-TB patients to social welfare schemes and community support; to address this, appropriate models have to be urgently developed11. Airborne Infection Control Airborne infection control (AIC) measures are lacking in the majority of healthcare facilities and laboratories in India, although the RNTCP has begun to pay attention to this issue. 12 There is a need to improve AIC to interrupt the chain of transmission, particularly in vulnerable settings. Engage all providers In India, private sector health care is the first point of contact for the majority of the population, including TB patients. Diagnosis and treatment quality in the private sector is still not uniform, and many do not follow treatment guidelines.13 There is an important delay in accurate diagnosis, registration of new TB patients, and initiation of appropriate treatment, which is significantly linked to “doctor shopping.” Treatment adherence is also important; because patients in the private sector may start treatment, then stop if they can no longer afford to pay for doctor visits or medications. With the current thrust on universal case detection, ways and means to effectively involve the private sector in accurately diagnosis or to refer patients to the public sector in a timely manner should be actively pursued. GoI has recognized this fact and has taken several practical steps to encourage private sector participation in RNTCP. However, a large proportion of the private sector healthcare providers remains outside the program and do not refer patients for various reasons.14 Advocacy, communications and social mobilization (ACSM) In 2005, CTD approved a national health communication strategy to develop awareness of TB symptoms, diagnosis and treatment services. The strategy encouraged health-seeking behaviors and treatment adherence through improved patient and community education and by strengthening 9 JMM 2009. SEA-TB-231. World Health Organization, New Delhi 10 USAID/India 5 Year Strategy. Draft 11 USAID/India 5 Year Strategy. Draft 12 JMM 2009. SEA-TB-231. World Health Organization, New Delhi; p 83-84. 13 Thakur, J. S., S. Sekhar Kar, A. Sehgal, S. Kumar. “Private sector involvement in TB control in Chandigarh,” Indian J Tuberc, 2006; 53: 149-153. 14 USAID/India 5 Year Strategy. Draft Evaluation of Tuberculosis Program in India: PATH Report 10 patient-provider interpersonal communication.15 To align itself with the Global Plan to Stop TB, CTD modified its own strategy to align with a new global initiative called Advocacy, Communication, and Social Mobilization (ACSM). Using this approach, CTD has sought to raise awareness about TB and encourage behavior change related to case detection and treatment adherence. On a national level, CTD provides states and districts with resources and capacity building, while states and districts are responsible for planning and for including ACSM activities into their action plans. The overall capacity to design and manage ACSM interventions, however, remains limited. According to the 2009 Joint Monitoring Mission (JMM) report, health officers are either not aware of the national strategy or do not know how to translate it into action. Many of the action plans that are prepared do not reflect an “analysis of needs, program data or existing KAP survey data.”16 As a result, the materials that are produced lack clear messages, are not well designed, and are often text￾heavy materials that do not communicate to low literate populations. Analysis and dissemination of effective methods was limited with a “notable disconnect between interventions envisaged at the national level and what was happening in the field.” Many district plans are not evidence-based and responsibility to carry out activities is placed on over-burdened program staff. Finally, there is limited assessment or documented evidence about the effectiveness of these activities and their contributions to improved TB control program performance. Intervention Theory The aims and objectives of the various activities undertaken by PATH, their conceptualization and design, their implementation in the field, and the intended results are detailed in subsections related to each of the activities. Purpose of the Evaluation USAID/India contracted Social Impact, Inc. to carry out this in-depth and thorough evaluation of the tuberculosis prevention and control activities implemented by PATH. The objectives of this review are to: • Determine the impact of PATH projects relative to stated objectives and achievements. • Make suitable recommendations for the future direction and priorities of the projects. The original statement of work (SOW) called for a comprehensive evaluation of PATH’s projects, the appropriateness of the project activities in achieving the objectives, the level of impact, cost￾effectiveness and future directions. Based on an in-country meeting with USAID/India, the main focus of the evaluation shifted to address the issue of the value for money, and answering the question of if, and to what extent, the money provided by USAID to PATH to support RNTCP activities has been well spent. Consequently, the focus of the evaluation became much more of a qualitative and descriptive exercise rather than on the level of targets reached, project performance, and on impact achieved. 15 CTD. “A Health Communication Strategy for RNTCP.” CTD, New Delhi, Nov 2005, pg. 180. 16 JMM Report, 2009 Evaluation of Tuberculosis Program in India: PATH Report 11 The evaluation examined project trends, sought to discover the “pull and push” factors that determined the level of achievements, and assessed the quality of activity implementation. The questions on the comparative advantage of PATH’s involvement and the specific niches it occupies constantly steered our search for evidence and the comparison of observed achievements against the hypothetical realization of those activities had they been carried out by GoI staff alone, or by other NGOs, rather than by PATH.17 This evaluation searched for the main successes and lessons learned from this project, and provide recommendations for improvement in the future. RESEARCH DESIGN AND EVALUATION METHODOLOGY The evaluators used a range of methods and approaches for collecting and analyzing the information required to address the evaluation objectives. The evaluation team conducted an initial planning meeting with USAID/India, and then prepared an overall framework, which was reviewed and approved by USAID/India (see Annex 1). Methods • Desk review of documents (see list in Annex 2) • Attendance and direct observation of two workshops, held on Feb 17–8 in Delhi on the following topics: DOTS-Plus and RNTCP3 • Adaptation of SOW (A “Framework-cum-questions”18 • Interview of key informants (see full list in Annex 3) was developed, and approved by USAID/India; see Annex 1) • Validation of received information • Field visits: Gujarat, Andhra Pradesh (AP), Uttar Pradesh (UP), Rajasthan and Maharashtra • Inventory of activities • Key Informant Interviews documenting views of authorities (DDG, DTOs, STOs, ex-STOs, etc.) • Consensus-building among evaluation team on findings, conclusions, and recommendations Team Planning Meeting (TPM) An initial teleconference, facilitated by the team leader, was held before the evaluation began. This provided the mission members with an opportunity to review the purpose, expectations and agenda of the assignment. In addition, the TPM also: • Clarified team members’ roles and responsibilities • Established the timeline, and shared experiences and thoughts on the evaluation methodology 17 The assumption, made for this hypothetical comparison, is that the important technical and operational expertise from PATH/India, the technical back-stopping by PATH/USA and the important resources provided by USAID provide significant advantages to PATH in the design, implementation and follow-up of the various lines of actions undertaken. 18 This refers to the framework for the review, including the specific questions and issues in each section that were raised by the experts. Evaluation of Tuberculosis Program in India: PATH Report 12 • Allowed the team to exchange ideas about the data collection tools and guidelines Site Visits and Interviews • A thorough review of the various projects was carried out through site visits and interviews of project staff, and key informants. • Interviewees included key members from all stakeholder groups, including RNTCP, WHO, PATH, other donors, partners in TB control, and beneficiaries. • An interview questionnaire was prepared and then presented to USAID/India for comments and their written approval. Various site visits were carried out, with special focus on the assessment of specific activities of PATH and/or WHO. Given that PATH carries out its activities in five states, site visits were carried out in all those states. • The team evaluated the state and district level periodic reports to take stock of the performance indicators. The Evaluation Framework The framework of this evaluation focused on addressing a general set of questions about the program’s management and operations, along with a set of specific technical questions regarding the level of achievement of PATH’s various program objectives, identifying contributing factors and barriers that determined this level of achievement of the objectives, and whether PATH acquired any specific niche for providing technical assistance in India. Based on the evaluation, the team prepared a set of recommendations for each of the activities. The details are given in Annex 1. FINDINGS PROCESS FINDINGS ABOUT THE MANAGEMENT AND IMPLEMENTATION OF THE PROGRAM Implementation Arrangements, Management, and Program Monitoring The most important factor influencing PATH’s ability to plan, manage, and monitor its TB project in India is that it is currently financed through a USAID/Washington-managed contract. This incrementally-funded task order (through the USAID/W TB IQC) has several implications for both PATH’s and USAID/India’s ability to successfully implement the project. First, it has resulted in a cumbersome internal management structure, as illustrated in Figure 1, below. Evaluation of Tuberculosis Program in India: PATH Report 13 Figure 1. Project Management cycle of PATH The management structure above seems unnecessarily burdensome for a project that needs to move forward swiftly with its implementation. Currently, PATH/India’s TB pipeline is approximately $4.3 million and it continues to have a low burn rate. While PATH/India and USAID/India both report a good working relationship and frequent communication, approvals for items, such as work plans, and activities, and lab improvements, remain slow. A move toward a bilateral agreement, or situating the TB project within a larger, bilateral health-systems-strengthening project, would remove some of these barriers without losing the quality control that PATH/HQ provides. It should be noted that while a move toward a bilateral agreement might provide USAID/India with additional management controls in the form of direct approvals, it is predicated on USAID/India having sufficient staff within both its technical unit and its contracting staff to manage the project. Additionally, the decision to fund the project incrementally has hindered PATH’s ability to undertake long-term planning, hire and retain high-quality staff, and even develop a performance monitoring plan (PMP). Currently, project monitoring is done through an annual activity matrix that consists almost solely of output-level indicators. Thus, the ability to measure PATH’s larger contribution to the RNTCP has been limited from the start. In addition, staff can be hired only on one-year contracts. PATH reports reluctance by many individuals to apply for a position or stay for a long period. Whether this has affected the quality (and quantity) of the staff hired by the project is unknown. Likewise, PATH’s internal management of the project appears to be hampered by a number of factors. It should be noted that this project started primarily as the initiative of one individual (currently, the project director) a few short years ago. As it has grown during the last two years (the first official approval of the work plan was given in January 2009), it has suffered growing pains. As the internal management structure has increased, it does not appear that there has been a concurrent Evaluation of Tuberculosis Program in India: PATH Report 14 delegation of authorities. It was observed by several members of the evaluation team that requests for technical and administrative information from PATH staff often resulted in being referred back to the project director. Given the growth of the PATH in-country staff, the project needs a management structure with more devolved responsibilities and authorities. Indeed, PATH’s in￾country staffing (see Annex 4) is currently insufficient and the addition of a deputy director and/or team leaders might alleviate some of these challenges. Finally, while some of the staff has been trained on managing USAID contracts (PATH as a non-profit usually enters into grant agreements with USAID and other partners), there is an urgent need to continue to strengthen this knowledge. The lack of delegation has resulted in another observation by the Evaluation Team and external stakeholders, namely, that PATH’s decision-making process is not always transparent. While PATH has developed internal guidance for roles and responsibilities, the flow of information between PATH and its external partners (and even within its own office) has not been ideal. For example, when questioned as to whether they understood why PATH had made particular technical or geographic choices, in many instances partners could not provide an answer. Information bottlenecks may have also affected CTD’s ability to comprehend PATH’s role in TB control in India, with the result that the CTD did not give greater priority to understanding PATH’s work plan or grant it unequivocal approval. The lack by PATH’s TB project (which is based in Hyderabad) of a senior-level person based in Delhi also limits PATH’s ability to have face-to-face interaction with CTD. While some of PATH’s staff believes that the need for a presence in Delhi is overestimated, it seems that the common sense rule of “out of sight, out of mind” at least partially applies. As noted previously, the ability of either USAID or PATH to monitor this project for higher-level impact is limited due to the lack of a PMP, which impedes attributing results to either USAID’s TB strategy log frame or to the RNTCP. While PATH does send some of its results, such as PPM referrals and lab accreditations, to local counterparts, none of them reported using the results for programmatic decision-making. To date, monitoring and evaluation (M&E) generally has been given a low priority in the project. There is currently no M&E officer; functions are split between the project’s administrator and the technical staff, with backstopping from PATH’s headquarters. An M&E officer position was advertised in July 2010, but remains still unfilled. Likewise, there is no project-specific M&E plan. Within PATH’s Activity Matrix, currently more than forty indicators (derived from work plan activities and state implementation plans) are routinely reported to USAID, and these are also utilized for internal program monitoring. This number of indicators is excessive. Further, the activity matrix is not structured so that the targets can be easily compared with the results achieved. Routine monitoring efforts do appear to be sufficient for the level of data needed, although there does not appear to be any great effort to move into higher-level data analysis. Data is collected, collated, and disseminated, but is not rigorously analyzed. No programmatic evaluations have yet taken place and only basic data quality assurance is performed. Core Recommendations 1) PATH should consider both increasing its staff and changing staffing patterns. There is an urgent need for PATH to increase its staffing, ensure that staff is given sufficient Evaluation of Tuberculosis Program in India: PATH Report 15 autonomy, and ensure that information flows improve. This should include, at a minimum, hiring additional senior-level staff while providing them sufficient authority to make programmatic decisions. This delegation of responsibility should, by its nature, also improve information flows. Additionally, PATH must seriously examine whether the lack of Delhi presence has hindered its ability to build its clout and the perception of its technical capacity with in-country partners and, if so, must remedy this by hiring a well-respected, highly￾qualified Delhi-based senior advisor. Finally, PATH HQ must ensure that it is providing regular and more frequent in-country technical and managerial supervision, and USAID must be supportive of these visits. 2) PATH must strengthen its M&E function. While it is noted that PATH’s current structure does not require a large amount of effort for M&E, a number of gaps still exist. PATH needs to rationalize its M&E functions within an M&E officer position, give that person the task to develop an M&E plan, and revisit and reformulate the supporting documentation, particularly the activity matrix. As noted previously, there may be an even greater need for dedicated M&E resources, if PATH is requested to develop a PMP and subsequently provide baselines and systematic monitoring of targets for those indicators. Additional Recommendations 1) USAID/India must move away from incremental funding toward a longer-term project. As noted several times previously, the current project has no PMP and thus cannot measure any higher-level outcomes. Given the move within USAID toward evaluating its “value for money” in financing development work, it would behoove USAID/India to establish a structure that allows for the design, monitoring, and evaluation of activities in which it has invested, from the very beginning. Even if USAID/India decides to move toward a new design for its TB efforts, it can still request that PATH establishes a PMP and subsequent baselines while the new project is designed. This would also require that USAID be supportive of a shift in PATH’s budget toward M&E such that it can obtain baselines for higher-level indicators as well as establish sufficient M&E capacity to collect the on-going monitoring data. 2) USAID/India should consider moving to a competed, bilateral cooperative agreement or explore the possibility of longer-terms task orders. A bi-lateral relationship should afford the project fewer approval delays as well as give USAID/India more direct oversight and input into its technical approach and management decisions. Shifting from a contract to a cooperative agreement would provide the project greater flexibility to refine activities in response to evolving in-country needs (currently, a contract modification can take between four to six months), in addition to alleviating the pressure on both PATH and USAID staff of managing a contract. 3) The trend in international public health during the past several years has been to move away from vertical programs toward more integrated programming. The vertical stove-piping of well-funded TB (and, more importantly, HIV) programs has resulted in a depletion of resources for other health sectors. USAID/India should consider embedding its TB efforts in a larger, and longer-term, health-systems-strengthening project that would draw upon a consortium of partners, both to work at a larger scale and to seek multiplicative synergies. Evaluation of Tuberculosis Program in India: PATH Report 16 4) PATH should consider forming a Technical Steering Group. This group could assist PATH in several ways including: (1) building its in-country technical credibility; (2) providing further transparency in its decision-making; (3) ensuring that PATH’s work plans have the full backing of multiple partners, including the CTD; and (4) providing a forum from which it can better coordinate its efforts with other organizations. 5) At some point in the near future, PATH should develop stronger linkages with the Tuberculosis Research Centre (TRC), possibly through WHO or with other reputed research centers in the country, so that the results it obtains from its activities, including its pilots, might provide the research center with information to conduct Operational Research (OR) on PATH’s activities, in addition to PATH developing its own, internal OR capacity. MONITORING AND EVALUATION (TECHNICAL ASSISTANCE TO GLOBAL FUND ROUND 9) As part of its current work plan, PATH will provide M&E technical support to Global Fund Round 9 (GF R9) Principal Recipients (PR). While areas of support have been broadly outlined in a concept note, it is not clear what specific technical assistance PATH will provide, since this activity is still in the planning stages. Further, given PATH’s current need to build its own internal M&E capacity, it is questionable whether this is an area in which PATH should be asked to provide assistance to other organizations. However, at least one group, which had attended and participated in PATH’s training, interviewed during this evaluation was very positive about the instruction they had received and recommended that PATH continue in these efforts. In broad terms, PATH proposed to identify gaps within the PR, Sub-Recipients (SR) and Sub-Sub￾Recipients (SSR) M&E systems and strengthen their systems. This may include developing and then jointly implementing supportive supervision tools, performing data quality audits, and conducting facilitation meetings. If providing M&E technical support does become a core responsibility for PATH, then it should further consider supporting the RNCTP’s efforts in these areas. Some of these efforts, as noted by both the RNCTP3 documentation and the JMMs, could include: • Capacity building for district and state managers, in conjunction with WHO, to interpret data and problem-solve • Serving as external consultants to assess the validity and reliability of data • Strengthening the feedback system • Piloting efforts to include private sector data • Assisting with creating a body of evidence to justify the greatly increased funding as part of RNTCP3 Core Recommendations 1. If PATH could prove that it has, or will have in the near future, sufficient in-country M&E capacity, it should then move forward with the provision of M&E technical assistance to Global Fund Principal Recipients, with a primary focus on training. However, PATH should also have biannual reviews to perceive if this TA continues to be needed or desired Evaluation of Tuberculosis Program in India: PATH Report 17 by Global Fund partners and, if so, if PATH’s provision of this technical assistance is effective and the best modality. Additional Recommendations 1. If PATH’s M&E provision to Global Fund PRs appears to be successful, USAID/India should consider entering into discussions with PATH and other partners, such as CTD, about whether PATH should expand its role in M&E technical assistance provision to RNTCP stakeholders. As outlined above, the needs are great and, if PATH can provide sufficient evidence that it possesses the internal capacity to provide this kind of assistance, it should be encouraged to expand its role. 2. Utilize PATH’s experience sharing workshops as a forum for lessons-learned and best practices on M&E. Currently, within India there are several states and organizations implementing innovative practices in M&E, but there is no coordinated forum to share these lessons. Most workshops, PATH-hosted or otherwise, focus on other TB technical disciplines, with M&E in a supportive role. PATH could organize a workshop that focuses on M&E as its own technical specialty. OUTCOME AND RESULTS FINDINGS ABOUT THE PROJECT’S ACHIEVEMENTS AND CONSEQUENCES, CONCLUSIONS AND ACTIVITY SPECIFIC RECOMMENDATIONS The activities undertaken by PATH intend to: • Improve the national capacity to provide high-level expertise on infection control. • Strengthen intermediate reference laboratory capacity to attain and maintain accreditation • Build an evidence base to monitor effective expansion of advocacy, communication, and social mobilization (ACSM) activities. • Effectively engage other providers and segments of society in TB control activities to support RNTCP goals and objectives • Support effective expansion of MDR-TB control activities by identifying and addressing gaps in the DOTS-Plus program • Test innovative approaches to improving TB case detection, treatment success and preventing MDR-TB PATH Airborne Infection Control (AIC) Risk of Noso-comial Transmission and Control Measures TB patients are a source for both noso-comial (patient-to-patient) and occupational (patient-to￾healthcare worker) M. tuberculosis transmission. Infectious MDR-TB patients serve as even greater Evaluation of Tuberculosis Program in India: PATH Report 18 potential infectious sources because they often remain AFB smear and culture positive for months, even years. Case reports from India highlight noso-comial transmission in health care settings, leading even to fatal incidences, as happened to a young health care worker infected with noso￾comial XDR-TB. 19,20 Several outbreaks in the United States demonstrated the role that hospitals can play as focal points of MDR-TB transmission, as there remains a substantial risk for noso-comial transmission of MDR-TB at tertiary care hospitals, due to intra-hospital delays in diagnosis and start of treatment, and the length of time those patients remain infectious during their hospital stay. The infectiousness of MDR strains is still subject to discussion; molecular epidemiological studies comparing the spread of drug-resistant strains to that of drug-susceptible strains have yielded conflicting results: MDR strains can be up to ten times more transmissible than pan-susceptible strains, although cases of equal,21 or even ten times lesser transmissibility have been reported. 22 Inadequately treated MDR-TB patients play an important role in the transmission. The effectiveness of implementing environmental control measures in Outpatient Department (OPD) and MDR wards has been documented.23,24 In general, the primary focus of national TB programs in high￾prevalence, low-income countries is to expand the national TB control program, but limited attention is given to preventing noso-comial transmission. Three types of AIC measures have been recently adopted by the Indian Government: 25 administrative control measures, which identify persons with respiratory symptoms and separate them into appropriate environments; fast-track them through the health care facility to reduce exposure time to others; and diagnose/treat them with minimal delay. Environmental control measures are the second line of defense for preventing the spread of TB in health care settings; they include ventilation (natural and mechanical), ultraviolet germicidal irradiation, filtration and other methods of air cleaning. The use of personal protective equipment constitutes the third line of defense, which is especially required in high-risk situations, such as while handling drug resistant tuberculosis patients/sputa, and during high-risk aerosol-generating procedures such as bronchoscopy or sputum induction. Need for AIC Measures in TB control The effectiveness of the protection for both patients and health care workers from M. tuberculosis infection depends on a good understanding and an adequate implementation of the AIC guidelines. The optimal combination of those control measures should be based on the assessment of the 19 USAID/India. “5-Year Tuberculosis Strategy 2010-2014,” (Draft) 20 Francis J. Curry. Tuberculosis Infection Control: A Practical Manual for Preventing TB, National Tuberculosis Center, 2007. 21 Yoshida, S., K. Suzuki,K. Tsuyuguchi, et al. “Molecular Epidemiology of Mycobacterium tuberculosis--Comparison between Multidrug￾Resistant Strains and Susceptible Strains. Kekkaku. 2007 ;82(6): 531-538 22 Borrell, S., S. Gagneux “Infectiousness, Reproductive Fitness and Evolution of Drug-Resistant Mycobacterium tuberculosis.” International Journal of Tuberculosis Lung Disease, 2009; 13(12): 1456–66. 23 Natural methods of encouraging airflow (e.g., opening doors and windows) work well and in theory could reduce the likelihood of TB being carried from one person to another. Some aspects of the design of wards in old hospitals (such as large windows and high ceilings) are also likely to achieve better airflow and reduce the risk of infection. In poor countries, where mechanical ventilation systems might be too expensive to install and maintain properly, rooms designed to achieve naturally good airflow might be the best choice. Another advantage of natural ventilation is that it is not restricted by cost to just high-risk areas, and can therefore be used in many different parts of the hospital, including emergency departments, outpatient departments, and waiting rooms, and it is here that many infectious patients are to be found. 24 Roderick Escombe A., D. A. Moore, R. H. Gilman, et al. “The Infectiousness of Tuberculosis Patients Co-infected with HIV,” PLOS Med 2008; Sept 5(9): e188. 25 “Guidelines on Airborne Infection Control in Healthcare and Other Settings,” Directorate General of Health Services, New Delhi, April 2010. Evaluation of Tuberculosis Program in India: PATH Report 19 infection risk and on the local epidemiological, climatic and socioeconomic conditions. Implemented correctly, these measures can reduce, and even eliminate M. tuberculosis transmission to both patients and health care workers. 26 It is important to realize that environmental control measures by themselves are insufficient to eliminate all risks. Even isolated administrative measures for infection control can significantly reduce the risk for TB infection among health care workers in high-burden countries and should be implemented, even when resources are not available for engineering infection control measures. 27 Interventions to reduce nosocomial transmission of TB are useful and cost-effective preventive measures to control TB.28 To improve the capacity of the RNTCP to prevent nosocomial transmissions, PATH proposed to develop an innovative AIC program in collaboration with the GoI. The project would pilot test guidelines, establish AIC checklists, and provide technical assistance trainings. PATH’s Objectives • To improve the national capacity to provide high-level expertise on airborne infection control and provide support for AIC scale-up. • To complement the work being performed under TASC2 TB TO2 and Year 1 of TB TO 2015. Findings PATH has done a lot of work in this area, but much of it consisted mainly in assisting WHO and GoI with the development of guidelines, studies and piloting. The activities of PATH in this domain have been carried out in a phased manner: for the year FY ’08 planning included building collaborative efforts and field activities, and in subsequent years FY ’09 and FY ’10), these activities were expanded to include training activities. Major partners in all their activities were CTD, State AIC committees of Andhra Pradesh, Gujarat and West Bengal, WHO-SEARO, and Partners-In-Health (PIH). PATH participated in the National Airborne Infection Control Committee as an “invitee.” PATH, along with the WHO and GoI, was instrumental in conducting a national-level workshop at New Delhi for pilot states. PATH facilitated the development of the action plan on AIC for AP, which was approved by CTD. AIC Capacity Development: Training on AIC is one of PATH’s main activities. In 2010, trainings were conducted in Kolkata (West Bengal), Ahmadabad (Gujarat) and Hyderabad (AP) for a total of 26 Jensen P. A., L. A. Lambert L A, et al. “Guidelines for Preventing the Transmission of Mycobacterium tuberculosis in Health Care Settings,” CDC, 2005. MMWR Recomm Rep 2005; 54: 1–141. 27 P. Albuquerque da Costa, A. Trajman, F. Carvalho de Queiroz Mello, et al. « Administrative Measures for Preventing Mycobacterium tuberculosis Infection Among Healthcare Workers in a Teaching Hospital in Rio De Janeiro, Brazil,” J Hosp. Infect., 2009; 72: 57-64. Evaluation of Tuberculosis Program in India: PATH Report 20 130 participants, including members of state AIC committees from pilot sites. A first of its kind training for architects, engineers, and state health officials was conducted in January 2011. The training focused on building design and environmental approaches to AIC for five states (Andhra Pradesh, Gujarat, Maharashtra, Karnataka and West Bengal). For this activity, PATH recruited two staff that were technically very qualified and experienced in infection control and especially in AIC. PATH, with CTD and WHO support, conducted systematic baseline assessments of AIC measures (administrative, environmental and personal protective measures) and practices at the thirty-four selected health care facilities (HCF) in three states. This activity was trailed closely by capacity building for district officials, HCF administrators and IC focal points, and provided specific recommendations for implementation. Unfortunately, there has been limited post-training supportive supervision. PATH’s partners that had provided technical support on AIC piloting, did the procurement of AIC assessment kits and acted as external facilitators for baseline healthcare facility risk assessments, the state-level AIC workshop, and the in-country training of engineers on AIC at Hyderabad, and they assisted in the preparation of the reports. PATH supported the training of four engineers at Harvard School of Public Health, Boston, USA. Implementation of AIC Guidelines: The team found that AIC guidelines and assessment reports of facility risk assessments carried out in three states (West Bengal, AP and Gujarat) were in place. Field visits to the labs and MDR-TB wards showed that the health facilities had made use of the AIC assessment results and had introduced appropriate modifications in their infrastructure through civil works initiatives. However, those modifications were limited in scope due to constrained funds. In the IRL, the lab personnel follow infection control measures. However, knowledge of personnel protection measures and practices of health staff working in the MDR/TB wards were not optimal; we found no evidence that staff working with MDR-TB patients wear N95 masks. MDR/TB patients also were found not adhering to control measures. State officials expressed their full appreciation of this AIC activity and wished to do similar assessments of other facilities in their respective state. The CTD wants PATH to expand this activity further in other states. PATH confirmed having plans to expand AIC assessments, subject to CTD’s explicit approval. Two of the planned activities under AIC had been cancelled (RIPC module and KAP survey), due to delayed approval and late communication from CTD (Risk assessment reports were not opportunely finalized by CTD, pending discussion with CTD and National Centre for Disease Control, and the draft protocol is still awaiting ethics committee clearance). The frequent turnover of WHO consultants at CTD produces delays in communicating decisions and follow-up from CTD. As suggested by the CTD, a follow-up assessment of health care facilities to evaluate the impact will be carried out in September–November 2011. A plan was developed to scale up this activity to the entire initial states; a decision will be taken in the next National AIC Committee meeting. Evaluation of Tuberculosis Program in India: PATH Report 21 During the field visits it was noted that the AIC engineering activities focused only on air exchanges, overlooking the need to protect for vector-borne diseases, such as P. falciparum malaria, hemorrhagic dengue and chikungunya. Recommendations To ensure that the achievements of PATH’s AIC project are sustained and expanded, USAID/India should consider the following recommendations: Core recommendations: 1. PATH should develop training material on the subject of AIC, and field-test and validate these materials. These materials must be translated and made available in those languages spoken in PMDT trial sites. PATH should assess the impact of these training materials and revise as and when needed, since this will help to expedite planning and execution of infection control at the community level. 2. Follow-up action at healthcare facilities for assessing the impact should be considered. Pilot sites’ findings should be discussed at national AIC committee meetings for further action. Given the risk of vector-borne diseases for patients admitted in MDR/TB wards, the AIC measures should be holistic in their approach, and not overlook the need for bed nets and other appropriate protection measures in function of the epidemiological risk profile of the area. Additional recommendations: 1. USAID/India should support and encourage PATH to take a leadership role in the implementation of the AIC measures. 2. To make this AIC activity sustainable, PATH should consider involving more partners in future dates, thus extending the scope and the geographical coverage of the AIC activities. 3. PATH should consider using NRHM funds to facilitate scaling up of AIC at HIV/TB facilities and at those institutions managed by the Indian Medical Association. Strengthen Intermediate Reference Laboratory Capacity The RNTCP has endorsed the ambitious scale-up targets for MDR-TB services: • By 2012, provide universal access under RNTCP to laboratory-based and quality-assured MDR-TB diagnosis and treatment for all smear positive re-treatment cases and new cases who have failed an initial first-line drug treatment • By 2015, provide universal access to MDR-TB diagnosis and opportune treatment for all smear positive TB (new and retreatment) cases registered under RNTCP To meet the above targets, there is an urgent need to strengthen the laboratory capacity to undertake LPA and C&DST. Evaluation of Tuberculosis Program in India: PATH Report 22 The IRL network forms the backbone of the diagnosis at state level, as it performs the necessary reference functions of monitoring the quality of the RNTCP smear microscopy services and providing quality-assured LPA, culture and DST facilities. PATH’s objectives in the domain of strengthening IRL capacity • To strengthen the laboratory network’s capacity to diagnose TB • To accelerate the accreditation of selected IRLs and build the capacity of their staff to perform reference laboratory functions, including solid culture and DST Findings Based on three work plans (FY’08, FY’09, and FY’10) and their respective revisions/carry-overs, PATH proposed the following activities: • Upgrade/renovate laboratories (FY’08) • Assess IRLs (FY’08, FY’09, FY’10) • Develop and implement an IRL accreditation action plan (FY’08, FY’09) • Engineering assessment of all IRLs to identify infrastructure needs (FY’09) • Complete infrastructure upgrades, including essential equipment, at selected IRLs based on engineering and architectural evaluations (FY’09, FY’10) • On-site capacity building for IRL activities (FY’08, FY’09, FY’10 ) • Follow-up monitoring at accredited IRLs to maintain quality and assure re-accreditation (FY’09, FY’10) • Provide TA for proper operation and maintenance of laboratory equipment (FY’08, FY’09) • Perform Lab HR assessment for IRLs and prepare lab staffing plan (FY’09) • Support and facilitate ongoing IRL experience-sharing meetings (FY’09, FY’10) • Participate in the national lab committee (FY’08, FY’09, FY’10) The details of their implementation are given in Table 1. The accreditation process has been slow, as a result of a number of barriers, including infrastructure deficiencies, lack and turnover of technical staff, lack of adequate training in culture and DST techniques, and deficits in political commitment at the respective states. PATH has noted that a negative factor for implementing the accreditation process is the lack of infrastructure to perform culture and DST accurately—particularly lack of power supply, in spite of the provision of back-up generators. PATH is of the opinion that a thorough assessment of the infrastructural problems is needed, and that a concrete action plan has to be developed in order to speed up the accreditation process. Therefore, PATH’s 2009-2010 action plan showed that these barriers would be addressed in consultation with CTD, WHO and the respective states in order to accelerate the accreditation and ensure sufficient IRL capacity to diagnose TB, and especially MDR-TB. Evaluation of Tuberculosis Program in India: PATH Report 23 In the field, it was observed that the accredited are still not fully proficient in QA, and need ongoing support to be re-accredited at the end of the second year of operation. Table 1: Implementation of IRL strengthening activities Activities Progress Eventual deviations from the planning Upgrade/renovate laboratories IRL/Hyderabad required minor renovations and procurement of biosafety equipment necessary to meet the Biosafety Level 3 (BSL3) standards. Tamil Nadu (TN) State Government requested CTD to shift Chennai IRL from Chetpet to Tamabaram sanatorium; consequently, upgradation is still pending. Assess IRLs In FY’08: 8 IRLS selected following priority criteria in FY’08 (Cuttack, Dehradun, Karnal, Kolkata, Lucknow, Ranchi, Chennai and Raipur). Assessment visits carried out in April-May 2009 In FY’09: IRLs assessed in Arunachal Pradesh & Assam Engineering assessment IRL Kolkata assessed CTD did not give consent for a separate engineering assessment; PATH was asked to visit only those IRLs that require upgrading support. The visits will be scheduled after the finalization of the upgrading Infrastructure upgrades Upgradations were carried out in State TB training and demonstration center (STDC), Hyderabad. Initial renovation work completed in Dec 2009, further work was done, including completion of BSL-3 upgrades29 LPA upgrades has been completed recently in the IRLs in Cuttack, Puducherry and Lucknow , installing air curtains and putting a roof over the handling units; this was finalized in August 2010 Critical renovations at four IRLs (Cuttack, Puducherry, Lucknow & Indore to ensure that accreditation/re￾accreditation activities can move forward, and that LPA can be carried out Equipment installation for Vizag Medical college Laboratory (AP) & Regional medical research center in Patna planned by Sept 2011. Work is progressing well Approval from CTD received on February 3, 2011 for LPA upgrades for IRL Patna, IRL Hubli medical college On-site capacity building In FY’08: capacity building for IRL activities was planned In FY’09: Capacity building for IRL staff was planned, and was intended to be carried out jointly with the engineering and architectural evaluation supported under TASC2 TB TO2. PATH intended to adapt the training to the needs of each lab, and carry out such training at the IRL meeting and through on-site training of 5 accredited labs (Cuttack, In FY’08: No reports on such capacity building is available and no documents were shown to this review team about written EQA, QA and QC procedures; hence there are serious doubts that any such training was provided. This activity was delayed, because CTD wanted to discuss it first in the lab committee (which did not meet) 29 All laboratories are Biosafety Level 2 (BSL-2) facilities. Only two IRLs (the one in Hyderabad and in BPHRC, Hyderabad) were designed and constructed to reach the BSL-3 level Evaluation of Tuberculosis Program in India: PATH Report 24 Lucknow, Dehradun, Ranchi and Hyderabad). Experience-sharing workshop organized in Nov. 2009 and March 2011, with the participation of NRLs as supervising entities, Follow-up monitoring at accredited IRLs to maintain quality and assure re￾accreditation FY09: 5 IRLS attained accreditation: Ranchi, Kolkata, Lucknow, Chennai & Cuttack. Accreditation of IRL Dehradun and Karnal is in process. In all those IRLs assessment visits and on-site trainings took place >95% concordance between the IRL and NRL results were obtained, except in IRL Hyderabad, Gujarat and Kerala where the concordance reached a 90% concordance TA for proper operation and maintenance of laboratory equipment Manuals have only recently become available; Preventive maintenance checklist of equipment was developed in 2010. FY09: PATH had planned to train the lab staff in diagnostic equipment maintenance SOPs at IRL semi-annual meetings, as part of the ongoing experience sharing workshops; and reinforce the messages during on-site TA visits by PATH laboratory officers and consultants in conjunction with the respective NRLs having supervisory responsibility for those labs. The implementation of this activity through experience-sharing workshops has been postponed, pending approval from CTD Lab HR assessment for IRLs and prepare lab staffing plan This activity had been approved by CTD in the work planning stage; however CTD halted its implementation mid-way, and requested HR elements to be incorporated into the overall IRL assessment checklist. Support and facilitate ongoing IRL experience￾sharing meetings It was intended to hold two meetings per year, each meeting focusing on one or more technical issues related to lab performance, providing newest information on technologies that will be introduced at the IRL level, and organizing a hands-on session on SOPs and lab practices. A workshop was organized in Nov 2009 and another in March 2011 Only 2 meetings were held: 1 in 2009 and another in 2011, because CTD requested to limit this activity to an annual rhythm. However, during the meeting, there was no real discussion amongst the participants; hence there has been no real experience￾sharing. Participate in the national lab committee The target was to attend 3 such meetings; however, no formal lab committee meetings have been conducted since Feb 2010. Nevertheless, PATH staff participated in lab co￾ordination meetings between CTD, WHO, FIND and PATH The participation in the national lab committee is dependent on its level of functioning, which is still pending for the moment Evaluation of Tuberculosis Program in India: PATH Report 25 Conclusions 1. PATH’s involvement in lab strengthening is characterized by good effectiveness, and certainly by grateful recognition by the authorities and the beneficiaries. Its cost-efficiency is more difficult to judge, as there are no benchmarks yet against which to compare PATH’s involvement, although the money seems to have been spent adequately. 2. PATH’s role as capacity builder for laboratories is recognized by the partners, including CTD, WHO, and the Foundation for Innovative New Diagnostics (FIND). 3. Despite the bureaucratic hurdles, PATH has been able to catalyze laboratory accreditation and promote Laboratory upgrades. 4. The initiatives taken by PATH, in consultation with FIND, WHO, and CTD, in the establishment of molecular biology laboratories for LPA and liquid culture have been effective. This is evident in the establishment of four such accredited laboratories and two BSL level-3 laboratories. 5. The manuals developed by PATH30 are of high technical quality and will be very useful for other laboratories as well. 6. So far more than 375 cases of MDR-TB have been diagnosed in the IRL-Hyderabad and all started treatment. Blue Peter Public Health and Research Centre (BPHRC-Hyderabad) also initiated treatment for 175 MDR-TB cases. This is a significant achievement for the RNTCP, and PATH has been instrumental in establishing these two laboratories. Barriers and Challenges • The delay in rehabilitating a site in Tamil Nadu for LPA and BSL-3 facility has hampered the process of establishing the laboratory. • The decision by the CTD not to assess the engineering work in the IRLs became a setback for PATH to accomplish the plan. • Laboratory committee meetings have not been conducted by CTD since February 2010; this has led to delays in executing the work plan. • PATH’s support to strengthen the IRLs requires coordination between WHO, CTD, FIND and state TB cells. Delays in coordination between these agencies affect PATH’s ability to accomplish many of the objectives. • The stability of human resources is evident in all the laboratories visited except in Tamil Nadu, where there is an excessive turnover of contractual staff due to administrative issues. 30 (1) “Preventive maintenance and troubleshooting for equipment at IRLs”, (2) “Biosafety practices and infection control for laboratories” Evaluation of Tuberculosis Program in India: PATH Report 26 Recommendations 1. Given the important role other actors, such as FIND, play in lab strengthening in India, it is imperative that PATH defines its proper niche better and demonstrates its comparative advantage in a more convincing way. 2. PATH should ensure that it has sufficient senior technical staff capacity in-country to guide its technical assistance and to provide the expected services in an undisputable, expert manner. 3. PATH should hire a technical expert or an instrumentation engineer to rectify equipment issues in IRLs, such as problems with biosafety cabinets, walk-in incubators, autoclaves and deep freezers. 4. PATH should organize annual meetings, inviting consultant microbiologists of all the IRLs in the country along with other stakeholders, to get feedback on situation analysis and troubleshooting. 5. PATH should take care to prove that its activities are being carried out in a time- and cost￾effective manner 6. USAID should continue to provide financial support for PATH to take an active role in strengthening IRLs. ACSM Activities In 2005, CTD approved a national health communication strategy to develop awareness of TB symptoms, diagnosis and treatment services. The strategy encouraged health-seeking behaviors and treatment adherence through improved patient and community education and by strengthening patient-provider interpersonal communication.31 To align itself with the Global Plan to Stop TB,32 produced by WHO in 2005, CTD modified its own strategy in line with a new global initiative called “Advocacy, Communication, and Social Mobilization” (ACSM). Using this approach, CTD would continue to raise awareness about TB and encourage behavior change related to case detection and treatment adherence. In addition, ACSM strategies would now be used to create, facilitate and forge political, administrative and community-level commitment to TB control in India and to encourage patients and their families to become advocates for the program. On a national level, CTD provides states and districts with resources and capacity building, while states and districts hold the responsibility to plan and include ACSM activities into their action plans (see Table 2). Overall capacity to design and manage ACSM interventions, however, remains limited. Many district plans are not evidence-based;33 31 CTD. “A Health Communication Strategy for RNTCP. CTD, New Delhi,” Nov 2005, pg. 180. responsibility to carry out activities is placed on over￾burdened program staff; and there is limited assessment or documented evidence about the 32 This is a WHO initiative, taken in 2005 33 JMM Report, New Delhi, 2009 Evaluation of Tuberculosis Program in India: PATH Report 27 effectiveness of these activities and their contributions to improved TB control program performance. Table 2. Government Roles for Planning and Conducting ACSM CTD States Districts • Overall leadership • Procure services of ACSM agency • Coordinate activities • Arrange national mass media • Assess and support, capacity building for ACSM at state level • Develop state plan for ACSM • Support district planning and implementation • Monitoring and supervision • Mobilize support/resources and involve other government departments • Develop materials in local languages • Organize events for advocacy • Capacity building of districts • Develop plan for ACSM activities • Use local appropriate medium for dissemination of information • Seek involvement from local organizations, leaders, panchayats, and NGOs • Organize minimum number of activities PATH’s Objectives To support the CTD, state and district personnel, as well as non-governmental organizations (NGO) funded by the Global Fund Round 9, to develop their capacity to plan, implement and evaluate ACSM activities, and to develop evidence of the role of ACSM to improve TB Control program performance, PATH proposed the following objectives: • FY’08— Support RNTCP health communication efforts by moving the existing comprehensive ACSM strategy forward into results-oriented field activities. • FY‘09 — Test and scale up an approach to developing, implementing and evaluating ACSM interventions to address TB control issues in poor-performing Tuberculosis Units. • FY ’10—Expand effective, evidence-based ACSM interventions to improve TB control outcomes. Capacity development will be achieved through training, providing technical assistance, facilitating experience sharing, and finally, by providing evidence that ACSM contributes to improved TB control performance and outcomes. Findings PATH proposed six implementation strategies to achieve its overall ACSM objectives: • Build ACSM capacity in RNTCP staff at national and state levels (FY ’08; FY ‘09). • Provide ACSM support to the NGO TB Consortium (NTC) and support Global Fund Round 9 Principal Recipients in capacity-building for ACSM M&E (FY ‘09) (FY ‘10). • Work with USAID/India partners to identify synergies between projects and develop a plan for collaboration (FY ‘08 and ‘09). • Expand ACSM implementation in five states to support improvement in TB control indicators (FY ‘10). Evaluation of Tuberculosis Program in India: PATH Report 28 • Conduct experience-sharing workshops to share best practices, review data and M&E practices, and provide peer support for problem-solving (FY ‘10). • Build an evidence base to inform effective expansion of ACSM activities (FY’09, to 2015). By using the three work plans (FY’08, FY’09, and FY’10), a total of nineteen sub-activities were identified. (See Table3). PATH has completed, or initiated, ten of these sub-activities. PATH supported four national training initiatives, and developed and implemented its eight training workshops on ACSM, reaching a total of 197 persons. PATH’s training activities were favorably reviewed through course evaluations and this was confirmed by participants interviewed during the evaluation. A number of NGO attendees report using the ACSM training materials for training other staff and partners about ACSM and for planning ACSM activities. For example, the Union34 trained an additional eighty persons. While PATH did not survey workshop participants prior to the trainings to identify pre-workshop practices of planning and implementing ACSM practices, it did conduct pre- and post-tests which show knowledge improvements. PATH is currently conducting a survey of participants which may help to identify practice change. During the evaluation, one district senior treatment supervisor stated that the trainings helped him to improve his communication with patients, while other districts report using program data for planning ACSM activities. The evidence for capacity￾development, however, is limited. While PATH is viewed as an effective provider of basic trainings on ACSM, some organizations reported that more practical and advanced trainings are required. Specifically, they cited the need for more training on advocacy. Another issue that emerged during the evaluation is that CTD funded another organization, Social^Rural Direction, 35 a subsidiary of RK Swamy BBDO, to provide similar workshops for all states (state IEC officer, STOs, DTOs, communication facilitators, WHO consultants and NGOs). While the assessment of Social^Rural Direction’s trainings fell outside the ToRs of this evaluation, it raises some concern about duplication. 34 The UNION, means the International Union for TB and Lung Diseases, with headquarters in Paris. The Union organises each year a big international TB meeting and publishes the International of TB and Lung Diseases, which is THE forum to publish papers related to TB control. 35 Social^Rural Direction is a private firm that develops communication strategies and social marketing campaigns. Evaluation of Tuberculosis Program in India: PATH Report 29 Table 3. Implementation of ACSM Strategies and Activities Program Strategies Activities Progress Build ACSM capacity in RNTCP staff at national and state levels (FY ’08; FY ‘09) 1. Support CTD focal point 2. Act as resource persons during 4 trainings provided to state-level RNTCP staff. 3. Provide ongoing support as requested, to develop, implement, and evaluate ACSM activities. 4. Replicate workshops for additional staff in charge of ACSM activities. 5. Support the IEC Officer in AP and staff in 4 poorly-performing TUs. 6. Document lessons learned and best practices in collaboration with the State IEC Officer and support TU staff to present findings 7. Develop recommendations for scaling up the process based on experience 8. Participate as a member of the RNTCP’s advisory board on ACSM 1. Limited due to lack of CTD focal point; new staff recently hired 2. Assisted in 4 national workshops 3. No requests received 4. PATH conducted 4 trainings for state and district staff in AP and other states. 5. Pilot activities initiated in 2 districts, Warangal and Cuddapah, but no detailed documentation about PATH contributions. 6. Lessons learned not yet documented. 7. Initiated planning for scale up; identified new states (Maharashtra, Gujarat, Uttarakhand, UP, MP, Kerala); clarified TA activities 8. PATH is a member of RNTCP ACSM advisory board. (Provide ACSM support to the NGO TB Consortium (NTC) and support Global Fund Round 9 Principal Recipients in capacity￾building for ACSM M&E (FY‘09) (FY‘10) 9. Conduct a ToT in ACSM using the curriculum developed in AP 10. Provide ACSM support to the NGO Consortium to develop ACSM plans 11. Provide support at local level for specific trainings and implementation of ACSM plans. 12. Work with the NTC to document best practices and lessons learned. 13. Develop M&E plan to measure progress and outcomes of ACSM initiatives implemented by civil society agencies. 14. PATH and Initiatives Inc. will work closely with NGO consortia representatives to develop standardized supportive supervision guidance. 9. PATH conducted four ToTs for NGO partners. 10. Convene meeting with other NGOs to review IEC materials 11. No trainings/support provided. 12. Activity not conducted. 13. Activity not conducted 14. Sub-contracted with Initiatives; no commitment from partner to proceed; decision postponed to May 2011. Work with USAID/India partners to identify synergies between projects and develop a plan for collaboration (FY08 and ‘FY09) 15. Coordination with ABT’s Market-Based Partnerships (MBP) project to assist the RNTCP in improving linkages with the private sector, 16. Develop a plan for collaborative activities in coordination with USAID, MBP, and CTD, 15. Preliminary discussions with ABT, but no action taken yet 16. No action taken yet Expand ACSM implementation in five states to support improvement in TB control indicators (FY10) 17. Expand ACSM activities into six new states 17. Activities scheduled to begin in April 2011 Conduct experience-sharing workshops to share best practices, review data and M&E practices, and provide peer support for problem￾solving (FY‘10) 18. Convene joint semi-annual ACSM experience-sharing workshops stakeholders to ensure that lessons learned and best practices are well￾documented and disseminated throughout the RNTCP network. 18. No activities conducted so far Build an evidence base to inform effective expansion of ACSM activities (FY09 to 2015) 19. Use the Global Fund and State level ACSM work plans as the basis for monitoring progress, identifying specific challenges, and use group problem-solving and expertise to develop and implement specific actions to overcome them. 19. No activities conducted so far Evaluation of Tuberculosis Program in India: PATH Report 30 PATH initiated a pilot project to develop ACSM capacity in government staff working in two poorly-performing districts in Andhra Pradesh (Warangal and Cuddapah). Two tuberculosis units per district were selected, based on low case detection. PATH did not conduct a baseline study at the onset of this project to assess staff and program practices related to planning, implementing and conducting ACSM prior to implementation. PATH has not adequately defined the scope of technical assistance (TA) offered, how needs are identified, or determined PATH’s possible response to these identified needs. Moreover, PATH has not sufficiently documented the provision and potential results of TA it provided. Despite the fact that PATH did not assess the effectiveness of the pilot or identify potential operational or design issues, PATH prematurely proposed and initiated scaling up this project to six other states. On a positive note, PATH recently provided a draft document that describes the types of technical assistance services it will provide, but has not yet developed the mechanisms for assistance provision and monitoring (See Annex 5). PATH planned to initiate several activities later this year, including experience-sharing workshops and supportive supervision activities, but has not provided specific start dates. PATH has contracted with the NGO Initiatives Inc. to provide technical assistance for the supportive supervision activities. Initiatives, Inc. is currently recruiting a staff member to lead this activity. Several of PATH’s proposed activities, such as monitoring and evaluation (M&E) and supportive supervision projects, are dependent upon securing partner commitments to proceed, which PATH has to secure in a timely manner so far. For example, PATH initially identified World Vision, India (WVI) as a partner for the M&E technical assistance project, but WVI has developed its own M&E activities and does not appear interested in utilizing PATH support. As a result, PATH has been required to find new partners, such as the Union, for collaboration. The Union expressed interest in the M&E and supportive supervision projects offered by PATH, but will not make any decision to proceed until after it evaluates its own M&E system and sub-recipient activities in May 2011. In regards to another proposed collaboration with ABT’s Market-Based Partnerships (MBP) project, PATH has had limited contact with ABT Associates and reports an inability to identify a synergy between ABT and its own organization, although both are working with similar target populations (private-sector providers). Conclusions Given RNTCP3’s ambitious goal of universal access to treatment and scale-up treatment for MDR￾TB, there is an urgent need for ACSM activities to strengthen community awareness, generate demand for TB services, and support treatment adherence. The capacity of states and districts—and even GF R9 sub-recipients— to effectively plan and use different communication strategies and methods remains limited. PATH has responded to these needs by providing basic ACSM training and implementing technical assistance interventions. There is some evidence that PATH’s interventions have contributed to knowledge change about ACSM, but there is insufficient evidence to draw firm conclusions about positive contributions of PATH’s technical assistance approach to build capacity and improve ACSM practice. Evaluation of Tuberculosis Program in India: PATH Report 31 Multiple projects have also been delayed due to management issues, reducing the potential of PATH’s contributions to building ACSM capacity. In addition, many proposed activities listed “as requested” have simply not been conducted. It is not clear why these non-specific activities are included in PATH’s work plans, nor if PATH has a viable strategy for responding to any requests. Some of the delays can be attributed to delayed institutional reviews or contract approvals, but others were created by insufficient staff or partner commitments, which PATH should have foreseen or responded to by now. Weak partner agreements and commitments also adversely affect PATH’s ability to efficiently use its USAID resources. There are several factors that need to be addressed. First, PATH needs to ensure that it has senior staff capable of designing and planning its ACSM strategy. Second, the overall management of PATH in-country needs to dedicate resources to negotiating agreements with proposed partners that have clear scopes of work and implementation timelines. PATH has shown progress in adapting and delivering its international training curriculum within India, but it has not yet created a unique model or approach to carry out ACSM. Many alternative organizations have conducted similar activities. With the implementation of the Global Fund Round 9, national attention will be diverted further to other organizations. It is critical that PATH determines and defines what is unique, or qualitatively better, about its approach to show a comparative advantage. It would be useful for PATH to meet with these other organizations to understand the differences, or similarities, in the various training approaches and to identify gaps or unmet needs in the different trainings. The proposed scale up of technical assistance activities is premature, given that PATH first needs to resolve and clarify several key issues. In particular, PATH needs to improve these activities’ overall design. Specifically, PATH needs to: (1) clarify the scope of technical assistance it will and can provide given current staff capacity, (2) define how it intends to identify and respond to needs; and (3) improve how it documents and measures support provided. There is an urgent need to finalize plans for baseline assessments, secure and train staff—particularly field-based staff— and to develop a full operational management plan for how it will manage, conduct and supervise interventions across twelve districts and six states. Given the aforementioned management and design issues, PATH may not have sufficient capacity at this time for working in more than one district per state; PATH can clearly benefit from the guidance of senior, technical staff. A dedicated focus on the government sector may provide that strategic difference if PATH can support districts to:36 (1) carry out a situational analysis; (2) plan ACSM using rational planning and decision-making; (3) improve access to more funding for ACSM, to better ACSM models, and ACSM materials and more rational management of those extra resources; and (4) conduct community-based ACSM activities more effectively. For PATH’s activities to be successful, PATH needs to strengthen its overall design, ensure more rigorous implementation, and conduct thorough monitoring and evaluation with relevant outcome indicators. In addition, greater, and earlier, involvement of national, state and district leadership is needed to ensure that these critical partners understand and support the interventions. 36The focus of PATH specifically is to develop capacity of the government sector to better plan and implement ACSM. To do this, PATH needs to develop and use methods that are more rigorous, rather than follow its current ad hoc process. It does involve planning based on needs assessments and evidence. Evaluation of Tuberculosis Program in India: PATH Report 32 Recommendations ACSM provides important strategies to develop vital knowledge about TB and TB services, encourage positive health behaviors and to foster supportive public attitudes needed to address stigma, gender disparities and equality issues. With RNTCP3’s focus on universal access, addressing TB/HIV co-infection, and implementation of DOTS-Plus, significant scaling up of communication activities will be required. Additional resources and capacity will be needed to develop effective IEC materials, improve patient-provider interpersonal communication, and address community concerns. Ensuring capacity and effectiveness of new initiatives will test an already over-burdened system. Core Recommendations In order to improve the quality of PATH’s ACSM interventions, USAID/India should work with PATH to address the following recommendations: 1. Continue trainings to increase core understanding about ACSM strategies and methods, and provide more advanced and targeted training on other issues (e.g., advocacy-specific focus). 2. Improve overall partner coordination, formalizing agreements for technical assistance services it will provide. 3. Delay scale-up activities until PATH addresses management and design issues, namely, assessing the resource requirements for providing district-based TA, and defining how technical assistance needs will be determined, how responses will be provided, and how outcomes will be assessed and documented. In addition, baseline assessment plans need to be finalized prior to commencing activities in order to determine the efficacy of its approach to build capacity and improve ACSM practice. 4. Ensure that PATH has sufficient senior, technical staff capacity in country to guide its technical assistance project and to provide adequately the expected services. Engage Other Providers In India, the private sector is the dominant provider of health services. A recent national Family Health Survey37 Because the private sector is the first point of contact for most TB clients, it can increase early case detection, opportune treatment, and treatment adherence. The private sector can also reduce diagnostic delays and cost to patients. Therefore, limiting TB control to only a public-sector approach cannot reach the whole population afflicted with TB, particularly those that bypass public found that more than two-thirds of households (sixty-five percent) generally seek health care from private medical providers, and that the private medical sector remains the primary source of health care for the majority of the population, both in urban and rural areas. 37 The 2005-06 National Family Health Survey (NFHS-3) is the third in a series of national surveys; earlier NFHS surveys were carried out in 1992-93 (NFHS-1) and 1998-99 (NFHS-2). All three surveys were conducted under the stewardship of the Ministry of Health and Family Welfare, Government of India, with the International Institute for Population Sciences, Mumbai, serving as the nodal agency. NFHS-3, like NFHS-1 and NFHS-2, is a household survey which will provide estimates of indicators of population, health, and nutrition by background characteristics at the national and state levels. In NFHS-3, information is collected about households, and individual interviews are conducted with women age 15-49 and men age 15-54 Evaluation of Tuberculosis Program in India: PATH Report 33 health care delivery system- the majority of the population. Collaboration with private providers is important if RNTCP wants to reach a wider audience. Pharmacies and private clinics are widely accessible to patients and are generally perceived to provide better quality services. Patients trust private doctors. These private providers are therefore best poised to initiate first-level screening for effective case detection, which is an essential element of any TB control programs. PATH’S Objectives To strengthen the role of the private sector, PATH proposed the following program objectives: • Increase referral system from private chemists to designated microscopy centers (DMC) • Increase TB case detection and reduce diagnostic delays for appropriate TB treatment • Improve access to quality DOTS service • Discourage the sale of TB drugs without prescription Findings PATH piloted its pharmacy initiative in Ongole Tuberculosis Unit (TU), Prakasam district, Andhra Pradesh (AP) based on a suggestion made by the Principal Secretary, Health, Medical and Family Welfare of the government of AP, in consultation with DTO. Ongole TU was chosen to pilot this project, in view of the high number of migrant workers and urban poor and generally poor living conditions of the people in its catchment area. The rationale for choosing Ongole is not convincing.38 A notable achievement of this initiative is that PATH succeeded in forging the cooperation and collaboration of partners in PPM, bringing partners like the Ongole Drug and Chemist Association, the District TB Office, and the Drug Control Office and various pharmacies together to engage in a productive dialogue on issues regarding their involvement in TB care. Instruments for partnership develop by PATH As a way to bind partners to the program, PATH developed tri-partite agreements, signed by the District TB Office, the Ongole Drug and Chemist Association, and the Drug Control Administration. PATH also developed a number of promotional materials to facilitate the work and gain the acceptance of pharmacies. Materials produced include: (1) general information about TB; (2) “No to self-medication” flyers; (3); facility maps; (4) job aids on how to detect TB suspects; and, (5) referral slips. A random check of ten pharmacies showed that pharmacies have the IEC materials and are using them. The tri-partite agreement lists the responsibilities of the Ongole Drug and Chemist Association, but does not specify the responsibilities of the office of the DTO to the program. The pharmacists who joined the program signed a “willingness form” to refer clients and refrain from selling TB drugs 38 The reason given by PATH for the choice of ONGOLE is that Principal Secretary, Health, Medical and Family Welfare of GoAP in consultation with Ongole DTO suggested it . There was no other evidence presented. Will the results of the pilot in Ongole be applicable to the entire state? Does Ongole share the same profile as Prakasam District or the entire AP state?. The evaluation team did not find any evidence. Evaluation of Tuberculosis Program in India: PATH Report 34 without prescription. Referral forms are given to clients suspected of TB. The client submits the form to the DMC and a duplicate copy is kept at a drop box located at the DTO office. These “willingness forms” and the tri-partite agreement are rather weak “instruments of consenting participation.” The willingness form does not specify responsibilities of the pharmacy as a partner. The Tripartite Agreement specifies only Ongole Drug and Chemist Association’s responsibilities, but not of the DTO office. Agreements such as these should be entered into in the true spirit of partnership where the terms clear and are mutually beneficial. No baseline study has been carried out in the Ongole Tuberculosis Unit (TU) to understand the practices of pharmacists, their level of knowledge and attitudes towards TB suspects/patients, and their referral habits. How many TB suspects seek the pharmacist or private provider (PP) for initial consultation also remains unknown, as does the number of diagnosed TB cases stay with them until completion of treatment. The project was started in August 2010. Out of 175 pharmacies in Ongole TU, 110 attended the sensitization training, and seventy-nine signed the “willingness form”. Sixty were trained to identify TB suspects. Currently, though, only twenty-five pharmacies are referring suspects to DMC. Referral rates are not consistent. In the month of September 2010, a total of five suspects were referred to DMC, and in the month of December 2010 (after the training program) referrals increased to twenty-six suspects. Referrals averaged thirteen TB suspects per month (Figure 2). The five pharmacies outside Ongole TU, including those in Tanguturu and Chimakurthy TU, refer patients to the DMC located in these districts. There were five TB symptomatics referrals made by these pharmacies since the start of the program. The Designated Microscopy Center (DMC) in Ongole under the DTO is currently processing an average of forty sputum smears a day or a total of 280-300 a month. Most of the DMC referrals come from ICTC (Integrated Counseling and Testing Center), RIMS (Ranjo Gandhi Institute of Medical Services) and the India Medical Association. The pharmacy as a source of referral represents less than three percent of the DMC monthly total. It is difficult to judge the level of referral in the absence of a baseline study. Ideally, baseline or KAP studies are done prior to the pilot and then strategies are developed based on the results of the baseline. However, the low referral from pharmacies participating in PATH’s initiative is not convincing in demonstrating that it has increased the case detection rate. And, there is no credible evidence that PATH has successfully discouraged the sale of TB drugs. In the Ramareddy TU, sixty pharmacists joined the desensitization training and forty-three signed the willingness form. There is no data available on the performance of pharmacies in the Rangareddy TU. Evaluation of Tuberculosis Program in India: PATH Report 35 0 5 10 15 20 25 30 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 # referred by pharmacies Figure 2: Number of Pharmacy referrals of TB suspects in Ongole TU, Andhra Pradesh 2010-11 Other interventions by PATH PATH has prepared the ground for scale up of the pharmacy initiative in six states: Gujarat, Maharashtra, Uttar Pradesh, Karnataka, Uttarakhand, and Madhya Pradesh. In Andhra Pradesh, two districts (Nagfur and Yawatmal) will be the focus of their intervention. Reports on the scale up plans in these areas are limited so far. Additional Reasons for Low Referral from Pharmacies: - PATH’s training program excluded sales clerks. The training done by PATH was mainly through the pharmacists and pharmacists in some stores do not deal with the public at all. Sales clerks or other front line staff deals with the public. - PATH has not devised incentives for profit-motivated pharmacies. Due to this lack, pharmacies are not motivated to refer clients. There are various incentive programs that have been developed for private sector participation. Implementation of non-monetary incentives would be ideal to motivate pharmacies. - Supportive supervision of the pharmacy project is deficient. Currently, the staff assigned to Ongole visits sixty pharmacies in only two days. This does not allow for productive discussion with pharmacies on issues relevant to their responsibilities. Conclusions Evaluation of Tuberculosis Program in India: PATH Report 36 PATH’s attempt to implement a Public Private Mix (PPM) through a stand-alone pharmacy initiative is rather weak. While the pharmacies’ role in TB prevention and control is important, pharmacies by themselves are limited in reaching TB suspects. Many seek private providers, and to a limited extent the pharmacy, for diagnosis and treatment. PPM is the engagement of the full private sector which ideally includes: (1) private practitioners; (2) the corporate sector or large companies; (3) NGOs; and, (4) pharmacies. In some PPM programs like those in the Philippines, pharmacies are an ideal starting point for PPM, but private providers (PPs) were later included to complement the pharmacy initiative. There appears to be little comparative advantage in engaging PATH in PPM. Limited core competence was found in PATH’s implementation of the pilot project in Ongole TU, and the activity is missing a number of fundamental steps needed for effective implementation. PATH should draw lessons learned from the pilot and apply them to the planned scale up. Recommendations USAID/India should consider taking the following actions to strengthen PATH’s PPM project: 1. Expand coverage to include private providers: Private providers should be an integral part of the PPM initiative. Mapping of these providers should be done in Ongole TU and any assessment should include information obtained from the clients they serve. A KAP study should be undertaken on PPs to find out current attitudes and practices, including an assessment of their motivation to join the program. 2. Baseline study: Consider conducting a baseline study among the participating pharmacies in Ongole, to understand the number of TB that uses them and their reasons for doing so. The assessment should also examine the behavior of pharmacies in referring clients. These baseline values will be helpful in developing behavior change communication to address behavioral problems encountered. 3. Effective Supervision and Monitoring: There is a need for effective supervision and monitoring in Ongole and other pilot sites. It is important that PATH knows the pharmacy practices when TB suspects consult with TB symptoms. This monitoring is usually done through a “mystery client study.” Responses of pharmacies are evaluated based on their interaction with the client. Such a study is ideal to evaluate behavior of the pharmacy given an anonymous client. Currently, PATH is using a checklist filled in by the PATH representative based on his/her interview with the pharmacist. This is not an accurate measure of pharmacist practices, as responses are subject to “courtesy bias.” 4. Develop an incentive system: There are many non-monetary incentive systems already developed for pharmacy outlets, such as: (1) signage for trained pharmacy as referral points; (2) a certificate of appreciation for joining the program signed by DTO and President of Pharmacy Association; and, (3) a plaque or letter of recognition for referring clients to DMC. Prizes for referring clients to the DMC can also be included. Quarterly prize awards can be used as a PR event to motivate participation. It is important that these incentives be studied carefully to ensure that implementation has no adverse effects on the program. Evaluation of Tuberculosis Program in India: PATH Report 37 5. Additional time/ knowledgeable staff to attend to PPM activities: A maximum of ten pharmacies per day should be set as a quota to allow for academic detailing of the program. Also, PATH staff needs additional training to understand the fundamentals of PPM implementation. The PPM intervention is so complex that additional training is absolutely necessary before PATH can lend technical assistance to DTO staff. 6. Include sales clerks in the training: Consider revising the training program to include sales clerks. Sales clerks, who are the front lines in most pharmacies in dealing with the public, are the first point of contact with TB suspects, while the pharmacist is out of reach. 7. Revising contracting instruments: Consider revising the contracting instruments. Use a Memorandum of Understanding (MOU) between the individual pharmacy and the DTO. Include in the MOU specific responsibilities of PATH and other signatories in the partnership. Consider making the MOU mutually beneficial. Expansion of MDR-TB control Drug resistance in TB is a serious problem, compromising both treatment and control programs. Abuse of available anti-TB drugs has led to strains of TB that show progressive drug resistance￾multi-drug resistance (MDR), extensive drug-resistance (XDR), and even total drug resistance (TDR). While drug-sensitive TB is easily curable, MDR-TB is difficult to treat, and XDR and TDR are often fatal. Non-availability of new drugs to treat drug resistant cases further complicates the problem. In 2008, an estimated figure of 390,000–510,000 cases of MDR-TB emerged globally. Among all incident TB cases globally, 3.6% were estimated to have MDR-TB, and MDR-TB caused an estimated 150,000 deaths. Alarmingly, 5.4% of MDR-TB cases were found to have XDR-TB. 39 Almost fifty percent of MDR-TB cases worldwide are estimated to occur in China and India. There is recent evidence of increasing incidence of drug resistant TB cases in India.40 WHO identified twenty-seven countries as “high burden countries” (countries to have at least 4,000 MDR-TB cases arising annually and/or at least ten percent of newly registered TB cases with MDR￾TB); India is one of them. So far, in those high MDR-TB burden countries, only one percent of new TB cases and three percent of previously treated TB cases underwent DST, and 1.3 million MDR￾TB cases will need to be treated between 2010 and 2015. The EXPAND-TB Project is a multi￾country initiative that aims to scale-up and accelerate access to MDR-TB diagnostics in twenty-seven countries through a network of partners, such as WHO, the Global Laboratory Initiative, the Foundation for Innovative New Diagnostics (FIND), the Stop TB Partnership’s Global Drug Facility, and UNITAID. The Project is funded by UNITAID and has a budget of US$87 million over five years. 39 “Multi-Drug and Extensively Drug-Resistant TB (M/XDR-TB),” 2010 Global Report on Surveillance and Response, World Health Organization, Geneva. 40 Rawat J., G. Sindhwani, et al. “Five-Year Trend of Acquired Anti-Tubercular Drug Resistance in Patients Attending a Tertiary Care Hospital at Dehradun (Uttarakhand),” Lung India. 2009; 26: 106–108 doi: 10.4103/0970-2113.56342 Evaluation of Tuberculosis Program in India: PATH Report 38 The emergence of drug-resistant TB, particularly MDR-TB, has created significant obstacles to effective TB control in India. The RNTCP launched DOTS-Plus41 in 2007 and has since scaled up implementation from two to twelve states, and has twenty-one accredited laboratories equipped for culture and DST. RNTCP is working to address issues related to DOTS-Plus implementation, including identifying MDR suspects; strengthening mechanisms to reduce the gap between diagnosing cases and beginning treatment; drug costs42 and delays in procurement; maintaining the success of laboratory scale-up and second-line drug resistance; and lowering the initial default and treatment default rate. In October 2009, PATH received funds from USAID/India to continue the work in MDR-TB control, started under TASC2 TB, Task Order 2; and especially to support effective expansion of MDR-TB control activities by identifying and addressing gaps in the existing DOTS-Plus program. PATH’s Objectives • Support effective expansion of MDR-TB control activities by addressing gaps in the DOTS-Plus program (PMDT), in collaboration with RNTCP, WHO, and NGOs. • Test innovative approaches to improving TB case detection, treatment success and prevention of MDR-TB.Develop MDR-TB guidelines and training materials for community care • Conduct health facility readiness assessment • Prepare directory of DOTS-Plus sites • Conduct experience-sharing workshops to identify best practices and document common challenges, with recommendations for addressing them. Findings PATH conducted two national experience-sharing workshops on PMDT in 2010 and 2011 (with participation from nine and six states, respectively) and documented the proceedings. PATH participated twice in 2010 in DOTS-Plus site assessment and appraisals. In addition, PATH supported state-level micro-plan preparation for scaling up of DOTS-Plus in New Delhi. In collaboration with Institute of TB and Lung Diseases in Delhi (LRS) and other national institutes, PATH facilitated the preparation of a Nurses Counseling module. PATH, in collaboration with Lilly, APSACs and the STO of AP, conducted four preparatory meetings, starting in May 2010 with AP State AIDS Control Society officials, to pursue the proposal to train nurse practitioners. It was planned to take this training in MDR-TB patient care to additional staff in lower-level facilities closer to patients. There were some delays in starting the training, because the nurse practitioners’ contract was terminated. As of mid-September 2010, their 41 Based upon DOTS, DOTS-Plus is a comprehensive management strategy under development and testing that includes the five tenets of the DOTS strategy. DOTS-Plus takes into account specific issues (such as the use of second-line anti-TB drugs) that need to be addressed in areas where there is high prevalence of MDR-TB. Thus, DOTS-Plus works as a supplement to the standard DOTS strategy. By definition, it is impossible to conduct DOTS-Plus in an area without having an effective DOTS-based TB control programme in place. 42 Given that the treatment of an MDR case is much longer than for primary cases, and the drugs are much more expensive; therefore the total costs for the drugs for one treatment are much higher than for those of a primary case Evaluation of Tuberculosis Program in India: PATH Report 39 contracts were renewed, and this training activity is moving forward in collaboration with the AP government. Community-level DOTs is in the discussion and planning phase with partners. In close collaboration with CTD and WHO, PATH is planning to develop guidelines and materials to build the capacity of NGOs participating in MDR TB control schemes to strengthen community support for MDR-TB patients. In collaboration with TB ALERT and LEPRA, a project is planned with the objective of strengthening case detection and treatment completion in hard-to-reach areas and in populations with poor access to services. PATH will set up sputum collection sites in collaboration with other field-level NGOs. Some of the community DOTS and infection control activities identified were not taken up due to a variety of reasons, some beyond the control of PATH. Frequent changes in CTD’s priorities throughout the project year presented significant challenges to PATH and partners’ progress. A number of specific activities were initially approved by CTD, but subsequently cancelled or postponed. Community level activities like development of training materials are yet to be started. Conclusions Although success stories, such as successfully organizing workshops for experience sharing, can be mentioned, significant hurdles impeded completing the mandate. To mention the major ones: PATH does not have enough staff or does not seem to have enough expertise to carry out some of the listed activities and there is poor coordination with CTD resulting in either delays in getting their approval or complete cancellation of their activities. The absence of long term planning and lack of commitment from some partners resulted in questionable quality of their performance. The absence of an advisory body resulted in poor synergy with national level implementation. Recommendations 1. PATH should plan and coordinate their activities well in advance with CTD, WHO and FIND. Activities should be implemented in a phased manner and be conducted with clearly defined roles and responsibilities for all partner organizations. 2. PATH should strengthen its staff and increase their expertise. To develop community-level activities and to involve patients for long-term treatment compliance, PATH needs to have the necessary community-level communication capabilities. 3. PATH should involve the private practitioners for early diagnosis, creating NGO networks for effective community-level DOTS, and developing a strategy along with NGOs to decrease the defaulting are all necessary. PATH needs to involve medical colleges for managing problem cases of MDR, and build capacity to increase program reach to all sections of the society. 4. Community DOTs counseling issues and infection control continue to be a source of concern with PMDT. Hence PATH needs to take a proactive role in the expansion of Evaluation of Tuberculosis Program in India: PATH Report 40 MDR-TB control activities by developing strategies for community DOTS, involvement of the private sector and sensitizing the community thru ACSM for early detection and cure. 5. More Operational Research, and especially action research studies, are needed to find effective ways for optimizing early case detection, and decreasing initial default. PATH should coordinate such studies with recognized research institutes within the country. 6. The guidelines developed by CTD, PATH and WHO need to be disseminated to all states and NGO partners, need to be field tested and revised, if needed, on the basis of the lessons learned from field implementation experience. Test Innovative approaches PATH has not yet started this activity. GENERAL CONCLUSIONS AND RECOMMENDATIONS Given the nature of this evaluation mission and the particularities of the USAID support to RNTCP, the main evaluation approach towards PATH support was qualitative and descriptive, focusing on the following key questions: 1. Analysis of the quality of technical expert support services provided to the Revised National Tuberculosis Control Program; 2. Assessment of the level of achievement of the specific objectives of the various support activities, such as Public-Private Mix (PPM), capacity building, lab strengthening, airborne infection control, innovative strategies, Multi-Drug Resistant TB/Extensively Drug-Resistant TB (MDR-TB/XDR-TB), and Advocacy, Communication, and Social Mobilization (ACSM). 3. Examination of PATH’s ability to coordinate and the coordination practices, given the fact that PATH produced multiple interventions working with a broad range of partners. 4. Analysis of cost-effectiveness or “value for money” of the PATH approach and project activities. The evaluation has identified three domains of recognized expertise: Airborne Infection Control (AIC), lab strengthening, and training. In AIC PATH has been an innovator in this area and continues to be active in it. PATH is recognized by multiple sources as providing high quality support to lab strengthening. PATH has developed a methodology for training which is recognized by alumni as very effective and the handouts developed as a component of these trainings are being used by other institutions today. Little evidence was found for the specific niche that PATH could occupy in the other program areas such as ACSM, PPM (Pharmacy approach), and Multi-Drug Resistance Tuberculosis (MDR-TB), but PATH/India may not have been in the field long enough to demonstrate significant results. In the latter three areas, PATH may eventually have the potential to be recognised as an authority through the originality and quality of its approach and activities, but in order to reach that level of national recognition, it needs methodological strengthening, to hire top level staff, and to be at the forefront of testing innovative methods and approaches, and it should reduce the spectrum of activities to those in which its senior staff members have a recognized expertise. Evaluation of Tuberculosis Program in India: PATH Report 41 The question of the value for money invested by USAID/India in PATH can be positively answered for the program areas of AIC, lab strengthening and training. The evaluation team recommends continuing and if possible increasing the level of funding of these activities, and encourages PATH to take more initiatives in these domains; however, the administrative structure of the selected funding mechanism should be looked upon critically by USAID and should be improved. In the remaining program areas of ACSM, PPM and MDR-TB the available evidence is not very convincing, although it is may-be too early to judge. The projects in those activity areas seem very ambitious and complex. Some parsimony in the design could be very beneficial. If the operational challenges of intense and supportive supervision and monitoring, and improvement of the program and HR management can be adequately addressed, then the cost-effectiveness of the projects in those three areas would likely increase. Therefore, further financial support in these three program areas should be made conditional on a clear program strategy for improvement in planning, project and HR management, increased levels of technical expertise of the senior staff and recruitment of such senior staff, and improved coordination with all key partners. Evaluation of Tuberculosis Program in India: PATH Report 42 Appendix 1: SCOPE OF WORK43 USAID/INDIA Office of Population Health and Nutrition EVALUATION OF TUBERCULOSIS PROGRAM IN INDIA STATEMENT OF WORK I. Identification of the Task: USAID/India seeks to evaluate the performance, impact, and lessons learned of tuberculosis (TB)- related programs implemented through its partners, the World Health Organization (WHO) and Program for Appropriate Technologies in Health (PATH). I. BACKGROUND TB is the leading cause of death among curable infectious diseases worldwide. A disease caused by Mycobacterium Tuberculosis, TB has affected mankind for over 5000 years, and still continues to be a leading cause of morbidity and mortality. Though the bacilli was discovered in 1882 by Sir Robert Koch and effective drugs for treatment have been available for more than half a century, more than 1.3 million people die of the disease every year. In 2008, there were an estimated 9.4 million new TB cases, which is equivalent to 139 cases per 100,000 people. Provisional estimates indicate that women account for about 3.6 million cases. Though globally the incidence of TB is decreasing, the absolute number of TB cases is still on the rise due to population growth. Most of the estimated cases in 2008 occurred in Asia (55%) and Africa (30%). The 22 high burden countries account for 80% of all estimated cases worldwide, and India and China alone account for an estimated 35% of TB cases worldwide.1 TB-HIV co-infection and drug resistant tuberculosis aggravate the TB situation globally. TB is a leading cause of death in HIV infected persons and HIV infection is the most potent risk factor for developing active TB disease from a latent TB infection. Of the 9.4 million incident cases in 2008, an estimated 1.4 million (15%) were HIV positive.2 Globally, multi drug resistant (MDR) TB is emerging as a major health challenge. Multi drug resistance occurring primarily as a consequence of poor treatment services could lead to the emergence of Extensively Drug Resistant (XDR) TB if MDR TB is not managed properly. There were an estimated 0.5 million cases of MDR-TB in 2007. The countries that ranked first to fifth in terms of total numbers of MDR-TB cases in 2007 were India (131,000), China (112,000), the Russian Federation (43,000), South Africa (16,000) and Bangladesh (15,000).3 TB Burden in India 43 This is the original Scope of Work, as provided to Social Impact by USAID in the original Request for Task Order Proposals. Evaluation of Tuberculosis Program in India: PATH Report 43 On an annual basis, India reports more new TB cases than any other country in the world. In 2008, out of the estimated global annual incidence of 9.4 million TB cases, 1.98 million were estimated to have occurred in India, of which 0.87 million were infectious cases, thus catering to a fifth of the global burden of TB. About 40% of the Indian population is infected with TB bacillus4 ; the incidence of TB in India is estimated based on findings of the nationwide Annual Risk of Tuberculosis Infection (ARTI) study conducted in 2000-2003. In 2000, an expert group of Government of India (GOI) estimated that the prevalence of TB at 3.8 million, while the more recent World Health Organization’s (WHO) estimate gives a prevalence of 2.186 million. Strategy for Tuberculosis Control Global TB control is guided by the Stop TB Partnership’s Second Global Plan and the WHO Stop TB Strategy. In line with the Millennium Development Goals, the Second Global Plan aims to halve TB prevalence and deaths by 2015 relative to 1990 levels. The Stop TB Strategy identifies the six main components required to achieve these targets, including the provision of high-quality DOTS expansion and enhancement; address TB/HIV, MDR-TB, and other challenges; contribute to health system strengthening; engage all care providers; empower people with TB and communities; and enable and promote research. DOTS is at the heart of the Stop TB Strategy. For countries to successfully implement Directly Observed Treatment Short-course (DOTS) they must demonstrate political commitment to fund and implement an effective national TB control program. DOTS also requires an integrated network of capable laboratories, surveillance through a standardized recording and reporting system, and a robust logistics system that ensures a secure supply of drugs. In addition, implementation of DOTS requires improved access to primary care services that are affordable, equitable, committed, and well￾organized. Education and training are essential elements to ensure the availability of human resources. The WHO established the Green Light Committee to help control and prevent MDR-TB through access to quality-assured second-line anti-TB drugs and prevention of the development of resistance to anti-TB drugs by assuring the appropriate use of these drugs. Revised National TB Control Program (RNTCP) Recognizing that the previous national TB control program was ill-equipped to achieve its mandate, the GOI initiated the RNTCP in 1998. Thus began the most rapid scale-up of any DOTS-based TB control program in the world. Nationwide implementation of DOTS was achieved in March 2006, and the RNTCP is now treating over 1.5 million persons annually. Enormous barriers remain, however, for the RNTCP to implement all components of the STOP TB Strategy, and now the rise of disease-resistant strains TB and TB-HIV co-infection threaten to turn back tremendous gains made in the past ten years if these challenges are not addressed effectively. The RNTCP’s targets are aligned with the global STOP TB Partnership’s targets of 70% case detection rate and 85% cure rate by 2005 and halving prevalence and deaths by 2015. Case detection and cure rate targets have been achieved on a national scale, however the quality of DOTS implementation remains quite poor in many areas. In order to achieve these targets and sustain performance, the RNTCP must increase the reach and quality of DOTS while addressing rising challenges in drug resistance and TB￾HIV co-infection. Since a well-managed DOTS program remains the best line of defense against drug resistance, improving the quality of DOTS remains a top priority. II. OVERVIEW OF USAID ACTIVITY USAID/India has been supporting the RNTCP for over a decade. The major areas of support include enhancing DOTS services, improving lab capacity to diagnose drug-resistant TB, operations research, Evaluation of Tuberculosis Program in India: PATH Report 44 TB- HIV collaboration and health systems strengthening. The RNTCP, one of the best managed disease control programs in the country, has now entered a crucial phase of implementation. There are changes in the global Stop TB Strategy5 that now advocate for universal access to care as opposed to the earlier objective of 70% case detection. The country has also set in motion its ambitious plan to provide Programmatic Management of Drug Resistant TB (PMDT) which involves setting up a chain of laboratories for undertaking Culture and Drug Sensitivity Testing (C&DST), and initiating treatment services for Multi-Drug Resistant TB (MDR TB) patients. There are other new initiatives which include civil society mobilization for TB awareness under the Global Fund Round 9 grant, and introduction of new diagnostics. Traditionally, USAID has always focused more on TA, and covers the entire spectrum of TB services, from ensuring quality and access to monitoring and evaluation. USAIDfunded operations research (OR) has led to many significant policy changes in the program. USAID/India had earlier supported RNTCP implementation in the state of Haryana till the year 2008. The other projects supported by the country mission include a civil-society mobilization project through World Vision, a CDC-USAID inter￾agency project to provide TA in the areas of TB-HIV and MDR-TB, and an innovative private sector engagement project through ABT Associates. The WHO TB Technical Assistance Project The USAID partnership with WHO started in the year 1999, with support of research activities undertaken by Tuberculosis Research Centre (TRC) in Chennai, which is a WHO collaborating center and a Supra-National Reference Laboratory. In 2003 USAID started supporting WHO TA through a field network of consultants, along with other donors like DFID and SIDA. The objectives were to provide TA to TB control efforts in India via the following: • Strengthening of the laboratory network for mycobacterial culture (solid and liquid) and drug susceptibility testing (for first and second line drugs), and introduction of line probe assay; • Large scale demonstration study of nucleic-acid amplification testing (NAAT) for early and improved TB case detection; • Strengthening of the involvement in RNTCP of health care providers of other sectors, focusing on medical colleges and professional medical societies; • Technical support to all RNTCP activities, via the RNTCP consultant network; and • Collaborative activities with TB Research Centre, Chennai, on epidemiological impact assessment, drug resistant TB and HIV-associated TB. While the geographic focus is the entire country, however for the consultant network the RNTCP have identified the states which would be supported by USAID. Though the objectives have remained the same for the past three years, the activities have differed slightly each year. The specific activities include: • To speed up the accreditation process, and to enable much wider availability of quality-assured culture and Drug Sensitivity Testing (DST) services for the rapid expansion of RNTCP Category IV services for MDR-TB cases through the placement of laboratory expert staff at strategic positions within the country, including 1) an international Laboratory Focal Point based at the WHO South-East Asian Regional Office in Delhi, with 70% work-time allocated for India-related activities, and 2) a National Laboratory Specialist based in the Central TB Division (CTD), Ministry of Health and Family Welfare in Delhi. • Establishing a “RNTCP Laboratory Task Force” for comprehensive TA needed to successfully utilize the expanded international support provided to RNTCP for laboratory scale-up, and successfully deliver TB diagnostic services nationwide by 2012. Evaluation of Tuberculosis Program in India: PATH Report 45 • Hire an international laboratory expert from a WHO Supra-National Reference Laboratory to strengthen diagnostic capacity for XDR-TB through second-line DST and build capacity of India labs • Strengthening involvement of Medical Colleges and medical professional bodies through the ‘task￾force’ mechanism and also creating forums like the IMPACT (Indian Medical Professional Associations Coalition against TB) • WHO direct TA to the program through field network of consultants and Central TB Division based experts. There are 49 field consultants based in the states of Haryana, Maharashtra, Tamil Nadu, North￾Eastern States, Chhattisgarh, Delhi, Gujarat, Himachal Pradesh, Jammu and Kashmir, Jharkhand, Karnataka, Kerala, Punjab, Rajasthan and Uttar Pradesh. • Operations Research and clinical research through Tuberculosis Research Centre, Chennai. The PATH India TB Project Since 2008, PATH has been a major partner of USAID/India in the area of TB Care and Control. The main objectives of the partnership are to provide TA to TB control efforts in India by strengthening the laboratory network’s capacity to diagnose TB and to identify drug-resistant strains of TB; facilitating the introduction of improved infection control practices (focused on reducing nosocomial infections in health care settings, especially with regards MDR-TB and TB-HIV), and assist the RNTCP in strengthening its approaches and methodologies related to advocacy, communication, and social mobilization (ACSM) to improve ACSM’s contribution to improved TB control program performance The mandate was to work in any part of the country, based on the requests made by Central TB Division regarding laboratory strengthening. Though the objectives have remained unchanged over the years, certain elements like piloting or field trials of new diagnostics and piloting novel approaches in engagement with the private sector has been added to the original scope of work. The specific activities include: 1. Strengthen the intermediate reference laboratory network through strategic provision of technical assistance, training, equipment, and upgrading facilities etc. 2. Accelerate accreditation of Intermediate Reference Laboratory network, and ensure the maintenance of accreditation through periodic site visits and mentoring. 3. Establish pilot studies to test improved infection control practices, participate in the National Airborne Infection control committee meetings, pilot test the National Airborne Infection Control Guidelines in Andhra Pradesh. 4. Support RNTCP health communication efforts by translating existing comprehensive ACSM strategy into results-oriented field activities; develop the capacity of the different stakeholders to design, implement and monitor needs based ACSM activities. 5. Design and implement community level activities to engage private sector providers, both formal and informal, in RNTCP activities USAID/India intends to examine these projects in-depth, and undertake a thorough evaluation of the PATH activities. III EVALUATION SCOPE Evaluation of Tuberculosis Program in India: PATH Report 46 Purpose and Objective USAID/India intends to carry out an in-depth and thorough evaluation of the tuberculosis prevention and control activities implemented by WHO and PATH. The objectives of the review are to: Determine the impact of the WHO and PATH projects relative to stated objectives and achievements. Make suitable recommendations for the future direction and priorities of the projects. ii) Statement of Work This statement of work (SOW) is for a comprehensive evaluation of the WHO and PATH projects, the appropriateness of the project activities in achieving the objectives, the level of impact, cost-effectiveness and future directions. Critical stakeholders will be involved during various stages of the review process as appropriate. The team will gather both qualitative and quantitative data based on the following specific objectives. Overarching issues: • Determine the impact of activities. • The extent to which the projects have achieved the objectives and met targets in the Performance Monitoring Plan (PMP) • Discuss contributing factors and barriers to achievement for objectives that were not fully met • Determine how the project is filling the gaps and collaborating with the Revised National TB control program (RNTCP) • Describe the main successes and lessons learned from this project • Provide recommendations for improvement in the future Technical: • Evaluate the quality of the technical expertise being provided to RNTCP by the project • Evaluate to what extent the project has met the technical and programmatic objectives • Evaluate whether the project-funded research strategy was developed in conjunction with the country program. To what extent were the results of such research utilized by the program (e.g. informing policy formulation)? • Evaluate to what extent the project contributed to the overall capacity building of the RNTCP Management: • Determine the cost-effectiveness and efficiency of the management and administration of the project; what has been the return on USAID’s investment to date in this assistance to the RNTCP? • Evaluate the sustainability of the projects: o Whether the project is effectively transferring organizational development and technical skills at international standards to the local partners/RNTCP o Would the GOI be able to support the activities undertaken by the project by itself in the future – in terms of funds, human resources? What recommendations could be made regarding an exit strategy for USAID’s extensive support to human resources under RNTCP (e.g., the consultants’ network)? Coordination: Evaluation of Tuberculosis Program in India: PATH Report 47 • Determine how effectively the project has collaborated with other partners working in the field of TB control, including USAID-funded projects • Determine how effectively the projects have advanced recommendations on RNTCP delivered through other evaluations and assessments, e.g., from the Joint Management Missions with the World Bank and the Green Light Committee? • Determine whether and how effectively the projects have coordinated and collaborated with the host government at all levels iii) Methodology The evaluators should consider a range of possible methods and approaches for collecting and analyzing the information which is required to assess the evaluation objectives. Data collection methodologies will be discussed with, and approved by the USAID/India TB team prior to the start of the assignment. Desk review of documents USAID/India will provide the team with all relevant country and project specific documents such as proposals, reports, etc. The evaluation team is expected to collect and collate relevant international documents, reports, and data, and all team members are expected to review these documents in preparation for the team planning meeting. This desk review will help to organize the materials for external evaluation team analysis and review of progress to date. It will allow the team to quickly digest a wealth of information, maximizing their time. The Mission point of contact will provide the evaluation team with project reports, analyzed information and summaries as well as all other documents needed for conducting this desk review. Team Planning Meeting (TPM) A two-day team planning meeting will be held by the team at an offsite location before the evaluation begins. This will be facilitated by the team leader, and provide the Mission with an opportunity to present the purpose, expectations and agenda of the assignment. The evaluators shall come prepared with a draft set of tools and guidelines and preliminary itinerary for the proposed evaluations. In addition, the TPM will also: • Clarify team members’ roles and responsibilities • Establish the timeline, share experiences and thoughts on the evaluation methodology • Finalize the data collection tools and guidelines Site Visits and Interviews • Conduct a thorough review of the project through site visits and interviews • Interviewees will include key members from all stakeholder groups, including RNTCP, WHO, PATH, other donors, partners in TB control, and beneficiaries • Interview questionnaire to be prepared in advance and finalized during the TPM • Site visits will be planned taking into consideration factors like geographical diversity, representation of various beneficiary groups, and scale of interventions The Team will evaluate the state and district level periodic reports to take stock of the indicators iv) Timeline USAID/India anticipates that the period of performance of this review will be from November 2010 to January 2011 for about six weeks. v) Team Composition and Technical Qualifications and Experience Requirements Evaluation of Tuberculosis Program in India: PATH Report 48 of the Evaluation Team USAID seeks a six-member assessment team composed of a Senior Technical (TB) Expert, Senior Public Health Specialist, Evaluation Specialist, Senior Laboratory Expert and two Public Health Specialists. Since two different projects will be evaluated in multiple states, it is envisioned that the teams will separate to conduct field analyses. All team members must have relevant prior experience in India, familiarity with USAID’s objectives, approaches, and operations, and prior evaluation/assessment experience. In addition, individual team members should have the technical qualifications and required experience identified for their position below: 1. Senior Technical Advisor: This Senior Technical (TB) Expert in the field of international tuberculosis control has an excellent understanding of the global strategy and its implementation. S/he should have significant experience monitoring and evaluating various TB programs throughout the world. The expert should not be directly affiliated with WHO or PATH. A minimum of 12 years of experience in the design and management of tuberculosis control programs, particularly with regard to DOTS services, lab systems, TB-HIV collaboration, and health systems strengthening. (LOE up to 50 days) 2. Health and HIV/AIDS Analyst: This Senior Public Health Specialist has extensive experience with USAID project design, implementation, and evaluation. The person should have an excellent understanding of USAID operational, management, and technical approaches. Knowledge and experience of tuberculosis control activities would be an added advantage. A minimum of 12 years of experience in the design and management of tuberculosis control programs, particularly with regard to DOTS services, lab systems, TB-HIV collaboration, and health systems strengthening. (LOE up to 50 days) 3. Evaluation Methods Specialist: This expert will have deep knowledge of evaluation methodologies and their practical applications. A minimum of 7 years of experience in strategic planning, surveillance, operations research, monitoring and evaluation of global and national tuberculosis programs. (LOE up to 45 days) 4. Health and HIV/AIDS Analyst: This Health Analyst /Senior Laboratory Expert is a medical microbiologist, with a minimum of 7 years experience in Mycobacteriology. S/he should have extensive experience in setting up laboratories for culture and sensitivity testing for TB, and should be well-versed with the modern developments in the field of TB Diagnostics and techniques. If a local expert is not available, an international expert could be considered. (LOE up to 45 days) 5. Health and HIV/AIDS Analysts: Two experts in international public health with expertise in program management and strategic planning. They should have experience with the Stop TB Strategy and its approaches. A good understanding of human resource and institutional development is desired. A minimum of 7 years of experience in the design and management of tuberculosis control programs, particularly with regard to DOTS services, lab systems, TB-HIV collaboration, and health systems strengthening. (LOE up to 45 days) Summary Table: Labor Labor Category Level Illustrative LOE Senior Technical Advisor - Senior Technical (TB) Expert, Team Leader 1 50 Evaluation of Tuberculosis Program in India: PATH Report 49 Health and HIV/AIDS Analyst- Senior Public Health Specialist, Co-Team Leader 1 50 Evaluation Methods Specialist 1 45 Health and HIV/AIDS Analyst-Senior Laboratory Expert 1 45 Health and HIV/AIDS Analyst 2 45 Health and HIV/AIDS Analyst 2 45 In addition, each team member should have, at minimum, the following skills and experience: 1. An understanding of the country context. 2. An advanced degree in Public Health, Social Sciences, Business Administration, or other relevant course of study. 3. Demonstrated skill in written and oral communication. 4. Demonstrated knowledge of USAID policies and procedures. 5. Ability to work effectively in, and communicate with, a diverse set of professionals. The Senior Technical (TB) Expert and Senior Public Health Specialist will serve as Team Leader and Co￾Team Leader, respectively, and will be responsible for coordinating evaluation activities and ensuring the production and completion of quality reports, in conformance with this scope of work, which may become a public document for distribution among the program’s key stakeholders, including high-level U.S. government policy makers and officials, host country government officials, private sector and NGO leaders, and other audiences. In addition to proven ability to provide this leadership role, involving a technically and logistically complex program, he/she should have substantial and demonstrated expertise in evaluation techniques involving projects with technical assistance, training, advocacy, and partnership components. The Team Leader and the Co-Team Leader will be a senior expatriate with extensive experience in tuberculosis control programs and must have excellent English language skills (both written and verbal) as they will have the overall responsibility for pulling together the different elements of the assessment for the two separate final reports. They will agree to fulfill their responsibilities in approximately six weeks, spending up to four weeks in-country, and will play a central role in guiding the evaluation process. The Team Leader may hold a conference call with core team members and USAID/India representatives before and after the visit to India, if needed. vii) Relationships and Responsibilities Overall Guidance: The Health Evaluation Specialist in conjunction with the USAID/India Activity Manager and other key TB team members and CO will provide overall direction to the assessment team. Responsibilities: • Contractor will be responsible for obtaining visas and country clearances for travel for consultants. • Contractor will be responsible for coordinating and facilitating assessment-related field trips, interviews, and meetings in conjunction with the USAID, WHO and PATH Project officials. • Contractor will be responsible for submitting a budget for all estimated costs incurred in carrying out this review. The proposed cost may include, but not be limited to: (1) international and in-country travel; (2) lodging; (3) M&IE; (4) in-country transportation; and (5) other office supplies and Evaluation of Tuberculosis Program in India: PATH Report 50 logistical support services (i.e., laptop, communication costs, etc.) if needed. • In-country logistics to include transportation, accommodations, communications, office support, etc. viii) Reports and Deliverables The Team will provide separate sets of the deliverables mentioned below, for each of the Projects (WHO and PATH). 1. Draft Work Plan and Pre-Departure Briefings. The evaluation team will develop a draft work plan prior to departure from Washington, DC. The team will meet with USAID and other contractor staff for at least three working days prior to departure for the field. 2. Oral Presentation. The evaluation team will provide an oral briefing of its findings and recommendations to relevant staff in the field as well as to the respective country coordinators, GOI officials and other USAID staff at the conclusion of the visits to the various implementing partners. 3. Draft Report. The evaluation team will present a draft report of its findings and recommendations to the TB POC/Activity Manager before return to the United States. 4. Final Report. Ten paper copies of the final report as well as an electronic version in Word X version shall be submitted within five working days following receipt of comments from USAID and its implementing partners. Ten copies of each report will be provided to the USAID/India TB POC and two copies will be provided to PPC/CDIE/DI. The final report should include an executive summary of no more than three pages, a main report with conclusions and recommendations not to exceed 20 pages, a copy of this scope of work, evaluation questionnaires used to collect information on each of the program components, and lists of persons and organizations contacted. The final report, with executive summary and electronic files, must be received by the USAID/India TB POC within the seven working days after receiving the final comments on the draft evaluation report from USAID/India team. Evaluation of Tuberculosis Program in India: PATH Report 51 Appendix 2: FRAMEWORK of REVIEW QUESTIONS Below is a framework of Evaluation Questions that was used through the course of data collection. This is a revision to the original Scope of Work, and was agreed upon between the Evaluation Team and USAID staff during the Team Planning Meeting in New Delhi in first week of field work. Framework for Evaluation of USAID-funded TB Programs: WHO and PATH Core Evaluation Topics 1. General administrative and management issues Methods/ Info Sources 1.1.Rationale of decision-making for support 1.1.1. What was the rationale for PATH to enter India as a technical assistance organization? 1.1.2. What was the rationale for the USAID buy-in of TO2 and TO15? 1.1.3. How were the various fields of assistance selected? 1.1.4. What strengths do PATH and WHO bring to India? 1.1.5. What are their main weaknesses, if any? Key informant interviews; USAID documents 1.2. Planning and implementation issues 1.2.1. What is WHO’s process for designing work plans? What evidence/information is used to plan proposals? 1.2.2. What is PATH’s process for designing work plans? What evidence/information is used to plan proposals? 1.2.3. How has the funding mechanism (PATH=contract, WHO=grant) affected your organizations ability to implement the project? 1.2.4. How has USAID’s funding on an annual basis impacted the ability of the programs to deliver results in the short-, medium-, and long-term? 1.2.5. How are proposed activities/interventions reviewed and approved by CTD, USAID, and WHO/PATH? 1.2.5.1.1.What is working well with this process? 1.2.5.1.2.What concerns do agencies have about the review process? 1.2.5.1.3.Should the process be improved; and if yes: how? 1.2.6. How does CTD/WHO coordinate planning with PATH? 1.2.7. Has the Mission reviewed or provided feedback on your work plans? 1.2.8. Who at USAID provides technical and administrative oversight? 1.2.8.1. How often does the program interact with this person(s)? 1.2.9. How other agencies are consulted when designing interventions? 1.2.10. How are the needs of states/districts and other organizations identified when developing program plans? 1.2.11. How would you change the annual planning process to ensure coordination between funder and implementing partners? 1.2.12. What recommendations do you have to improve the planning and implementation of PATH activities? 1.2.13. What are the primary bottlenecks for implementing PATH’ portfolio in India? 1.2.14. Have these bottlenecks been overcome; if yes: how? 1.2.15. Have there been any USAID administrative delays? If so, what have these delays been and how have they effected implementation? 1.2.16. What role should the Mission play to resolve bottlenecks? 1.2.17. Have the current USAID funding mechanisms provided enough flexibility to respond to changing needs? 1.2.18. What is the absorptive capacity of your organization in India? Key informant interviews; program reports 1.3. Monitoring and evaluation related of supported activities 1.3.1 How is program progress measured? 1.3.3 Who receives data and results from the program? How is this done and how Key informant interviews, program Evaluation of Tuberculosis Program in India: PATH Report 52 often? 1.3.4 How is feedback given on data and information generated by the programs? 1.3.5 How does WHO and PATH conduct DQA on their results? 1.3.6 Give examples of how project results and data have been used for decision￾making. 1.3.7 How could the project M&E systems be strengthened? 1.3.8 Who on your staff does M&E? 1.3.9 Approximately what proportion of your project’s budget is dedicated to M&E? 1.3.10 What M&E support has been provided to other organizations? 1.3.11 Give examples of how that support might eventually have improved the other organizations’ capacity to report its results. reports 1.4.Coordination and communication between implementers and USAID Mission 1.4.1. Describe your communication strategy with the USAID Mission? 1.4.1.1. How often do you meet with Mission staff? 1.4.1.2. What do you report to the Mission about your activities? 1.4.1.3. What support has the Mission provided? 1.4.1.4. What suggestions do you have for strengthening this relationship? 1.4.1.4.1.Has USAID provided clear direction? 1.4.1.4.2.Are there clear channels for reporting? 1.4.1.4.3.How often does the project communicate with USAID/W, USAID/India, and their own headquarters? 1.4.2. If you think that the communications between your program and the Mission could be improved, what recommendations do you have for improving communications between your program and the Mission? Key informant interviews; program reports 1.5.Coordination and communication between implementers and other organizations 1.5.1. Describe your communication strategy with the CTD and State govt. 1.5.1.1. How often do you meet with these partners? 1.5.1.2. What do you report to them about your activities? 1.5.1.3. How would you describe the quality of this communication? 1.5.2. How are results of interventions shared with primary partners? 1.5.2.1.1.Are these methods sufficient? 1.5.2.1.2.What changes would partners like to see with the dissemination of results? 1.5.3. If you think that the communications between implementing organization and partners should be improved, what practical recommendations do you have? Key informant interviews; program reports 1.6.Program sustainability (Transfer of core program responsibilities) 1.6.1. Has WHO/PATH considered the issue of sustainability or succession planning? 1.6.2. What plans does WHO/PATH have to transfer responsibility for core programmatic activities to RNTCP counterparts? 1.6.3. Which activities, processes, or products developed by PATH will continue beyond current funding? 1.6.4. How will this happen? 1.6.5. What tools have been developed to support this? 1.6.6. Is there any evidence that this has occurred? 1.6.7. What are PATH’s long-term plans for working on TB in India? Key informant interviews; program work plans/ Reports 1.7.Principles for continuation of USAID funding 1.7.1. If USAID funding ended now, how would the partners carry on their activities in￾country? 1.7.2. What alternative funding sources might replace existing resources? 1.7.3. What would be your strategy if USAID funding ended? 1.7.4. What principles should USAID Mission follow to guide continuation of funding? Key informant interviews 2. Support to WHO 2.1. Laboratory strengthening and involvement of other sectors Key informant Evaluation of Tuberculosis Program in India: PATH Report 53 2.1.1. What TA was provided for LAB EQA? 2.1.2. What TA and capacity-building is being provided for implementation and analysis of DRS surveys? 2.1.3. What support is provided for capacity building culture and DST? 2.1.4. What TA is provided for strengthening of IRLs? 2.1.5. What is your role in the “RNTCP Laboratory Task Force” for short-term intensive technical support for new laboratory establishment? 2.1.6. Through “Expand TB initiative” setting up of labs for rapid diagnosis was planned. Under this support how many labs were strengthening? 2.1.7. How many BSL3 labs for TB were set up under your initiatives? 2.1.8. What TA for new diagnostic technology was provided by whom and was it evaluated? 2.1.9. Was there a training program for the lab personnel and by whom was it done? 2.1.9.1. Was the quality of training assessed; if yes: how, and what was the outcome? 2.1.10. What support, if any, was provided for state level IRL staff? 2.1.11. What are the functions of lab task force? 2.1.11.1. Will they be involved in monitoring lab activities? 2.1.11.2. What is their role in introducing newer technologies? 2.1.12. What is the mechanism of coordination of the task force with CTD and WHO? 2.1.13. Is there a TA for establishing DST for 2nd line drugs and detection of XDR TB? 2.1.14. The proposed NAAT validation study will be guided and supervised by whom? 2.1.15. What is the plan of WHO in RNTCP 3 to support lab related activity? 2.1.16. What were the measures taken to reduce the delay in establishing IRLs? interviews, program reports 2.2. Collaboration with Partners/PPM 2.2.1.Strengthen relationship NTF 2.2.1.1. How are strategies developed? Are strategies developed collaboratively? 2.2.1.2. How many RNTCP zonal centers were established using this mechanism? 2.2.1.3. Are these zonal centers the best way to reach private sector clients? 2.2.1.4. What are the results of this activity? 2.2.1.5. How are results monitored? 2.2.2.Relationship and engagement of Medical College Task Force 2.2.2.1. What is the contribution of the Medical College Task mechanism in enforcement of standard and establishing DOTS center? 2.2.2.2. Has the quality of TB care in the private sector improved because of this activity? 2.2.2.3. What funds were allocated/used for this activity? Was the result commensurate with the funds spent? 2.2.3.Achievements of PPM activities supported with USAID funding 2.2.3.1. What contribution did WHO make in expanding PPM activities? 2.2.3.2. Is there an increase in involvement of private sector as a result of WHO initiatives? 2.2.3.3. Is there an increase in the quality of private sector services as a result of these activities? 2.2.3.4. If USAID withdraws funding, could the government continue supporting PPM activities? 2.2.4.Documentation and dissemination of PPM activities 2.2.4.1. Has WHO documented the performance of its strategies? 2.2.4.2. Is the result disseminated to a wider audience? Key informant interviews, program reports, field observations 2.3. Operational Research 2.3.1. What OR has been done with USAID support? Key informant interviews, Evaluation of Tuberculosis Program in India: PATH Report 54 2.3.2. What was the process for choosing topics for the USAID-funded OR? 2.3.3. How was it linked to the RNTCP OR agenda? 2.3.4. Were any of the WHO consultants involved in the design and/or implementation of the OR? 2.3.5. Do any of the WHO consultants serve on the expert panels? 2.3.6. What kind of support does WHO provide to the national and state-level OR committees? 2.3.7. Does WHO provide a mentoring process for junior-level researchers? 2.3.8. How were the USAID-funded OR results disseminated? 2.3.9. Give examples of how the findings and recommendations of OR were used by the national or state level programs to influence policies, programs, and practices. 2.3.10. What areas of OR could use future support? 2.3.11. How were the TB disease prevalence surveys developed? 2.3.12. Is the OR being done still within the framework of Model DOTS? 2.3.13. Are there examples of data from other organizations being used for OR? 2.3.14. What is the status of other field-OR? program reports 2.4.WHO TA to RNTCP 2.4.1. Selection criteria; preparation for the job. Is there a shift over time? 2.4.2. ToRs: is there a change over time? 2.4.3. Role and impact of WHO Consultants at State/National Level 2.4.3.1. What were the areas covered under TA by the consultants employed by WHO? 2.4.3.2. Is technical support provided to the STO, DTO or lab personnel? 2.4.3.3. Their role in TA to OR? 2.4.3.4. How are TA activities supervised & monitored? 2.4.3.5. Was the impact of TA measurable, in low & high performing states, and at CTD level? 2.4.3.6. What changes in RNTCP implementation have occurred due to the TA? 2.4.3.7. What is the role of WHO consultants in poor performing and well performing districts/states? 2.4.3.8. Measures adopted to build the capacity of consultants in the field? 2.4.4. Adaptation of their role to the changing environment (well trained STOs & DTOs, RNTCP3 plans) 2.4.5. Transfer of knowledge & skills to RNTCP staff (to whom, benchmarks?). How could this transfer be affected by instability of the senior government health staff? 2.4.6. Supervision, M & E practices 2.4.7. Accountability(for processes vs. outcome) 2.4.8. Their own capacity development. What were the past practices? What are the current practices? 2.4.9. Career development perspectives 2.4.10. Future directions Interviews FGD Field Observations 2.5. Model DOTS Project 2.5.1. Completion of prevalence studies 2.5.2. Documentation of declining TB prevalence and contributing factors Key informant interviews, program reports 3. Support to PATH 3.1.PATH staffing and technical assistance capacity 3.1.1. How has PATH determined staffing needs for the Task Order implementation? 3.1.2. Who approves staffing decisions within your organization? 3.1.3. What role, if any, has the funder had over personnel decisions? 3.1.4. How are technical needs of staff positions determined? 3.1.5. How are new staff screened or evaluated to ensure a match with your program’s technical activities and deliverables? Key informant interviews, program reports Evaluation of Tuberculosis Program in India: PATH Report 55 3.1.6. How is staffed recruited in-country? 3.1.7. What oversight of staff hiring and performance is provided by your HQ? 3.1.8. What challenges have you encountered to recruit/retain staff? 3.1.9. How do you plan to address these challenges? 3.1.10. What strengths (technical and others) does your organization bring to India? 3.1.11. Does your organization currently require additional capacity: if yes: which type? 3.2. Strengthening IRL 3.2.1. Status of infrastructure upgrades for IRLs 3.2.2. Does your lab have the capacity to carry out EQA checking? 3.2.3. What is the role of PATH and WHO in developing lab infrastructure, drug supply and Technical assistance? 3.2.4. Does the lab follow the Q.C for media (LJ) preparation? 3.2.5. Does the lab have NRL trained microbiologist? 3.2.6. Does the Lab have NRL trained LTs? 3.2.7. Safety issue – Does the lab have proper Fire safety measures? 3.2.8. Does the lab have proper waste management system? 3.2.9. Program issues: Do you have a cross-referral system? 3.2.10. Do you have proper specimen transport system? 3.2.11. Sustainability of infrastructural investment? 3.2.12. Sustainability of the staff in the lab? 3.2.13. What are the main successes and lessons learnt? 3.2.14. Provide recommendations for improvement in the future? 3.2.15. Utilization of policy and programmatic resources? 3.2.16. Plans to address bottlenecks affecting implementation 3.2.17. What were the procedures adopted for AMC? 3.2.18. Does the lab have SOP for Stain, media preparation? 3.2.19. Does the lab have N95 mask for liquid culture? 3.2.20. Does the lab have research activities in relation to the Program? 3.2.21. Does the lab have shower facilities? 3.2.22. Does the lab have emergency evacuation plan? 3.2.23. If PATH or WHO would withdraw its support, what would be the sustainability of the IRL activities? Key informant interviews, program reports, field observations 3.3.Airborne control & AIC scale-up 3.3.1. Status to document best practices 3.3.2. Development of AIC materials 3.3.3. Introduction AIC in high-risk congregate settings 3.3.4. Establishment AIC engineering training program in India 3.3.5. Utilization of programmatic materials 3.3.6. Plans to acquire technical capacity and personnel for program implementation 3.3.6.1. Plans for addressing bottleneck affecting implementation 3.3.7. Status for scaling up AIC activities in AP 3.3.8. How much support was given by PATH and what is the current status with regard to accreditation? 3.3.9. AIC activity done in AP: How was the quality of training given on AIC assessed and by whom? 3.3.10. Were the materials developed for dissemination of AIC messages validated in the field? 3.3.11. What are the barriers in introducing concepts of AIC under field conditions and was it documented? 3.3.12. What are the barriers in scaling up of AIC activity? 3.3.13. What was the quality of the training and how and who validated the modules on AIC? 3.3.14. In the context of DOTS plus, what were the measures taken to scale up AIC at the level of community? Key informant interviews, program reports, field observations Evaluation of Tuberculosis Program in India: PATH Report 56 3.4.ACSM 3.4.1. What information does PATH use for planning its ACSM interventions? 3.4.2. How many PATH staff are dedicated full-time/part-time to ACSM? 3.4.2.1.1.What is their experience and expertise related to ACSM? 3.4.2.1.2.What are the recruitment and selection processes? 3.4.2.1.3.How does PATH develop staff ACSM capacity? 3.4.2.1.4.Are there concerns or barriers to achieve this? 3.4.3. What resources does PATH have for implementing ACSM? 3.4.3.1. What was the budget for ACSM activities for FY ’08, FY ’09 and FY ’10? 3.4.4. Describe the role of PATH HQ to support ACSM activities? 3.4.5. What are PATH primary technical skills for ACSM in-country? 3.4.6.Coordination with NGO networks and contributions to GFATM implementation 3.4.6.1. How does PATH assist CTD and the NGO networks? 3.4.6.2. How many requests have members of the NGO Consortium made? 3.4.6.3. What agreements does PATH have with its partners? 3.4.6.4. What type of support did PATH provide in response? 3.4.6.5. How many workshops for ACSM has PATH conducted? 3.4.6.5.1.Where were these held and who attended them? 3.4.6.5.2.What follow-up, if any, did PATH provide following the workshops? 3.4.6.6. How many ACSM projects has PATH evaluated? 3.4.6.6.1.What were the results of these evaluations? 3.4.6.7. What follow-up activities were required or supported by PATH? 3.4.7.Linkage with national and state strategies 3.4.7.1. How has PATH integrated the RNTCP’s communication strategy into its curriculum? 3.4.7.2. How has PATH linked ACSM strategies and activities with TB control priorities? What was the result of this linkage? 3.4.7.3. What were the results of the training of trainer (TOT) workshops? 3.4.7.4. How many of those trained provided training in the home districts? 3.4.7.5. How does PATH collaborate with the RNTCP? Which staff does PATH interface with? 3.4.7.6. How has PATH provided supportive supervision and TA to develop or adapt site-specific ACSM materials? What were the outcomes of these activities? 3.4.7.7. How has PATH contributed to IEC staff at the State and District level ability to analyze TB program data, develop appropriate ACSM interventions, and implement and evaluate the impact of those interventions? What methods has PATH used? Which staff is involved in these activities? 3.4.8.Technical Assistance Project for States/Districts 3.4.8.1. What is the status with PATH’s placement of ACSM Advisors in five states to provide ongoing technical support to build the capacity of State IEC Officers, Communication Facilitators, and civil society organizations in ACSM? 3.4.8.2. What assistance has PATH provided to develop capacity-building in supervision, monitoring, data collection and analysis, and intervention planning? 3.4.8.3. What have been the main achievements of the expansion? 3.4.8.4. How has PATH developed ACSM capacity at the State/District level? What evidence is there that this has occurred? What changes have resulted due to PATH’s work? 3.4.8.5. For the scale up project, has PATH developed a plan for modifications? Has PATH evaluated the pilot? 3.4.8.6. How is the status with documenting best practices and lessons learned Key informant interviews, observations, document review Evaluation of Tuberculosis Program in India: PATH Report 57 documented, to inform recommendations for ACSM implementation through NGOs? How was this document used? 3.4.8.7. Did PATH develop a comprehensive supportive supervision and M&E plan, based on a framework designed to measure progress and outcomes of ACSM initiatives implemented by civil society agencies? How will the plan enable continuity and sustainability of ACSM efforts in India? 3.4.9. Has PATH provided training in interpersonal communications and counseling at district or TU level, and facilitation of meetings and workshops where TUs can share progress, successes, and challenges? What were the outcomes of these activities? 3.4.10. General Questions 3.4.10.1. Some of the activities in FY ‘10 plan were previously proposed in ’09, such as develop standardized supportive supervision guidance, training materials, and tools? How do these activities differ over time? What happened to the development of these materials and tools? 3.4.10.2. What challenges did PATH identify and how did it support problem￾solving to ensure their activities are effective and timely? Partners/Beneficiaries 3.5. Has your organization participated in any PATH ACSM training? 3.5.1.1. Was the training relevant for your work? 3.5.1.2. What changes to your ACSM planning or activities, if any, occurred following this? 3.6. What materials and information has PATH provided to your organization? 3.6.1.1. Have you used these materials, and if yes: how and how frequently? 3.7. Has your involvement with PATH changed your knowledge and insight about ACSM theories and practices; if yes: how? 3.8. Has your organization made any requests for TA? 3.8.1.1. What was the response? 3.8.1.2. How was the TA used? 3.8.1.3. Has PATH/WHO conducted any field visits with your organization to review ACSM activities? 3.8.1.4. If no assistance, why have you not made any request? 3.9. How has the work of PATH changed or strengthened your program? 3.10. What else would you like to see PATH accomplish? 3.11. What are PATH’s technical strengths for ACSM? 3.12. Where do you think they have weaknesses, if any, for ACSM? 3.13. How do you access materials for ACSM? Do you use materials provided by RNTCP? If yes, which ones? How do you adapt or review these materials? 3.14. Do you have an evaluation plan for your ACSM activities? How are the results of this evaluation used? 3.15. Does PATH inform you about implementation of its ACSM program (For States/CTD/Districts); if yes: how? 4. Expansion of MDR activities 4.1. Status of NGOs community support 4.2. Assessments of health facility readiness 4.3. Strengthen communications between all levels 4.4. Organize experience-sharing workshops 4.5. Innovative strategies for case detection and treatment completion in hard-to-reach areas and in populations with poor access to services 4.6. Development of community level counseling messages and development of training materials 4.7. Measures taken to reduce initial defaulters? 4.8. Measures taken to reduce mortality? Key informant interviews, program reports, field observations 5. PPM 5.1. Achievements of partnerships with pharmacies and pharmacies Key informant interviews, Evaluation of Tuberculosis Program in India: PATH Report 58 associations. 5.1.1. How many pharmacies and pharmacy organization joined the program at the district level and city level? 5.1.2. What incentives were provided to pharmacies to join the program? 5.1.3. What activities were implemented to attract their participation? 5.1.4. What mechanisms were used to bind pharmacies to the program? 5.1.5. Were pharmacies involved in developing strategies for their involvement? 5.1.6. Was there any consultation with USAID/RNTCP in the development of pharmacy strategies? 5.2. Achievements related to OTC sales of TB drugs. 5.2.1. How many pharmacies/pharmacies organizations refrained from selling OTC TB drugs to suspect/patients at the district and city level? 5.2.2. How was the sale OTC TB drugs monitored? 5.2.3. What strategies were implemented to discourage sale of OTC TB drugs? 5.2.4. Was there any consultation with other agencies in the development of these strategies? 5.2.5. What skills does PATH bring towards the development of these strategies? 5.2.6. What materials were developed to discourage sale of TB drugs? Which once were found effective? 5.3. Methods used to engage and train pharmacists and pharmacist associations. 5.3.1. What training did the pharmacies get for their involvement in the program? 5.3.2. Were pharmacies trained in preparing and submitting reports? 5.3.3. Is PATH able to provide this training? 5.3.4. Does PATH staff involved in this possess the requisite skills in training? 5.4.Achievements for identification and referrals of TB suspects/patients 5.4.1. How many pharmacies are involved in referring TB suspects/patients? 5.4.2. How many patients were referred to public sector for treatment? 5.4.3. Of those referred by private sector, how many stayed until completion of treatment? 5.4.4. Did PATH use any mechanism to monitor patients who do not return for treatment? 5.4.5. How did PATH monitor referrals among TB suspect/clients using the private sector? 5.4.6. What is pharmacy project component contribution to CDR? How is this monitored? 5.5.Engagement of traditional healers for referrals of TB suspects/patients 5.5.1. How many traditional healers joined the program? 5.5.2. How many suspects/patients were referred by traditional healers? 5.5.3. What activities were undertaken to promote participation of traditional healers? 5.5.4. What materials were produced to support this activity? Which of these materials were found to be effective? 5.6. Achievements for monitoring suspect/patients using private sector? 5.6.1. How is the performance of the private sector monitored? 5.6.2. Is the performance evaluation done on the private sector done on a regular basis? 5.6.3. Did the private sector partner assist in the monitoring of patient? 5.6.4. Did the private provider follow up patients who did not return for medication? 5.6.5. Was there any consultation with USAID/RNTCP on the development of strategies to monitor suspect/patients in the private sector? 5.7.Achievements of worksite interventions 5.7.1. What companies are involved the work based program? On what basis are these companies selected? 5.7.2. What TB services are performed by these companies? Do they conform to ISTC standard? 5.7.3. Are these companies able to complete the treatment of their patients? program reports, field observation Evaluation of Tuberculosis Program in India: PATH Report 59 5.7.4. What are the barriers to the work-based programs? 5.7.5. How does the project minimize these barriers? 5.7.6. Are the lessons learned from work-based program? 5.7.7. Are best practices documented? What strategies were used to improve TB care in the work place? 5.7.8. Are these strategies effective? 5.7.9. Did PATH develop these strategies collaboratively with other agencies involved in the program? 5.7.10. Did the quality of TB care improve in the workplace because of these strategies? 5.7.11. Did the facility mapping help in improving services in the work place? 5.8.Analysis of program data 5.8.1. Did PATH get any assistance in the analysis of program data? Who helped? 5.8.2. Were partner pharmacies involved in the analysis? 5.8.3. Were results of program data disseminated to partners? 5.8.4. Are results used for program planning and developing onward strategies? 5.8.5. Was there any consultation with USAID/RNTCP and other agencies on the achievement of results? 5.9.Plans for scaling up activities (geographic and program) PPM 5.9.1. What activities were implemented to scale up PPM in more geographic locations? 5.9.2. What is the result of these activities? 5.9.3. Were more i) pharmacies; ii) private doctors; iii); work based program involved as a result of scaling up of PPM? How many? 5.9.4. What materials were produced and used for scaling up PPM? Which ones? Were they effective? 5.9.5. Were PATH objectives in scaling up PPM achieved? 5.9.6.If USAID would consider withdrawing its support for these activities, will the government on its own be able to continue the necessary support? 5.10. Achievement on Referrals from Private Doctors 5.10.1. How many private doctors are involved in the program in: Chennai, Pune, and Lucknow? 5.10.2. What strategies are used to attract private doctors to participate in the program? 5.10.3. Who were involved in developing strategies for doctors’ participation? 5.10.4. How many referrals have private doctors made in Tamil Nadu, Maharashtra, and Uttar Pradesh, since the start of the PPM program? 5.11. What districts were involved in the pilot? What proportion of DMC sputum smears are coming from private doctors? 5.12. What are the challenges faced by private providers in referring TB suspects/patients? 5.13. How does the project minimize these challenges or at least their impact? 5.14. What is the quality of TB services provided by private doctors? Do they conform to ISTC standard? 5.15. Are the strategies involved in promotion of private doctors within the institutions’ core competencies? 5.16. Are private practitioners able to follow-up patients who do not return for medication? 5.17. Do they routinely undertake contact tracing of their patients? 5.18. What baseline data was used to assess private doctors’ performance? 5.18.1. Has KAP study been done on private doctors? If yes, when and how was it done? 5.18.2. Who did the KAP study? What was the level of Knowledge, Attitudes and Practices prior to the pilot? 5.18.3. What districts were involved in the pilot? Was the selection of those districts justified? Evaluation of Tuberculosis Program in India: PATH Report 60 5.18.4. Has there been any performance evaluation done on private doctors? 5.19. Achievements on NGO participation 5.19.1. How many NGOs are involved in the provision of TB services? 5.19.2. How are they funded? 5.19.3. What is their contribution to early case detection and referrals? 5.19.4. What strategies are used to attract their participation? 5.19.5. Who monitors their performance? 5.19.6. Are they preparing and submitting reports about their program? 5.20. Achievements of Government Facilities Outside Health Department ( e. Railway, Prisons, State Insurance, Central Gov Health Services) 5.20.1. Who are involved in the provision of TB services? 5.20.2. How are these institutions invited to be involved? 5.20.3. Who monitors their performance? 5.20.4. What is quality of their TB services? 5.20.5. Do they comply with ISTC standards? Evaluation of Tuberculosis Program in India: PATH Report 61 Appendix 3: PERSONS CONTACTED ANDRA PRADESH Lepra, Aparna S- Microbiologist J. Subbanna, Director BPHRC Ong ole TB Unit Sailaja, Medical Officer T. Ramesh, DTO Kresna Veri, Lab Technician Ongole Chemist and Druggist Association K Rama Murthy, Secretary Y, Sri Rama Murthy, Secretary Narasimha Rao, President Ongole Pharmacies Ganish Medicals PPS Medicals Ravi Medical Rekha Medicals Sai Ankita Medical Saidurga Medicals Sri Venkata Ramana Medicals Sudheer Medicals Suresh Medicals PATH Pravakar Adhikaree Consultant (LABS) Naveena Ambatipudi, Administrator Shanta Ghatak, MDR-TB Officer Mayank Ghedia, AICO Satish Kaipilyawar, Project Director Dyson Misguitta, AICPA Uma Shankar Shiva Shrestha, ACSM and M&E Officer Sidhartha Srikar, PPM Officer STC Kishore Reddy -Microbiologist P.H.Vishnu S.T.D.C Sai Babu, Former STO Sreenivas S.T.O State TB IEC Officer TB Alert Ramya Anantakrishnan. Project Officer Arun Kumar, Program Manager Sunith Mathew, District. Coordinator Khasim Sayyad, District. Coordinator Evaluation of Tuberculosis Program in India: PATH Report 62 Mr. Vikas, Program Manager Warangal District S. Murali Krishna, CF Additional Persons Met Sai Babu, former STO Santosha, former WHO consultant NEW DELHI ABT Associates George Oommen, Deputy Chief of Party Avinosh, Consultant Avik Bhansal, Consultant Deepanjali Bhas, Consultant Kamaldeep Kaur, Consultant Ajay Kumar, Consultant Malik Parmar, Consultant K S Sachdeva, Medical Officer M Sangata, Consultant Shanti Sehgal, Consultant Sharad, Consultant Department for International Development Ms Sabina Barnes, DFID, Delhi eNVisions Varsha Chanda Initiatives, Inc Rebecca Furth International Union Against Tuberculosis and Lung Disease Sreenivas A, M&E Coordinator Subrat Mohanty, Project Coordinator, PMU, GF R9 RK Swamy BBDO Manisha Singh Development Strategy Director Nemdeikem KS, Development Strategy Manager Strategic Alliance, Delhi Gautam Nath, Country Director USAID Elizabeth Callender, Program Officer K. Hemachandran, Advisor for TB Care and Control Sanjay Kapor, Division Head, HIV/AIDS Charushila Lal, Program Development Specialist World Bank Patrick Mullen, Senior Health Specialist Evaluation of Tuberculosis Program in India: PATH Report 63 World Health Organization Puneet Dewan, Medical Officer for Tuberculosis M Hyder, RA Ranjani Ramachandran, Consultant World Vision, India Amit Gordon, M&E Officer Subodh Kumar, Program Manager Rajdeep Srivastava Additional Persons Met Meena Som, former WHO consultant GUJARAT Apollo Hospital Aruna Gautam, Consultant for Microbiology Sujata Naidu, Deputy General Manager of Operations Premila Robert, Chief Coordinator for Infection Control Abhijat Sheth, Director of Medical Services BJ Medical College Rajesh Solanicz, Professor of Pulmonary Medicine Bharat Shah, Dean M.M. Phrabhakar, Medical Superintendent CBCI. Father Thomas Indian Medical Association Kanodia Ashok, Coordinator State Office K.R. Pujara, Chief Medical Officer STDC Vijya Amin, Medical Officer Divit Kapadiya, TB -HIV Coordinator Mitesh Nayak, Medical Officer Purvi, Nayak, Medical Officer Pankaj Nimavat, Medical Officer Nikunj Patel, Statistical Assistant Pradip Patel, Director Rajesh Solandi, Professor, TB & Chest Department World Health Organization Kiran Rade, Consultant MAHARASHTRA CHAI D.R. Shinole, District Coordinator Evaluation of Tuberculosis Program in India: PATH Report 64 IMA Suhas Shingte, Shree Clinic, Sandeep Nagar Thergaon, Pune Nagpur District Staff RM Criri, MO Medical College Motiram Kamble, Chief Medical Officer RN Warjuker, Medical Officer, DTL, Nagpur Population Services International Rari Bhetnafer Arizuddhai Pathak RNTCP K. Subhakar, Chairman State Task Force SAI Pathology Lab (Private Lab) Charusheela Gore, OM State Office Ashish Doddomai, IEC Officer Khilare, City TB Officer for Pune S.B. Rajyoe, DTO STDC B.D. Bhalekar Narendrakumaz Birare, Medical Officer (Major) Pradeep Gaikwad Uddhav Gavande V.S. Ingale, Superintendent of the Chest Hospital Kanchan Jagtop, Chief Medical Officer Shri V.D. Kathale, Administrative Officer S.R. Khezat, Medical Officer Balgani Shilva, Consultant Microbiologist Y.G. Yeole, A.P.O. World Health Organization Rajesh Deshmukh, Consultant Avinash Jadhav, Consultant Jyoti Salve, Consultant Sanjay Suryawanshi, Consultant Manoj Toshniwal, Consultant Amol Wankhede, Consultant ORISSA (PPM CONFERENCE) IMA Shruti Seghal, IMA -GFATM, Bhubaneswar Kajal Krishna, Editor, IMA Journal Sai Prasad, IMA, Hony Sec. Gen, Suresh Gupta, Medical Coordinator, IMA Davendra Shirole, State Coordinator, IMA Evaluation of Tuberculosis Program in India: PATH Report 65 World Health Organization Chetair, WHO Consultant, Hyderabad Syer Imran, WHO Consultant, Harkland Durha Paul, WHO Consultant, W. Bengal Suchendo Roy, WHO Consultant, Uttar Pradesh Soumya Samkap, WHO Consultant, Orissa Sudhi, WHO Consultant, Haryana R.D. Yeale, WHO Consultant, Uttarakhand RAJASTHAN Bundi District DK Mathur, DTO Jaipur I District Vinod Garg, DTC Bharti Malhotra, IRL Sarthak Maklav Kushtashram, Medical director (NGO) BK Meghwal, DTO State Office Gupta, STO World Health Organization Sanjay Kumar, Consultant Pankaj Dhinga, Consultant Lalit Mehandroo, Consultant Vivek Mishra, Consultant TAMIL NADU Chennai DTC Dharani Latha, DTO, Chennai IMA Kailaash, President Pharmacies Guru Medicals Moorthy Medicals Pondicherry sites Muthuraj – Microbiologist S.Prabhu Medical superintendent (Government Hospital) K.V.Raman S.T.O STC Madhu Mathi, IRL Microbiologist C Udayasankar, STO TB REACH Evaluation of Tuberculosis Program in India: PATH Report 66 M. Terence Aldrin, Program Manager Ramya Ananthakrishnan Ms. M. Chitra Sheila, PPM Officer Tuberculosis Research Centre R Balambal, Specialist V. Chandrasekaran, Scientist Nirupa Charles Gomathi NS, TO Gopi, Head statistical department C Kolappan, Scientist AK Hemanth Kumar, TO MM K Jaggagarajam, TO Vanaja Kumar, Scientist M Muniyandi, Health Economist G. Navendran, Scientist Padmaprya, Physician Banu Reka, Physician N.Selvakumar Scientist F, (Bacteriology) Selva Kumar, Head MDR TB lab S. Sivakumar, Research Assistant Mohane Suhadi, Technical Officer (TO) Aleyamma Thomas, Director In -charge Beena Thomas, Director World Health Organization Rajan Sreenivas, Consultant Additional Persons Met Dr Santha, ex -WHO consultant, Chennai UTTAR PRADESH Chandauli District P Tiwari, DTO Ferozabad District (via telephone) A.K. Mithal Janpur District (via telephone) Ashok Kumar, Deputy DTO Rastogi, BCG Technician, Jaunpur Kaushambi District Atul Kumar, District Magistrate Krishna Kumar, CMO S Mishra, DTO D R Verma, CMO Nagar District V K Dubey, CMO Rahul Singh, DTO Evaluation of Tuberculosis Program in India: PATH Report 67 RTCP Technical Assistance Project D K Gupta Bharati Kalottee, Medical Consultant Ashu Pandey, Medical Consultant Y N Prabhakar Sukhendu Roy Sanat Jh Kovid Sharma STC Prof. Amitha Jain, Director, Microbiology Lab V K Koyal, STO Sonali Malashuwari, TB IEC Officer Urmila, Microbiologist Tambaram Sanatorium R Krishna Rajasekar, Associate Professor Varanasi District Swami Hridayanand, Medical Officer B N Singh, DT G N Srivastava, Associate Professor Swami Varishthanand Ji, In-Charge officer Evaluation of Tuberculosis Program in India: PATH Report 68 Appendix 4: REFERENCES Government of India Documents • A Health Communication Strategy for RNTCP, Central TB Division, Directorate General of Health Services, Ministry of Health and Family Welfare in collaboration with DANTB. 2005 • How can Private Medical Practitioners and TB Program work together to help control TB • Revised Schemes for NGOs and the Private Provided, Central TB Division, Directorate General of Health Services, Ministry of Health and Family Welfare. August, 2008. New Delhi • S.P. Agarwal, Saroj Dhingra, L. S Chauhan. The Role of IEC in the RNTCP. Chapter 19, Tuberculosis Control in India • Guidelines on Airborne Infection Control in Healthcare and Other Settings, Central TB Division, Directorate General of Health Services, Ministry of Health and Family Welfare. April 2010. New Delhi • Impact Assessment of RNTCP II communication campaign on KAP of Target Audience. SEDC, Synovate • Social Assessment Study for RNTCP Final Report. ORG Centre for Social Research. New Delhi • Training Module for Medical Practitioners, Central Tuberculosis Division, December 2010 • Technical and operational guidelines for Tuberculosis Control. October 2005 • Terms of Reference (TOR) for engagement Communication Facilitator (CF) by State Health Society • TB India, 2010. RNTCP Status Report, Central TB Division, Directorate General of Health Services, Ministry of Health and Family Welfare, March 2010. New Delhi. USAID • 5-year Tuberculosis Strategy2010-2014,Working in Partnership with the Government of India’s National Tuberculosis Control Program, USAID/India, Draft Strategy Report PATH • Annual Work Plans o TASC2 TB, Task Order 2, India Work Plan FY 2008, December 2, 2008 o TASC2 TB, Task Order 2, India Work Plan FY 2009, September 1, 2009 o TASC2 TB, Task Order 2, India Work Plan FY 2010 o TB TO 2015 Final Revised FY09 India Work Plan, February 11, 2010 o TB TO 2015 Final Revised FY10India Work Plan, September 2, 2010 • Annual and Semi-Annual Reports o Report submitted November 1, 2010 o Semi-Annual Report, TASC2 TB, Task Order 2, October 1, 2008 - March 31, 2009 • Materials and Other Reports • Overcoming Barriers to TB Control: The Role of Advocacy, Communication and Social Mobilization (ACSM) Training Report, 2010 Evaluation of Tuberculosis Program in India: PATH Report 69 World Health Organization • Advocacy, communication and social mobilization (ACSM) for tuberculosis control: a handbook for country programmes. WHO, 2007 • Advocacy, communication and social mobilization to fight TB: a 10-year framework for action. World Health Organization 2006 • Aide Memoire. The Revised National Tuberculosis Control Program, Joint Monitoring Review Mission, May 17 - 28, 2010. • Joint Monitoring Mission, Revised National Tuberculosis Control Programme (RNTCP), India, 15- 28 April 2009, World Health Organization, New Delhi. • The STOP TB STRATEGY Building on and enhancing DOTS to meet the TB-related Millennium Development Goals, 2006 Scientific Articles • Indian Journal of Tuberculosis, Published by The Tuberculosis Association of India, January 2011 • Intensified Scale-Up of Public Private mix: A systems approach to Tuberculosis care and control in India., SS Ll, S. Sahu, F. Wares, K. Lonroth, LS Chauhan, M. Uplekar, 2011 • Tuberculosis Management of Private Practitioners in Mumbai, India: Has Anything changed in two decades, Zarir F. Udwandia, Lancilot M. Pinto, Mukund Uplekar, August 2010 • Private patient perceptions about a pubic program; what do private Indian Tuberculosis patients really feel about directly observe treatment, Lancelot M Pinto and Zarir F Udwandia, 2010 • Health seeking behavior of new smear-positive TB patients under a DOTS programme in Tamil Nadu, India 2003. • Private Pharmacies and Tuberculosis control a neglected link, R, Rajeswari, R. Balasubramian, MSC Bose, L Sekar, F. Raman. TRC 2002 • Evaluation of Directly Observe Treatment Providers in Revise National Control Program, .C. NIrupa G. Sudha, T. Santha, C. Ponnuraj., et.al. , Dec, 2004 • Public Partnership in Tuberculosis Control experience in Hyderabad, India , KVR Murthy, T.R. Frieden, A. Yazdani, P Hreshikesh • Feasibility and Effectiveness of Public Private Mix Project for Improved TB Control in Delhi, India, V.K. Arora, R. Sarin. K. Lonroth, 2003 IUTLD • The private-public divide: Impact of conflicting perceptions between private and public health care sectors in India, RM Vyas, PM Small, K. De Riemer, 2003 IUTLD • Tuberculosis Control in rural India: Lessons from public private Collaboration, S.G. Rangan, SK Juvekar, SB Rasalpurkar, et al. 2004 IUTLD • Every Provider Counts; effect of a comprehensive public-private mix approach for TB control in large metropolitan area in India, G. Ambe, K. Lonroth, Y Dholakia, J. Copreux, M. Signol, et.al. 2005 IUTLD Other Organizations • Promising Practices in TBACSM. World Vision India. December 2010. New Delhi. • Training of Trainer Manual on Soft Skills. World Vision India. August, 2010. New Delhi. • Training Manual for Rural Health Care Providers. World Vision India. August, 2010. New Delhi. Evaluation of Tuberculosis Program in India: PATH Report 70 Appendix 5: LIST of the SENIOR STAFF of PATH as of MARCH 2011 Name of staff member Age Gender Degree Role at PATH Training & Qualifications (not exhaustive) Any additional training that is specific to your job at PATH? Years of professio nal experien ce What is your background? What were you working on before coming to PATH? Dr. Satish Kaipilyawa r 49 Male MBBS; Master in Health Administ ration Director – India TB project Overall leadership and management of the PATH India TB project. Project management and Health promotion AIC in the WHO collaborating center at Sondalo, Italy; AIC National Level training by CTD; MDR training; ACSM training at national level; M&E, MDR and advanced TB by USAID 24 years Served in Ministry of Health for 13 years. Associated with DFID for evaluating the country’s first RNTCP implementation in AP Was lead in handling the Gates Funded Children’s vaccination Project and also coordinated the state activities on HPV project. Brings to PATH a wealth of varied experience of handling different projects on Leprosy, Immunization, School Health, Bio-medical waste management, TB, diagnostics and injection devices. Mr. Uma Shankar 39 Male Post graduate diploma in Rural Develop ment ACSM and PPM Officer Support to Advocacy Communication and Social Mobilization and PPM projects at three states of India. Project Planning & management Participatory approaches Essential new born care Immunization Monitoring & evaluation Formative study & implementatio n of HPV demonstration project. Immunization essentials, injection safety, behavior change 15 years ARAVALI Rajasthan & CARE India Rajasthan Human & Institutional capacity building, managing natural resource management, livelihood programs, community development, tribal welfare, Immunization, injection safety, behavior change and design and scale up of new tools for behavior change. Advocacy at State and national level on program & policies. Evaluation of Tuberculosis Program in India: PATH Report 71 ACSM & PPM HSBC training communication Dr. Dyson Misquitta 29 Male Masters in Health Administ ration Program Associate for Airborne Infection control Implementation of AIC activities at PATH. Facilitator for capacity building workshops, baseline healthcare facility risk assessments, Preparation of final draft reports Certifications & training in Epidemiology, Qualitative methods; Training of Rapid Response Teams, ‘Course in Protection of Human Subjects Curriculum’, Doctors working in Prisons: human rights and ethical dilemmas Workshop on Pilot Implementatio n of AIC Activities in Health Care Facilities at Gujarat and West Bengal (March-2010) Building Design and Environmental Approaches to AIC – Short Course’ at Hyderabad (Jan-2011) 4 years Working with the Government health programs. Epidemiologist with the Integrated Disease Surveillance Project, responsible for the management of Pandemic Influenza A (H1N1) in its planning, training, epidemiological investigation, drug inventory management, supervision, documentation. Provided technical support to Vector Borne Disease Control Programme in capacity building, supportive supervision, creating liaison with partners, and assisted central nodal team for planning of the Malaria Elimination Pilot Project in Goa. Worked with the NACO as a Sr. Technical Officer for Mainstreaming AYUSH where he was responsible for coordinating with Technical Resource Group, networking with NACO partners and assisting in the planning the national strategy for AYUSH inclusion into the NACP phase III. Evaluation of Tuberculosis Program in India: PATH Report 72 Taught courses on Public Health and Basics of Hospital Management at Marian Institute of Health Care Management. Research assistant at Jawaharlal Nehru University for a Joint Participatory Mid-term evaluation of the Arogya Project of Aarohi Dr. Mayank Ghedia 30 Male MBBS,M D Microbiol ogy AIC Officer TA to pilot Project at 3 states. Supporting IRL for up-gradation and accreditation in various states of India. Third International Short Course in Clinical Tropical Medicine on 2- 14/02/ 2009 at Infectious Disease Training and Research Centre, Christian Medical College, Vellore. Training on Culture and DST at TRC, Chennai on 16 July – 2 August Training on External Quality assurance in sputum smear microscopy at TRC, Chennai on July – August 2009 Research Dissemination Work shop on TB and Lung Disease organized by WHO and TRC on 9- 10/12/ 2009 Workshop on Pilot Implementatio n of AICl Activities in Health Care Facilities at Gujarat and West Bengal (March-2010) Building Design and Environmental Approaches to Airborne Infection Control – Short Course’ at Hyderabad (Jan-2011) 2,8 years TRC, Chennai as consultant Microbiologist in WHO-APW project. As consultant Microbiologist he conducted training of Microbiologist & Laboratory Technician on EQA in Sputum Smear Microscopy and Culture & DST. Experience in coordinating Drug resistance surveillance and routine laboratory work at TRC, Chennai. Christian Medical College, Vellore as Scientist at managing Antimicrobial Resistance Surveillance for Gram negative pathogen (ICMR-Multi Centric Study), Clinical Trials, Nursing TB study and Sr. Medical officer of ART center. As Faculty at Infectious Disease Training and Research Centre, Department of Medicine at Christian Medical College, Vellore for NACO programme and Counselor training Evaluation of Tuberculosis Program in India: PATH Report 73 Dr. Shanta Ghatak 48 Female MBBS, M Phil MDR-TB Officer Expansion of MDR TB efforts in scaling up. She has been part of the appraisal team for scaling of DOTS plus in the states of Maharashtra and Madhya Pradesh and has done 2 PMDT experience sharing workshops in the country. Clinician Specialization in Health Economics, Biostatistics, Epidemiology, Health Systems Strengthening , Strategic Management, MIS, Environmental Health sciences, Field Epidemiology, Organizational Behavior , OR , Accountancy and costing management , Organizational behavior, General management - principles Trained at the national level on DOTS Plus guidelines and participated in WHO SEARO regional trainings on PMDT. Coursework through the WMA: DRTB, CDC courses on TB USAID e￾trainings on Advanced TB, HIV Certified from CDC and John Hopkins Bloomberg School of Public health on Tobacco control 27 years WHO medical consultant (10+ yrs) Strong background in TB control implementation and expansion - in the public sectors, private sectors, medical colleges, 4 large TB sanatoriums, corporate sectors and NGOs. During RNTCP I and II : Implementation , Expansion and Consolidation phases , collaborating extensively with the corporate sector , urban municipalities including second largest slum in the country , private providers and the public sectors One of the first to incorporate the TB/HIV networking in DMCs and ICTCs in the NRHM policy developed formulating the reporting matrix at the national level through NACO. Contributed extensively in developing the Quality Assurance protocol for smear microscopy and coordinated with CTD, WHO actively for the RNTCP recording and reporting system and has helped focus on initial defaulters and delays in diagnosing TB. Dr. Pravakar Adhikaree 52 Male M.B.B.S., M.D.(Mic robiology ) Consultant (Labs) Upgrading IRLs for BSL – 3 facility and LPA clean rooms, TA for accreditation of PG Certificate Course in Health & Family Welfare Management; WHO Fellowship on ‘HIV/AIDS 4 weeks’ training at Blue Peter Public Health & Research Center 26 years Worked for 25 yrs in Indian Railways Medical Services in different Administrative & Professional capacities. Jan 2005-Established a Bact. Lab. & worked as I/C for 10 yrs with Evaluation of Tuberculosis Program in India: PATH Report 74 C&DST labs, support installation of equipments in IRLs Surveillance & Awareness’. Master trainer training in RNTCP at LRS, New Delhi (BPHRC) on “Culture & DST for M. tuberculosis” outstanding quality certificate for consecutive 4 years (2005- 2009) by Bio- rad laboratories under EQAS Sept 2005-Established a Telemedicine Centre at Bilaspur, SEC Railway). Ms. Naveena Ambatipu di 41 Female Bachelor s in Commer ce; Diploma in Architect ure Administrator for India TB Project and also Hyderabad Site Administrator Preparation of project budgets, track expenditures, Procurement Prepare draft Project reports Assist the Project director in recruiting Participated as a facilitator and note taker in various Research studies on HPV, Safe Water Project, Rabies Intra Dermal User adaptability etc Commerce with specialization in personnel management and Industrial relations Architecture Dip in French and Russian In house trainings - Project Administrator’ s training in PATH Seattle Grants management and Foreign exchange rules for NGOs SAP HR Module 18 yrs Worked in International NGOs like Save the Children (Water and Sanitation Project) and Swiss Agency for Development and Cooperation (Natural resources Management) as an Administrator looking after administration and finances of the Projects and also the Partner NGOs Successfully handled International audit both in PATH and Swiss Agency for Development and Cooperation Worked in an architectural consulting firm. Mr. Shiva Shrestha 30 Male Masters in Health Administ ration ACSM, M&E and PPM Officer Implementing TA project on ACSM and PPM in selected states of India. Support building capacity Monitoring and Evaluation ACSM and PPM Proposal development 4 years ACSM implementation of USAID Jump start (LEPRA) Global Fund Round 9 TB project implementation at Sub recipient level (with LEPRA) European union project on TB and Evaluation of Tuberculosis Program in India: PATH Report 75 and providing TA to sub recipients and primary recipients of Global fund Round 9. Project Professional TOT Life Skill education RNTCP, HIV AIDS, Leprosy GIS and Health care HIV AIDS (LERPA) Global fund Rd 6 PACT, RCC project implementation at SSR level (community care center) Public health consultant – technical assistance to Govt. of MP Mr. Srikar Siddarth 39 Male Masters in Social work Post graduate in Marketin g and sales managem ent ACSM and PPM Officer ACSM, PPM Pharmacy Pilot and work place interventions for TB control in AP. Livelihood for poverty elimination Social marketing of condoms ACSM Work Place intervention and PPM 15 years Worked with AP State AIDS Control Society (APSACS) for strengthening workplace response on HIV/AIDS with ILO and Department of Labor through Public Private partnerships Worked with various Government Departments, Corporates, Trade unions and NGOs on HIV/AIDS Worked with Society for Elimination of Rural Poverty (SERP) on Strengthening livelihoods, Hindustan Latex Family Planning Promotion Trust (HLFPPT) on condom Promotion, NAANDI Foundation on Safe drinking water programme. Ms. Sujata Rao 26 Female Post graduate in Health Managem ent Program Assistant Program assistance to the project and also responsible for providing TA in scaling up of pharmacy project in Rangareddy district, Andhra Pradesh. Trained on ACSM & PPM, Programme management and NRHM MIS. Training on OR Course on Protection of Human subjects in research 3 years Worked with UNICEF Orissa in the capacity of State consultant Adolescent Anemia Control to design state plans of implementation, modules for training, reporting tools, conducting trainings, monitoring the implementation of project & analysis of data to provide feedback. Worked in the Govt. of Gujarat as a District Programme Coordinator Evaluation of Tuberculosis Program in India: PATH Report 76 and was providing programme management and technical support in implementation of national programmes like RCH, Malaria, AIDS, and RNTCP etc Evaluation of Tuberculosis Program in India: PATH Report 77 Appendix 6: INTERVENTION MODEL DEVELOPED by PATH