SLGD WORK PLAN November 14, 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by The Mitchell Group, Inc. (TMG) Tuberculosis Health Action Learning Initiative (THALI) – Final Performance Evaluation FINAL EVALUATION REPORT This document was prepared for the United States Agency for International Development, AID￾OAA-I-15-00028/72038619F00002: Tuberculosis Health Action Learning Initiative by The Mitchell Group, Inc. Principal Contacts: Jenkins Cooper, Vice President, TMG Email: jenkinsc@the-mitchellgroup.com Phone: +001-202-567-1097 (Mobile) 202-350-0025 (Direct) Abou Kone, Program Manager, TMG Email: abouk@the-mitchellgroup.com Phone: +001-202-350-0021 The Mitchell Group, Inc. 1816 11th Street, NW Washington, DC 20001 Tel: 202-745-1919 FINAL PERFORMANCE EVALUATION FOR TWO TUBERCULOSIS HEALTH ACTION LEARNING INITIATVE (THALI) PROJECTS DRAFT EVALUATION REPORT AID-OAA-I-15-00028/72038619F00002 NOVEMBER 14, 2019 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. i TABLE OF CONTENTS ACRONYMS.................................................................................................................................. 1 EXECUTIVE SUMMARY.............................................................................................................. 2 INTRODUCTION......................................................................................................................... 4 METHODOLOGY......................................................................................................................... 6 PHASE 1: DESK REVIEW AND TEAM INDUCTION............................................................................................................6 PHASE 2: FIELD PHASE .........................................................................................................................................................8 PHASE 3: ANALYSIS AND REPORT WRITING....................................................................................................................8 FINDINGS, DISCUSSION AND RECOMMENDATIONS ....................................................... 9 FINDINGS: NOTIFICATION/DIAGNOSTIC PHASE I ..........................................................................................................9 FINDINGS: TREATMENT - PHASE II...................................................................................................................................15 RECOMENDATIONS................................................................................................................. 24 GOAL I: ESTABLISH PROOF OF CONCEPT FOR PATIENT-CENTERED CARE................................................................24 RECOMMENDATION #1: ENSURE THAT ANY THALI FOLLOW-ON SOLICITATION DEMANDS DEMONSTRATED EXPERIENCE OR CAPACITY IN SOCIAL WORK, CASE WORK, OR QUALIFIED CARE EXPERIENCE ........................25 RECOMMENDATION #2: ESTABLISH METRICS FOR TREATMENT OUTCOMES AS WELL AS INTERMEDIATE RESULTS ..............................................................................................................................................................................................25 RECOMMENDATION #3: ESTABLISH RIGOROUS STANDARD OPERATING PROCEDURES FOR PATIENT-CENTERED CARE WITH SUPPORTING DOCUMENTATION ..............................................................................................................25 RECOMMENDATION #4: ADVANCE THE DIFFERENTIATED CARE MODEL ...............................................................25 RECOMMENDATION #5: PROVIDE IPS SUFFICIENT TIME TO MEET SOLICITATION EXPECTATIONS.....................26 RECOMMENDATION #6: PROVIDE TECHNICAL ASSISTANCE.....................................................................................26 GOAL II: ESTABLISH A PRIORI GOVERNMENT COMMITMENT....................................................................................26 RECOMMENDATION #7: ENGAGE BOTH THE NATIONAL AND STATE GOVERNMENTS IN A ROADMAP TO IMPROVED TB TREATMENT OUTCOMES PRIOR TO LAUNCH.....................................................................................26 RECOMMENDATION #8: ENSURE THAT THALI FOLLOW-ON IPS HAVE A DIRECT MANDATE AND MECHANISM FOR ADVOCACY AT THE NATIONAL AND STATE LEVELS. ..........................................................................................26 RECOMMENDATION #9: ADVOCATE FOR PATIENT-CENTERED CARE AT EVERY STEP...........................................27 GOAL III: IMPROVE GOI TB INVESTMENT MECHANISMS .............................................................................................27 RECOMMENDATION #10: USE THALI FOLLOW-ON DATA TO ENSURE THAT PIPS ARE STRATEGICALLY TARGETED ..........................................................................................................................................................................27 RECOMMENDATION #11: ASSIST STATE GOVERNMENTS IN DESIGNING COMPETITIVE PROCUREMENTS..........27 GOAL IV: CONTINUE TARGETING KEY AFFECTED, MARGINALIZED POPULATIONS................................................28 RECOMMENDATION #12: CONTINUE TO SEEK OUT AND SERVE THE MOST VULNERABLE ..................................28 ANNEX 1: SCOPE OF WORK.................................................................................................. 29 ii ANNEX II: EVALUATION METHODS ................................................................................................................................32 ANNEX III: SOURCES OF INFORMATION ........................................................................... 35 ANNEX IV: DATA COLLECTION INSTRUMENTS.............................................................. 47 ANNEX V: EVALUATION SCHEDULE ................................................................................. 56 ANNEX VI: SUCCESS STORIES .............................................................................................. 58 1 ACRONYMS BCC Behavior Change Communication CBNAAT Cartridge Based Nucleic Acid Amplification Test CDCS Country Development Cooperation Strategy DMC Dedicated Microscopy Center DR-TB Drug-Resistant Tuberculosis DTO District TB Officer ET Evaluation Team FGD Focus Group Discussion HIV Human Immunodeficiency Virus HV Health Visitor IDAT Integrated Drug Adherence Technology IP Implementing Partner IR Intermediate Result JEET Joint Effort to Eliminate TB KHTP Karnataka Health Promotion Trust KII Key Informant Interviews LGBT Lesbian, Gay, Bi-sexual, and Transgender MELP Monitoring Evaluation and Learning Plan NGO Non-Government Organization NSP National Strategic Plan 2017 - 2025 PIP Project Implementation Plan PPSA Patient Provider Support Agency RNTCP Revised National Tuberculosis Control Program RT Research Team STO State TB Officer STS Senior Treatment Supervisor TB Tuberculosis THALI Tuberculosis Health Action Learning Initiative TMG The Mitchell Group TOCOR Task Order Contracting Officer’s Representative TPM Team Planning Meeting USAID United States Agency for International Development WHP World Health Partners 2 EXECUTIVE SUMMARY The Tuberculosis Health Action Learning Initiative (THALI) is a four-year program funded by USAID/India that has sought to improve tuberculosis diagnoses, notifications, and treatment in nine states in India. Launched in 2016, THALI has been implemented by two contracted consortia. One consortium has been led by World Health Partners in the west and north on India, while the second consortium has been led by the Karnataka Health Promotion Trust in the south. In August 2019, USAID/India contracted with The Mitchell Group and its local partner, New Concept Information Systems, to conduct a ten-week evaluation of THALI achievements and challenges to help USAID/India better understand the nuances of its investment in THALI in terms of what worked, what did not work, what are the internal and external variables and assumptions that influenced those results. Over the course of the evaluation, a five-member Evaluation Team conducted a rigorous desk review of THALI source literature as well as conducted primary qualitative data through key informant interviews and focus group discussions with over 380 individuals including THALI staff, clients, partners, government officials, and medical practitioners in five states targeted by the program including Gujarat, Jharkhand, Karnataka, Telangana, West Bengal. Quantitative data from source documents and qualitative data from field interviews was analysed by the Evaluation Team using thematic coding and triangulation to produce the findings and recommendations provided in this report. A summary of these findings and recommendations is presented below. FINDINGS The evaluation results concluded that the Phase I of the THALI program, which lasted from inception to December 2017, was successful in its primary mission to increase private sector notifications for presumptive TB cases in urban slums in the targeted cities of Bengaluru, Hyderabad, and Kolkata. Source data from THALI implementing partners and the Government of India’s Nikshay database confirmed that private sector notifications rose by 577% between 2016 and 2017 in the municipality of Kolkata and by 562% in the city of Hyderabad. Further, data confirmed that the THALI program had provided the Government of India’s Revised National Tuberculosis Control Program (RNTCP) with proof of concept that active case finding methods can be effective in raising private sector notification rates. Unfortunately, just as THALI had proven a model for urban private notifications, the United States government passed the Protecting Life in Global Health Assistance policy, which disallowed the THALI program partners from working with private health providers to increase notification and improve treatment due to the provision of abortion-related services. However, as a result of THALI’s proof of concept, the Government of India agreed to additional investment from the Global Fund for the Joint Effort to Eliminate TB to continue private sector notification services as well as began funding its own Patient Provider Support Agency program to expand up THALI’s successful model of private sector TB notification services. The evaluation concluded that THALI’s efforts to implement, prove, and transfer a successful model for urban private sector TB notifications had been successful, despite the curtailed timeline. The RNTCP has also adopted THALI’s innovative geomapping strategy for TB hotspots and medical care providers. In 2018, the THALI implementing partners worked with USAID/India to develop a new Phase II strategy that would improve patient-centered care for the TB afflicted in both urban and rural areas in an expended geography including six new states. The evaluation findings note a significant delay in the launch of the new patient-centered care model due to the time required to revise THALI strategy, engage a significant number of new partners, develop and pilot models for patient support, and adapt information systems to track data. These challenges delayed the full launch of patient services to the end of 2018, less than one year before this evaluation. Despite this short implementation period, THALI demonstrated an impressive ability to reach over 9,493 TB patients in the north and 12,464 patients in the south with patient-centered 3 services. Further, THALI activities raised awareness of TB prevention and treatment through community outreach, linked vulnerable households afflicted by TB with valuable government services such as the direct benefit transfer, and innovatively targeted key affected populations such as commercial sex workers, LGBT community members, people living with HIV, and tribal populations, each of which is marginalized from the government TB health system. Qualitative data from interviews and focus groups demonstrated resoundingly that government TB services providers, clients, and community groups appreciated and saw value in THALI’s at-home case work model for assisting in treatment adherence. Further, THALI’s development and deployment of a differentiated care model for assessing treatment default risks has shown promise in assisting RNTCP medical treatment staff in more efficiently allocating their strained human resources to identify and address the needs of the most at-risk TB patients in their jurisdictions. The differentiated care model has already been adopted and is being piloted by RNTCP treatment staff in the states of Karnataka and Telangana. Finally, in a show of partnership, THALI has been contracted by the RNTCP to pilot integrated drug adherence technology in three states targeted by THALI. The evaluation concluded that THALI implementing partners had created an innovative system of in-home and community patient services that have the potential, if properly implemented and documented, to improve treatment outcomes. RECOMMENDATIONS Due to current US policy, any follow-on THALI activity will continue to be barred from targeting private sector health providers. For this reason, the evaluation recommendations below focus on continuing and strengthening the Phase II THALI activities, which the data has shown to have the potential to improve treatment outcomes. The 12 recommendations below are categorized by four main goals to ensure effective implementation, documentation, replication, and scale of a proven patient-centered model impacting treatment outcomes for eventual transfer to the RNTCP. Goal I: Establish Proof of Concept for Patient-centered Care Recommendation #1: Ensure that Any THALI Follow-on Solicitation Demands Demonstrated Experience or Capacity in Social Work, Case Work, or Qualified Care Experience Recommendation #2: Establish Metrics for Treatment Outcomes as well as Intermediate Results Recommendation #3: Establish Rigorous Standard Operating Procedures for Patient-centered Care with Supporting Documentation Recommendation #4: Advance the Differentiated Care Model Recommendation #5: Provide IPs Sufficient Time to Meet Solicitation Expectations Recommendation #6: Provide Technical Assistance Goal II: Establish A Priori Government Commitment Recommendation #7: Engage both the National and State Governments in a Roadmap to Improved TB Treatment Outcomes Prior to Launch Recommendation #8: Ensure that THALI Follow-on IPs have a Direct Mandate and Mechanism for Advocacy at the National and State Levels. Recommendation #9: Advocate for Patient-Centered Care at Every Step Goal III: Improve GOI TB Investment Mechanisms Recommendation #10: Use THALI Follow-on Data to Ensure that PIPs are Strategically Targeted Recommendation #11: Assist State Governments in Designing Competitive Procurements Goal IV: Continue Targeting Key Affected, Marginalized Populations Recommendation #12: Continue to Seek Out and Serve the Most Vulnerable 4 INTRODUCTION PURPOSE, AUDIENCE, and TASK In August 2019, USAID/India contracted with The Mitchell Group (TMG) and its local partner New Concept Information Systems to conduct a Final Performance Evaluation for the Tuberculosis Health Action Learning Initiative (THALI) project. Per USAID’s solicitation, the THALI Final Performance Evaluation sought to answer seven key questions including: 1) What are the key results and outcomes of the USAID TB intervention? Did the program accomplish its objectives and achieve its targets? 2) To what extent have the activities increased the TB and DR-TB case finding and improved the treatment outcomes? 3) To what extent have the innovations demonstrated by THALI been scaled up and replicated? 4) To what extent have the projects been successful in addressing equity (including gender) in the delivery of TB, prevention, diagnostic, care and support services? 5) To what extent has the program activity strengthened capacity of the national and subnational (province and district) government, private sector, community, and other stakeholders? 6) How replicable, adaptable/adoptable, and sustainable are the programs/program components? 7) What is the contribution of the THALI project to the overall USAID/India TB program goals? The audience for the evaluation report includes the USAID/India Mission, and specifically the Health Office and the Program Support Office; USAID/Washington’s Global Health and Asia Bureaus; and the Mission’s implementing partners, World Health Partners (WHP) and the Karnataka Health Promotion Trust (KHTP). USAID will leverage lessons learned from the report to inform the future design of the TB program and the design of new projects and amendments to existing projects and/or activities. TMG launched the final performance evaluation on September 3, 2019 with a project planning meeting at USAID/Delhi, which included the five-member THALI Evaluation Team (ET) and pertinent USAID staff. Over the course of the project planning meeting, USAID clarified the terms of reference for the evaluation, explored options for site visits and meetings, and established a timeline for evaluation activities. An official Project Plan was submitted by TMG on September 13th, 2019 and the evaluation was officially launched on September 15th, 2019. The evaluation scope included qualitative data collection in five of the key states in which THALI is implemented with a final report submission date of November 8th, 2019. The evaluation Scope of Work can be found in Annex I. THALI BACKGROUND AND OBJECTIVES According to the Central TB Division’s 2019 Annual TB Report, India has the largest TB burden of in the world, with approximately 2.7 million cases. In 2017, India’s TB notification rate for new cases was an astonishing 2.15 million. Despite a 16% growth rate in new notifications in 2019, the Government of India’s (GoI) Revised National TB Control Program (RNTCP) has renewed its efforts to identify and stamp out the “missing cases” through the National Strategic Plan (NSP) for TB Elimination 2017 – 2025. To support the GoI’s efforts, USAID/India launched the four-year THALI project in January 2016. The THALI program was designed not only to meet India’s needs as stated in the pre-2017 NSP, but also to meet the strategic goals established and aligned with the United States Government’s (USG) Treat, Cure, and Prevent TB Strategy, the USG’s 2015 National Action Plan for Combatting Multi-Drug Resistant TB, and USAID/India’s Country Development Cooperation Strategy (CDCS) 2012-2016. At inception, the THALI project sought to establish a holistic approach to TB control efforts in districts in the states of Karnataka, Telangana, and West Bengal to identify, apply, and scale up successful, innovative 5 approaches to addressing TB and multi-drug resistant TB. USAID chose two distinct THALI implementing partners (IPs) to launch THALI activities in three distinct urban centers. A consortium led by WHP was contracted to implement THALI activities in Kolkata, West Bengal, while a consortium led by the KHTP and TB Alert/India was engaged to implement TB-notification activities in the cities of Hyderabad in Telangana and Bengaluru in Karnataka. In its first two years, THALI focused exclusively on targeting under- or inadequately served urban slum dwellers and at-risk populations. The original THALI framework targeted support and technical assistance to better integrate private sector health providers and the populations they serve into the greater system of diagnostics, notification, and treatment through India’s RNTPC. Additionally, the project included activities to (a) develop treatment support systems for patient-centered care in vulnerable communities, (b) strengthen the utilization of TB resources by municipalities and the RNTCP, and (c) improve evidence￾based decision making through improved monitoring and evaluation systems. These activities were intended to lead to an “urban model” for public/private integration of TB prevention and control. However, with the advent of the Protecting Life in Global Health Assistance (PLGHA) policy in 2017, the THALI program was no longer eligible to work directly with private sector health providers. Instead, the THALI project’s targets were re-oriented to establish and enable a patient-centered, family-focused TB prevention and care initiative supporting vulnerable populations’ access to quality TB care services from the health care providers of their choice to ensure better treatment outcomes. The revised THALI program, which was formally re-prioritized in mid-2018, seeks to develop Patient Treatment Support Systems and works with and through, (1) communities, especially key populations affected by TB and DR￾TB, and (2) state and local governments and program managers of India’s RNTCP. In addition, the revised THALI program expanded into new states and rural areas in an effort to address new TB needs. The THALI program led by WHP expanded to the new states of Gujarat, Uttar Pradesh, Jharkhand, Bihar, and Punjab in late 2018, while THALI-KHPT expanded to additional rural districts in Telangana and Karnataka with the addition of the state of Andhra Pradesh in the south. Each THALI program has its own distinct set of objectives which contribute to USAID’s overarching goal in TB control. Table 1 below lists out the specific objectives of each THALI component as detailed in the individual IP monitoring and program documents post 2017. TABLE 1: THALI OBJECTIVES BY IP THALI – KHPT THALI - WHP 1. To demonstrate community-centered and community-driven interventions to improve (a) health seeking behavior and service delivery to increase TB/DR-TB case finding, and (b) treatment outcomes among people diagnosed and initiated on TB/DR-TB treatment. 1. To increase community engagement to facilitate early diagnosis of TB and sustained care for treatment among high-risk groups. 2. To support and develop capacity of RNTCP, and leverage public resources to improve access to acceptable and quality TB services from public and private care providers, thereby increasing TB case finding and improving treatment outcomes. 2. Deliver patient-centered care for TB patients engaged in the THALI network in selected geographies of West Bengal, Gujarat, Uttar Pradesh, Jharkhand, Bihar, and Punjab. 6 3. To translate THALI learning and innovations demonstrated to enable scale-up and replication. 3. Strengthen evidence-based decision making 4. Engage with Municipal Corporation and State Health Department and RNTCP 5. Unlock Government Resources for Patient Support In August 2019, USAID/India contracted TMG and its local partner New Concepts “to help USAID/India to better understand the nuances of its investment in [THALI] in terms of what worked, what did not work, what are the internal and external variables and assumptions that influenced those results.” Based upon the guidelines issued by USAID, TMG and New Concepts developed an evaluation methodology that sought to analyze THALI achievements and explore opportunities for further TB programming. METHODOLOGY This section details the TMG approach and methodology for the THALI Final Performance Evaluation including sampling framework, data collection instrument development, data collection methods, and analytical framework. TMG chose a mixed-method data collection strategy that included descriptive quantitative analysis of source data supported by direct qualitative data collection. Quantitative and qualitative data findings were thematically coded and triangulated to reveal findings and explore recommendations. The THALI evaluation was conducted in three phases: (1) Desk Review and Evaluation Work Plan, (2) Field Phase, and (3) Analysis and Report Writing. PHASE 1: DESK REVIEW AND TEAM INDUCTION Phase 1 of the evaluation consisted of a desk review of key project documents provided by USAID/India and the THALI IPs, which included: THALI Cooperative Agreements; Monitoring Evaluation and Learning Plans (MELPs); Annual Workplans; THALI Quarterly and Annual Reports; Bi-Monthly Updates; research reports; technical briefs; case studies; behavior change communication (BCC) materials; and others. In total, over 200 documents, videos, presentations, and reports were reviewed. A full bibliography of reviewed documents is included in Annex III. The desk review formed the basis for the evaluation approach and lines of inquiry which were explored and clarified further during the ET induction with USAID and the IPs. On September 6, six days after contract execution, the ET led a Team Planning Meeting (TPM). The TPM provided USAID/India an opportunity to present the purpose, expectations, and agenda for the evaluation assignment. The ET used the TPM to refine the evaluation approach, to establish contact with the IPs through USAID/India introductions, and to establish a working protocol with the Task Order Contracting Officer’s Representative (TOCOR). GEOGRAPHY AND STAKEHOLDER SAMPLING Across both components, THALI covers a total of nine states and over 30 districts. The scope of the evaluation did not afford the ET the opportunity to visit all target districts. Over the induction period, the 7 ET worked with USAID and the IPs to establish a reasonable geographic sampling framework, which allowed for investigation into standard THALI operational programming as well as into states and districts with critical demonstration projects and innovations. The final geographic scope for the evaluation included Kolkata and the nearby district of Hoogly, Ahmedabad and Surat in the state of Gujarat, Ranchi and Jamshedpur in the state of Jharkhand, Bengaluru and Belgaum in the state of Karnataka, and Hyderabad and Warangal in the state of Telangana. Further, the ET worked with USAID and the IPs to develop a key informant sampling matrix. The key informant matrix included a representative sample from all THALI stakeholders including government partners, clinicians, community engagement and mobilization agents, as well as beneficiaries. Prior to the field phase, the ET finalized a list of targeted key informants for the IPs to begin scheduling meetings. The IPs provided the ET logistical support, contacts with key informants, meeting scheduling, local transportation, and trouble shooting. The ET recognized that employing THALI IPs to establish KI and FGD contacts and meetings introduces a significant bias into the sample population. As suspected, almost all interviews provided positive responses to THALI activities. When pressed to provide contact information from KIs who had challenging interactions with THALI, THALI IPs attempted to expand the KI list. USAID and TMG discussed the challenges this bias posed during the TPM and acknowledged that this was an unavoidable flaw in design and scope given the parameters of the evaluation. In total, the ET conducted interviews with 375 key informants (KIs) across the five states as well as conducted 5 focus group discussions (FGDs), with groups ranging from 4 to 10 respondents, with clients and THALI staff. A graphic representation of the key informant and focus group breakdown by state and by groups can be found in Table 2. TABLE 2. BREAKDOWN OF KEY INFORMANT INTERVIEWS/FOCUS GROUPS BY STATE AND GROUP Totals by State Clients THALI Staff Govt. Other FGD Total West Bengal 44 31 19 6 2 102 Gujarat 5 6 31 1 0 43 Jharkhand 10 3 10 2 0 25 Karnataka 22 28 31 20 1 102 Telangana 32 54 15 5 2 108 Totals 113 122 106 34 5 380 DATA COLLECTION INSTRUMENTS During the induction period, the ET drafted data collection instruments for individual KIIs and FGDs. KII protocols were designed using semi-structured questionnaires. The semi-structured questionnaires solicited information from KIs pertaining to specific activities in which they engaged within the THALI framework. The “structured” nature of the questionnaires ensured that the interviews covered issues consistently across KIIs with similar THALI interaction profiles for comparative purposes. The “semi” nature of the questionnaires ensured the enumerator and/or interviewer had the opportunity to explore and probe unique or unanticipated topics which may inform evaluation findings. FGD protocols were tailored to issues relevant to the type of participants in each FGD. Each FDG protocol contained a list of topics common to FGD participants and upon which each participant could informatively comment or elaborate. For example, FGD protocols for client focus groups solicited 8 viewpoints on the common causes of treatment avoidance, allowing them to debate and/or discuss their unique experience during the interview. Where possible, particularly among government officials and medical officers, English and Hindi protocols and questionnaires were used. For other KIs, such as outreach workers, self-help groups, and end-user clients, protocols were translated into each of the local languages including Bengali, Urdu, Kannada, and Telegu. A sample of KII questionnaires and FGD protocols for the four main tiers of key informants (government officials, private sector practitioners, field health functionaries, and client) can be found in the Data Collection Methodology section in Annex II. Due to time constraints and the lack of THALI operations in Delhi, the ET was unable to pilot test these instruments prior to deployment. These protocols were piloted, refined, and redeployed for various intermediary KIs in the field over the course of data collection. PHASE 2: FIELD PHASE The field phase began immediately following USAID’s approval of the Project Plan. The ET Lead provided regular progress updates to USAID during the Field Phase through Weekly Updates provided on the Friday of each week. Data collection in the field was conducted over four weeks from September 16, 2019 to October 1, 2019 and from October 9, 2019 to October 13, 2019. Field data was not collected from October 2 – 8, 2019 due to national and local holidays. The field visits involved two teams working simultaneously: (1) the ET (which includes the four key personnel (Team Lead, Evaluation Specialist, TB Specialist, and BCC Specialist), and (2) local Research Teams (RT)—hired local enumerators fluent in the local language and trained in research methods for Kolkata, Bengaluru, Hyderabad, and Warangal. The Evaluation Schedule can be found in Annex V, and a full list of KII and FGD Interviews can be found in Annex III. PHASE 3: ANALYSIS AND REPORT WRITING The ET organized the data by relevant emergent themes and sub-themes from the KII and FGD records and transcripts, when available. A comparison of interview reports during daily ET debriefs elicited deeper insights and findings, particularly through the cross-fertilization of ideas shared. A triangulation analysis was then conducted across the analysis from the KIIs and FGDs and secondary data gleaned during the desk review. Triangulation of all data and evidence determined the robustness, validity, and reliability of the findings. The ET’s evaluation framework analyzed data across three dimensions. The first dimension was the two distinct phases of the THALI program, pre-PLGHA and post-PLGHA. Though the Program Description remained the same, these two phases of the THALI program were found to be distinct in both objectives and activities. The second dimension included the 7 research questions and sub-questions posted in the RFTOP solicitation. For the third dimension, all data was analyzed on the ability of THALI follow-on potential to contribute to improvement of India’s RNTCP capacity to provide TB services to vulnerable groups. After field data collection was complete, the ET provided USAID with the required Debrief Presentation on October 15, 2019. The Debrief Presentation was attended by approximately 10 members of the USAID staff from the Contracting Office, Health Office, Program Support Office, and the Asia Bureau. 9 FINDINGS, DISCUSSION AND RECOMMENDATIONS The current section breaks down the findings and recommendations for each phase of the THALI program. Phase I from 2016 to PLGHA enforcement in December 2017 is labeled as the Notification/Diagnosis Phase, as its primary goal was to improve private sector notification rates. Phase II, from December 2017 through to data collection, is labeled the Treatment Phase, as it focused on patient-centered care. While there is overlap between the two phases, the breakout and evaluation of each phase separately lends itself more readily to recommendations. It should be noted that the data provided below is illustrative and not summative or cumulative. Each of the two THALI programs had nuances in their respective program approaches, their activities, the indicators they tracked, the methods by which they tracked data, and their methods of reporting. Due to these nuances, quantitative and qualitative data below is illustrative of various achievements across the two programs, not cumulative indicators of both programs’ efforts. This section provides the findings for each phase delineated by the seven evaluation research questions posed by USAID. All findings articulated are based on emergent dominant themes that surfaced during thematic coding of qualitative data. Where figures and graphs are provided these findings are taken from secondary quantitative datasets acquired from IPs or from desk review sources. Recommendations for further USAID investment in TB control and treatment follow in Section 4. FINDINGS: NOTIFICATION/DIAGNOSTIC PHASE I THALI Phase 1 was targeted specifically to increase the contribution of private sector health practitioners to RNTCP diagnostics and notification. With side activities that included TB community awareness in targeted slums in Kolkata and Bengaluru. The evidence below speaks to THALI Phase I activities in these areas. 1) What are the key results and outcomes of the USAID TB intervention? Did the program accomplish its objectives and achieve its targets? The ET found that THALI did indeed demonstrate success in the improvement of private sector case findings from 2016 – 2017. Despite the fact that life-of-program targets were made irrelevant due to the introduction of the PLGHA policy, the THALI program did succeed in establishing an urban model for private sector TB case findings and referral to government TB treatments systems. The innovation of this model was the deployment of systematic manpower and “boots on the ground” in the engagement of private sector practitioners from hospitals all the way down to chemists and local allopathic healers. THALI outreach staff for both programs were able to engage local private sector practitioners in identifying the key warning signs for presumptive TB (persistent cough and fever over a two-week period) for the collection of sputum samples to be sent to the local Dedicated Microscopy Center (DMC) or Cartridge Based Nucleic Acid Amplification Test (CBNAAT) center for diagnostics, and if registered positive, for notification into the RNTCP Nikshay reporting systems. These centers also tested for MDR-TB when appropriate. This deployment of human resources was beyond the capacity and scope of RNTCP services and resources. Both THALI programs were creative in their approach to private sector practitioners, many of whom mis-diagnosed the symptoms of TB as a cold or a common virus, through systematic and consistent outreach. THALI staff provided private sector practitioners with the contact information for local testing centers, informed practitioners and patients of test results, and assisted private practitioners 10 in the provision of the appropriate treatment in line with the local RNTCP Senior Treatment Supervisor (STS). It should be noted, however, that THALI private sector outreach staff did not have medical training on TB treatment regimens prescribed by individual practitioners and was therefore not able to comment on the appropriateness of these regimens. Several of the private sector practitioner evaluation respondents noted that they preferred their own treatment regimens based upon their lengthy experience to those recommended by the RNTCP. Graphs 1 provides a visual of the increase in private sector notifications for both the WHP and KHPT/Telangana THALI programs. GRAPH 1: TB NOTIFICATION RATES 2016 – 2018 IN THALI TARGET GEOGRAPHIES Kolkata Municipal Corporation Hyderabad, Telangana *WHP, 2019 *KHPT, 2019 In both geographies, there is a marked increase in private sector notifications from the start of THALI in 2016 to full implementation in 2017. For the Kolkata Municipal Corporation, private sector notifications went from 819 total in 2016 to 4,732 in 2017, a 577% increase. Hyderabad private sector notifications rose similarly drastically from 498 in 2017 to 2801 in 2018, a 562% increase. In conversations with the THALI, medical, and government staff on the ground, the ET was unable to find any plausible alternative or supplementary program that would have contributed to these dramatic increases aside from THALI. Further, the impact of THALI’s private sector notification systems seems to have been sustainable as numbers for 2018 did not revert to the previously low levels, but remained largely steady in the KMC and rose again significantly in Hyderabad. 2) To what extent have the activities increased the TB and DR-TB case finding and improved the treatment outcomes? As demonstrated through the graph and narrative above, THALI activities seem to have contributed to an increase in TB and DR-TB case findings. However, as THALI was in its infancy at that time and was only targeting notification, there is no data on whether the THALI program impacted treatment outcomes from 2016 – 2017. 3) To what extent have the innovations demonstrated by THALI been scaled up and replicated? 11 As stated previously, the key innovation that THALI introduced in private sector TB notifications was the addition of systematically deployed manpower to engagement private sector health care providers across the spectrum with detailed documentation on location, services provided, and number of TB cases notified. This mapping and documentation allowed for THALI in both the west and south to provide the RNTCP with proof of concept that private sector engagement is key to identify previously missed cases in private care. As was demonstrated, this proof of concept encouraged the RNTCP to adopt parts of the THALI private sector engagement model such as private practitioner engagement and outreach, replicate them, and scale them up using its own resources. This testifies to the power of the evidence supporting increased investment in private sector notification and engagement. Graphs 2 and 3 below provide a visual representation of how THALI engaged with the spectrum of private sector health care providers. GRAPH 2: KOLKATA PRIVATE PROVIDER MAPPING AND ENGAGEMENT 2016 - 2017 *WHP, 2019 As can be seen from Graph 2 above, in Kolkata, WHP was able to map over 10,000 private sector providers in slum areas for potential engagement. Recognizing that it lacked the resources to engage all 10,000 providers, WHP strategically deployed its resources to target the 20% of private providers who had previously noted the highest TB caseload or the highest number of referrals. This allowed WHP to actively engage with over 1800 individual practitioners, a significant number given the resources available and the challenges of working in urban slums. 12 GRAPH 3: BENGALURU PROVIDER MAPPING AND ENGAGEMENT 2016 – 2017 *KHPT, 2019 Similarly, in Bengaluru, KHPT was successful in mapping out approximately 7,600 private practitioners, facilities, and clinicians by the end of 2017, with at least 42% of these having referred at least one TB case to the RNTCP diagnostic system for testing. Interestingly, KHPT also reported that of the 2,059 presumptive TB cases referred as a result of THALI engagement, approximately 44% returned with a positive diagnosis. This is an unusually high number of cases testing positive. The average ranges between 10 – 15% as TB is not a high-prevalence disease. This positive diagnosis percentage is either in error, unfairly attributed to THALI, or evidence of unusually successful outreach targeting by THALI staff. The ET was unable to understand the reason for this abnormally high diagnosis percentage. The main finding for replication is that the proof of concept for private sector notification was successful and was replicated by the RNTCP. With the advent of PLGHA policy, THALI was forced to abandon its original mission to increase private sector notifications. Recognizing the contribution THALI had demonstrated through concrete notification indicators, the RNTCP at the national and state levels actively engaged and endorsed the Global Fund’s Joint Effort to Eliminate TB (JEET) program to replaced THALI in the target areas and beyond. Not only did the RNTCP actively endorse the JEET program, but individual states such as West Bengal, Gujarat, Karnataka, Telangana, and even Jharkhand actively began funding the JEET program in expanded areas through the use of the RNTCP’s Project Implementation Plans (PIPs), the RNTCP’s annual allocation for outsourcing TB control and diagnostic services to the private sector. In Gujarat alone, which had not been one of the THALI target geographies in Phase I, allocated a full 59 million rupees to private sector efforts in six municipal corporations across the state through a Patient Provider Support Agency (PPSA) contract issued through the PIP. This RNTCP investment speaks volumes to the THALI program’s accomplishments, and more importantly, documentation of how to actively and successfully engage private sector health care providers in TB notification and diagnosis. 4) To what extent have the projects been successful in addressing equity (including gender) in the delivery of TB, prevention, diagnostic, care, and support services? The ET did not find any evidence that there were systematic biases in equity for the notification and diagnosis process in the target areas. TB cases for men and women were largely biased in favour of men, which is typical of epidemiology of TB. TB traditionally affects more men than women due to a combination of workplace transmission and poor nutritional and health behaviours such as smoking and alcohol use, both of which are higher in male populations. However, it is important to note that THALI’s theory of change lends itself to an increase in equity of diagnosis and notification for poor and marginalized populations. As THALI specifically targeted urban 13 slums, the target cohort was not the middle class or more affluent classes who have access to more expensive and thorough health care options. Rather, THALI targeted the destitute populations who traditionally prefer the private health sector, including traditional, unqualified, and dispensary health care providers, as their first recourse when ill. There are no specific comparative numbers to prove that THALI increased equity of access to diagnosis and notification for impoverished populations, but traditional social science theory indicates that by targeting the impoverished and marginalized, one can increase their access to more qualified and appropriate medical care. However, THALI’s re-orientation to patient-centered treatment and care in Phase II of the program provided evidence that increased targeting of marginalized populations (commercial sex workers, HIV+ individuals, miners, tribal dwellers, LGBT, etc.) did increase their awareness of TB control and access to services. This will be explored further in the following section - Findings: Treatment Phase. 5) To what extent has the program activity strengthened capacity of the national and subnational (province and district) government, private sector, community, and other stakeholders? The GoI’s RNTCP system does not lack for financial resources. In fact, two of the District TB Officers (DTOs) specifically mentioned that “if we need money, we get it.” What the RNTCP did lack was planning, training, and manpower capacity to fulfil its mission for maximum outputs. THALI did contribute to each of these three deficiencies to varying degrees. With regards to planning, THALI was instrumental in providing the RNTCP with both private sector mapping of health care providers as well as with geomapping of urban slums to identify TB “hotspots,” both of which contributed to the RNTCP’s ability to better target it resources. Specifically, THALI’s geo￾mapping of slum hotspots was named as particularly helpful for the RNTCP to understand the epidemiology of TB in slum settings. Picture 1 below provides an illustration of TB Geomapping conducted by WHP in West Bengal during both Phases I and II of the THALI program. PICTURE 1: EXAMPLE OF THALI WHP GEOMAPPING IN WEST BENGAL How replicable, adaptable/adoptable, and sustainable are the programs/program components? 6) What is the contribution of the THALI project to the overall USAID/India TB program goals? *WHP, 2019 14 The geomap above overlays 9,077 TB cases reported in West Bengal with 472 private and public health care facilities in the surrounding areas. As a first attempt to geolocate TB prevalence relative to health care access, the WHP geomapping exercise was distinctly impressive to the State TB Office (STO) in West Bengal. In fact, as a result of both private sector health care mapping and geomapping, the STO offered WHP THALI program a contract for 1.5 million rupees to conduct further geomapping in several more slum districts in Kolkata and the surrounding area. For unknown reasons, WHP did not take advantage of this contract offer. Nonetheless, the information and capacity building for RNTCP planning was demonstrated and appreciated by the STO and DTOs interviewed and was included into the subsequent Kolkata JEET Program. Similarly, THALI was widely recognized for its ability to provide the state and district RNTCP offices with the timely training resources that were needed. While the RNTCP has a formal training and roll-out program for new initiatives and new policies, this training system was noted to be slow and cumbersome in reaching reach staff on the ground. For example, in December 2017, the RNTCP released the guidelines for the Program Management of Drug-Resistant TB (PMDT) in India. Recognizing that the official training program for RNTCP state and district staff would be long in coming, KHPT organized its own training on PMDT for local practitioners. Similarly, THALI made its most lasting contribution to government TB programming in the area of staffing and additional human resources. Often, DTOs, STSs, and STOs noted that THALI staff provided additional manpower for tackling the critical issue of increased private sector notification. Local RNTCP staff noted that they were uncertain how to ensure this increased notification without additional manpower, which THALI provided. The ET noted that the RNTCP did not have the proper staffing systems to be able to affect private sector engagement to the extent that THALI did. Recognizing the need for additional, targeted staffing, the national RNTCP and STOs endorsed and funded the JEET program to fill the gap that THALI left when it shifted to patient-centered care in Phase II. Similarly, albeit less well documented, THALI contributed to the capacity of private sector through both integration into the RNTCP systems as well as through training and other exercise. Under KHPT and TB Alert, 2688 health facilities were mapped, and 1144 providers were networked into the RNTCP system. Approximately1652 private providers were enrolled into Nikshay and were registered for a Health Establishment ID. Finally, KHPT conducted 4 continuing medical education seminars for the private sector, in which 198 providers were trained on Standards for TB Care in India (STCI) guidelines. However, the ET noted that this private sector capacity building was more opportunistic than systematic. Finally, and least systematically, THALI peripherally contributed to the capacity building of non-TB focused, non-governmental organizations (NGOs) through partnership and engagement over the course of the program. THALI built this capacity through trainings, TB awareness raising, and the introduction of referral systems through partnership on the ground. While recognized as “valuable” and “innovative” in providing TB awareness services to non-profit NGOs, the ET found no robust evidence to prove that sustainable capacity was built. 7) How replicable, adaptable/adoptable, and sustainable are the programs/program components? THALI Phase 1 was very well targeted and focused to increasing diagnosis/notification from private sector providers as its primary goal. As can be seen above under Question 1, THALI directly contributed to sustainable increases in private sector diagnosis/notification. In fact, it created a model that was adopted in the modified JEET program by the RNTCP to maintain momentum in this area. For this reason, the ET feels that the THALI Phase I program activities targeting private sector notification have proven themselves replicable and sustainable. However, THALI’s secondary goals of increasing awareness and capacity building were less systematically addressed and were more opportunistic than designed for systematization and replication. A more 15 rigorous and documented approach around capacity building and increased TB awareness would be required to demonstrate that these components were indeed replicable. 8) What is the contribution of the THALI project to the overall USAID/India TB program goals? The USAID/India TB program goals were centered around the objectives of three documents: the Mission Country Development Cooperation Strategy (CDCS), the USG’s Reach, Cure, Prevent TB Strategy, and the GoI’s National Strategic Plan 2017 – 2015. As outlined below, Phase I of the THALI program contributed in whole or in part to each of these documents’ stated objectives. THALI Phase I did contribute to the CDCS Development Objective 1: Increase the capacity of India’s health system to improve the health of vulnerable populations in India. By directly targeting the underserved slum populations, THALI Phase I directly addressed Intermediate Result I.1: Increase access to priority health services. Similarly, THALI Phase I contributed to Development Objective 3: Development innovations impact people’s lives at the base of the pyramid (BOP) in a range of sector in India. Specifically, THALI Phase I effectively addressed Intermediate Result 3.1: Effective health solutions identified, demonstrated, and scaled. The RNTCP’s investment in JEET and PPSA effectively demonstrate that THALI identified an effective health solution, demonstrated proof of concept, and scale up private sector notification. THALI Phase I was less effective in targeting priorities for the USG’s Reach, Cure, Prevent TB Strategy. It can be assumed through the logic model that THALI Phase I addressed the Strategy’s Objective I: Improve Access to High-Quality, Patient-Centered TB, DR-TB, and TB/HIV Services. While THALI definitely improved access through increased private sector reporting, the ET does not have evidence to support the assumption that THALI Phase I contributed to “high-quality” services. However, the ET comfortably asserts that THALI Phase I did contribute to the Strategy’s Objective 3: Strengthening TB Service Delivery Platforms, specifically Outcome B: Comprehensive Partners and Informed Community Involvement. THALI’s successful efforts to increase government/private sector partnership and involve community-based NGOs in TB control efforts both directly speak to Outcome B. Finally, THALI Phase I contributed to several chapters of the GoI’s National Strategic Plan. These include Chapter 5, which speaks to increased case findings; Chapter 6, which directly addresses the needs of presumptive TB patients in the private sector; Chapter 12, which speaks to improving urban TB control systems; and Chapter 15, which is about improved surveillance. Overall, the ET finds that THALI Phase 1 was well defined, well-administered, and innovative in providing the GoI, RNTCP, and USAID with the sustainable private sector notification model that was intended. This is no small achievement given the reduced timeline from 4 years to less than 2 years with the introduction of PLGHA. FINDINGS: TREATMENT - PHASE II The THALI program was directly impacted by the advent of PLGHA in December 2017. Forbidden to work with private sector health providers, THALI IPs worked with USAID over the course of five months to adjust and adapt their strategies, resources, staffing, and metrics to shift to patient-centered care models to assist in TB treatment. These patient-centered care models involved deep-dives into the local slum communities, working with established community groups such as self-help groups and local NGOs, and even expending to more states and districts than during Phase I. In June 2018, WHP THALI expanded its programs beyond the districts of West Bengal to double the number of districts across the six states in total including Gujarat, Uttar Pradesh, Jharkhand, Bihar, and Punjab. This required a significant redeployment of resources, time for set up and staffing, and engagement and relationship building with local government stakeholders and new rural communities. This additional planning delayed the launch of THALI Phase II to October – December 2018. Similarly, in July 2018, 16 THALI KHPT and TB Alert expanded their services to 8 new rural and urban communities across Telangana and Karnataka with the addition of communities in the neighboring state of Andhra Pradesh. Not only did the THALI programs have to expand their geographic targeting to more states and new rural districts, but upon agreement with USAID, THALI began a strategy to address not just the TB treatment needs of impoverished slum populations, but marginalized populations such as commercial sex workers (CSW), LGBT persons, HIV+ individuals, miners who have a high vocational risk for TB, and rural tribal dwellers. Combined, these drastic changes to the approach, and resources of the THALI programs created formidable challenges to effective programming and reduced THALI’s ability to measure meaningful impact on treatment over time. In the words of one government official familiar with THALI programming: “The Mexico Agreement [PLGHA] deprived THALI of their strongest wicket from which they have yet to recover.” Nonetheless, the ET has determined that there is overwhelming qualitative, if not quantitative, evidence that THALI’s community and patient-centered engagement has made a difference to adherence and treatment support that needs to be further built up, standardized, and measured in any future TB engagement. The Evaluation unearthed several qualitative findings that support the endorsement of the potential of THALI’s Phase II patient-centered approach. The ET’s findings show qualitatively measured impact as a result of THALI Phase II’s holistic community engagement, targeting of marginalized populations, and the patient-centered focus on treatment adherence. These three activities have had the greatest potential impact on treatment indicators. The following section outlines the challenges and potentially positive impacts of the THALI program on patient-centered care structured by the seven evaluation questions posed by USAID. 1) What are the key results and outcomes of the USAID TB intervention? Did the program accomplish its objectives and achieve its targets? The ET noted that the outcomes expected and placed before the THALI programs during Phase 2 were marginal and largely unmeasurable given the time allotted. Per their final approved workplans, the expected outcomes of the two THALI programs are represented in Table 3 below. TABLE 3: EXPECTED THALI PHASE II OUTCOMES BY IP THALI – KHPT THALI – WHP 1. To demonstrate community-centered and community-driven interventions to improve (a) health seeking behavior and service delivery to increase TB/DR-TB case finding, and (b) treatment outcomes among people diagnosed and initiated on TB/DR-TB treatment. 1. To increase community engagement to facilitate early diagnosis of TB and sustained care for treatment among high-risk groups. 2. To support and develop capacity of RNTCP, and leverage public resources to improve access to acceptable and quality TB services from public and private care providers, thereby increasing TB case finding and improving treatment outcomes. 2. Deliver patient-centered care for TB patients engaged in the THALI network in selected geographies of West Bengal, Gujarat, Uttar Pradesh, Jharkhand, Bihar, and Punjab. 3. To translate THALI learning and innovations demonstrated to enable scale-up and replication. 3. Strengthen evidence-based decision making 17 THALI – KHPT THALI – WHP 4. Engage with Municipal Corporation and State Health Department and RNTCP 5. Unlock Government Resources for Patient Support Both THALI programs were successful in developing strategic and systematic methods for addressing patient-centered care. Through the use of outreach workers in the form of field workers, community health workers, key opinion leaders, and TB Champions, both THALI programs engaged individuals suffering from TB applying personalized services through regular contact, information provision, and a basic form of case work. This case work involved psych-social assistance, nutritional support, infection prevention support, financial assistance, and referral to medical services. Further, the THALI field officers were often called upon to bolster the lack of staff capacity in monitoring treatment at the district and block levels. In Jhamshedpur, the DTO noted that there was currently a 36% vacancy rate in RNTCP staffing. Further, in Hyderabad, one local STS at a slum-based TB Unit noted that he and the one, lone TB-Health Visitor were in charge of monitoring TB treatment adherence of an unmanageable case load with 375 patients to visit and monitor. Such staffing shortages and large caseloads were not uncommon in the RNTCP system at the state and district levels and THALI health workers made the difference in meeting patient demand to the best of the system’s ability. It should be noted that the THALI field workers received united praise from their RNTCP colleagues in assisting in tracking treatment and supporting treatment adherence. Several district RNTCP officers noted that THALI served as a dedicated member of the RNTCP team and helped bolster the lack of staff capacity at the local TB Unit or DMC levels. Often, the THALI field worker joined hand-in-hand with the district STS and TB-Health Visitor (TB-HV) to cover the more challenging cases. Several STSs and TB-HVs noted that THALI had more flexibility in its engagement strategy in that it could use telephone calls, repeat visits, and community-based groups to help monitor at-risk cases and treatment defaults. Graph 4 below provides an illustration of the WHP THALI program’s outreach efforts and indicators for first time and follow-up visits for counselling and patient support services since July 2018. 18 GRAPH 4: WHP THALI INDICATORS OF ENGAGEMENT IN PATIENT-SUPPORT SERVICES *WHP, 2019 In total, WHP THALI was able to service 9,493 unique TB patients approached for contact tracing, counselling, treatment adherence, and screening of their family members. Contract tracing through community engagement allowed THALI to identify1,664 presumptive TB patients through June 2019. Out of those, 231 cases converted as TB patients. THALI KHPT patient-centered engagement indicators were no less significant. In the year from July 2018 when the new patient-centered care services began, THALI KHPT was able to engage 12,464 TB patients and identify 5,385 cases with heightened risk of treatment default. They were able to do this through innovative support services such as Differentiated Care Model (DCM) and the use of an innovative Risk and Needs Assessment (RANA), which are discussed under innovations below. Further, the THALI patient-centered care model provided demonstrable impact in linking marginalized and affected populations to necessary government and community services. The key driver for this linking was the launch of the 500 rupee per month Direct Benefit Transfer (DBT) program to support nutritional consumption for TB affected individuals. The DBT was launched in early 2019 by the RNTCP and became a time-consuming tasks for local RNTCP staff in assisting TB patients to receive the appropriate identification cards, open a bank account at a local bank, and finally link the patient’s identification card and bank account to the DBT system. THALI Phase II’s patient-centered model fit well with helping impoverished and marginalized groups to access services that they did not know were available to them. Graph 5 below shows the efforts of TB Alert in Telangana to refer patients to community-based services including the DBT. 1528 2472 2786 2707 241 742 1186 2107 0 500 1000 1500 2000 2500 3000 JUL-SEP'18 OCT-DEC'18 JAN-MAR'19 APR-JUN'19 Number No. of TB patients counselled during contact tracing and follow-up visits (N=9493) First visit Follow-up visit 19 GRAPH 5: PERCENTAGE OF PATIENTS LINKED TO SERVICES BY NEED IN TELANGANA *KHTP, 2019 The case work approach adopted by KHTP and TB Alert staff identified 43% of the TB patients under their care as in need of one form of assistance or another. DBT linkage was by far the greatest need at 26% followed by nutritional support, bank account support, and referrals to health-related services. The programs were less demonstrable in their goals to impact the RNTCP, local government structures, and strengthen evidence-based decision making. These are the areas in which THALI follow-on activities should be significantly strengthened. The recommendation section below provides more justification and detail on how to improve these services. 2. To what extent have the activities increased the TB and DR-TB case finding and improved the treatment outcomes? Due to the short timeline and the lack of a THALI Phase II baseline on community-based treatment outcomes, the ET was unable to determine whether the THALI patient-centered care model had a traceable quantitative impact upon treatment outcomes, unlike its demonstrated impact on private sector notification. One of the key challenges around tracking treatment outcomes is that they are not entered into the authoritative Nikshay government TB database until one full year after treatment is completed and these statistics are not available for the district level at which THALI work. As many THALI patient￾centered care activities began only in late 2018 and 2019, the current data on treatment outcomes in THALI areas is not available. This does not mean that THALI did not attempt to record and tracks its own treatment outcomes. KHPT THALI attempted to track treatment outcomes for its patients registered for patient-centered care as can be seen in Picture 2 below. Unfortunately, these outcomes have yet to be verified in the Nikshay database for confirmation with RNTCP-tracked outcomes and should be considered unverifiable at present. 1 2 2 3 3 4 7 16 26 43 Any other schemes Link to free medicines Ration Card ( APL/ BPL) Livelihood support Aadhaar Health related schemes Bank Account Nutrition support DBT linkage Any need % of TB patients (n=7555) 20 PICTURE 2. TREATMENT OUTCOMES FOR TB PATIENTS RECEIVING PATIENT￾CENTERED CARE IN KARNATAKA *KHPT, 2019 Having recognized the impediments to quantifiable treatment outcomes for patient-centered care, the ET found universal approval for and use of THALI home-based interventions and case work through which TB-infected individuals and their families self-reported improvements in treatment adherence. No client sampled noted that THALI outreach workers and home visitors had negatively impacted their ability to adhere to treatment. The ET found it warranted to note that this may be due to sampling bias. As the THALI programs themselves chose the client sample, there is suspect that the ET was only shown best-case scenarios and outcomes. Nonetheless, the power of the attention and care demonstrated by THALI home-visit workers was repeatedly emphasized in client visits as can be demonstrated by the following three testimonials. Many more testimonials, all along a similar vein, can be provided. Client Testimonial #1: Guna Kalindi - Male - 37 years - TB patient, Location: Ulidhi, Jamshedpur, Jharkhand Patient Registered for PCS Care - Karnataka 10734 Treatment completed 7457 Favourable outcome (TSR) 6443 (86%) Death/ Failure 441 (6%) Others (LFU/Not Evaluated/ Regimen change) 573 (8%) Treatment completed outcome not yet entered in NIKSHYA 1166 On Treatment 2111 “Initially I went to a private doctor when I started coughing. When I got no relief even after 3 weeks, my sister contacted "bhaiya" [brother, as he refers to the Field Officer of THALI] and he put us in touch with the Government center for TB treatment. I have been on medication since then; my weaknesses have slowly disappeared, and I have been gaining weight steadily. I am grateful to Bhaiya, he brings me a nutrition basket every month. Bhaiya is Bhagwan (God) for me; he is my family's savior. I am the only bread-earning member of this family. If bhaiya had not pointed us to the Government schemes, I would not have been aware of the benefits I can get." 21 Client Testimonial #2: Harini, Female, 18 years, TB Survivor, Location: Vidyanranpura, Bengaluru (Declined Photograph) Client Testimonial #3: Paresh Rameshbhai Variya, Male, 36 years, TB Patient Location: Surat, Gujarat State 3. To what extent have the innovations demonstrated by THALI been scaled up and replicated? Innovations under THALI Phase II have yet to proven, scaled, or replicated outside of the program. This is primarily due to two factors. First, the THALI Phase II programming was only active for one year from late 2018 to the present. This is not long enough to track treatment indicators and verify them in the Nikshay. Further, there was a lack of rigorous documentation that would lead to proof of concept for replication. The recommendations section below provides more information on the activities required for replication and scale under a THALI follow-on activity. 4. To what extent have the projects been successful in addressing equity (including gender) in the delivery of TB, prevention, diagnostic, care and support services? As noted under the findings for THALI Phase I, the ET did not notice any stark or glaring inequities in access to diagnostics/notification. Government services for diagnostics/notification are readily accessible to those who present themselves as presumptive cases, regardless of gender, poverty level, or marginalized status. However, the ET noted that THALI Phase II’s attention to marginalized groups encouraged and supported them to engage with the formal health care sector for TB services. By definition, marginalized groups are not targeted by government outreach and engagement efforts. THALI appears to be the only program working in communities that are traditionally health care averse. This direct targeting of marginalized groups directly addresses the issue of equity of engagement. For example, THALI WHP placed strong emphasis on outreach to what the program has labelled as “key affected populations” such as commercial sex workers, LGBT individuals, people living with HIV and, most importantly, migrant workers, to bring TB services to their doorsteps. Table 4 below shows a graphic representation of THALI WHP efforts in reaching marginalized populations across their 6 states. “It is only with the support and counselling of Savitha (THALI Field Staff) that I could fully recover from TB. She constantly visited me and gave me guidance on the nutritious food I should eat and the drug regime I should follow. She gave me a lot of emotional support by reassuring me that TB is curable, and I will be back to leading my normal life soon. I attribute my recovery to her and I am happy to be back in college to complete my studies.”(Translated from Kannada) "Mayur bhai (THALI field staff in Surat) is like my brother. He meets me so very often and counsels me constantly. He treats me like a friend and visits me both at home and at the shop frequently. It is only because of Mayur Bhai that I have been able to continue medication, even though I had severe side effects. Mayur Bhai is a pillar of strength for me." 22 TABLE 4: THALI WHP PHASE II OUTREACH TO KEY AFFECTED POPULATIONS Slum (Population) 847,333 CSW 2810 LGBTQ 152 PLHIV 382 Migrant labor work force (KMC) 3576 During outreach to these groups, THALI identified 28 presumptive TB cases, two of which were confirmed, diagnosed, and began treatment. These numbers may seem small by comparison with the larger efforts in engaging slum populations, but the awareness raised during this outreach and knowledge of TB prevention and treatment options may have a lasting effect, particularly amongst these more marginalized populations. Further, the fact that THALI is directly addressing they key affected populations speaks to equity of access and equity of engagement, which these groups critically need. Despite this success in engaging key affected populations, there appear to be issues in the equity of services provided by both THALI and the formal health care system. While these inequities are anecdotal, a single example of discrimination in treatment for a transgender TB patient in the testimonial below belies the need for increased attention to reducing stigma across both the NGO and formal health care communities. Client Testimonial #4: Name and Photo Withheld, Transgender TB Patient, Location: Sonogachi, Kolkatta 5. To what extent has the program activity strengthened capacity of the national and subnational (province and district) government, private sector, community, and other stakeholders? Capacity building for government and the private sector were not priorities under THALI Phase II as both programs concentrated their efforts and resources on communities and patients. Nominal, but perhaps important, trainings were provided to government health care workers and a limited number of private practitioners on the use of “soft skills” in patient care and support, but there is no documentation to evidence the use of these “soft skills” in practice. The most formalized effort to build government capacity was demonstrated by THALI KHPT through the attempted transfer of the Differentiated Care Model (see below) in preparation for more efficient resource allocation after THALI exit in January 2020. More effort was placed in the capacity building of local community groups as mechanisms for TB awareness raising, prevention, contact tracing, and treatment adherence. THALI KHPT made a concerted effort to engage local self-help groups, support groups, existing community structures such as labor groups, and health information centers (HICs) to improve the geographic scope and reach of its TB awareness and prevention efforts. Table 5 below provides a summary of THALI KHPT’s efforts at community outreach, engagement, and TB capacity building in Andhra Pradesh and Telangana State with concrete outcomes in notification and diagnosis. “Thanks to this program, we are more aware of TB. However, your field workers do not enter our homes, they do not even enter our colonies. The field worker is OK to come to a group meeting in a common location, but will not enter our homes. We are treated like untouchables at the health centre. The doctor refuses to ‘touch’ us for treatment. I would suggest you train and place transgenders as field workers to address our health needs.” 23 TABLE 5: KHPT COMMUNITY AND HIC ENGAGEMENT IN ANDHRA PRADESH AND TELANGANA Particulars State Number of units No. of TB Presumptive Referred No. of TB Presumptive tested No. Diagnosed TB CSs (Community Structures) since Jan 2019 AP 51 202 96 7 TS 145 264 188 44 Total 196 466 284 51 HICs (Health Information Centres) since Jan 2018 AP 27 48 24 2 TS 152 248 181 21 Total 179 296 205 23 *KHPT, 2019 6. How replicable, adaptable/adoptable, and sustainable are the programs/program components? In just the single year of implementation from late 2018, THALI has demonstrated two concrete innovations that have the potential for scale up and replication. The first of these is the KHPT Differentiated Care Model (DCM) and accompanying patient Risk and Need Assessment. The second is the use of Integrated Drug Adherence Technology (IDAT). The DCM for patient-centered services is a common tool used by social workers and case workers around the world. The DCM uses a risk and need assessment (RANA) to identify risk factors for treatment default including extreme poverty, substance abuse, mental health disorders, co-morbidities, living conditions, and other factors to identify those patients who will most likely need targeted support. The majority of TB cases will require simple regular check in and follow-up. The more severely challenged cases will require additional effort. The DCM is useful in helping case workers such as THALI field staff, STSs, and TB-HVs to prioritize TB cases for additional treatment and more efficiently allocate time and resources to high risk individuals. The DCM was piloted by KHPT beginning in October 2018. Since that time, 12,464 TB patients were assessed through RANA and 5,385 patients were enrolled into the DCM approach. Evidence collected by the ET supports the finding that the DCM is an innovative mechanism for THALI patient-centered care, but the ET was unable to obtain documentation on how it was deployed after the risk assessment. Questions remain as to how much time was dedicated to non-risk TB patients versus those with multiple risks, which risk factors have proven most difficult to tackle, and how the RANA has affected time and resource allocation. This lack of comparative documentation belies the fact that DCM was a pilot initiative and has not yet been integrated into a rigorous indicator framework. Nonetheless, KHPT THALI has begun training RNTCP staff in Hyderabad and Bengaluru on the use of RANA and DCM to prepare them for when THALI services are no longer available in January 2020. Preliminary indications from interviews with 6 STSs and 3 TB-HVs have shown them to be receptive to the DCM approach if not fully trained on how to use it to best deploy limited time and resources. One STS and TB-HV at a TB Unit in Hyderabad were able to show us physical copies of the RANA that they themselves had deployed. The integrated and rigorous implementation of DCM as a mechanism moving forward figures prominently into the recommendations for future programming. The second potential THALI component for replication is the use of Integrated Digital Adherence Technology (IDAT). IDAT is commonly used around the world to integrate the use of cellular phone and internet technology as a method of verification of adherence to particular treatment regimes. The two most common IDAT technologies are 99DOTS and the Medical Event and Reminder Monitoring (MERM). Both technologies make use of cellular networks to inform the RNTCP when medicines have been accessed, but does not verify that they have been ingested. 24 The RNTCP launched the pilot IDAT initiative in May 2019 across seven districts in the THALI states of Karnataka and Gujarat. Due to its emphasis on treatment adherence, the RNTCP chose THALI as the pilot implementing partner rather than implement IDAT on its own through government structures. This may hint that the GoI values THALI’s potential role to improve treatment outcomes. In mid-September 2019, the RNTCP conducted an initial assessment of IDAT implementation in parallel to this THALI Final Performance Evaluation. The results of the RNTCP initial assessment have yet to be released, but the government is already looking to THALI’s efforts at IDAT implementation for potential scale up and replication. 7. What is the contribution of the THALI project to the overall USAID/India TB program goals? As with THALI Phase I, Phase II was designed, albeit hastily, to address the three main documents governing USAID/India’s TB program goals including the Mission Country Development Cooperation Strategy (CDCS), the USG’s Reach, Cure, Prevent TB Strategy, and the GoI’s National Strategic Plan 2017 – 2015. Unfortunately, due to its lack of rigor, scale, and treatment quality interventions, THALI Phase II did not structurally contribute to any of the CDCS’s three development objectives in a substantive way. The emphasis on patient-centered care did not target IR 1.1: Increased Access to Priority Health Services. By not targeting health service provision, THALI Phase II also did not contribute to IR1.2: Improve the quality of priority health services. Due to a lack of scale, THALI II was also not able to contribute to IR3.1: Effective health solutions identified, demonstrated, and scale. This conclusion does not imply that THALI’s patient-centered approach does not have the potential to contribute to these indicators, but rather that the time and measurement limitations did not allow for demonstrable impact towards the DOs and IRs. THALI Phase II was more effective in targeting priorities for the USG’s Reach, Cure, Prevent TB Strategy. THALI Phase II directly addressed the Strategy’s Objective I: Improve Access to High-Quality, Patient￾Centered TB, DR-TB, and TB/HIV Services by targeting Outcome C – Patient-centered Care and Treatment. Further, THALI Phase II built upon the foundation of Objective 3: Strengthen TB Service Delivery Platforms through targeting Outcome B – Comprehensive Partnerships and Informed Community Involvement. Finally, the ET concluded that Phase II design contributed to several chapters of the GoI’s National Strategic Plan. These contributions include Chapter 8, which speaks to the need to engage priority populations; Chapter 9, which speaks to strengthening patient support systems; and Chapter 14, which supports advocacy, communication, and social mobilization. RECOMENDATIONS With a few exceptions, the current PLGHA policy prohibits THALI and other USAID/India TB implementing partners to work directly with private sector health care providers. As a result, the recommendations section specifies the steps the ET feels necessary to build upon and improve THALI Phase II contributions to patient-centered TB treatment, care, and support. Overall, as demonstrated above, the ET feels that there is compelling evidence to suggest that THALI patient-centered care activities have the potential to contribute to treatment outcomes. However, any follow-on activities in this area should focus on four overarching goals: rigorous proof of concept, a priori government commitment, improved government TB investment mechanisms, and continued emphasis on key affected marginalized populations. The recommendations are structured below to address each of these four goals. GOAL I: ESTABLISH PROOF OF CONCEPT FOR PATIENT-CENTERED CARE As noted consistently throughout the findings, THALI Phase II showed the qualitative potential to impact treatment outcomes through proper implementation of patient-centered care. However, the lack of 25 experience, rigor, proper measurements, and time did not allow for proof of concept. Any THALI follow￾on activity that targets patient-centered care should address each of these issues to ensure that patient￾centered care, a hallmark of the USG Treat, Cure, Prevent TB Strategy, is given proper due. The following recommendations speak directly to establishing proof of concept. RECOMMENDATION #1: ENSURE THAT ANY THALI FOLLOW-ON SOLICITATION DEMANDS DEMONSTRATED EXPERIENCE OR CAPACITY IN SOCIAL WORK, CASE WORK, OR QUALIFIED CARE EXPERIENCE THALI Phase II largely flew by instinct and opportunity, creating a new activity for patient-centered care where available and appropriate. However, patient-centered care has been a hallmark of progressive health care systems for quite some time. There should be no need to recreate the wheel, but rather adapt existing social work, case work, and qualified care systems to the local conditions. They key criteria in any social work sphere is “do no harm.” Only trained and qualified staff can abide that maxim to the best of their abilities. USAID should request that any IP participating in THALI follow-on solicitation have a documented history of case work, identified credentials, and demonstrated outputs. If no applying organization can demonstrate qualified care experience, but maintain other competitive advantages, USAID would be well placed to provide them start-up time for training and supervision provided by a qualified partner institution. RECOMMENDATION #2: ESTABLISH METRICS FOR TREATMENT OUTCOMES AS WELL AS INTERMEDIATE RESULTS Proof of concept not only requires metrics to establish treatment outcomes, but also metrics for the intermediate results that lead to those outcomes. USAID should require follow-on IPs to establish outcome metrics and data collection systems that align with and parallel the government’s Nikshay reporting system, but also metrics used in standard social work that allow for the collection of overwhelming evidence that these outcomes were achieved through USAID-funded TB investment. Sample metrics include patient risks diagnostics, data collection mechanisms for the number of visits/assistances provided, assistance outcomes, and other relevant data. There are numerous resources for tracking social and case work on the web or the IP can be connected to a qualified professional association to receive help. Where possible, the IP should integrate the use of IT such as tablets, smart phones, and other devices to collate data in a standardized manner, but also to disaggregate treatment outcomes by level and form of care and assistance. Only by providing a support theory of change documenting input-process-outcome can the THALI follow-on activity provide overwhelming evidence of its contribution. RECOMMENDATION #3: ESTABLISH RIGOROUS STANDARD OPERATING PROCEDURES FOR PATIENT-CENTERED CARE WITH SUPPORTING DOCUMENTATION The current THALI IPs have minimal documentation requirements and guidelines for providing patient￾centered care. These have served the program well during the tumultuous Phase II implementation. Any patient-centered follow-on should seek to make standard of care more rigorous and documented. For example, traditional and robust social and case work systems include standards of ethics, qualified supervisory and training capacity, cultural and gender sensitivity standards, and standard operating procedures. USAID should require any passing IP to ensure that they have demonstrated capacity in the area of systems documentation or can develop this capacity through partnership with a qualified organization. RECOMMENDATION #4: ADVANCE THE DIFFERENTIATED CARE MODEL DCM is used throughout the world as a standard for social and case work, each DCM tailored to its own target population needs. The ET noted that the DCM was by far the most advanced system developed by THALI IPs to assist in the identification patient default risks, the allocation of resources to those patients evidencing higher risk, and the tracking of specific risk interventions and their outcomes. USAID should encourage follow-on IPs to adopt, expand, and push the boundaries of the DCM model within their local 26 contexts. This will require that IPs continually search for new risk categories, use the metrics established under Recommendation #2 to prioritize risks, and use the documentation established under Recommendation #3 to identify which interventions worked and which did not. Only by pushing the boundaries of the DCM will IPs be able to hone their services to increasingly higher at-risk groups and document their progress. RECOMMENDATION #5: PROVIDE IPS SUFFICIENT TIME TO MEET SOLICITATION EXPECTATIONS It is unlikely that any future USAID solicitation will get responses from qualified local organizations that meet all of the criteria established in these recommendations. That is to be expected. However, many organizations such as KHPT, the Child In Need Institute, and TB Alert have demonstrated that they have other competitive advantages in addressing community-based TB needs. Should an attractive and competent applicant lack some of the capacity for rigorous case work, monitoring, and evaluation, USAID should require that the applicant organization demonstrate how they will achieve these capacities within a reasonable time frame and from authoritative partners and sources. RECOMMENDATION #6: PROVIDE TECHNICAL ASSISTANCE By funding activities in the social service sphere, USAID is obligating itself to provide the highest standards of care and to the best of its ability ensure the maxim of “do no harm.” It is unlikely that all IPs will have the experience, track record, and capacity to implement THALI patient-centered care models to the rigorous standards required for proof of concept. USAID’s Health Office should make itself and additional funding available to provide IPs with additional technical assistance to advance organizational capacity and improve service performance where possible. IP budgets can be very tight once a contract or cooperative agreement is signed. If USAID is to require the most rigorous services available to advance the TB treatment agenda, it would be well placed to allocate minimal additional funds to assist in IP treatment service development. GOAL II: ESTABLISH A PRIORI GOVERNMENT COMMITMENT As demonstrated by its adoption of THALI Phase I, the GoI and RNTCP are willing to invest both finances and political will to activities that demonstrate concrete impact on TB control indicators. THALI Phase I fit nicely with the GoI priority of identifying missing cases in the private sector and was willing to invest funding through the JEET Program and PIP tenders in targeted regions. However, such commitment for proven activities in patient-centered care model should not be taken for granted. The three recommendations below provide a road-map for how government commitment and investment can be ensured over the lifetime of follow-on projects. RECOMMENDATION #7: ENGAGE BOTH THE NATIONAL AND STATE GOVERNMENTS IN A ROADMAP TO IMPROVED TB TREATMENT OUTCOMES PRIOR TO LAUNCH THALI is now a known entity in India. The adoption and investment in post-THALI Phase I private sector notification demonstrates that the government recognizes the value and input of activities that contribute to metrics and indicators. However, this was achieved post-hoc as THALI’s private sector notification contributions were under threat and ready to be terminated. Prior to award, USAID should engage the RNTCP and local state TB offices in key districts in developing a roadmap for TB treatment outcomes using the THALI Phase II model. RNTCP officials should be aware that USAID investment will contribute directly to improved treatment outcomes and should support USAID activities with both political will and a promise to allocate funding once proof of concept has been established. This government commitment can be garnered through memoranda of understanding, partnership agreements, or letters of commitment that should form the cornerstone of THALI follow-on activities. RECOMMENDATION #8: ENSURE THAT THALI FOLLOW-ON IPS HAVE A DIRECT MANDATE AND MECHANISM FOR ADVOCACY AT THE NATIONAL AND STATE LEVELS. THALI Phases I and II were noticeably remiss in advocating the needs of their constituents and their achievements to national and state governments. THALI follow-on programming should have a dedicated 27 mechanism or several mechanisms to promote program activities using data and indicators. These may include quarterly meetings, policy briefs, position papers, rigorous research, or other methods appropriate for the situation. The key criteria here is that any THALI follow-on programming should have a hard-sell approach to government advocacy that continually keeps achievements and lessons learned at the forefront of government policy makers over the life of program. RECOMMENDATION #9: ADVOCATE FOR PATIENT-CENTERED CARE AT EVERY STEP Local and international IPs will often be seen as partners and subordinates to government institutions and policies. USAID commands the convening power and authority to advance its programs’ agendas at every step. The ET concluded that rigorous documentation and advocacy for improved patient-centered care by IPs will not be enough to advance the agenda at the national level to effect policy to the fullest extent of the data. The evaluation recommends that USAID take an active leadership role in collating, dissecting, and advancing the data collected by THALI follow-on IPs for its own advocacy purposes at the national and state levels. USAID advocacy will complement IP advocacy by attacking the policy makers from both ends, local and international. Additional USAID firepower to the advocacy agenda can only help to make the data, documentation, and outcomes stronger. GOAL III: IMPROVE GOI TB INVESTMENT MECHANISMS As demonstrated by JEET and PPSA, the GoI is willing to invest in priorities that produce outcomes, such as private sector notification. However, the JEET Program and PPSA are radically different from THALI Phase I in both budget and structure. There is no guarantee that these mechanisms will parallel THALI outcomes. The GoI and RNTCP have funding available to invest in proven TB control mechanisms through their annual PIPs. Current practices suggest that they are not using this investment to its fullest potential for maximum TB impact. The following three recommendation can help the GoI and RNTCP better adapt their procurement mechanisms to ensure effective investment in patient-centered care after USAID’s follow-on programming has ended. RECOMMENDATION #10: USE THALI FOLLOW-ON DATA TO ENSURE THAT PIPS ARE STRATEGICALLY TARGETED Evaluation data suggests that state-level PIPs are poorly targeted. When asked why the 2020 PIP did not seek to invest in THALI Phase II services, the STO replied “we never thought of that. That’s a good idea.” USAID and THALI follow-on IPs would be well headed to assist the state governments in strategically investing PIPs in evidence-based interventions that demonstrably improve indicators. THALI-based qualitative evidence should support initial investment in patient-centered care. As THALI follow-on IPs generate new evidence on what does and what does not work, this evidence can be used to encourage state governments to target their investment priorities more wisely and strategically. This strategic use of the PIP funding can serve as a welcome exit strategy for THALI follow-on programming. The National AIDS Control Organization (NACO) has extensive experience in the employment of local NGOs that could be easily adopted to serve the needs of TB control. RECOMMENDATION #11: ASSIST STATE GOVERNMENTS IN DESIGNING COMPETITIVE PROCUREMENTS While some PIP investments such as that in Jharkhand have been issued on a competitive bases, the allocation of PIP resources in West Bengal and Gujarat seem to have been handed to current THALI implementing partners on the basis of last year’s memorandums of understanding. This allocation of scarce TB control resources is not recommended as it can lead to inefficient and wasted opportunities. In line with Recommendation #10, USAID and IPs can assist the state RNTCP offices and State Technical Assistance Units to design more rigorous and strategic tenders for government-funded TB services. This will also include improvement to current PIP funding practices. As noted by government officials interviewed, the majority of PIP activities tendered to local NGOs have a reimbursement system for specific indicators. For one PIP, the local NGO is provided a minimal payment for every person that it 28 refers who is clinically diagnosed with TB. Local NGOs lack the funds to effectively advance the GoI money on a contract and then provide proof for reimbursement. USAID and its THALI follow-on IPs should assist the state governments in crafting more appropriate financial mechanisms to ensure maximum participation and output from public tenders for TB services. GOAL IV: CONTINUE TARGETING KEY AFFECTED, MARGINALIZED POPULATIONS RECOMMENDATION #12: CONTINUE TO SEEK OUT AND SERVE THE MOST VULNERABLE One of the key lessons learned from the THALI Final Performance Evaluation was that very few vulnerable and marginalized communities receive TB access and treatment services. Those marginalized population targeted under THALI Phase II demonstrated extreme gratitude at the attention they were given and were extremely impressed with THALI outreach worker services. The ET was hard pressed to identify any companion organizations that targeted these marginalized populations with specific TB awareness, diagnosis, and treatment options. While the number of infected maybe small within these marginalized groups, it is a number that will most likely go untreated without the targeted assistance that can be provided by THALI follow-on IPs. 29 ANNEX 1: SCOPE OF WORK PURPOSE OF EVALUATION USAID/India Mission will evaluate THALI’s ( KHPT and WHP activities) contributions towards India’s TB Control Program against program objectives set in the cooperative agreements and the subsequent revisions in work plans for both activities. The evaluation will help USAID/India better understand the nuances of its investment in the TB Control Program in terms of what worked, what did not work, what are the internal and external variables and assumptions that influenced those results. The evaluation will also look at the overall TB strategy of the Mission while focusing on the THALI project and provide recommendations for future direction. This external final performance evaluation of THALI comes after three and half years of the project implementation. The evaluation will provide findings, statistics, and recommendations that will assist THALI Project, its sub-partners, and USAID/India to confirm what has been accomplished; to help determine what components and project aspects worked well, which did not and why; and will help to make informed decisions for future designs. While evaluating the THALI project activities, the evaluation will also assess the broader TB strategy of the USAID/India Mission and suggest modifications in its approach, strategy and implementation for effective management and added value. In summary, the evaluation will help USAID better understand its investment in TB Control Program results and help focus and strengthen the TB Control Program in India. Evaluation Questions The evaluation will address the following questions. In particular, the evaluation shall address the sub￾questions while addressing the main questions. 1- What are the key results and outcomes of the USAID TB intervention? Did the program accomplish its objectives and achieve its targets? a. To what extent have the project activities improved TB prevention and care in project districts and states in terms of increased utilization of the Standards for TB Care in India that enables case notification and treatment outcomes? b. To what extent have the community-centered and community-driven interventions improved (a) health seeking behavior and service delivery to increase TB/DR-TB case finding, and (b) treatment outcomes among people diagnosed and initiated on TB/DR-TB treatment? c. To what extent did the project achieve the targets set for the project and improve the quality of services? d. What is the impact of PLGHA and how it has affected the objectives and expected results of the project in comparing the pre and post ALGHA developments? 2- To what extent have the activities increased the TB and MDR TB case finding and improved the treatment outcomes? a. Has there been an improved uptake of RNTCP services and adoption of standards for TB care by all health care providers? b. To what extent have the interventions increased the number of TB and MDR TB patients/families being counselled and are linked to social entitlements and other benefits from the government, and/or of other support, in cash or kind? c. Has there been a decrease in health system delays in diagnosis and treatment initiation among adult new smear positive TB patients accessing treatment under RNTCP? d. Have the activities mobilized resources for TB prevention and care from government and non￾government sources? 3- To what extent have the innovations demonstrated by THALI been scaled up and replicated? 30 a. Has there been an uptake of these innovations demonstrated by THALI by the public and/or private sector? b. To what extent have the research studies, evaluations, articles and poster presentations, published in peer-reviewed scientific publications, or in oral or made at state, national or international conferences been effective in improving the service outcomes? c. How effective has the collaboration/coordination among the programs been in maximizing efforts and achieving greater results? 4- To what extent have the projects been successful in addressing equity (including gender) in the delivery of TB, prevention, diagnostic, care and support services? 5- To what extent has the program activity strengthened capacity of the national and subnational (province and district) government, private sector, community, and other stakeholders? a. How effective has been the strategies to engage non-traditional partners (i.e., the private sector, commercial entities, associations, etc.) in investing in TB control? 6- How replicable, adaptable/adoptable, and sustainable are the programs/program components? a. To what extent have the models, innovations and the pilots developed by THALI been mainstreamed by the Government, both at national and state level, leading to improved TB treatment outcomes? What have been the barriers and assumptions inhibiting uptake? What variables have hastened up take? b. To what extent have the interventions in strategic communication, monitoring, evaluation, research and capacity building support to the Government service providers contributed to improved treatment outcomes? 7- What is the contribution of the THALI project to the overall USAID/India TB program goals? a. Does the THALI project align with the Mission’s overall TB strategy and efforts and, if not, what are the lessons to be learned and possibilities for improvement? b. What are the comparative similarities and differences between the approaches of both the partners that can inform future USAID programming? 8- Based on the findings from the above questions, the evaluation team should make recommendations regarding future direction of TB portfolio and programing. Audience and Intended Uses The audience of the evaluation report will be the USAID/India Mission, and specifically the Health Office and the Program Support office but also the support offices, USAID/Washington and Global Health; the Asia Bureau; and the Mission’s implementing partners, WHP and KHPT. USAID will use the report’s lessons learned to inform the future design of the TB program, and the design of new projects and amendments to existing projects and /or activities. EVALUATION DESIGN A) Evaluation Design The evaluation will be mixed method (qualitative and quantitative) in nature with secondary literature and quantitative data analysis. The evaluation team shall start its work with a literature review of all the documents cited below. Should the evaluation team deem it necessary to collect quantitative data using a sample survey, the evaluation team will need to include a section in the evaluation plan that clearly depicts how the survey will be conducted, the sample frame to be used, sample size, quality assurance, data analysis plan, etc. The evaluation team will review documentation provided by USAID and the THALI implementing partners, and any relevant research they collect. An instrument will be developed to codify and organize data from the document review for analysis according to the evaluation questions. The evaluation team will be 31 accountable for ensuring data analysis methods are in line with best practices. For both quantitative and qualitative data, the evaluation team will need to articulate methodologies for analyzing collected information, including any qualitative data analysis software programs to be used. For qualitative data specifically, the evaluation team will need to ensure that data are recorded and transcribed. It is expected that the evaluation team will present initial findings from the document review against the evaluation questions as part of the Team Planning Meeting at the beginning of the evaluation. i. Desk review of documents: The evaluation team will review USAID/India strategies on TB and its future direction project reports, and any relevant assessments. USAID/India will provide the team with all relevant country and program specific documents including proposals, Project Monitoring & Evaluation plan (PMEP), progress reports, monitoring indicators data and other relevant documents for conducting this desk review. The evaluation team is expected to collect and collate relevant documents, reports, and data, and all team members are expected to review these documents in preparation of the evaluation design and for the team planning meeting. Specific Tasks Specific tasks to be undertaken by the evaluation team in carrying out the evaluation include, but not limited to: ● Review of the program’s Contract documents. ● Review of all program reports and annual work plans. ● Review of baseline data, Project Monitoring & Evaluation Plan (PMEP), targets and performance reports as provided in the quarterly reports. ● Review of USAID/India’s Country Development Cooperation Strategy, the relevant Development Objective and TB’s role therein. ● An in brief and out brief with USAID/India’s Mission Director, Health, Program Office, and USAID’s Agreement Officer’s Representative (AOR) of the TB activities and other related USG representatives. ● Meetings and interviews with the THALI implementing partners ● Meetings and interviews with the GOI and State Government counterparts and partners. ● Meetings and interviews with associated institutions and other relevant stakeholders associated in the program in each region of the country. Field Visit and Interviews: The evaluation team will undertake field visits to THALI project sites, and also meet with USAID/India staff in New Delhi, to review activities and meet with USAID management and technical staff, implementing partners, Ministry of Health officials, donors, and other stakeholders, as appropriate. B) Data Analysis Methods Prior to the start of data collection, as part of the evaluation work plan, the evaluation team will develop and present, for USAID review and approval, all protocols for data collection; what procedures will be used to analyze qualitative data; and how the evaluation will weigh and integrate qualitative data from these sources with quantitative data from project performance monitoring records to reach conclusions about the effectiveness and efficiency of the TB Control Program implementation in India. 32 ANNEX II: EVALUATION METHODS This annex details the TMG approach and methodology for the THALI Final Performance Evaluation including sampling framework, data collection instrument development, data collection methods, and analytical framework. TMG chose a mixed-method data collection strategy that included descriptive quantitative analysis of source data supported by direct qualitative data collection. Quantitative and qualitative data findings were thematically coded and triangulated to reveal findings and explore recommendations. The THALI evaluation was conducted in three phases: (1) Desk Review and Evaluation Work Plan, (2) Field Phase, and (3) Analysis and Report Writing. Phase 1: Desk Review and Team Induction Phase 1 of the evaluation consisted of a desk review of key project documents provided by USAID/India and the THALI IPs, which included: THALI Cooperative Agreements; Monitoring Evaluation and Learning Plans (MELPs); Annual Workplans; THALI Quarterly and Annual Reports; Bi-Monthly Updates; research reports; technical briefs; case studies; behavior change communication (BCC) materials; and others. In total, over 200 documents, videos, presentations, and reports were reviewed. A full list of these documents is included in Annex III. The desk review formed the basis for the evaluation approach and lines of inquiry which were explored and clarified further during the ET induction with USAID and the IPs. On September 6, six days after contract execution, the ET led a Team Planning Meeting (TPM). The TPM provided USAID/India an opportunity to present the purpose, expectations, and agenda for the evaluation assignment. The ET used the TPM to refine the evaluation approach, to establish contact with the IPs through USAID/India introductions, and to establish a working protocol with the Task Order Contracting Officer’s Representative (TOCOR). Geography and Stakeholder Sampling Across both components, THALI covers a total of nine states and over 30 districts. The scope of the evaluation did not afford the ET the opportunity to visit all target districts. Over the induction period, the ET worked with USAID and the IPs to establish a reasonable geographic sampling framework, which allowed for investigation into standard THALI operational programming as well as into states and districts with critical demonstration projects and innovations. The final geographic scope for the evaluation included Kolkata and the nearby district of Hoogly, Ahmedabad and Surat in the state of Gujarat, Ranchi and Jamshedpur in the state of Jharkhand, Bengaluru and Belgaum in the state of Karnataka, and Hyderabad and Warangal in the state of Telangana. Further, the ET worked with USAID and the IPs to develop a key informant sampling matrix. The key informant matrix included a representative sample from all THALI stakeholders including government partners, clinicians, community engagement and mobilization agents, as well as beneficiaries. Prior to the field phase, the ET finalized a list of targeted key informants for the IPs to begin scheduling meetings. The IPs provided the ET logistical support, contacts with key informants, meeting scheduling, local transportation, and trouble shooting. The ET recognized that employing THALI IPs to establish KI and FGD contacts and meetings introduces a significant bias into the sample population. As suspected, almost all interviews provided positive responses to THALI activities. When pressed to provide contact information from KIs who had challenging interactions with THALI, THALI IPs attempted to expand the KI list. USAID and TMG discussed the challenges this bias posed during the TPM and acknowledged that this was an unavoidable flaw in design and scope given the parameters of the evaluation. 33 In total, the ET conducted interviews with 375 key informants (KIs) across the five states as well as conducted 5 focus group discussions (FGDs), with groups ranging from 4 to 10 respondents, with clients and THALI staff. A graphic representation of the key informant and focus group breakdown by state and by groups can be found in Table 2. TABLE 2. BREAKDOWN OF KEY INFORMANT INTERVIEWS/FOCUS GROUPS BY STATE AND GROUP Totals by State Clients THALI Staff Govt. Other FGD Total West Bengal 44 31 19 6 2 102 Gujarat 5 6 31 1 0 43 Jharkhand 10 3 10 2 0 25 Karnataka 22 28 31 20 1 102 Telangana 32 54 15 5 2 108 Totals 113 122 106 34 5 380 Data Collection Instruments During the induction period, the ET drafted data collection instruments for individual KIIs and FGDs. KII protocols were designed using semi-structured questionnaires. The semi-structured questionnaires solicited information from KIs pertaining to specific activities in which they engaged within the THALI framework. The “structured” nature of the questionnaires ensured that the interviews covered issues consistently across KIIs with similar THALI interaction profiles for comparative purposes. The “semi” nature of the questionnaires ensured the enumerator and/or interviewer had the opportunity to explore and probe unique or unanticipated topics which may inform evaluation findings. FGD protocols were tailored to issues relevant to the type of participants in each FGD. Each FDG protocol contained a list of topics common to FGD participants and upon which each participant could informatively comment or elaborate. For example, FGD protocols for client focus groups solicited viewpoints on the common causes of treatment avoidance, allowing them to debate and/or discuss their unique experience during the interview. Where possible, particularly among government officials and medical officers, English and Hindi protocols and questionnaires were used. For other KIs, such as outreach workers, self-help groups, and end-user clients, protocols were translated into each of the local languages including Bengali, Urdu, Kannada, and Telegu. A sample of KII questionnaires and FGD protocols for the four main tiers of key informants (government officials, private sector practitioners, field health functionaries, and client) can be found in Annex IV. Due to time constraints and the lack of THALI operations in Delhi, the ET was unable to pilot test these instruments prior to deployment. These protocols were piloted, refined, and redeployed for various intermediary KIs in the field over the course of data collection. Phase 2: Field Phase The field phase began immediately following USAID’s approval of the Project Plan. The ET Lead provided regular progress updates to USAID during the Field Phase through Weekly Updates provided on the Friday of each week. 34 Data collection in the field was conducted over four weeks from September 16, 2019 to October 1, 2019 and from October 9, 2019 to October 13, 2019. Field data was not collected from October 2 – 8, 2019 due to national and local holidays. The field visits involved two teams working simultaneously: (1) the ET (which includes the four key personnel (Team Lead, Evaluation Specialist, TB Specialist, and BCC Specialist), and (2) local Research Teams (RT)—hired local enumerators fluent in the local language and trained in research methods for Kolkata, Bengaluru, Hyderabad, and Warangal. The Evaluation Field Research Schedule can be found in Annex V. Phase 3: Analysis and Report Writing The ET organized the data by relevant emergent themes and sub-themes from the KII and FGD records and transcripts, when available. A comparison of interview reports during daily ET debriefs elicited deeper insights and findings, particularly through the cross-fertilization of ideas shared. A triangulation analysis was then conducted across the analysis from the KIIs and FGDs and secondary data gleaned during the desk review. Triangulation of all data and evidence determined the robustness, validity, and reliability of the findings. The ET’s evaluation framework analyzed data across three dimensions. The first dimension was the two distinct phases of the THALI program, pre-PLGHA and post-PLGHA. Though the Program Description remained the same, these two phases of the THALI program were found to be distinct in both objectives and activities. The second dimension included the 7 research questions and sub-questions posted in the RFTOP solicitation. For the third dimension, all data was analyzed on the ability of THALI follow-on potential to contribute to improvement of India’s RNTCP capacity to provide TB services to vulnerable groups. After field data collection was complete, the ET provided USAID with the required Debrief Presentation on October 15, 2019. The Debrief Presentation was attended by approximately 10 members of the USAID staff from the Contracting Office, Health Office, Program Support Office, and the Asia Bureau. 35 ANNEX III: SOURCES OF INFORMATION 1- List of KII and FGD Interviews West Bengal Evaluation Team KIIs 16th September to 18th September Evaluation Team Member - Chris Cavanaugh,Mr.Raghavan Srinivasan,Mr.Tushar Kanti Ray,Ms.Bhavani Giddu,Ms.Sohini Chaudhuri Sno. Name Designation Place Date Time 1 Dr. Milan K Dinda Project Director THALI Officer 16-Sep-19 09:00 - 13:00 2 Isita Ray Chakraborty Community Process Advisor THALI Officer 16-Sep-19 09:00 - 13:00 3 Dr. Abhijit Dey Senior Medical Consultant THALI Officer 16-Sep-19 09:00 - 13:00 4 Dr. Pallav Capacity building Advisor THALI Officer 16-Sep-19 09:00 - 13:00 5 Rangan Sengupta Supplie Chain Advisor THALI Officer 16-Sep-19 09:00 - 13:00 6 Anuradha Chandra Program Support Manager THALI Officer 16-Sep-19 09:00 - 13:00 7 Vivek Sharma M&E Advisor THALI Officer 16-Sep-19 09:00 - 13:00 8 Tanusree Dasgupta Community Liaison Officer THALI Officer 16-Sep-19 09:00 - 13:00 9 Abhishek Anand District Coordinator THALI Officer 16-Sep-19 09:00 - 13:00 10 Dibyashree Sengupta District Coordinator THALI Officer 16-Sep-19 09:00 - 13:00 11 Biplab Kundu Operation Lead THALI Officer 16-Sep-19 09:00 - 13:00 12 Satabdi Dey Operation Lead THALI Officer 16-Sep-19 09:00 - 13:00 13 Amit Bhattacharjee IT Support THALI Officer 16-Sep-19 09:00 - 13:00 14 Dr. Biplap Operation Lead - JEET WHP Officer 16-Sep-19 09:00-13:00 15 Dr. Bipra Bishnu WHO State Consultant THALI Officer 17-Sep-19 10:30 - 11:00 16 Dr. Brojo Kishor Saha State Tb Officer (STO) Health Department 17-Sep-19 11:30 - 12:00 17 Dr. Bijay Kar Ex. City TB Officer (CTO) Kolkata THALI Officer 17-Sep-19 13:00 - 14:00 18 Sri Atin Ghosh Deputy Mayor (KMC) Kolkata Municipal Corporation 17-Sep-19 15:00 - 16:00 19 Dr.T.K. Mukherjee Health Advisor Kolkata Municipal Corporation 17-Sep-19 15:00 - 16:00 20 Dr. Mollah Chief Municipal Health Officer (KMC) Kolkata Municipal Corporation 17-Sep-19 15:00 - 16:00 21 Dr. Chandra Shekhar Das City Tb Officer (CTO) Kolkata Kolkata Municipal Corporation 17-Sep-19 15:00 - 16:00 22 Dr. Samsuddin Ahamed Private Provider (Doctor) Kolkata Municipal Corporation 17-Sep-19 16:30 - 17:00 23 Dr. Prabir Hazra Block Medical Health Officer (BMOH) Jangipara (Hugli) 18-Sep-19 11:30 - 12:30 24 Mr. Abhishek Pramanik Senior Treatment Supervisor (STS) Jangipara (Hugli) 18-Sep-19 11:30 - 12:30 25 Dr. Ranjan Kumar Dey Block Medical Health Officer (BMOH) Tarkeshwar (Hugli) 18-Sep-19 13:30 - 14:30 26 Mr. Bidyut Kumar Das TB Home Visitor (TBHV) Tarkeshwar (Hugli) 18-Sep-19 13:30 - 14:30 27 Mr.C.R. Chakraborty Senior Treatment Supervisor (STS) Tarkeshwar (Hugli) 18-Sep-19 13:30 - 14:30 28 Smt.Soma Pal Public respresentative (Pradhan) Tarkeshwar (Hugli) 18-Sep-19 15:00 - 16:00 36 29 Mr. Tushar Kartidas Public respresentative (Vice Pradhan) Tarkeshwar (Hugli) 18-Sep-19 15:00 - 16:00 30 Dr. Pradip Kumar Das Private Provider (Doctor) Srirampore (Hugli) 19-Sep-19 11:30 - 12:30 31 Dr. Prakash Kumar Baag District TB Officer (DTO) Chinsura (Hugli) 19-Sep-19 14:00 - 15:30 32 Mr. Arnav Biswas District Public Private Mix Coordinator (PPM) Chinsura (Hugli) 19-Sep-19 14:00 - 15:30 33 Mr. Pradip Mondal District DRTB Coordinator Chinsura (Hugli) 19-Sep-19 14:00 - 15:30 34 Mr. Amalendu Manna District DRTB Coordinator Chinsura (Hugli) 19-Sep-19 14:00 - 15:30 35 Indrani Sarkar FO in Hugli Srirampore (Hugli) 19-Sep-19 11:30 - 12:30 36 Sanchita Dey FO in Hugli Chinsura (Hugli) 19-Sep-19 14:00 - 15:30 37 Punam Dubey FO in Hugli Tarkeshwar (Hugli) 18-Sep-19 13:30 - 14:30 38 Piu Naskar FO in Hugli Jangipara (Hugli) 18-Sep-19 11:30 - 12:30 Gujarat Evaluation Team KIIs 20th September to 21st September Evaluation Team Member - Chris Cavanaugh, Mr. Raghavan Srinivasan, Mr. Tushar Kanti Ray, Ms. Bhavani Giddu Sno. Name Designation Place Date Time 1 Dr Paresh V Dave Additional Director-Public Health and State TB Officer, Gujarat. Ahmedabad 20-Sep-19 11:30 - 13:00 2 Dr Pranav G Patel Director-STDC, Ahmedabad Ahmedabad 20-Sep-19 11:30 - 13:00 3 Dr Dixit Kapdiya District TB Officer Ahmedabad 20-Sep-19 11:30 - 13:00 4 Dr Pujara TB Officer, Government of Gujarat Ahmedabad 20-Sep-19 11:30 - 13:00 5 Dr Pankaj Nimavat SMO, STDC Ahmedabad 20-Sep-19 11:30 - 13:00 6 Dr Jignesh Patel MO, STDC Ahmedabad 20-Sep-19 11:30 - 13:00 7 Dr Purvi Nayak MO, STDC Ahmedabad 20-Sep-19 11:30 - 13:00 8 Dr Rajendra Acharya MO, STDC Ahmedabad 20-Sep-19 11:30 - 13:00 9 Dr Vijay Amin MO, STDC Ahmedabad 20-Sep-19 11:30 - 13:00 10 Mr Girish Parmar State Finance Manager Ahmedabad 20-Sep-19 11:30 - 13:00 11 Mr Mukesh Kumar State IEC Consultant Ahmedabad 20-Sep-19 11:30 - 13:00 12 Dr Hardik Solanki RNTCP WHO Consultant Ahmedabad 20-Sep-19 11:30 - 13:00 13 Dr Jignesh Patel MO (STDC) Surat 21-Sep-19 9:30 - 10:30 14 Dr. K N. Sheladia CTO (SMC) Surat 21-Sep-19 9:30 - 10:30 15 Dr. Dhaval Rathod DTO(Surat-R) Surat 21-Sep-19 9:30 - 10:30 16 Dr. Pravin Thummar MO Hirabaug UHC Surat 21-Sep-19 9:30 - 10:30 17 Dhaval Trivedi DPPMC Surat 21-Sep-19 9:30 - 10:30 18 Narendra Shirsath DPC Surat 21-Sep-19 11:00 - 13:00 19 Roshan Patel DPS Surat 21-Sep-19 11:00 - 13:00 20 Parash Raval STS Surat 21-Sep-19 11:00 - 13:00 21 Shailesh Solanki STS Surat 21-Sep-19 11:00 - 13:00 22 Vinay Chauhan STS Surat 21-Sep-19 11:00 - 13:00 23 Krunal Vyas STS Surat 21-Sep-19 11:00 - 13:00 24 Chirag Patel STS Surat 21-Sep-19 11:00 - 13:00 37 25 Dixit Gohil STS Surat 21-Sep-19 11:00 - 13:00 26 Dinesh Wagh STS Surat 21-Sep-19 11:00 - 13:00 27 Jintal Bhagvawala STS Surat 21-Sep-19 11:00 - 13:00 28 Mitesh Vanra TBHV Surat 21-Sep-19 11:00 - 13:00 29 Dharmesh Mehta TBHV Surat 21-Sep-19 11:00 - 13:00 30 Vinod Rana TBHV Surat 21-Sep-19 11:00 - 13:00 31 Prakash Dafda TBHV Surat 21-Sep-19 11:00 - 13:00 32 Dr Tirumala Rao Technical Consultant-IDAT Surat 21-Sep-19 11:00 - 13:00 33 Vivekanand Pandey Teal Leader-Gujarat Ahmedabad 20-Sep-19 11:30 - 13:00 34 Dharmistha Field Officer Surat 21-Sep-19 13:30 - 14:30 35 Mayur Pathar Field Officer Surat 21-Sep-19 13:30 - 14:30 Jharkhand Evaluation Team KIIs 23rd September to 24th September Evaluation Team Member - Mr. Tushar Kanti Ray, Dr. Sanjeev Kumar Sno. Name Designation Place Date Time 1 Md Fakre Alam TB-HIV Coordinator Jamshedput 23-Sep-19 12.30 -14.00 2 Mr Rajib Singh THALI state coordinator Ranchi 23-24-Sept-2019' Accompaning all two days 3 Mr Subash Kumar THALI M&E Associate Ranchi 23-24-Sept-2019' Accompaning all two days 4 Mr Gautam Ghosh THALI Field Officer Jamshedpur 23-24-Sept-2019' Accompaning all two days 5 Aslam Khan MDR patient Jamshedpur 24-Sep-19 10.00 -10.45 6 Dr Smita Kapata CSC MO cum MO-TU, Mango Jamshedpur 24-Sep-19 11.30 -11.40 7 Mr Nagendra Singh STS Jamshedpur 24-Sep-19 11.30 - 1145 8 Dr Rajesh Dayal STO, Jharkhand Ranchi 24-Sep-19 17.30 - 18.30 9 Dr A Mitra Director, STDC Ranchi 24-Sep-19 16.45 - 18.30 10 Dr Rajabhau Yoeley WHO-RNTCP Consultant Ranchi 24-Sep-19 16.45 - 18.30 11 Mr Pran Ranjan Mishra State IEC officer Ranchi 24-Sep-19 17.30 - 18.33 12 Dr Sankar Alert India Ranchi 24-Sep-19 17.30 - 18.34 Karnataka Evaluation Team KIIs 26th September to 28th September Evaluation Team Member - Mr. Chris Cavanaugh, Mr. Raghavan Srinivasan, Mr. Tushar Kanti Ray, Ms. Bhavani Giddu, Ms. Sohini Chaudhuri Sno. Name Designation Place Date Time 1 Dr Prakash KH Project Director THALI KHPT Office 26-Sep-19 10:00 - 15:00 2 Dr Reynold G Washington Sr. Technical Advisor KHPT Office 26-Sep-19 10:00 - 15:00 3 Prakash Javlakar M & E Specialist KHPT Office 26-Sep-19 10:00 - 15:00 4 Sunitha B J CB & Counselling Specialist KHPT Office 26-Sep-19 10:00 - 15:00 5 Sandeep H M & E Officer KHPT Office 26-Sep-19 10:00 - 15:00 6 Poornima B S Project Coor. Banglaore KHPT Office 26-Sep-19 10:00 - 15:00 7 M Suresh Project Coordinator, Bellary KHPT Office 26-Sep-19 10:00 - 15:00 8 Tejaswini Hiremath Project Coordinator, Bagalkot KHPT Office 26-Sep-19 10:00 - 15:00 38 9 Dr K Karthikeyan Tech Specialist KHPT Office 26 -Sep -19 10:00 - 15:00 10 Dr Prarthana BS TC KHPT Office 26 -Sep -19 10:00 - 15:00 11 Joseph Francis Munjattu DPD KHPT Office 26 -Sep -19 10:00 - 15:00 12 Arin Kar DD, M&E KHPT Office 26 -Sep -19 10:00 - 15:00 13 K V Balasubramanya IEC/Ado.Co KHPT Office 26 -Sep -19 10:00 - 15:00 14 Dr Troy KHPT Office 26 -Sep -19 10:00 - 15:00 15 Raghavendra T Project Coordinator KHPT Office 26 -Sep -19 10:00 - 15:00 16 Rajaram S MERL Specialist KHPT Office 26 -Sep -19 10:00 - 15:00 17 Mohan H L MT KHPT Office 26 -Sep -19 10:00 - 15:00 23 Dr. John Stephen Project Coordinator KHPT Office 26 -Sep -19 10:00 - 15:00 24 Dr. Balu State Medical College KHPT Office 26 -Sep -19 16:00 - 17:00 25 Dr. Troy Cunningham Project Director, JEET JEET Office 26 -Sep -19 16:00 - 17:00 26 R. Rajendra Operation Lead, JEET JEET Office 26 -Sep -19 16:00 - 17:00 27 S. Bhagya Hub Agent JEET Office 26 -Sep -19 16:00 - 17:00 28 Jyothi Patagar SCT Agent JEET Office 26 -Sep -19 16:00 - 17:00 29 M.L. Harish Treatment Coordinator JEET Office 26 -Sep -19 16:00 - 17:00 30 Dr. Seenappa Joint Director, STO STO Office 27 -Sep -19 10:30 - 11:30 31 Dr. Anil DDSTO STO Office 27 -Sep -19 10:30 - 11:30 32 Dr. Nirmala Senior Specialist, STO STO Office 27 -Sep -19 10:30 - 11:30 33 Dr. Manjula Additional Project Director KS AIDS Service Center 27 -Sep -19 12:15 - 13:00 34 Mr. Srisala Assistant Health Inspector KR Puram Hosp. TB Unit 27 -Sep -19 14:30 - 15:15 35 Ms. Akshatha CHW KR Puram Hosp. TB Unit 27 -Sep -19 14:30 - 15:15 36 Mr. Ashok Project Technician KR Puram Hosp. TB Unit 27 -Sep -19 14:30 - 15:15 37 Mr. Mallesh Cluster Coordinator, THALI KR Puram Hosp. TB Unit 27 -Sep -19 14:30 - 15:15 38 Dr. Kumar DTO District TB Center 27 -Sep -19 16:00 - 17:00 39 Dr. Prasanna Senior Health Inspector District TB Center 27 -Sep -19 16:00 - 17:00 40 Bibi Jan Senior Lab Technician District TB Center 27 -Sep -19 16:00 - 17:00 41 Arun Kumari District Program Coordinator District TB Center 27 -Sep -19 16:00 - 17:00 42 Vinay Kamakar STS RNTCP 30 -Sep -19 10:00 - 11:00 43 Vijayalakshmi Gowda TBHV RNTCP 30 -Sep -19 10:00 - 11:00 44 Kahaspnavar DPC RNTCP 30 -Sep -19 10:00 - 11:00 45 Ashok ART Counselor ART 30 -Sep -19 10:00 - 11:00 46 Raju Kadagi ART DEO ART 30 -Sep -19 10:00 - 11:00 47 Hanumanth Marate Outreach Workers Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 48 Kamith Kittur Outreach Workers Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 49 Sreedevu Peer Educator Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 50 Myalawwa Peer Educator Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 51 Renuka Peer Educator Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 52 Roopa Peer Educator Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 53 Tara Peer Educator Shakthi Mahila Sangha 30 -Sep -19 11:30 - 13:00 39 54 Vaibava Counselor Shakthi Mahila Sangha 30-Sep-19 11:30 - 13:00 55 Shalini Counselor Shakthi Mahila Sangha 30-Sep-19 11:30 - 13:00 56 Renuka Counselor Shakthi Mahila Sangha 30-Sep-19 11:30 - 13:00 57 Kalmesh Kabbur Program Coordinator Mahesh Foundation/KHPT 30-Sep-19 11:30 - 13:00 58 Rohini Shinde Outreach Workers Mahesh Foundation/KHPT 30-Sep-19 11:30 - 13:00 59 Roopa Program Coordinator KHPT 30-Sep-19 11:30 - 13:00 60 Ashok Tolin Supervisor KHPT 30-Sep-19 11:30 - 13:00 61 Siddappa Byakod Link Worker KHPT 30-Sep-19 11:30 - 13:00 62 Sunitha Link Worker KHPT 30-Sep-19 11:30 - 13:00 63 Basavaraja Project Office TSU 30-Sep-19 11:30 - 13:00 64 Zakhir Community Coordinator KHPT 30-Sep-19 11:30 - 13:00 65 Meenakshi Community Coordinator KHPT 30-Sep-19 11:30 - 13:00 66 Ganapathi Community Coordinator KHPT 30-Sep-19 11:30 - 13:00 67 Sharanappa Community Coordinator KHPT 30-Sep-19 11:30 - 13:00 68 Shivappa Community Coordinator KHPT 30-Sep-19 11:30 - 13:00 69 Malappa Shivanagol TB Patient Advocate Client 30-Sep-19 11:30 - 13:00 70 Mr. Bharat District Program Lead KHPT 30-Sep-19 Accompanied Telangana Evaluation Team KIIs 9th - 12th October Evaluation Team Member - Mr. Chris Cavanaugh, Mr. Raghavan Srinivasan, Mr. Tushar Kanti Ray, Ms. Bhavani Giddu, Dr. Satish Sno. Name Designation Place Date Time 1 Salma CHW TBAI office 10-Oct-19 14:00-17:00 2 K Uma Maheswari CHW TBAI office 10-Oct-19 3 K Prameela CHW TBAI office 10-Oct-19 4 B Tulasi CHW TBAI office 10-Oct-19 5 J Lakshmi CHW TBAI office 10-Oct-19 6 D Padma CHW TBAI office 10-Oct-19 7 Mubin begum CHW TBAI office 10-Oct-19 8 G Prem Kumar CHW TBAI office 10-Oct-19 9 P Nagamani CHW TBAI office 10-Oct-19 10 Renuka CHW TBAI office 10-Oct-19 11 G Ravinder ACC` TBAI office 10-Oct-19 12 Anil District Project Coordinator TBAI office 10-Oct-19 13 Dr Rajesham STO STO office 10-Oct-19 14:30-15:00 14 Mr Jitender IEC officer STO office 10-Oct-19 15:00-15:30 15 Mr Ramesh State Program Coordinator Accompanied 16 Mr Mahadev TBHV Bowenpally-DMC 11-Oct-19 10.30-11:00 17 Dr Prasidh MO-TC DTC-Hyderabad 11-Oct-19 11.30-12:10 18 Mr Ameer Khan Patient Karwan 11-Oct-19 13.00-13:30 19 Ms Samreen Khan Family member (Sister) Karwan 11-Oct-19 40 20 Ms Sakeena Begum Family member (Mother) Karwan 11 -Oct -19 21 Mr Jangaiah Patient Karwan 11 -Oct -19 22 Ms Satyamma Family member (Wife) Karwan 11 -Oct -19 13:45 -14:30 23 Mr Prem kumar CHW Karwan 11 -Oct -19 Accompanied 24 Mr Venkat care Coordinator Karwan 11 -Oct -19 Accompanied 25 Mr Laxmi Narayana care Coordinator Bowenpally 11 -Oct -19 Accompanied 26 Mr Ravinder ACC` Hyderabad 11 -Oct -19 Accompanied 27 Dr Madhavi MO -TC Addagutta -DMC 11 -Oct -19 10:30 -11:00 28 Ms Tulasi CHW Addagutta -DMC 11 -Oct -19 11:00 -11:10 29 Mr Sridhar STS Addagutta -DMC 11 -Oct -19 11:10 -11:30 30 Mr Giribabu Patient Addagutta -DMC 11 -Oct -19 31 Ms Laxmi MDR -TB Addagutta -DMC 11 -Oct -19 11:10 -11:30 32 Mr Venkaesh MDR -TB Addagutta -DMC 11 -Oct -19 33 Support group meeting See the next sheet 11 -Oct -19 11:30 -13:00 34 Mr ABN Chary STS Jangammet -TU 11 -Oct -19 35 Mr Jeevan Reddy TBHV Jangammet -TU 11 -Oct -19 14:30 -15:30 36 Mr Babu Miya Patient Bandlaguda 11 -Oct -19 16:00 -16:30 37 Ms Sabiya Family member (Wife) Bandlaguda 11 -Oct -19 38 Ms Ruksar Family member (Daughter) Bandlaguda 11 -Oct -19 39 Ms Zahera Fathima Patient (DRTB) Barkas 11 -Oct -19 16:45 -17:15 40 Ms Umera Family member (Mother) Barkas 11-Oct-19 41 Mohioddin Twins (6 years) Barkas 11 -Oct -19 42 Moinuddin Barkas 11 -Oct -19 43 Ms Harathi care Coordinator Addagutta -DMC 11 -Oct -19 Accompanied 44 Mr Anil Kumar care Coordinator Addagutta -DMC 11 -Oct -19 Accompanied 45 Ms Suguna CHW Addagutta -DMC 11 -Oct -19 Accompanied 46 Mr Anil District Project Coordinator Addagutta -DMC 11 -Oct -19 Accompanied 47 Mr Satya Prasad care Coordinator Jangammet -TU 11 -Oct -19 Accompanied 48 Ms Parveen CHW Accompanied Accompanied 49 Ms Veena CHW Accompanied Accompanied 50 Ms Anjum care Coordinator Accompanied Accompanied 51 Veena CHW TBAI office 12 -Oct -19 10.45 -12:45 52 Vijaya N CHW TBAI office 12 -Oct -19 53 Vijaya K CHW TBAI office 12 -Oct -19 54 Suguna CHW TBAI office 12 -Oct -19 55 Narmadha CHW TBAI office 12 -Oct -19 56 Janabai CHW TBAI office 12 -Oct -19 57 Padma Sree CHW TBAI office 12 -Oct -19 58 Reshma CHW TBAI office 12 -Oct -19 59 Ravinder ACC` TBAI office 12 -Oct -19 41 60 Ramesh State Program Coordinator TBAI office 12-Oct-19 61 Mr Vikas CEO TBAI office 12-Oct-19 14:20-15:10 62 Ramesh State Program Coordinator TBAI office 12-Oct-19 63 Mr Manohar KOL Osmania hospital 12-Oct-19 11:45-12:00 64 Ms. Shainaz Begum Patient NS Kunta 12-Oct-19 15:30-16:00 65 Ms Tarunam Begum Family member (Daughter in law) NS Kunta 12-Oct-19 66 Ms Hafsha sultana Family member (Mother) NS Kunta 12-Oct-19 67 Ms Mansuri Patient Fathima Nagar 12-Oct-19 16:30-17:00 68 Ms Latifa Family member (Daughter in law) Fathima Nagar 12-Oct-19 69 Ms Nagma Family member ( Fathima Nagar 12-Oct-19 70 Mr Venkat care Coordinator Accompanied 12-Oct-19 Accompanied 71 Ms Subadra CHW Accompanied 12-Oct-19 Accompanied 72 Mr Anil Kumar Care Coordinator Accompanied 12-Oct-19 Accompanied 73 Mr Prem kumar CHW Accompanied 12-Oct-19 Accompanied 74 Mr Satya Prasad Care Coordinator Accompanied 12-Oct-19 Accompanied 75 Mr Anil District Project Coordinator Accompanied 12-Oct-19 Accompanied 76 Ms Anjum care Coordinator Accompanied 12-Oct-19 Accompanied 77 Ms Parveen Care Coordinator Accompanied 12-Oct-19 Accompanied 42 2- Bibliography Bibliography - Background Research 6250311444TB India Report 2018.pdf Background State Oversight Guide to Supervision Monitoring and Evaluation of HIV TB Collaborative Activities.pdf Standards of TB care in India CTD WHO_book.pdf States Initiatives for Patient Support Systems for TB Elimination in India.pdf Supervision and Monitoring Strategy Document.pdf Guideline for PMDT in India 2017 Guideline for PMDT in India 2017 with messages.pdf India TB Report 2019.pdf India_CDCS.pdf national_action_plan_for_tuberculosis_20151204_final.pdf NSP Draft 030717 Final.pdf Reach-Cure-Prevent-2015-2019-TBStrategy.pdf Bending the curve WHO Annual Report 2017.pdf Final-Technical-Assessment-Program-Towards-Elimination-of-Tuberculosis-P167523.pdf Framework of reviewing TB programms.pdf Global Framework for TB research WHO.pdf Global US government TB strategy.pdf GP-ORF Eradicating TB in India.pdf Highlights of THALI's Knowledge Cafe - Partnerships for TB Prevention.pdf ICMR_India TB Research Consortium_2 Nov 2017_0 Press release.pdf India country TB profile WHO Global TB report 2018.JPG India_Case_Study TB Alliance.pdf International Health - John Snow, Inc_.pdf KHPT_AR_final.pdf KOLKATTA THALI Tackling TB among India’s Urban Populations – The Pump.pdf Lancet Shifting from control to elimination.pdf Landscape+of+Inclusive+Business+Models+in+Healthcare+in+India-Final-Pres+++.pdf Mandatory TB notification in Mysuru.pdf monitoring_and_evaluation_toolkit-_hiv, _tb_and_malaria_and_health_systems_strenghtening_ (2009). compressed.pdf national family health survey - India fact sheet.pdf NFHS 4 INDIA.pdf NHP 2017.pdf NHP SA 2017.pdf PM Modi speech on 13 March 2018 in Vigyan Bhawan in Delhi.pdf PPM_Roadmap.pdf PPMLandscapeAnalysis.pdf Pradhan Mantri Jan Arogya Yojana about.docx Project-JEET-PPT.pdf Report of the Joint Monitoring Mission 13_Feb_2017.pdf Riddhi Foundation THALI Kolkatta.pdf Role of medical colleges in TB.pdf STOP TB Private sector FG_10.pdf TB Notification in a private sector hospital in South India.pdf TB project eval Afganistan.pdf 43 TB REACH - EVALUATION REPORT.pdf tbevaluationhandbook_tagged.pdf THALI-brief_FINAL_PRINT.pdf THALI-RFP for Endline survey.pdf THALI-Updates-Jan-Feb-2019-1.pdf THALI-Updates-Mar-Apr-2019.pdf Tuberculosis Health Action Learning Initiative JSI Kolkatta.docx WHO end_tb_essential.pdf World Bank 4596327277Environmental and Social Systems Assessment (ESSA)_P167523.pdf Bibliography - KHPT Background Research 01_KHPT_THALI_Agreement and MERL Plan _01-A-Cooperative Agreement Signed Pages 1 and 2_KHPT.pdf _01-B-Cooperative Agreement_KHPT_01 12 2016.pdf KHPT THALI MERL Plan.pdf 01_KHPT_THALI_Event reports Highlights of ACF in Karnataka -2018.pdf Satellite Training for ASHAs on TB.pdf THALI Community-level campaigns brochure.pdf World TB Day 2018 - HYD.pdf 02_KHPT_THALI_Qquarterly Progress Report 01_THALI (KHPT) Quarterly Report - 2016-Q1.pdf 02_THALI (KHPT) Quarterly Report - 2016-Q2.pdf 03_THALI (KHPT) Quarterly Report - 2016-Q3.pdf 04_THALI (KHPT) Quarterly Report - 2016-Q4.pdf 05_THALI (KHPT) Quarterly Report - 2016-Q5.pdf 06_THALI (KHPT) Quarterly Report - 2016-Q6.pdf 07_THALI (KHPT) Quarterly Report - 2016-Q7.pdf 08_290118-THALI (KHPT) Quarterly Report -2017 -Q8- final.pdf 09_300418-THALI (KHPT) Quarterly Progress Report -2018 -Q9- final.pdf 10_30072018-THALI (KHPT) Quarterly Progress Report -2018 -Q10 (final).pdf 11_THALI (KHPT) Quarterly Progress Report -2018 -Q11.pdf 12_30012019-THALI (KHPT) Quarterly Progress Report -2018 -Q12 (final).pdf 13_30042019-THALI (KHPT) Quarterly Progress Report -2019 -Q13 (final).pdf 14_30072019-THALI (KHPT) Quarterly Progress Report -2019 -Q14-final Apr_june 2019.pdf 03_KHPT_THALI Bimonthly Updates THALI Bimonthly Update (Jan-Feb 2018).pdf THALI Updates- March-April 2018_KHPT.pdf THALI Updates- March-April 2018.pdf THALI Updates- May to June.pdf THALI Updates-Jan - Feb 2019.pdf THALI Updates-July - Aug 2018- final.pdf THALI Updates-July - Aug 2018.pdf THALI Updates-Mar - Apr 2019.pdf THALI Updates-May-June 2019.pdf THALI Updates-Nov - Dec 2018.pdf THALI Updates-Sept - Oct 2018.pdf THALI_ May- June 2018 update.pdf THALI_update_Jan-Feb 2018.pdf 44 04_KHPT_THALI_Materials submitted to CTD An Elected Representative’s Handbook.pdf Brochure for MPs.pdf TB English Poster Final.pdf TB Hindi Poster Final.pdf 04_List of officials and stakeholders Andhra_Telangana_TBAI THALI Project list of Stakeholders for Evaluation Aug 2019.xlsx Karnataka_List of stakeholders and govt. officers.docx 05_KHPT_THALI_M_E forms PCS Codes Kannada _ English.pdf PCS Codes Telugu.pdf RANA Kannada _ English.pdf Slum mapping Kannada _ English.pdf Slum mapping Telugu _ English.pdf 08_KHPT_THALI_Research Reports Bengaluru Delay Study report 2017.pdf Hyderabad Delay Study report 2017.pdf TB_Blore Knowledge and Health Seeking Report (FINAL).pdf TB_Blore Knowledge Study Report (FINAL).pdf TB_HYD Knowledge and Health Seeking Report (FINAL).pdf TB_HYD_Knowledge study report.pdf 09_KHPT_THALI_Technical Briefs DCM SOP with RANA.pdf Specimen Collection and Transportation System.pdf TB Careline technical brief_2019.pdf 10_KHPT_THALI_Case studies Case Study - Patient-Support-Groups THALI (1).pdf Case Study - Positive-Stories-at-Patient-Support-Groups_THALI_TBAI_KHPT.pdf 11_KHPT_THALI_Technical resources Counselling in TB- module (November 2018).pdf PMDT Ready Reckoner (Draft-July 2018).pdf RNTCP DRUG DOSAGE CHART.pdf RNTCP-TOG- Abridged (2016).pdf Abbigere Profile.pdf BCC English Bullet - Piano Folder.pdf Disclosure.pdf Ishte 17-01-2019.pdf Nutrition flashcards.pdf TB Vruksha.pdf The Right Step (flashcards).pdf Vada-Idly TB poster set.pdf Weigh the consequences.pdf Final draft USAID evaluation TS and AP slides Oct-with comments .pptx IEC English Basics of TB Handout English.pdf Community Handout English (for Karnataka).pdf Leaders to End TB- Seven posters (2018).pdf Patient Information Brochure English.pdf Job aids 45 CHW Diary English 2019.pdf Counselling DCM Card English.pdf Counselling Skills English Final.pdf TSG 2nd August 2019.pptx KPHT Method Docs 18-01-08 - KHPT THALI Work Plan Narrative - Year 3 - Submitted V4.docx DCM SOP with RANA.PDF Specimen Collection and Transportation System.pdf TB Careline technical brief_2019.pdf Meeting with KHPT team.docx PDF USAID evaluation Sep 2019 _ 26th September_Karnataka.pdf RNTCP Profile of TS.pptx TB Notification 2017-19.xlsx Telengana case notification data 2017 and 2018.xlsx THALI Films Press Clippings Project Documentation Radio and audio spots Technical and Communication Resources THALI MIS Presentation.pptx USAID evaluation Sep 2019 _ final draft - 6.30 pm.pptx Bibliography - WHP Background Research World TB Day Report_2019.pdf Fast tracking _data_summary 2016 1st QPR 2016.docx 1st QPR 2016.pdf 2nd QPR 2016.pdf 3rd QPR 2016.pdf 4th QPR 2016.pdf 2017 QPR_Apr-June_2017.pdf QPR_Jan-Mar 2017.pdf QPR_July-Sept 2017.pdf QPR_Oct-Dec 2017.pdf 2018 QPR_ Jul -Sep_ 2018.pdf QPR_ Oct -Dec_ 2018.pdf QPR_Apr-Jun-2018.pdf QPR_Jan-Mar_2018.pdf 2019 QPR_ Jan -Mar_ 2019.pdf QPR_Apr- June_2019.pdf Annual Work Plan.pdf Final_Updated_Logframe_2015-2020 with Formula_09-SEP-2019.xlsx Gujarat Presentations IDAT Presentation_21st Sep'19.pptx 46 PPSA Project data till Aug-19.pptx THALI Evaluation Team Visit Participants detail.xlsx THALI Gujarat data till Aug 2019.pptx Jharkand Interview Schedule Sept 23 - 24.xlsx Kolkata case notification.xlsx Linkages between STC_THALI RNTCP.pptx List of concerned Govt. officials and other stakeholders.pdf M&E Plan.pdf Milan sir-2.pptx Organogram and TOR_THALI.pdf Presentation to THALI Evaluation team-Sep 9th 2019.pdf State report July18-June 19.xlsx THALI Agreement.pdf THALI Brochure.pdf West Bengal Interviews Sept 16 - 19.xlsx 47 ANNEX IV: DATA COLLECTION INSTRUMENTS The section below provides KII interview and FGD protocols across the four main category of respondents: government/RNTCP officials, private practitioners, field functionaries, and clients. These protocols were tested and refined in the field. They served as the basis for additional protocols when warranted. The use of a common template for each of the four main category of respondent ensured consistency of topic coverage across multiple stakeholders who have a shared experience of interaction with the THALI program. THALI Evaluation Protocol Template (RNTCP Officials, STO, DTO, MO) Name of KI: _________________________________________ Position: _________________________________________ State/District: _________________________________________ Date/Time: _________________________________________ Location: _________________________________________ Interviewer(s): _________________________________________ Introduction and Purpose Small talk. Introduction of interviewers. Purpose of visit/assessment. Mention confidentiality and anonymity. Mention expectations from interview. Ask if interviewee has any questions. Taping and Informed Consent (Only for beneficiaries, families, and community members) Have a text here about informed consent. Taping of interview. Formal Interview Block 1: Overall Knowledge of THALI (5 questions max.) For officials: Ask about what they do in TB control, length of interaction with THALI, who they interact with at THALI, etc. a. What do you know about THALI project? Explain/ discuss b. What are the activities carried out under this project? c. Have you participated in ant THALI activities in your area? Please describe. d. Please describe in which function of RNTCP, THALI complement of supplement. e. What support STC get under THALI project Block 2: Sensitization/Mobilization and expectation from THALI intervention For officials: The RNTCP has protocols and programs for mobilization/detection/sensitization. What does the THALI program do to help you reach the vulnerable populations? Etc. a. Please describe ant community mobilization activities carried under RNTCP budget and its objective? b. What are the community mobilization activities carried out under THALI project and how it supplements the routine RNTCP activities? 48 c. Have the community-centered and community-driven interventions improved (a) health seeking behavior and service delivery to increase TB/DR-TB case finding, and (b) treatment outcomes among people diagnosed and initiated on TB/DR-TB treatment? d. How Community mobilization activities address delay in reporting RNTCP facilities by a presumptive TB cases from the onset of TB symptoms. e. What activities are carried out to reach vulnerable populations such as PLHIV, CSW, IDU, LGBT etc.? In what extent it has been achieved? f. What are your expectations of THALI? What are the areas where you think THALI has contributed to RNTCP? i. Case finding and Notification ii. Treatment iii. Adherence iv. Community engagement () v. Capacity building/training vi. Innovations – DCM, vii. Technology - care line, IDAT viii. IEC/BCC Block 3: Diagnostic/Notification a. How THALI project support RNTCP in diagnosis of DS and DR TB cases? b. For officials: How does your current diagnostic system work? What does THALI do to help you diagnose and notify? How do you follow up after a diagnosis is made? How does THALI help you follow up? c. Specimen transportation system and role of THALI Block 4: Treatment and Adherence a. In what extent THALI activities helped in reduction of treatment delay and improving treatment adherence of TB and MDR TB patient. b. In your opinion, to what extent have THALI activities assisted government in improving the TB and MDR TB treatment outcomes? Block 5: a. How THALI is contributing towards addressing equity (including gender) in the delivery of TB, prevention, diagnostic, care and support services? b. THALI program activities in strengthening capacity of the national, state, district and block level among government, community, and other stakeholders Block 6: Sustainability a. Which THALI practices have you or do you want to make part of your TB control program? b. Contribution of THALI program activities in strengthening capacity of the national, state, district and block level among government, community, and other stakeholders c. Your view on how replicable, adaptable/adoptable, and sustainable the programs/program components are d. To what extent have the models, innovations and the pilots developed by THALI been mainstreamed by the Government, both at national and state level, leading to improved TB 49 treatment outcomes? What have been the barriers and assumptions inhibiting uptake? What variables have hastened up take? e. Please indicate two important achievements/positive points of THALI project. f. Pl give two suggestions for improvement. Block 7: Missed Opportunities Officials: What are your current challenges in TB control? How can innovative programs help? THALI Evaluation Protocol Template (Private Practitioner) Name of KI: _________________________________________ Position: ______________-__________________________ State/District: _________________________________________ Date/Time: _________________________________________ Location: _________________________________________ Interviewer(s): _________________________________________ Introduction and Purpose Small talk. Introduction of interviewers. Purpose of visit/assessment. Mention confidentiality and anonymity. Mention expectations from interview. Ask if interviewee has any questions. Taping and Informed Consent (Only for beneficiaries, families, and community members) Have a text here about informed consent. Taping of interview. Formal Interview Block 1: Overall Knowledge of THALI (5 questions max.) For officials: Ask about what they do in TB control, length of interaction with THALI, who they interact with at THALI, etc. a. How long you are treating TB patient and how long you are participating in RNTCP? ` b. Why are you participating in RNTCP, and how it is different? c. Are you trained on RNTCP and for how many times and by whom? d. Do you have any exposer on Standard TB care in India handbook? Block 2: Sensitization/Mobilization (if appropriate) (5 questions max.) For officials: The RNTCP has protocols and programs for mobilization/detection/sensitization. What does the THALI program do to help you reach the vulnerable populations? Etc. Block 3: Diagnostic/Notification For officials: How does your current diagnostic system work? What does THALI do to help you diagnose and notify? How do you follow up after a diagnosis is made? How does THALI help you follow up? 50 a. What type of chest symptomatic patient you see in your clinic? b. At what stage of the disease the TB patient report you in your clinic? c. Anyone refer the patient, or the patient come in its own to your clinic? d. What is the main reason for patient come to you instead of going to other places such as govt hospital? e. What protocol you follow to diagnosis TB? Describe it. f. Where you refer them for GeneXpert, Sputum examination and Chest X-ray? g. What is your preferred diagnosis method? X-ray, Sputum AFB, GeneXpert Block 4: Start of Treatment a. After the confirmation of diagnosis what do you advice your patient? To get medicine from Govt hospital or buy it from private pharmacy? b. Is DOTS regimen adequately address the pulmonary and extrapulmonary cases? If not, what is your advice? c. Do you manage a DOTS clinic? d. Do you keep RNTCP supplied drug? Block 5: Adherence a. Who supervise the treatment of your TB patient? b. What you do if your patient having any side effect or drug reaction during the treatment? c. Who educate your patient at the start and during the treatment? d. Do you face any problem on irregularity or discontinuation of treatment by your patient? How you address this problem. e. Who maintain the treatment card and TB register? Block 6: Sustainability a. Please indicate two important achievements/positive points of THALI/ JEET project. b. Please give two suggestions for improvement. Block 7: Missed Opportunities a. What are your current challenges in TB control? How can innovative programs help? b. What are your expectations of THALI? What are the areas where you think THALI has contributed to RNTCP/ TB control? 51 Focus Group Template for Field Functionaries (CHWs, ORWs, SHGs, etc.) Name of FDG: _________________________________________ Number of Participants: _________________________________________ State/District: _________________________________________ Date/Time: _________________________________________ Location: _________________________________________ Moderator: _________________________________________ Confidential List of Participants to Tracked Separately FGD Guide Field Functionaries Below is a general guide for conducting our focus groups. We may modify this guide as needed as each focus group will inform the subsequent groups. Before the group begins, conduct the informed consent process. I. Introduction • Welcome participants and introduce yourself. • Explain the general purpose of the discussion and why the participants were chosen. • Discuss the purpose and process of focus groups • Explain the presence and purpose of recording equipment etc. • Outline general ground rules and discussion guidelines such as the importance of everyone speaking up, talking one at a time, and being prepared for the moderator to interrupt to assure that all the topics can be covered. • Address the issue of confidentiality. • Inform the group that information discussed is going to be analyzed as a whole and that participants' names will not be used in any analysis of the discussion. • This activity can take 45 min to 60 min. We would like the discussion to be informal, so there’s no need to wait for us to call on you to respond. In fact, we encourage you to respond directly to the comments other people make. If you don’t understand a question, please let us know. We are here to ask questions, listen, and make sure everyone has a chance to share. If we seem to be stuck, we may interrupt you and if you aren’t saying much, we may call on you directly. If we do this, please don’t feel bad about it; it’s just our way of making sure we obtain everyone’s perspective and opinion is included. We do ask that we all keep each other’s identities, participation and remarks private. We hope you’ll feel free to speak openly and honestly. Helping are my team members ______ and _______. They will be taking notes and be here to assist me if I need any help. Let’s begin. Let’s find out some more about each other by going around the room one at a time. Tell us your name and the job you have in the project and the communities you serve. II. Guide Questions/Probes/Cues/Triggers Q. What is most common obstacle /challenge/barrier in your job of working with communities and individuals in the (THALI) TB project? There could be common as well as some very different specific challenges? It could be related generating awareness, convincing people, interacting with different segment of audience etc. anything in particular addressing men or women, adolescents, elders? Other vulnerable populations such as differently abled, CSW, IDU, MSM, LGBTQ etc.? Q. How did you learn to overcome these challenges, obstacles, barriers? What support from the project staff, each other, and other functionaries helped you in different aspects such as Sensitization, mobilization, 52 early detection and testing, notification, treatment initiation and continuation, adherence, sustainability of people’s action and behaviors? How would you say project training and interaction processes helped you to carry out your job better? Did you tweak, design or come up with innovative responses and solutions for such challenges? Q. In your opinion how has the shift to people centric approach and support been useful? Has the different kind of support such as nutrition, direct benefit transfer, on site sputum collection, provided to people and patients been able to improve treatment initiation and outcomes? Q. What is the one most significant change have you seen during your time in the project in the community? And what is the one significant challenge that you see needs to be better understood and addressed? III. Closing remarks, Thank the participants 53 THALI Evaluation Protocol Template (Clients) Name of KI: _________________________________________ Position: _________________________________________ State/District: _________________________________________ Date/Time: _________________________________________ Location: _________________________________________ Interviewer(s): _________________________________________ Introduction and Purpose Small talk. Introduction of interviewers. Purpose of visit/assessment. Mention confidentiality and anonymity. Mention expectations from interview. Ask if interviewee has any questions. Taping and Informed Consent (Only for beneficiaries, families, and community members) Have a text here about informed consent. Taping of interview. Formal Interview Block 1: Overall Knowledge of THALI (5 questions max.) How did you know about TB and the symptoms, testing and treatment for TB? How many times did a health worker interact with you? (Prompts: When / Where / How) Did the health worker address all your questions on TB? (Prompts: What did they not address? What could have been addressed better?) Did the health worker enable you to seek benefits of Government schemes? (Prompts: Such as DBT) Block 2: Sensitization/Mobilization (if appropriate) (5 questions max.) Where did you get information on TB? (Prompts: Experience of first interaction with health worker) What is that one communication from health worker that helped you seek care/treatment? (Prompts: IEC / IPC / Emotional support / Linkages to Government schemes such as DBT, etc.) How did the health worker help you overcome the stigma, fears, concerns related to TB? (Specific to high risk groups) – Was information provided as per your needs? What could be improved for better sensitization of high-risk groups? Block 3: Diagnostic/Notification Where did you go when you thought you might have TB? 54 How did you find out about that place? Who did you turn to for information? What was the diagnosis experience like? What happened when they notified you that you had TB? What were you feeling at that time? How did you overcome the fears, apprehensions over TB? Block 4: Start of Treatment What was your initial experience of treatment? Was sufficient information, guidance and care provided to you during treatment? How many interactions did you have with the health worker during your treatment? What were some challenges you faced during treatment phase? How did the health worker help you overcome these challenges? Did you face any issues (stigma) during treatment? What was your overall experience during treatment phase? What can be done better to improve it? As a woman / LGBT / CSW, did you face any discrimination during any of this? How did the program help you overcome this? How did you overcome it? What can be done better to address this? Block 5: Adherence What motivated you to adhere to treatment and complete the course? What factors make it difficult to adhere to treatment? How did the health worker help you in adherence to treatment? What was your overall experience during the treatment, up to completion? How do you feel now that your treatment is completed? What could be done better to ensure all TB patients adhere to treatment? As a woman / LGBT / CSW, did you face e any discrimination during any of this? How did the program help you overcome this? How did you overcome it? What can be done better to address this? Block 6: Sustainability What are your next steps in TB control and treatment? 55 Now that you have been cured, how are you engaging with others in your community to sensitize them and inform them about TB prevention, testing and cure? According to you, how can issues like stigma about TB be overcome? Who in your community can play a major role to address this? Block 7: Missed Opportunities What was the hardest thing for you in dealing with your TB? What could have been done better? (Prompts: IEC, sensitization, awareness on cure, treatment & prevention, access, benefits, etc.) 56 ANNEX V: EVALUATION SCHEDULE Trip Schedule (September 15-October 1, 2019, October 9 – October 13, 2019) Date Day State District Covered Night Stay Objective 15-Sep-19 Sunday Travel from New Delhi to Kolkata Kolkata Travelling (evening flight) 16-Sep-19 Monday West Bengal Kolkata Kolkata Meeting with project team and training of researchers 17-Sep-19 Tuesday West Bengal Kolkata Kolkata Meeting with STO, CTO, KMC members and private providers (JEET) 18-Sep-19 Wednesday West Bengal Hooghly Kolkata Meeting with BMOH and PRIs 19-Sep-19 Thursday West Bengal Hooghly Kolkata Meeting with DTO, MOTC and private medical providers 20-Sep-19 Friday Travel from Kolkata to Ahmedabad (Go Air G8-701) - flight at 06:00 Surat Travelling (morning flight) 20-Sep-19 Friday Gujarat Ahmedabad Surat Meeting with STO to discuss on awarding WHP on PPSA project (6 MCs) 21-Sep-19 Saturday Gujarat Surat Surat Meeting with CTO and patients visit - MH and IDAT 21-Sep-19 Saturday Travel from Surat to Delhi (IndiGo 6E-5029) flight is at 21.30 Delhi Travelling (evening flight) 22-Sep-19 Sunday 23-Sep-19 Monday Travel from Delhi to Ranchi (IndiGo 6E-421) flight is at 07.20 Jamshedpur Travelling (morning flight) 23-Sep-19 Monday Jharkhand Ranchi Jamshedpur Meeting with STO, WHO Consultant and TB patients 24-Sep-19 Tuesday Jharkhand Jamshedpur Jamshedpur Meeting with DTO, MLA and TB patients (Tribal Population) - DR TB and facilitating various govt schemes 25-Sep-19 Wednesday Travel to Delhi/Bengaluru 26-Sept-19 Thursday Bengaluru Bangalore Bangalore Meeting KHPT, JEET, Training for Research Team 57 27-Sept-19 Friday Bengaluru Bangalore Bangalore Field Research, KIIs, FDGs, PSG Meeting, Patients, Families, Community Structures 28-Sept-19 Saturday Bengaluru Bangalore Bangalore Field Research, Training Observation, CHW and CC Interviews 29-Sept-19 Sunday Bengaluru Bangalore Bangalore Travel to Belgaum 30-Sept-19 Monday Bengaluru Belgaum Gokak KIIs -LSW, TI, Government Hospital, TI Office, DAPCU/DPS/ART Counselors 01-Oct-19 Tuesday Travel to Delhi 09-Oct-19 Wednesday Travel to Hyderabad 10-Oct-19 Thursday Hyderabad Hyderabad Hyderabad Meet with TBAI Team, KII with State Officers, KII with EIC Officers, Field Research Training 11-Oct-19 Friday Hyderabad Warangal Hyderabad KII-DTO Warangal, Community Structure, UCHC, DCM, Patients, MO TC, FDG Patient Support Group, FGD – CHW, KII Clients 12-Oct-19 Saturday Hyderabad Hyderabad/ Secunderabad Hyderabad KII – STS, TBHV, FDG w/ CHW, KII – Clients 13-Oct-19 Sunday Travel to Delhi 58 ANNEX VI: SUCCESS STORIES Success Story - Begum Mariam Begum’s articulation of what she went through as a TB patient belies her age. She is hardly 17 years old and comes from a depressed Hyderabad slum. This is a predominantly Muslim area, where her family had settled several centuries back. Her community is steeped in poverty and traditional practices. Mariam Begum tested TB positive a few months back. Her school final was also due leaving her to cope with both of these challenges in the face of the already difficult circumstances that her family was living under. When she had persistent cough for a few days, she thought it was just a passing fever and cold. But when she noticed blood when coughing, she decided to take action. Her semi-literate parents could only support her if she took the initiative to seek medical care. She came to know of the THALI community health worker, Subhadra, through one of her neighbours. Subhadra promptly took her to the Osmania Government Hospital for diagnosis and when she was confirmed to be positive as suspected, Mariam was initiated on the 6-month TB treatment regimen. Being undernourished and frail at the start of treatment, Mariam found the treatment very difficult to adhere to as she could hardly walk and had a depressed appetite. But her persistence and commitment soon showed results. She was so firm in her commitment that once when she vomited the tablets, she took them out and swallowed them again so that she did not miss even a single dose. This small and frail girl, wearing a burkha, is now a TB Champion. She regularly attends patient support group meetings at the hospital and encourages patients and their family members not to give up. The audience is all ears when she gets up and talks about her own experience and particularly that of her community. Many women in the Muslim community wear burkhas, stay indoors most of the time, and follow rigorous customs of fasting. They are more prone to anaemia and malnutrition and become potential victims for communicable diseases. TB prevalence is higher among women in this area than among men, which speaks for their vulnerability. So when a girl from this community stands up in a cramped room in the TB Unit, crowded with apprehensive patients and relatives, and cautions them to take care of their health, they lend their ears willingly. When Mariam speaks confidently and evocatively at the patient support meetings, her understanding of the problems of the women in her community is evident. The presence of the Senior Treatment Supervisors and prominent members of the community does not intimidate her. She advises 59 other Muslim women in the community to take extra precautions during fasting days and advises them not to hesitate to go for a check-up if they have cough for more than 2 weeks. She emphasises that “TB is definitely treatable and is nothing to feel guilty about.” She makes her earnest request to community members that families and neighbours accept TB as a normal disease, and not treat TB patients as untouchable and stigmatized people. She relates how she herself had initial doubts about her self-esteem among the community and the stigma that she would have to face among her fellow students. But she is now a cured patient and ready to move on in life. 60 Success Story – Babumiyam Mr. Babumiyam is a 50-year-old truck driver from Banglaguda, Hyderabad. He is the sole supporter of his five-person family living in the Muslim slums of outside Noori Palace. For years he had battled diabetes, which often crippled him to the point where he could no longer work. In February 2019, Mr. Babumiyam’s situation went from bad to worse when developed a cough and strong fever. As a first line of treatment, he sought help from a local private sector doctor. After two months of treatment, his condition had only deteriorated further as he lost 12 pounds and was unable to attend to the most daily of household functions. Through a friend and THALI Champion, Mr. Babumiyam was referred to the local TB Unit in Barkas where he underwent formal diagnosis through acid fast bacilli (AFB) microscopy and CBNAAT test to confirm drug sensitive TB. He initially suffered a host of side effects from the antitubercular treatment regimen that led to long-term unemployment compounded with several isolation and depression. Hearing about his case from a neighbour, the THALI community health worker and local cluster coordinator rushed to his aid. They held his hand in seeking appropriate treatment for the side effects, which immediately began to change his mood and outlook as well as improve the strained relations with his family members. Without the aid of THALI staff, Mr. Babumiyam would never have known that his side effects were treatable and doubts he would have remained on his six-month treatment regime through to his successful treatment outcome. Since August, 2019, Mr. Babumayim has been considered cured. He credits the cure not only to the medical attention he received, but to the constant attention and dedication of Mrs. Anjum, his THALI health worker. “She has become like a daughter to me. She has given me my life back,” Mr. Babumiyam beams. Mrs. Anjum, a devout Muslim woman and 20 years Mr. Babumayim’s junior, has become like a family member, often stopping by for afternoon tea as she completes her rounds in the community. The bond between the two is most likely life-long. As for Mr. Babumayim, he has now become a community TB Champion and advocate. He preaches the virtues of testing and treatment to the community at large. To celebrate his recovery, he has purchased two goats to host a community-wide feast to thank everyone for their part. Mrs. Anjum will undoubtedly be the guest of honor, Mr. Babumayim assures. Having put back on the 12 pounds he lost during his private sector treatment, Mr. Babumayim has returned to work, is providing for his family, and looks forward to helping fellow TB victims in his community. 61 Case Study – Taheb Mohammed Taheb, a young man of 24, belongs to a marginalized community in the Tangra block of Kolkata, West Bengal. He lives with his mother and two sisters. He is the only support system for providing food to his family. He was diagnosed with multi-drug resistant TB in March, 2019. His father and elder brother were also victims of TB. Unfortunately, they didn’t survive. Eight years ago, Mohammed, was diagnosed with TB for the first time, but it was not very serious and he survived. But again in March 2019, he noticed that he was coughing blood, sneezing, and had a recurrent fever. He immediately recognized the symptoms and rushed to the Tangra health center, which is one of the THALI-identified health centers in West Bengal. Ms. Shampa, a THALI health worker, came to know about Mohammed through the patient lists provided by the health center. She visited him and his family. She supported Mohammed through this illness for 6 months, which included 4 months of injection and two months of oral medication. She also ensured that his mother and sisters got their sputum tests. She supported the family financially by ensuring that they received a monthly food basket which contained rice, lentil, salt, sugar, soybean packets, biscuits, and a Nutrimix supplement (dry fruits), an innovative nutritional intervention provided by THALI local NGO partners. Mohammed shared that his most difficult times were when the side effects of medication started, including vomiting, dizziness, palpitation, itching, shivering, and chest pains. But he never thought of abandoning his course of treatment because of the constant visits, support and motivation provided by the THALI health workers. He was not subjected to any stigma and social discrimination, thanks to the health workers who constantly conducted awareness program in the community. THALI provided informational materials, such as a flip book on TB symptoms and treatment regimens and a food picture card, helped Mohammed and his family to learn about the types and causes of TB and how can it be prevented and cured. Mohammed is now completely recovered, he is healthy and has gained 15 pounds since his recovery. The voices of TB survivors like Mohammed provides important perspectives and are powerful tools for advocacy to accelerate the TB response and reach all affected people with prevention and care. “Shampa mam visits me every week and she has kept me motivated to continue my course of medication. I am very grateful to her.” - Md. Taheb, MDR TB survivor, Tangra, Kolkata,WB 62 Success Story: Tahir Mohammed Tahir lives with his parents and younger sister in a congested slum in Kolkata on GJ Khan Road. He was diagnosed with drug-sensitive TB about 6 months back and has been on medication ever since. While the initial months of medication gave him severe side effects, he is recovering well now and is waiting to bounce back and get back to work to support the family. Tahir's sister, Bilkis Jahan takes care of him. She cooks for him, she helps him with the medication and she gives him nutritious food to eat. He particularly likes the nutrition powder given by THALI, Bilkis says about her brother. “He mixes the Nutrimix with aata (flour) and we make rotis with it,” says Bilkis. The THALI field officer, "Champa didi “(didi = sister in Hindi) is a favourite in the household. She visits very often to check on Tahir and brings him nutrition baskets and the nutrition powder. There are times when she brings the medicine replenishment also, says Tahir. "Didi told me everything about TB, she even explained it to me in a very easy-to-understand way when I was diagnosed, using this booklet which is full of illustrations on how to treat and cure TB. That gave me the confidence that my condition is curable," says Tahir. Tahir likes this booklet of illustrations so much that he even tells his friends about TB using that book. “He explains every page of the book to us and showed us the different phases he went through in his TB diagnosis.” Tahir says there has been no stigma in his neighborhood or his family about TB. They are all now aware that it is curable “thanks to Champa didi,” says Tahir. 63 Success Story: Jhalpa Jhalpa is a young, vivacious, energetic, talkative, and lively young girl from Katar Gaon, Chikuwadi in Surat. She is from a lower middle-class family. Her father is a rickshaw puller and her mother is a home-maker. She lives in a modest one-bedroom home in Surat. Jhalpa was first diagnosed with TB when she was 18. This is the third time the TB has recurred and this time it is drug-resistant. During her previous bouts with TB, she had sought treatment in the private sector and it was cured after delayed diagnosis and a late start of treatment, but there was no treatment or after care provided by her private sector physicians. In 2018, when Jhalpa found herself once again having TB symptoms, she was very concerned about her health, education, career, and life. She took advise from some well-wishers in her colony who referred her to a THALI field officer - Mayur Pathar. Jhalpa was advised by Mayur to visit the government TB Unit near her home. She was immediately diagnosed with drug-resistant TB. And soon after the diagnosis, she was put on treatment. She has been on medication for 10 months now and has a few more months to go. During her treatment, she developed issues of stress, anxiety, and suicidal thoughts due to the side effects of medication and various other factors such as isolation, stigma, lack of confidence, and an inability to continue work/studies. She went into a deep depression. Jhalpa said that during this period, THALI field officer Mayur was very helpful, frequently met her, and gave her a lot of support and advise to keep her confidence high and to beat the depression. He also advised her to go outdoors more often, to take walks, read books, watch films that would cheer her up, and also watch TV shows that would put positive thoughts in her mind. Jhalpa said that all this personal support helped her immensely. Jhalpa showed us her log book of thoughts that Mayur asked her to maintain, advising her to keep it positive. Her log book tracks not only her thoughts penned in very articulate Gujarati, but also contains positive poems and short stories that were inspirational to read. Jhalpa hopes she will publish her "Book of Positive Thoughts" someday soon and help many more young girls like her living with TB to stay positive and beat mental health issues. Jhalpa had a strong message for society -- "TB is curable. Whether men or women have it, the suffering and the treatment process is the same. However, the stigma when a woman gets TB is higher - especially when it is a young woman like me. This stigma should be overcome. We are no different from any others in society. We just have a temporary setback. We will bounce back. Support us." 64 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523