April 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Michele Andina, Robertino Mera, Dipanjan Sujit Roy, and Anita Rego under the direction of Social Impact, Inc. with Management Systems International. EVALUATION AIDS Prevention and Control (APAC) Evaluation USAID/India Final Report AIDS PREVENTION AND CONTROL (APAC) EVALUATION USAID/INDIA FINAL REPORT April 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Michele Andina, Robertino Mera, Dipanjan Sujit Roy, and Anita Rego under the direction of Social Impact, Inc. with Management Systems International. AIDS PREVENTION AND CONTROL (APAC) EVALUATION USAID/INDIA DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document is available in printed or online versions. Documents are also made available through the Development Experience Clearing House (http://dec.usaid.gov). Additional information can be obtained from: Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com This document was submitted by Social Impact, Inc., with Management Systems International, to the United States Agency for International Development under USAID Contract No. RAN-I-00- 09-00019-00. ACKNOWLEDGMENTS The APAC Evaluation Team would like to acknowledge the USAID/India team for their guidance, support, and long-term commitment to HIV prevention in Tamil Nadu: Ms. Kerry Pelzman, Director, Office of Health; Mr. Arvind Kumar, COTR, APAC; Ms. Elizabeth Callender, COTR, Evaluations; Ms. Charushila Lal, Program Development Specialist, M&E. To Mr. William Hammink, Mission Director and Ms. Elizabeth Warfield, Deputy Mission Director, we thank you for your interest and questions during our presentation, and to Ms. Shanti Conly, Team Leader, HIV Prevention USAID Office of HIV/AIDS, Washington, D.C.: we were honored by your attendance at the findings presentation. The Chennai-based APAC team, led for many years by Dr. Bimal Charles, deserves special thanks for answering many questions and completing the detailed worksheets prepared by the APAC Evaluation Team. Additional thanks goes to Dr. A. Edwin Sam, Program Manager Research, for his invaluable assistance with data for the Evaluation Study; and to Dr. T. Ilan Chezhian, Gayatri Mishra Oleti, S. Lokabiraman, and Dr. P. Mahalingam for their guidance during field visits. To all APAC field partners in the TSUs and NGOs: we were inspired by your work and commend you for your commitment to HIV/AIDS prevention. We offer our gratitude to the many FSW and MSM/TG community members who spoke so proudly of their efforts and work. Please continue to care. Finally, because APAC was such a unique public-private partnership, we extend a special word of gratitude to the many collaborating government of India institutions, including: Tamil Nadu District Hospitals, Antenatal and STI Clinics, ICTC and ART centers, and to the Tamil Nadu and Kerala SACS staff and their DAPCU units. Ms. Girija Vaidyanathan, Principal Secretary, Ministry of Health and Family Welfare, Tamil Nadu, is a stellar example of India’s civil service system—highly knowledgeable and fully committed to improving the health and well-being of the people of Tamil Nadu.Thanks to your leadership (as well as your predecessors’), USAID’s 17-year investment has yielded high returns. Last, but most importantly, a special acknowledgement is due the National AIDS Control Organization (NACO), whose efforts have led India’s fight against HIV: Ms. Aradhna Johri, Additional Secretary; Dr. Rajan Khobragade, Team Leader NTSU; Dr. Neeraj Dhingra, Deputy Director General TI; and Dr. S. Venkatesh, Deputy Director General M&E. These dedicated professionals especially inspired the team to analyze the available data as rigorously as possible, irrespective of the outcome. Our thanks to all of you and to the Social Impact team for trusting and helping us with this important work. ACRONYMS ABC Abstinence, Be faithful, Condom AIDS Acquired Immune-Deficiency Syndrome ANC Antenatal Care APAC AIDS Prevention and Control Project ARC APAC Review Committee ART Antiretroviral Therapy AVERT HIV/AIDS Prevention and Control Project for Maharashtra (USAID) BCC Behavior Change Communication BMGF Bill & Melinda Gates Foundation BMW & IC Biomedical Waste Management and Infection Control BSS High-Risk Behavioral Surveillance Survey CBO Community-Based Organization CCLS Comprehensive Care and Livelihood Support CETC Continuing Education and Training Center CHAM Consortium of HIV/AIDS Mitigation Efforts CI Confidence Interval CHE Community Health Educator CMIS Computerized Management Information System CSM Condom Social Marketing DAPCU District AIDS Prevention Control Unit DPM District Program Manager DQA Data Quality Assurance ESRM Experience Sharing and Review Meetings FBO Faith-Based Organization FGD Focus Group Discussion FSW Female Sex Worker GH Tech USAID Global Health Technical Assistance Program GOI Government of India HCP Health Care Provider HIV Human Immunodeficiency Virus HIV+ HIV positive HRH Human Resource in Health HSS Health Systems Strengthening ICONHSS International Conference on Health Systems Strengthening ICTC Integrated Counseling and Testing Center IDU Injecting Drug User IEC Information, Education, and Communication IMRB Indian Market Research Bureau INGO International Non-Governmental Organization INP+ Indian Network of People Living with AIDS IRT PMCH Institute of Road Transport Perunderai Medical College Hospital KSACS Kerala State AIDS Control Society M&E Monitoring and Evaluation MARPS Most-At-Risk Populations MHC Master Health Check MSM Men who have Sex with Men NACO National AIDS Control Organization NACP-3 National AIDS Control Program phase 3 NGO Non-governmental Organization NHCRC National HIV Communication Resource Center NRHM National Rural Health Mission OI Opportunistic Infection ORW Outreach Worker OVC Orphans and Vulnerable Children PACS Puducherry AIDS Control Society PD Project Director PE Peer Educator PEPFAR The United States President’s Emergency Plan for AIDS Relief PLHIV People Living with HIV PMC Project Management Committee PPP Public Private Partnership PPTCT Prevention of Parent-to-Child HIV Transmission PSA Participatory Site Assessment PSV Participatory Site Visits RCH Reproductive and Child Health Program R&TC Resource and Training Center RFP Request for Proposal RNTCP Revised National TB Control Programme RUSE Reaching the Unreached Sex Workers SACS State AIDS Control Society SHCRC State Health Communication Resource Center SHDRC State Health Data Resource Center SHIRC State Health Information Resource Center SHRRC State Health Research Resource Center SI Strategic Information SIMS Strategic Information Management System SIMU State Information Management Unit SMC Social Marketing of Condoms SOW Scope of Work STD Sexually Transmitted Disease STI Sexually Transmitted Infection STRC State Training and Resource Center TA Technical Assistance TAC Technical Assistance Component TAI Tamil Nadu AIDS Initiative TANSACS Tamil Nadu State AIDS Control Society TCU Transition Coordination Unit TESPIM Thematic Experience Sharing and Performance Improvement Meeting TI Targeted Intervention TNHSP Tamil Nadu Health Systems Project TOT Trainer of Trainers TRG Technical Resource Group TSU Technical Support Unit TWG Technical Working Group USAID United States Agency for International Development VCTC Voluntary Counseling and Testing (Center) VHS Voluntary Health Services VDRL Venereal Disease Research Laboratory Test WHO World Health Organization APAC Evaluation CONTENTS EXECUTIVE SUMMARY ................................................................................................................................................... i INTRODUCTION ............................................................................................................................................................ 1 BACKGROUND OF HIV/AIDS IN INDIA ............................................................................................................ 1 THE STATE OF TAMIL NADU ................................................................................................................................ 3 APAC - HISTORY......................................................................................................................................................... 5 TARGETING THE EPIDEMIC ................................................................................................................................... 8 EVALUATION PURPOSE .............................................................................................................................................. 8 METHODOLOGY ............................................................................................................................................................ 9 Relative Ranking Methodology ............................................................................................................................ 10 Study of Health Outcomes and Contributing Factors ................................................................................... 10 Limitations ................................................................................................................................................................ 11 FINDINGS ......................................................................................................................................................................... 12 NGO-GOVERNMENT PARTNERSHIP MODEL .............................................................................................. 12 HEALTH OUTCOMES .............................................................................................................................................. 15 APAC TECHNICAL ASSISTANCE ........................................................................................................................ 24 Technical Support Unit (TSU) ............................................................................................................................. 27 Tamil Nadu Health Strengthening Project (TNHSP) ..................................................................................... 28 Master Health Check (MHC)............................................................................................................................29 APAC SUCCESS STORIES ............................................................................................................................................ 31 CONCLUSIONS ............................................................................................................................................................. 33 CONTRIBUTING FACTORS .................................................................................................................................. 33 RECOMMENDATIONS ................................................................................................................................................. 35 To USAID: Focus investments in “Health Systems Innovations”.................................................................. 35 To National AIDS Control Organization (NACO): ........................................................................................... 36 To State AIDS Control Society (SACS): ............................................................................................................... 36 FIGURES Figure 1: HIV Prevalence in India Among Different Population Groups .............................................................. 1 Figure 2: India HIV Prevalence by Districts, 2004-2006 ........................................................................................... 2 Figure 3: HIV Prevalence Among Different Population Groups, Tamil Nadu, 2007 ......................................... 4 Figure 4: Timeline of Major Events in HIV Scenario in India and Tamil Nadu in Relation to APAC ............. 4 Figure 5: APAC Program Management Committee .................................................................................................. 5 Figure 6: APAC Expenditures, Comparative, Phase I+II & III, 1995-2012............................................................ 7 Figure 7: Elements of a Targeted Intervention (TI) ................................................................................................... 8 Figure 8: Targeted Interventions in Tamil Nadu, 2007 ............................................................................................. 8 Figure 9: Overall Adjusted HIV Prevalence Rates, General Population HSS Tamil Nadu, 2002-2008 ....... 16 Figure 10: Adjusted HIV Prevalence Among MARPS, Tamil Nadu, 2002-2008 ................................................ 17 Figure 11: HIV Prevalence PPTCT, Tamil Nadu, 2005-2011 ................................................................................. 18 Figure 12: Adjusted STI Prevalence (VDRL), Gen. Pop. HSS, Tamil Nadu 2002–2008 .................................. 19 Figure 13: Adjusted STI Prevalence (VDRL) in MARPS, HSS, Tamil Nadu, 2002-2008 .................................. 20 Figure 14: Trends in Select BSS Indicators in Tamil Nadu, 1997-2008 ............................................................... 21 APAC Evaluation Figure 15: Comparison of Select BSS Indicators, APAC and non-APAC Districts, Tamil Nadu, 2009 ...... 22 Figure 16: Trends in ICTC Testing, MIS, Tamil Nadu, 2002-2010 ....................................................................... 23 Figure 17: Relationship between Health Agencies/Programs at National, State and District levels ............ 25 Figure 18: APAC Technical Assistance Activities .................................................................................................... 26 Figure 19: APAC Technical Support Unit .................................................................................................................. 27 Figure 20: APAC Tamil Nadu Health Systems Project ........................................................................................... 28 Figure 21: APAC Master Health Check ..................................................................................................................... 29 TABLES Table 1: Select Socio-demographic Statistics, Tamil Nadu vs. all of India ............................................................3 Table 2: AIDS Prevention and Control Project Overview, 1995-2012 ................................................................6 Table 3: Cumulative Relative Rankings by Six Groups - Result of Focus Group Discussions ......................12 APPENDICES APPENDIX A. SCOPE OF WORK ............................................................................................................................ 38 APPENDIX B. PERSONS CONTACTED ................................................................................................................ 47 APPENDIX C. RESEARCH STUDIES ........................................................................................................................ 57 APPENDIX D. FOCUS GROUP INSTRUMENT ................................................................................................... 60 APPENDIX E. FIELD VISIT SCHEDULE .................................................................................................................. 61 APPENDIX F. RELATIVE RANKING RESULTS .................................................................................................... 62 APPENDIX G. STATISTICAL ANALYSIS ................................................................................................................ 65 APPENDIX H. TECHNICAL ASSISTANCE PROVIDED BY APAC ................................................................. 66 i APAC Evaluation EXECUTIVE SUMMARY INTRODUCTION The AIDS Prevention and Control Project (APAC), a unique public private partnership, has been described as USAID/India’s HIV/AIDS flagship project. Operating in the southern state of Tamil Nadu from 1995– 2012 with a budget of $47.25 million, it has provided direct HIV-prevention services through local non￾governmental organizations (NGO) and technical assistance to both the Tamil Nadu State AIDS Control Society (TANSACS) and the National AIDS Control Organization (NACO). This evaluation provides a rigorous analysis of APAC’s impact on health outcomes in Tamil Nadu and of the elements contributing to APAC’s successful reputation. This report provides contextual information about HIV/AIDS in India and Tamil Nadu and describes APAC’s three phases. Evaluation methodology is followed by findings in three key areas of inquiry: (1) the NGO-government partnership model; (2) health outcomes; and (3) technical assistance. The report then offers the team’s conclusions and identifies factors that contributed to the findings. Finally, the report offers recommendations for future HIV/AIDS and health systems strengthening (HSS) programming to USAID as well as to NACO and TANSACS, as both remain active partners in APAC. HIV/AIDS AND APAC BACKGROUND The first case of HIV/AIDS in India was identified in Tamil Nadu in 1986. NACO, created in 1992, has led the tracking and management of this epidemic. Today, HIV/AIDS infection rates in India vary from state to state, with higher HIV prevalence in the four southern states, including Tamil Nadu. Female Sex Workers (FSW), Men￾having-Sex-with-Men (MSM), and intravenous drug users (IDU)—currently referred to as most-at-risk populations (MARPS)—are the most adversely affected. In India, the average HIV prevalence rates for women attending ante￾natal clinics is 0.48%; for MARPS, HIV prevalence rates range from 4.8% in FSW to 9.1% in IDU.1 Despite strong health and demographic indicators in Tamil Nadu, HIV prevalence rates, especially among MARPS, remained high. In 1992, as the first National AIDS Control Program (NACP-I) rolled out, USAID signed a unique “tripartite agreement” with Voluntary Health Services (VHS, a well-established Tamil Nadu medical NGO), and the government of India (GOI) to create APAC. A Project Management Committee (PMC) with full representation of all concerned stakeholders also was formed. APAC Phase I (1995–2002) pioneered targeted interventions (TI) and worked through local NGOs to target MARPS with HIV prevention messages, access to condoms, and sexually transmitted infection (STI) services in ten Tamil Nadu Districts. During APAC Phase II (2002–2007), the scope of TI expanded to include work with people living with HIV/AIDS (PLHIV), encouraging HIV testing and access to care services, including antiretroviral therapy (ART). In 2003, the Tamil Nadu AIDS Initiative (TAI), funded by the Bill & Melinda 1 Source: National AIDS Control Organization. HIV Estimations 2008-09. Dec 2010. ii APAC Evaluation Gates Foundation (BMGF) entered the HIV-prevention field, thereby assuring TI coverage in all Tamil Nadu districts. During APAC Phase II, besides implementing TI in 23 districts, APAC also provided technical assistance (TA) to TANSACS and NACO. During APAC Phase III (2007–2012), which coincided with NACP-III, TI continued in only seven districts; over 53% of APAC funding supported technical assistance (TA), including management of Technical Support Units for both the states of Tamil Nadu and Kerala. EVALUATION PURPOSE The purpose of this evaluation was to rigorously analyze three key areas of inquiry: (1) the NGO-government partnership model; (2) health outcomes; and (3) technical assistance; and to identify specific factors that contributed to each of the overall findings. The evaluation team was comprised of four team members – Dr. Michele Andina (Team Leader, Senior Technical HIV/AIDS Expert), Dr. Dipanjan Sujit Roy (Public Health Specialist), and Dr. Anita Rego (Senior Public Health Specialist) of Social Impact, Inc. (SI); and Dr. Robertino Mera (Evaluation Methods Specialist) of Management Systems International (MSI). Fieldwork and data collection in India took place over a four￾week period from January to February 2012. FINDINGS NGO-Government Partnership Model To better understand which aspect of APAC’s unique NGO-government partnership model facilitated project ownership at the state and national level, the evaluation team employed a “relative ranking methodology,” with six focus groups and 61 participants. The data were drawn from the USAID Global Health Technical Assistance Program’s (GH Tech) APAC documentation findings. The table below shows the final, overall rankings by the six stakeholder groups, with “10” indicating the most important element and “1” the least important element. iii APAC Evaluation RESULT OF FOCUS GROUP DISCUSSIONS Cumulative relative rankings by six groups ITEMS DESCRIPTION FINAL RELATIVE RANK Leadership Dynamic leadership that engages and supports government, CSOs and donors in project planning, management, implementation and monitoring 10 Capacity building Emphasis on building capacity of various entities (individuals, NGOs, government, training resource centers) with focus on need and mentoring 9 Systems Strong systems for monitoring and managing project progress (e.g. MIS for technical and financial, transparent systems) 8 Budget Adequate, timely and flexible budget 7 Community involvement Culture of participation as seen in involvement of community being served in project design and implementation 6 Innovation Opportunities and encouragement for innovation of new ideas (e.g. SMS) 5 Project Management Committee Project Management Committee with equal partnership between government and NGO (VHS) including regular meeting of all partners (ARC, tripartite agreement) 4 Evidence Strong focus on generation of “evidence” and using it for on-going project design (e.g. BSS, other studies) 3 Sustainable/ replicable models Creating replicable and sustainable models (e.g. State Health Data Resource Centre) 2 Private sector Engagement/ participation of the private sector 1 Leadership emerged as the factor most critical to the project’s success and shared sense of ownership. Participants highlighted three key requirements of successful leadership: 1. Leadership should be consistent and knowledgeable 2. The leader should have strong relationship building skills and the ability to sustain relationships 3. The leader should be accepted and considered “one of us” by all partners, especially the government Capacity building was designated the next most important component, since the knowledge and skills of many individuals were built through APAC’s work. The APAC Project Management Committee (PMC), which formed the basis of the unique public-private partnership governance structure, garnered a cumulative ranking of seventh; only APAC staff ranked it second. The APAC staff stressed the role of the PMC in creating the mechanism for shared ownership. For others it appeared to be a “given” that APAC, TANSACS and the State Health Society (SHS) were working as one. Innovations were ranked as the sixth most important element. Government informants appreciated APAC efforts to identify, implement, and evaluate new approaches and strategies and acknowledged the government’s inability to create an environment where innovation was possible. An important lesson drawn from this analysis is that a sound governance system, such as the Tripartite Agreement and the PMC, coupled with strong, acceptable leadership (at all levels, project and government) played a significant role in project success and shared ownership. iv APAC Evaluation Health Outcomes Using an epidemiological study, this evaluation offers quantitative evidence of the impact of APAC interventions over time in the seven Phase III districts. The evaluation team compared the seven Phase III districts to HIV interventions in the remaining 25 districts during the same period, through a retrospective non-intervention comparison group with repeated measures study. 2 The APAC Evaluation Health Outcomes Study analyzed the impact of APAC interventions on select behavioral indicators (self-risk perception, STI knowledge, condom use, and violence) service coverage (Integrated Counseling and Testing Center uptake) and prevalence (STI, HIV) among both the general population and MARPS in Tamil Nadu. Although APAC initiated services in 1995, 2002–2003 was set as the baseline (initial time point) because data prior to these years did not provide adequate statistical power. HIV Prevalence The evaluation team used a mixed-model, logistic regression analysis to determine HIV-prevalence trends among the general population and MARPS. The rate of decline in adjusted HIV prevalence for the general population was higher among the APAC districts when compared to non-APAC districts until 2007, with the curve overlapping for 2008. This overlap suggests a ripple effect (APAC was providing technical assistance to the Tamil Nadu State AIDS Control Society) or a “catch-up effect” in the 25 non-APAC districts. 2 See Appendix G, “Statistical Analysis,” for a full description. v APAC Evaluation The exploration of HIV prevalence among MARPS shows findings similar to the general population findings. The multivariate analysis shows significant decline in adjusted HIV prevalence among MARPS for APAC districts when compared to non-APAC districts. STI (VDRL) Prevalence The analysis of general population STI (VDRL) prevalence starting in 2002 showed a statistically significant decline with some degree of stabilization and variability noted in 2005. 3 Among APAC districts, an early and significant decline was observed and, by the year 2004, the STI/VDRL prevalence rate was 49% lower among APAC districts, (95% confidence interval [CI] 20% to 71%)4 compared to non-APAC districts. By 2008, a distinct tendency toward converging rates emerged, with no difference noted between APAC and non-APAC districts. Behavioral Indicators Behavioral indicators were taken from the High-Risk Behavioral Surveillance Survey (BSS) studies conducted collaboratively by TANSACS and APAC from 1997 to 2009. Due to BSS sampling methodology and data availability, comparison of APAC districts to non-APAC districts was only possible for the year 2009.5 The four behavioral indicators used for analysis were: (1) knowledge of STI prevention; (2) condom use during last sex act; (3) risk perception (self); and (4) exposure to violence (as a proxy/surrogate marker for stigma). 3 A Poisson statistical mixed model was used for analysis. 4 Definition of confidence interval (CI): A range of values so defined that there is a specified probability that the value of a parameter lies within it. 5 BSS data from the years 1997–2008 were collected to provide information for the entire state of Tamil Nadu. In 2009, however, data was collected separately for sites representing the seven APAC districts. Thus, comparison was possible for this one point in time. 0.0% 0.5% 1.0% 1.5% 2.0% 2.5% 3.0% 3.5% 2002 2003 2004 2005 2006 2007 2008 Estimated adjusted HIV Prevalence Year Non-APAC APAC 25 districts Ripple Districts made comparable by propensity Adjusted rates from a multivariate analysis 7 districts Overall Adjusted HIV Prevalence Rates, General Population HSS Tamil Nadu 2002-2008 vi APAC Evaluation Given the limitations of the BSS data, the team made use of conditional logistic regression to compare the proportion of all Tamil Nadu responders to three behavioral indicators for two points in time: 1997–1999 and 2007–2008. Analysis showed that among all subjects surveyed in Tamil Nadu, those who stated that STIs can be prevented grew significantly by 5.7% (95% CI from 3.3% to 8.0%) in 1997–1999 when compared to years 2006–2008. Reported condom use among all subjects surveyed in Tamil Nadu had also increased, significantly by 6.0% (95% CI1.9% to 9.9%) in 1997–1999 when compared to years 2007–2008. During the same period, ‘risk perception’ increased significantly by 34.8% (95% CI 18.6 to 48.1%). Using the point estimates for 2009, it was noted that two behavioral indicators, risk perception and reports of violence (used as a surrogate marker for stigma and discrimination), showed significant differences in 2009 between APAC and non-APAC districts. Among non-APAC districts in 2009, the proportion who reported moderate- to high-risk perception was 6.5% (95% CI 4.6 to 8.4%) compared to APAC districts with 12.5% (95% CI 9.2 to 15.7%). Reported violence was also lower in APAC districts (18.7%, 95% CI 14.9 to 22.5%) compared to non-APAC districts (27.0%, 95% CI 23.5 to 30.3%), also a significant difference. vii APAC Evaluation Service Coverage Integrated Counseling and Testing Center (ICTC) data were used as a proxy for service coverage. ICTC testing increased massively and significantly across the board for both APAC and non-APAC districts. This is consistent with the focus of the national program to increase HIV counseling and testing across the country. No significant difference in ICTC testing was seen in the rate of coverage when comparing APAC to non-APAC districts, but the declining trend observed in 2010 should be noted. The evaluation team’s findings for health outcomes indicate that APAC has had a significant positive impact on health outcomes, for the general population as well as for MARPS, in Tamil Nadu. The converging trends noted in 2007–2008 point to the “ripple effect” of APAC’s work throughout the state, and that of other agencies, including BMGF and TANSACS. TECHNICAL ASSISTANCE Approximately 53% of APAC’s Phase III expenditure was technical assistance (TA), with 24% of total TA going directly to the government. As identified by the GH Tech APAC documentation team, APAC has utilized multiple TA approaches: 1. Evidence generation/research 2. Strategy development 3. Experts/consultants 4. Experience sharing 5. Training and mentorship 6. Systems and procedures The evaluation team examined qualitative as well as quantitative findings to identify those most relevant for NACP IV and for future USAID HSS programming. All informants found evidence generation/research to be extremely useful. The Technical Support Unit (TSU) was credited with creating strong training links with the State Training Resource Center (STRC) and providing excellent opportunities for experience sharing. The Tamil Nadu Health Systems Project (TNHSP) was also acknowledged for their evidence generation, which resulted in the creation of a statewide infection control and biomedical waste management system. The TNHSP International Conference on HSS (ICONHSS) was also noted as an excellent example of international cross sharing and learning. Finally, the Master Health Check (MHC), described by FSW and MSM community members as a “full body check,” was credited with enhancing government STI services as well as increasing ICTC and VDRL testing among MARPS throughout the state. The work of the TSU, District AIDS Prevention and Control Unit (DAPCU), and MHC have significance for NACP-IV and the TNSHP was found to be a commendable example of a TA partnership, with great potential for future HSS endeavors. The impact of APAC TA can be seen in the transition of all TI to the government in March 2011; ICTC uptake; increased coverage of MARPs throughout the state; improved spending by both the Tamil Nadu and Kerala SACS; and the creation of sustainable systems and institutions, at both the state and national levels. In the field, the evaluation team also observed that the capacity of many local institutions (NGOs, DAPCU, service delivery points) and personnel to deliver effective HIV prevention programs throughout Tamil Nadu has been strengthened. viii APAC Evaluation However, the evaluation team observed that the use of data for decision-making remains limited. Looking forward to NACP-IV, increased technical assistance in the analysis and employment of data in program planning and management will be important for all levels of program personnel and service providers. CONCLUSIONS 1. The epidemiological study demonstrates that for the period 2002–2007, APAC has had a measurable, positive effect on health outcomes in the seven districts of Tamil Nadu where APAC has intervened since 1995. A secondary data source confirms a similar trend from 2005–2010. 2. Declining HIV and STI/VDRL prevalence in the general population and in MARPS, increases in service uptake, and improved behavioral indicators, over time, were noted in all 32 districts of Tamil Nadu. 3. A “catch-up effect,” caused by APAC and its technical assistance to the SACS, also contributed to the declining trends. Described by the APAC evaluation team as a “rising tide” with the ability to “lift all boats,” the work of APAC, other donors, and a proactive Tamil Nadu government has resulted in a decline in HIV and STI prevalence and an increased uptake of services, beyond those considered part of a natural disease progression. 4. Strong leadership and effective collaboration with the Tamil Nadu government were critical to APAC success and sense of shared project ownership. 5. The effect of APAC technical assistance at the NGO, state and national levels was evident in health outcomes, successful transitioning of all NGO-based targeted interventions to the government, and in the replication of state agencies, such as the Resource Communication Center, at the national level. 6. While APAC has been successful at generating evidence, additional work is required to improve the accurate use of data by program personnel—from SACS to DAPCU to NGOs—at all levels. 7. APAC and HIV/AIDS programming in Tamil Nadu has also had a positive impact on health systems strengthening, as demonstrated by the uptake of PPTCT antenatal care (ANC) testing and through successful initiatives like the master health checks that strengthened government STI clinics. 8. APAC allocations for HIV prevention in children were limited. The OVC Trust is an example of a successful APAC intervention, and its collaboration with, and support to, this TANSACS initiative serves as a potential model for furthering work with at-risk children. ix APAC Evaluation Contributing Factors Identifying the specific contributing factors to APAC’s success is not an exact science. Using both quantitative as well as extensive qualitative analysis, the evaluation team determined the following to be the most significant contributing factors: 1. Long term investment by USAID (1995–2012), for a total expenditure of $42 million, with $16.9 million in phase III focused on TA 2. Strong and effective collaboration with the government 3. Strong leadership, both of APAC and of TANSACS 4. Effective technical assistance by APAC to all levels—national, state, NGO, and the private sector, with particular reference to the following areas: a. Generation of evidence b. Technical Support Units (TSU) in Tamil Nadu and Kerala DAPCU financial and administrative management systems c. Health systems strengthening—most notably, support given by APAC to the TNHSP d. Experience sharing and the provision of opportunities for experiential learning e. Innovations designed to address ground realities and create an environment for innovation RECOMMENDATIONS To USAID Focus investments in health systems innovations • Strengthen generation, analysis, and use of evidence at all levels • Support management information systems for health • Encourage ground work efforts and innovations of NGOs through a “challenge grant mechanism” • Support opportunities for national and international cross sharing and experiential learning • Don’t forget the children—especially those children affected by HIV/AIDS (CABA) To NACO • SACS should be given greater autonomy; they will require the capacity and ability to respond to rapidly changing local environments • Develop an information management system that allows for greater flow of data down to program managers • Scale-up Master Health Checks, particularly in states with high HIV prevalence To TANSACS • Conduct health systems research to understand the changing nature of disease locally • Build capacity at all levels to analyze and use data for program planning • Strengthen DAPCUs to address full HIV/AIDS prevention-to-care continuum • Keep focus on “hard to reach” populations and be innovative in approaches • Encourage sharing within state to find solutions 1 APAC Evaluation INTRODUCTION The AIDS Prevention and Control Project (APAC), a unique public private partnership, has been described as USAID/India’s HIV/AIDS flagship project. Operating in the southern state of Tamil Nadu from 1995–2012 with a budget of $47.25 million, it has provided direct HIV-prevention services through local non-governmental organizations (NGO) and technical assistance to both the Tamil Nadu State AIDS Control Society (TANSACS) and the National AIDS Control Organization (NACO). This evaluation provides a rigorous analysis of APAC’s impact on health outcomes in Tamil Nadu and of the elements contributing to APAC’s successful reputation. This report provides contextual information about HIV/AIDS in India and Tamil Nadu and describes APAC’s three phases. Evaluation methodology is followed by findings in three key areas of inquiry: (1) the NGO-government partnership model; (2) health outcomes; and (3) technical assistance. The report then offers the team’s conclusions and identifies factors that contributed to the findings. Finally, the report offers recommendations for future HIV/AIDS and health systems strengthening (HSS) programming to USAID as well as to NACO and TANSACS, as both remain active partners in APAC. BACKGROUND OF HIV/AIDS IN INDIA The first HIV/AIDS case in India was identified in 1986 at Madras Medical College in the southern state of Tamil Nadu. The National AIDS Control Organization (NACO) formed in 1992 and launched the first National AIDS Control Program (NACP-I). Today, NACO estimates that two to four million (CI 1.8- 3.8 million) people are infected with HIV, making India the world’s third-largest infected population.6 As significant as that number appears, in 2001 the number of HIV infections in India was predicted to rise to 20–25 million by 2010, leading to demand for an enhanced response.7,8,9 6 UNAIDS report on the global AIDS epidemic. Geneva: UNAIDS, 2010. 7 US National Intelligence Council. The next wave of HIV/ AIDS: Nigeria, Ethiopia, Russia, India, and China. 2002. http://www.fas.org/irp/nic/hiv-aids.html (accessed February 5, 2012) 8 Stover J, Walker N, Garnett GP, et al. Can we reverse the HIV/AIDS pandemic with an expanded response? Lancet 2002;360: 73–77 9 Potts M, Walsh J. Tackling India’s HIV epidemic: lessons from Africa. BMJ 2003; 326: 1389–92 9.19% 7.30% 4.94% 2.46% 0.48% 0% 2% 4% 6% 8% 10% Injecting Drug Users Men having sex with Men Female Sex Workers Patients attending STI clinics Women attending Antenatal… Figure 1: HIV Prevalence in India Among Different Population Groups Source: National AIDS Control Organization. HIV Estimations 2008–2009. Dec 2010. 2 APAC Evaluation Post- 2001 saw a rapid increase in funding for HIV control in India. NACP-II (1999– 2007), with a budget of US$460 million, shifted the focus from raising awareness about HIV/AIDS toward interventions that promoted behavior change, especially condom use. In 2006, a revised statistical estimate lowered the number of people living with HIV/AIDS to 2.47 million from the 5.2 million estimated in 2005, but the government and donors’ commitment to fight the epidemic remained.10 NACP-III (2007–2012) allocated two-thirds of its $2.5 billion budget for HIV prevention while also seeking to integrate care, support, and treatment strategies.11,12,13,14 According to the 2010 United Nations General Assembly Special Session on HIV (UNGASS) country report, India’s epidemic is now concentrated within most-at-risk￾populations (MARPS).15 HIV prevalence is substantially higher among these groups when compared to the general population. HIV prevalence also varies dramatically by district, state, and region, with numerous isolated pockets of high prevalence. Approximately 60% of people living with HIV/AIDS (PLHIV) live in the four southern states of India (Andhra Pradesh, Karnataka, Maharashtra, and Tamil Nadu).16 Given the increased incidence among MARPS, NACO’s prevention efforts, referred to as “targeted interventions,” have focused on outreach to and services for at-risk populations. NACO also has strengthened the national health infrastructure (blood banks, hospital-based STI and ANC services), developed detailed systems for case identification, and worked to provide care, support, and treatment to people living with HIV/AIDS. The concerted efforts of NACO, the National Rural Health Mission (NRHM), multiple donors, hundreds of NGOs, and the work of thousands of community members representing at-risk populations have contributed to impeding the progress of the AIDS epidemic in India. 10 Technical Report on HIV Estimations 2006. National AIDS Control Organisation, Govt. of India, 2007. 11 National AIDS Control Organisation. Funds and expenditure. http://www.nacoonline.org/About_NACO/Funds_and_ Expenditures/ (accessed February 10, 2012) 12 John TJ, Dandona L, Sharma VP, Kakkar M. Continuing challenge of infectious diseases in India. Lancet 2011; 377: 252–69. 13 National AIDS Control Organisation. National AIDS Control Programme phase III (2006–2011). Strategy and implementation plan. New Delhi: Ministry of Health and Family Welfare, Government of India, 2006. 14 Marie Ng, Emmanuela Gakidou, Alison Levin-Rector, Ajay Khera, Christopher J L Murray, Lalit Dandona; Assessment of population-level effect of Avahan, an HIV-prevention initiative in India; www.thelancet.com Vol 378 November 5, 2011 pg 1644-1652 15 Most at Risk Populations - MARPS include female sex workers (FSW), men-who-have-sex-with-men (MSM), and intravenous drug users (IDU) 16 National AIDS Control Organisation. HIV Sentinel Surveillance and HIV Estimation 2007: A Technical Brief. Oct 2008 Figure 2: India HIV Prevalence by Districts, 2004-2006 Tamil Nadu Andhra Pradesh Maharashtra Karnataka -A districts (high) -B districts (mod) -C districts (low) -D districts (data NA) 3 APAC Evaluation The AIDS Prevention and Control Project, implemented from 1995-2012 in the southern state of Tamil Nadu with total budget of $47.25 million from USAID, is one example of the positive impact of this important work. THE STATE OF TAMIL NADU Tamil Nadu is a state with strong health and demographic indicators (Table 1). Since 2010 the state has been divided into 32 districts, 22 classified by NACO as A districts, five as B and three as C.17 Table 1: Select Socio-Demographic Statistics, Tamil Nadu vs. All of India CRITERIA TAMIL NADU INDIA Districts 32 640 Total population 72.1 million 1.2 billion Literacy rate 80.33% 74.04% Population density (per square km) 555 382 IMR (per 1000 Live births, 2010) 24 50 MMR (per 1000 Live births, 2007-09) 97 212 Annual per capita income (1 USD = INR 49.00) INR 72,993 (USD 1,489.6) INR 54,835 (USD 1,119.1) Sources 2011 Census Prov. Figs.; SRS Bulletin Dec 11; Tamil Nadu Statistical Handbook 2011 2011 Census Prov. Figs.; SRS Bulletin Dec 11; PIB Press release Id :73929 In the early 1990s, the Tamil Nadu state government entered into negotiations with USAID and agreed to form APAC, a unique private-public partnership between the government and a local medical NGO, Voluntary Health Services. APAC began in 1995 as a response to rising HIV prevalence among pregnant women. Although this early initiation of HIV prevention efforts slowed the overall rate of HIV infection, HIV prevalence rates among MARPS in Tamil Nadu remained high (Figure 3). Because of these high rates, additional donors, including the Bill and Melinda Gates Foundation (BMGF), joined Tamil Nadu’s efforts to slow the epidemic. The timeline in Figure 4 provides a graphic picture of the evolving HIV/AIDS scenario in the state. 17 This classification is based on HIV prevalence at ANC clinics as well as HIV prevalence among MARPS. A - districts have >1% ANC/PPTCT prevalence in any time in any sites in the last three years: B - districts have <1% ANC/PPTCT prevalence in all sites during last three years and >5% prevalence in any high risk group (HRG): C districts have <1% ANC prevalence in all sites during last three years with <5% in all STI clinic attendees or any HRG with known hotspots. 4 APAC Evaluation Figure 1: Timeline of Major Events in HIV Scenario in India and Tamil Nadu in Relation to APAC 16.80% 6.60% 4.68% 8.00% .25% 0.00% 5.00% 10.00% 15.00% 20.00% Injecting Drug Users Men having sex with Men Female Sex Workers Patients attending STI clinics Women attending Antenatal Clinics Percent Positive Figure 4: HIV Prevalence Among Different Population Groups, Tamil Nadu, 2007 1986 •1986: First case of HIV in India 1992 •1992: NACO established 1995 •1994:TANSACS Established 1999 2002 2004 •2004: Roll-out of free ART (GFATM) 2007 2011 NACP I: 1992 -1999 NACP II: 1999 -2007 NACP III: 2007 - 2012 APAC PHASE I: 1995– 2002 APAC PHASE II: 2002–2007 APAC PHASE III: 2007 - 2012  Targeted Interventions(TIs) among FSWs/Truckers - Direct Implementation  Focus on Condom Promotion Focus: 10 districts of Tamil Nadu  TI continued  Care and support (small scale)  MSM and migrants outreach Support: 23 districts of TN  TA - comprehensive prevention to care continuum in 7 districts  TA - NACO and TANSACS  TA to Tamil Nadu Health Systems Project Focus: 7 districts of TN  Prevention focus - with mass media campaigns  No state agencies  TI focus - MARPS  Testing -VCTC & PMTCT  Care, support and treatment  Program decentralized to SACS  Rapid scale-up of services  TI and HIV prevention focus - NTSU/TSUs  ART coverage to all (including children)  District AIDS Prevention and Control Unit (DAPCU) N A T I O N A L S T A T E  2003 Gates Foundation/TAI 5 APAC Evaluation APAC: HISTORY A tripartite agreement, signed in 1992 between VHS, USAID, and the government of India, serves as the foundation of APAC.18 This unique public-private partnership became one of the early pioneers in building viable working relationships between the private sector health NGOs and the government. The selection of VHS and the creation of the Project Management Committee facilitated sound mechanisms for regular coordination and shared ownership of project initiatives (see Figure 5). With initial funding of $9 million for Phase I (1995–2002), APAC was extended twice and its focus modified to respond to epidemic trends and emerging needs of the state and national government. Table 2 illustrates the highlights and modifications of each phase. 18 Tripartite agreement was signed in 1992, but APAC project activities began in 1995. Box 1: Voluntary Health Service Voluntary Health Service (VHS), established in 1958, follows the Gandhian philosophy to provide health and medical services to the poor with nominal or no payments. Today, VHS operates a 405-bed, multi-specialty hospital and has established 15 primary health centers and other community services in and around Chennai. 60% of services are provided free of charge. APAC, funded by USAID, and Tamil Nadu AIDS Initiative (TAI, funded by the Bill and Melinda Gates Foundation), are VHS HIV-prevention programs designed to reach most-at-risk populations, especially FSW and MSM. CHAIR HEALTH SECRETARY GOVT. OF TAMIL NADU USAID REPRESENTATIVE VHS (SECRETARY) NACO REPRESENTATIVE VICE CHAIR PROJECT DIRECTOR TANSACS Meets on a quarterly basis (four times a year or more if required) MEMBERS Figure 5: APAC Program Management Committee 6 APAC Evaluation Table 2: AIDS Prevention and Control Project Overview, 1995-2012 PHASE I: 1995–2002 PHASE 2: 2002–2007 PHASE 3: 2007–2012 Project Goal - Reduce HIV/AIDS Transmission - Care and support for affected populations - Scale up HIV prevention through SACS/NACO - Transition TI to government -Technical Assistance to Tamil Nadu (TN) - Health Systems Strengthening (HSS) for NACO and SACS Areas of Focus - Behavior change communication and condoms - Quality services to vulnerable populations - “Enabling environment” - Community mobilization - Strengthen/expand TI - Involve PLHA in initiatives - Manage Technical Support Units for TN & Kerala - Establish DAPCUs - Transition TIs to SACS - HSS through evidence and innovations Number of Targeted Interventions & Districts - 31 TI in 10 districts - 51 TI in 23 TN districts - 4 TI in 2 Puducherry districts - 27 TI in 7 TN districts - 4 TI in 2 Puducherry districts Communities Served - FSW, tourists, truckers, slum populations - Added MSM, IDU, Migrants, PLHIV - FSW, MSM, IDU, Migrants, PLHIV Highlights & Innovations - Created Targeted Interventions (TI) - Voluntary Peer Educators - Behavioral Change Communication tools (BCC) - Evidence based programs (BSS, mapping MARPS) - Public-Private Condom Social Marketing - Admin., finance, management, and capacity-building systems - Demonstration Centers - Added Puducherry TI - Home Based Care - Voluntary Counseling and Testing (VCT) at NGOs - Strengthened Positive Networks - Partnerships with industry and faith￾based groups - Teaching materials for BCC - Exposure visits for PD SACS/JD TI - Secretariat for NACO Technical Resource Group TI - Added Kerala SACS - SMS-based communications - Research Studies19, including Data Triangulation - National HIV/AIDS Communication Resource Centre - Livelihood Programs for MARPS - Orphans and Vulnerable Children Trust - Support to SACS for Master Health Check - State Health Resource Centers - for Data, Research and Communication - Training Centers for Biomedical Waste Management - Hello + hotline Funds Spent/Budget (in millions) $6.9 / $9 $16.7 / $18.2 $16.9 / $20.1 19 For a detailed list, refer to Appendix C, “Research Studies” 7 APAC Evaluation Figure 6 below offers financial evidence of the shifting focus of the project, from direct implementation through TI in Phases I and II to greater technical assistance to the government in Phase III. Further credit goes to APAC for leveraging a total of US$2.6 million from GOI for communications ($1.1 million), implementing the master health checks ($700,000) and for the Biomedical Waste Management Program ($800,000).20 20 Master health checks and the biomedical waste management program are discussed in the Findings section on technical assistance. 0 5 10 15 20 25 USD in Millions Phase I+II (10 Yrs) Phase III (5 Yrs) Total : USD 40.2 million 28% 7% 22% 6% 9% 8% 5% 15% APAC Phase III TI funding TSU Capacity building Govt. support Research/M&E Communication Care & support Admin Figure 6: APAC Expenditures, Comparative, Phase I+II & III 1995-2012 8 APAC Evaluation Figure 8: Targeted Interventions in Tamil Nadu, 2007 TARGETING THE EPIDEMIC APAC Phase I (1995–2003) is credited for promoting the concept of TI, which describes a focused approach to working with populations, including women in prostitution and truckers, who were suspected of being drivers of the disease. Over time, TI has become the accepted approach for reaching marginalized populations including migrants and other MARPS. In 2007, under NACP III, NACO created a standardized package of TI approaches adopted by all implementing partners (Figure 7).21 Since 1995, APAC has implemented TI in multiple districts of Tamil Nadu. During Phase III, a total of seven districts were covered by APAC. Figure 8 shows the coverage for each of the implementing partners, Tamil Nadu State AIDS Control Society (TANSACS), Tamil Nadu AIDS Initiate (TAI - funded by the Bill and Melinda Gates Foundation) and APAC. Of the seven districts covered by APAC, four are A districts (high) and three are B districts (moderate). In 2007, APAC began 27 TIs in the state, all of which successfully transitioned to TANSACS by March 2011. These seven districts have been TI sites for APAC since 1995, representing an intervention period of 16 years. This evaluation aims to assess the impact and outcomes of this long-term investment in HIV/AIDS prevention in Tamil Nadu by USAID. EVALUATION PURPOSE This final performance evaluation of APAC Phase III (2007–2012) analyzes the following program elements: 1. NGO-government partnership model: Analyze the effects of this innovative partnership model on the overall state of HIV/AIDS prevention and care programs and ascertain the specific elements, processes, and systems that facilitated improved ownership of the project at the state and national level. 2. Health outcomes and contributing factors: Assess select outcome and impact indicators (decided jointly by the evaluation team and the India Mission) among MARPs and PLHIV. The 21 The targeted intervention guidelines were developed based on the work of APAC and the Bill and Melinda Gates Foundation BMGF/Avahan Initiative. Figure 7: Elements of a Targeted Intervention (TI) 9 APAC Evaluation evaluation will focus on behavioral indicators (such as knowledge, risk-perception, service utilization) and will assess service coverage and HIV/STI prevalence among the target populations. To establish a comprehensive understanding of the interventions’ impact, an analysis of overall state performance, as well as select districts (both APAC and non-APAC) in Tamil Nadu, will be undertaken and the contributing factors that led to improved health outcomes and impact will be assessed. 3. Technical Assistance: Assess the process and effects of technical assistance support provided by the project to various stakeholders, including SACS, TNHSP, and NACO, and ascertain relevance of these TA areas for NACP-IV. Ideally, this evaluation will identify and assess the most significant technical assistance approaches and interventions that involve a target group and relevant comparison groups (that have not received USAID assistance), in order to yield substantive findings, conclusions, and recommendations. 22 The intended audience for this APAC Phase III evaluation is USAID/India. It is hoped that other USAID missions worldwide will also benefit from the evaluation. These findings, recommendations, and conclusions will feed into future HIV/AIDS program design in India, and have the potential to support other countries where similar HIV epidemics may exist (South-South collaboration). NACO and the SACS may also find this evaluation useful for the design of NACP-IV within the context of a changing HIV epidemic in India. METHODOLOGY USAID contracted Social Impact, Inc. (SI) and its partner, Management Systems International (MSI), to assemble a four-member team (two international and two national health specialists) to conduct an evaluation of APAC Phase III during a four-week period from January 9–February 11, 2012. The evaluation team included Dr. Michele Andina (Team Leader, Senior Technical HIV/AIDS Expert), Dr. Dipanjan Sujit Roy (Public Health Specialist), and Dr. Anita Rego (Senior Public Health Specialist) of SI; and, Dr. Robertino Mera (Evaluation Methods Specialist) of MSI. The methodology included (1) a team planning meeting between the team and USAID/India; (2) extensive desk review of all project-related documents; (3) interviews with key informants; (4) field visits to three of the seven APAC Phase III districts in Tamil Nadu, one BMGF/Avahan district, and one TANSACS district (control districts), where team members met select NGOs and community members and visited DAPCUs, ICTC, ART, and STI Clinics; (5) meetings with the SACS and TSU in Tamil Nadu and Kerala; (6) client briefings with SI and USAID through in-person meetings and teleconferences; and (7) presentations and discussion of findings with members of the APAC team, USAID and NACO. To enhance the quantitative rigor of the evaluation, the team undertook a separate epidemiological study to analyze health outcomes. The evaluation team conducted field visits to verify data collection and to inform subsequent findings qualitatively. Key informant interviews further enhanced the findings of the GH Tech APAC Documentation team and provided additional insights. 10 APAC Evaluation Relative Ranking Methodology A relative ranking methodology was used to quantify the findings of the GH Tech APAC documentation team to prioritize and better understand the elements of the NGO partnership model. The 10 elements used in the relative ranking, as seen in Table 3, were drawn directly from the GH Tech APAC documentation and their importance, (or lack thereof), was ranked by six groups: APAC staff, NGO representatives, APAC consultants, TANSACS staff, and TSU staff from both Kerala and Tamil Nadu.23 Each participant (N=61) first completed an individual ranking, then extensive group discussions were facilitated to create a group consensus of the final ranking. The evaluation team observed these sessions and noted the justifications given for the ranking of each item. The final ranking was determined by combining the results of the six groups. The evaluation team analysed the rankings using a non-parametric approach to establish whether the rankings were similar or different across groups.24 Study of Health Outcomes and Contributing Factors Study Design: Retrospective non-intervention comparison group with repeated measures. The objective of the study was to compare those health outcomes and contributing factors across districts in the state of Tamil Nadu that were subjected to HIV interventions. For the purpose of the study, the districts were divided into those that were exposed to APAC interventions (cases) and those that were not (controls). The unit of analysis was the individual district within Tamil Nadu. The seven districts subject to interventions throughout the life of the APAC project (1995 to 2011) were considered the basic exposed set, and the remaining districts (25) were considered as the basic not-exposed set. This definition of exposed sets established the basis for the statistical analysis. The design of this study nonetheless acknowledged and considered in the analysis the ebb and flow of the APAC interventions, from 10 districts in Phase I (1995–2002), to 23 in Phase II (2002–2007), to seven in Phase III (2007–2011). It is important to reiterate that the seven Phase III districts have been intervention sites since APAC began in 1995. The repeated-measures aspect of the study design allowed exposure to vary over time. Moreover, the statistical analysis also considered an alternate exposed set defined by the 10 districts that started the intervention in Phase I, were subject to intervention in Phase II, and are a superset of the seven basic sets. 23 APAC consultants are individuals regularly contracted to work on various APAC project elements, e.g., communications, training, etc. Many served as consultants for more than five years. 24 Non-parametric, or distribution-free tests, is a statistical term used to cover techniques that do not rely on data belonging to any particular distribution. See Appendix F for more details. Box 2: Propensity Score Propensity Score is defined as the probability of selecting a district given a set of baseline and pre-baseline covariates. These “covariates” are any available information about the districts—such as infant mortality rate, population size, literacy rate—that make the districts different. Once the probability of exposure (APAC intervention) is computed for each district, the propensity score is used as a statistical device to make districts comparable at baseline. 11 APAC Evaluation The baseline for the study, fixed at the years 2002–2003, was based upon data availability and quality. The study used the HIV Sentinel Surveillance Data for Tamil Nadu from 2000–2008 and programmatic PPTCT data from 2005–2011 as the two available data sets. To analyze behavioral indicators, the evaluation team used BSS data from 1997–2009. The most important issue when comparing districts at baseline (basic set exposed vs. not exposed) involved the initial criteria behind selection of the districts for intervention, which carried an unknown amount of selection bias, i.e. why they were chosen over other districts. To make the districts comparable, thereby allowing for proper comparison, a statistical device, known as a propensity score, was utilized (Box 2). The tertiles25 of propensity score adequately matched, on a variable ratio, all seven APAC districts (2,3,2 respectively) to non-APAC districts. Further analysis took this matching and weighting process into consideration. The repeated-measures portion of the study begins in 2002–2003 and ends in 2011. Indicators of impact (HIV and STI/VDRL prevalence), behaviors (condom use, STI knowledge, risk perception, stigma) and service utilization are the dependent variables subject to statistical analysis. Contributing factors are divided into two types: those which do not vary over time such as gender or occupation (“fixed” in statistical parlance) and those which do, such as age (“time-varying” in statistical parlance). Confounding variables are those that may be related both to exposure and to a given indicator, and can also be fixed, like human development index at baseline or time-varying, i.e., interventions over time by BMGF in some districts. Limitations Limitations of this study are those common to most retrospective studies and include the presence of unknown confounders at baseline (2002–2003) and the presence of time-dependent confounders—for example, other changes in population structure, etc.—throughout the duration of the repeated-measures study. Unknown baseline confounders will tend to produce selection bias and influence the comparability of districts. Unknown time-dependent confounders tend to bias the relationship between the prevalence rates at any single point in time and exposure. PPTCT data (used for additional evidence to cover the period from 2005–2011) could not be adjusted for changes of the structure of the population, since no subject-level information was available. Details of the statistical analysis can be found in Appendix G. 25 Statistical term used for dividing the distribution of propensity scores into thirds 12 APAC Evaluation FINDINGS NGO-GOVERNMENT PARTNERSHIP MODEL Building upon the findings of the GH Tech APAC documentation team, the evaluation team utilized a relative ranking methodology to ascertain the importance of specific elements, process, and systems that contributed toward improved ownership and the success of APAC at the state and national levels. Table 3 describes each element and shows the cumulative relative ranking given by the six groups during field level focus group discussions.26 Most important was ranked at “10” and least important at “1.” (See Appendix F for full details). Table 3: Result of Focus Group Discussions Cumulative relative rankings by six groups ITEMS DESCRIPTION FINAL RELATIVE RANK Leadership Dynamic leadership that engages and supports government, CSOs and donors in project planning, management, implementation and monitoring 10 Capacity building Emphasis on building capacity of various entities (individuals, NGOs, government, training resource centers) with focus on need and mentoring 9 Systems Strong systems for monitoring and managing project progress (e.g. MIS for technical and financial, transparent systems) 8 Budget Adequate, timely and flexible budget 7 Community involvement Culture of participation as seen in involvement of community being served in project design and implementation 6 Innovation Opportunities and encouragement for innovation of new ideas (e.g. SMS) 5 Project Management Committee Project Management Committee with equal partnership between government and NGO (VHS) including regular meeting of all partners (ARC, tripartite agreement) 4 Evidence Strong focus on generation of “evidence” and using it for on-going project design (e.g. BSS, other studies) 3 Sustainable/ replicable models Creating replicable and sustainable models (e.g. State Health Data Resource Centre) 2 Private sector Engagement/ participation of the private sector 1 26 Participants included: (1) APAC Staff, (2) APAC Consultants, (3) APAC NGOs, (4) TANSACS TSU, (5) TANSACS staff, and (5) Kerala TSU. 13 APAC Evaluation Leadership Three of the six groups chose leadership as the most important contributing element, and two groups identified it as the second most important item. Capacity building followed very closely behind; the only group offering different results were the APAC consultants who ranked community involvement and innovation as most important. Four of the six groups selected private sector involvement as the least important. Most participants felt that APAC's collaborations with government were of greater significance and had a greater potential for sustainability than initiatives undertaken in the private sector. Overall, informants believed that with strong project leadership, all other issues would be assured (capacity building, budget, staff, etc.). Without strong leadership, even a well-conceptualized program had greater potential for failure. When discussing the importance of leadership, numerous participants highlighted three key factors: 1) Consistent and knowledgeable leadership 2) Strong relationship-building skills 3) Ability to be accepted and considered “one of us” by all partners, especially the government Consistent and Knowledgeable Leadership Since 1995, APAC has been fortunate to have had strong and consistent leadership. Essentially, two individuals, Dr. Krishnamurti and Dr. Bimal Charles, have headed the project since its inception.27 Their in￾depth knowledge of technical aspects (HIV/AIDS, medical systems), as well as their familiarity and comfort with management, especially working with government systems, were considered integral to APAC’s success. Strong Relationship-Building Skills All stakeholders considered leadership personality to be a major factor in APAC’s sense of shared ownership. Both building and sustaining relationships were deemed critical to APAC’s harmonious working relationships with all partners. In addition, flexibility and adaptability were identified as important personality traits for the leader to possess. Examples were cited of how the leader’s model of adaptability and creativity in challenging situations “rippled” outward to many levels of the project. Acceptance by All Partners Acceptance of the leadership structure by all partners, especially the government, was identified by many as an important factor in APAC’s success in working with government agencies. While some mentioned the importance of a transparent and multi￾stakeholder selection process, most felt that having a leader accepted by the government was critical. “They are our heart,” explained the Kerala SACS Project Director (PD) when asked how she viewed the APAC Technical Support Unit. “Dr. Prashant Kumar [the team leader] is one of us.” The members of the Tamil Nadu TSU echoed this same philosophy. 27 Dr. B Chandramohan, IAS, served for one year. "WE ARE THE SONS AND DAUGHTERS OF APAC WEDDED TO TAMIL NADU STATE AIDS CONTROL SOCIETY." Tamil Nadu TSU Capacity Building Team Leader 14 APAC Evaluation A number of participants mentioned the existence of similarly strong leadership in the Tamil Nadu SACS (TANSACS) until 2010, after which the SACS experienced a period of rapid turnover—five project directors in a 16-month period. Participants reported that, since 2010, the effectiveness of TANSACS has significantly diminished. Each leadership change was said to cause disruption of services at the ground level. Many noted that strong leadership of both APAC and TANSACS contributed to a collective sense of ownership. Capacity Building and Systems All groups identified capacity building as a significant element of APAC’s success and ownership. All participants expressed that, as individuals, they personally had benefited from APAC’s capacity building efforts. Even the consultants said that they often learned more from APAC than from their work with other projects. APAC developed strong systems for project monitoring as well as for financial and administrative management. Training and building the capacity of all participants to use these systems helped APAC as well as their NGO partners and DAPCUs to withstand leadership and personnel changes. The effects of APAC capacity building have rippled beyond Tamil Nadu, not just in the implementation of targeted interventions or the use of training materials, but in terms of specific individuals’ increased capacity, built at APAC, who today participate at all levels (including NACO and SACS) throughout the country. In fact, the Project Director of TAI from 2004 to today worked with APAC from 1997–2004. She directly attributed TAI’s rapid scale-up of services to her previous experience working with APAC. Budget, Community Involvement, and Innovations Budget and community involvement ranked fourth and fifth, respectively. Budgets were described as adequate, flexible and dependable. Funding gaps, particularly for NGOs, tend to disrupt the ability to provide services. Sustained funding from APAC facilitated strong relationships with all field-level partners, including the community (FSW, MSM, IDU), who knew they could depend on the NGO to pay honorariums or conduct activities as promised. APAC’s involvement and commitment to the needs of the community was ranked as most important by the consultants; Kerala SACS deemed it the third most important element. It is worth noting that the APAC staff ranked community involvement as the second to last most important element. Innovations emerged as the sixth most important element. APAC was credited with creating a flexible and open environment that offered scope for creative ideas. Some of this “openness” was credited to leadership and some to the streamlined systems, which brought about opportunities to be creative preoccupied with daily operations. Project Management Committee The APAC Project Management Committee (Figure 5) while ranked seventh cumulatively, ranked second in priority for APAC staff, who stressed the role PMC played in creating a structure for shared ownership. To the other stakeholders, the seamless integration of APAC, TANSACS, and the state government was accepted as a given. Interestingly, while quarterly meetings were deemed important to shared ownership, the ability to pick up the telephone and discuss matters in a friendly manner was considered crucial. 15 APAC Evaluation The original selection of Voluntary Health Services, a well-established, experienced, and respected health NGO, may also have contributed to APAC shared ownership.28 Although APAC and TAI offices were located on the VHS hospital campus, both project directors reported little direct inputs or interference from VHS. Evidence APAC documentation stresses the generation and use of evidence for project management. In Phase III, nine percent of the budget specifically was used for research and studies. It is therefore interesting to note the ranking of evidence as only the eighth most important element of shared ownership. Participants never actually mentioned the use of evidence as important, despite the number of assessments performed for the government, or the Behavioural Surveillance Survey (BSS), conducted annually to inform on-going project implementation. In the field, the evaluation team noted a strong tendency to simply collect and submit data to the hierarchy, with limited analysis and use at lower levels. In fact, the evaluation team noted that at some DAPCU, data was presented in a manner that incorrectly demonstrated a significant decline in STI/HIV prevalence, creating an illusion of achievement. An important lesson to be drawn from this analysis is that a sound governance system, such as the Tripartite Agreement and the PMC, coupled with strong, acceptable leadership at all levels has played a significant role in project success and shared ownership. To conclude this set of findings, it was evident that a sense of ownership is shared across all APAC stakeholders. The majority of respondents rarely mentioned APAC alone, but referred to most activities in the personal plural form—i.e., “we” and “us.” HEALTH OUTCOMES Using an epidemiological study, this evaluation sought quantitative evidence of the impact of APAC interventions over time in the seven Phase III districts. The evaluation team compared the seven Phase III districts to HIV interventions in the remaining 25 districts over the same period, using a retrospective non￾intervention comparison group repeated measures study. 29 The objective of the APAC evaluation’s health outcome study was to analyze the impact of APAC interventions on select behavioral indicators (self-risk perception, STI knowledge, condom use and violence), service coverage (ICTC uptake), and prevalence (STI, HIV) among both the general population and MARPS in Tamil Nadu. Although APAC initiated services in 1995, 2002–2003 was set as the baseline (initial time point) because data prior to these years did not have adequate power and had wide confidence intervals that would have interfered with accurate analysis. 28 The role of VHS was considered part of the creation of the Project Management Committee and not included separately in the Relative Ranking. 29 Please refer to Appendix G for a full description. 16 APAC Evaluation Comparative Trends in HIV Prevalence: General Population and MARPS (HSS 2002–2008) among APAC and non-APAC districts of Tamil Nadu The evaluation team used a mixed-model logistic regression analysis to determine HIV prevalence trends among both the general population and MARPs with a start point of 2002–2003, fit to ‘propensity score’- adjusted HSS data. The confidence intervals of adjusted HIV-prevalence rates for APAC and non-APAC districts overlap for the year 2002, providing evidence that the propensity score method rendered the APAC and non-APAC districts comparable at baseline. Figure 9: Overall Adjusted HIV Prevalence Rates, General Population HSS Tamil Nadu, 2002-2008  Until 2007, the rate of decline in adjusted HIV prevalence was higher among the APAC districts when compared to non-APAC districts (statistically significant rate difference of 39.2%, 95% CI 15-48%). The overlapping curve for 2008 is evidence of a ripple effect, as APAC was providing technical assistance to TANSACS, or a catch-up effect in the 25 non-APAC districts. The ripple and catch-up effects could also be due to a variety of factors, including: NACP III launch in 2007; BMGF interventions in 12 districts beginning in 2004; and an overall rise in the level of TANSACS interventions across the state.  The analysis demonstrates a significant interaction between time and APAC-district interventions after adjusting for age, gender, education, and urban/rural status for HIV. 0.0% 0.5% 1.0% 1.5% 2.0% 2.5% 3.0% 3.5% Estimated adjusted HIV Prevalence 2002 2003 2004 2005 2006 2007 2008 Year Non-APAC APAC 25 districts Ripple Districts made comparable by propensity Adjusted rates from a multivariate analysis 7 districts 17 APAC Evaluation Figure 10: Adjusted HIV Prevalence Among MARPS, Tamil Nadu, 2002-2008  The HIV prevalence among MARPS shows findings similar to those for the general population, which adds substance to the propensity score method of making districts comparable in the absence of baselines and other coordinates.  The multivariate analysis shows significant decline in adjusted HIV prevalence among MARPS for APAC districts when compared to non-APAC districts. By 2007, a 30.4% difference in the APAC districts’ favor (95% CI 12.2% to 45.7%) emerged. A caveat: 2008 HSS data could not be used, as some factors were missing (e.g., patient level data). Thus, this finding is valid only until 2007. Additional Evidence – Comparative Trends in HIV Prevalence, APAC and Non APAC Districts, PPTCT Data, Tamil Nadu 2005-11 Because HSS data was not available beyond 2008, PPTCT district data from 2005–2011 (Figure 11) was used to establish whether trends observed in the HSS patient-level data were maintained over time, especially post￾2008. It is however important to note that less adjustment was possible due to the lack of patient￾level information in PPTCT data.30 30 Less adjustment for covariates (age, sex, education) and other time-dependent confounders was possible. 0% 5% 10% 15% 20% 25% 2002 2003 2004 2005 2006 2007 Estimated adjusted HIV Prevalence Year Non-APAC APAC Adjusted rates from a multivariate analysis 25 districts 7 districts 18 APAC Evaluation Figure 11: HIV Prevalence PPTCT, Tamil Nadu, 2005-2011  The statistical model displayed a significant decline of overall ANC HIV-prevalence adjusted rates over time, after taking into account both the wide variance in earlier years and increased PPTCT testing in later years.31  A simultaneous but significantly steeper decline was observed among APAC districts when compared to non-APAC districts. In the non-APAC districts, prevalence dropped by 62.7% (95% CI 53.5% to 80.5%) between 2005 and 2011, while among the APAC districts, prevalence dropped by 78.6% (95% CI 36.5% to 91.7%) during the same period of observation. Figuratively speaking, the prevalence rates were lower for APAC districts, but there was no significant difference at the endpoint, i.e., in 2011.  The uptake in PPTCT testing, performed in hospital-based antenatal clinics, is a prime example of HSS as secondary to an active HIV prevention program. Credit goes to NACO and SACS for this achievement. 31 PPTCT data was analyzed using a generalized linear model weighted by the inverse of the variance and adjusted for the (centered) increased testing. Variance is an indicator for the dispersion of data, the wider the variance, the lower the confidence of that data set. Antenatal Clinic (ANC) patients represent both general population and MARPS. ANC prevalence is the primary marker used for classification of districts as “at risk” or not (see footnote #15 in the Introduction section). 0.34 0.32 0.24 0.18 0.14 0.11 0.10 0.28 0.21 0.18 0.13 0.12 0.10 0.07 19453 19264 26065 29380 32122 28892 32086 0 5000 10000 15000 20000 25000 30000 35000 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 2005 2006 2007 2008 2009 2010 2011 HIV Prevalence Year Non APAC APAC #Tested 7 districts 19 APAC Evaluation Trends in STI (VDRL) Prevalence: General Population and MARPS Figure 12: Adjusted STI Prevalence (VDRL), Gen. Pop. HSS, Tamil Nadu 2002–2008  Analysis of general population STI (VDRL) prevalence, commencing in 2002 (Figure 12), showed a statistically significant decline, with some degree of stabilization and variability noted in 2005.32  An early and significant decline was observed among APAC districts and by the year 2004, the STI/VDRL prevalence rate was 49% lower among APAC districts (95% CI 20% to 71%) when compared to non-APAC districts. By 2008, a distinct tendency toward converging rates emerged, with no difference between APAC and non-APAC districts. 32 A Poisson statistical mixed model was used for analysis. The spikes noted in the chart for the APAC curve possibly are due to epidemiological trends, commonly seen with syphilis, resulting from “cluster outbreaks” secondary to the transmission dynamics of syphilis and other STIs. For further details, refer to A. De Schryver and A. Meheus, “Epidemiology of Sexually Transmitted Diseases: the Global Picture,” Bull World Health Organ, 1990; 68(5): 639–654. 0.0% 0.5% 1.0% 1.5% 2.0% 2.5% 3.0% 3.5% Estimated adjusted STI Prevalence 2002 2003 2004 2005 2006 2007 2008 Year Non-APAC APAC 25 districts 7 districts Adjusted rates from a multivariate analysis 20 APAC Evaluation Figure 13: Adjusted STI Prevalence (VDRL) in MARPS, HSS, Tamil Nadu, 2002-2008  Among MARPS, the statistical model showed no significant difference in STI prevalence between comparison districts in 2002; however, a wide confidence interval existed (Figure 13). By 2004, a significant difference between APAC and non-APAC districts was noted, with the APAC districts 51.1% lower (95% CI 17.8% to 74%) than non-APAC districts. It is important to note that this difference disappeared over time; nonetheless, when considering the complete period from 2002–2008, the rate among APAC districts was consistently and significantly lower than the rates among non-APAC districts (see linear trend line in Figure 13 above). Trends in Behavioral Indicators in Tamil Nadu Behavioral indicators were taken from the BSS studies conducted collaboratively by TANSACS and APAC from 1997 to 2009. Due to the BSS sampling methodology and data availability, it only was possible to compare APAC to non-APAC districts for the year 2009.33 The four behavioral indicators used for analysis were: knowledge of STI prevention, condom use during last sex act, risk perception (self), and exposure to violence (as a proxy/surrogate marker for stigma). Given the limitations of the BSS data, using conditional logistic regression, a comparison was made between the proportion of all Tamil Nadu responders to three behavioral indicators for two points in time: 1997–1999 and 2007–2008. 33 BSS data collected from the years 1997–2008 provide information for the entire state of Tamil Nadu. In 2009, however, data was collected separately for sites representing the seven APAC districts. Thus, comparison was possible for this one point in time. 0% 2% 4% 6% 8% 10% 12% 2002 2003 2004 2005 2006 2007 Estimated adjusted STI Prevalence Year Non-APAC APAC Linear (Non-APAC) Linear (APAC) 25 districts 7 districts Adjusted rates from a multivariate analysis 21 APAC Evaluation Figure 14: Trends in Select BSS Indicators in Tamil Nadu, 1997-2008 • Analysis showed that, among all subjects surveyed in Tamil Nadu, those who stated that STIs can be prevented grew by 5.7% (95% CI from 3.3% to 8.0%) in 1997–1999 when compared to years 2007– 2008. • Analysis showed that, among all subjects surveyed in Tamil Nadu (Figure 14), those who stated that STIs could be prevented grew significantly by 5.7% (95% CI from 3.3% to 8.0%) in 1997-99 when compared to years 2006-2008. • Reported condom use among all subjects surveyed in Tamil Nadu had also increased by 6.0% (95% CI1.9% to 9.9%) in 1997–1999 when compared to years 2007–2008. During the same period, risk perception increased significantly—by 34.8% (95% CI 18.6 to 48.1%). • The proxy indicator for stigma—violence—could not be used, as it was not consistently measured in the two time points (1997–1999 and 2007–2008) and, due to the lack of adequate sample size, did not have enough power for analysis. 93% 12% 88% 98% 19% 94% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% STI Knowledge Risk Perception Condom Use Proportion Responding Behavioral Surveillance Survey Tamil Nadu 1997-2008 1997-1999 2006-2008 22 APAC Evaluation • Using the point estimates for 2009 (Figure 15), it was noted that two behavioral indicators, risk perception and reports of violence (used as a surrogate marker for stigma and discrimination), revealed significant differences between APAC and non-APAC districts that year. Among non￾APAC districts, the proportion who reported moderate- to high-risk perception was 6.5% (95% CI 4.6 to 8.4%), compared to APAC districts with 12.5% (95% CI 9.2 to 15.7%) reporting moderate to high-risk perception, marking a significant difference. • Reported violence was also lower in APAC districts (18.7%, 95% CI 14.9 to 22.5%) compared to non-APAC districts (27.0%, 95% CI 23.5 to 30.3%), a significant difference. • The proportion of respondents with STI knowledge in non-APAC districts was 98.6% (95% CI 97.7 to 99.5%); among APAC districts it was just slightly higher at 99.3% (95% CI 98.4% to 99.9%), a non-significant difference. • Similarly, reported condom use presented a non-significant difference, that nonetheless figuratively favored the APAC districts (95.5%, 95% CI 93.5-97.5%) when compared to non-APAC districts (91.8%, 95%CI 89.7 to 94.0%). Trends in Service Coverage in Tamil Nadu, 2002-08 Integrated Counseling and Testing Center (ICTC) data served as a proxy of service coverage. As Figure 16 demonstrates, ICTC testing increased, massively and significantly, for both APAC and non-APAC districts. This is consistent with the national program’s focus to increase HIV counseling and testing across the country. No significant difference was noted in APAC districts, but the declining trend revealed in 2010 should be carefully observed. 27.0% 18.7% 0% 5% 10% 15% 20% 25% 30% 35% Non-APAC Districts APAC Districts Percent responding 2009 Tamil Nadu Districts Stigma: Proportion Reporting Violence 6.5% 12.5% 0% 5% 10% 15% 20% 25% 30% 35% Non-APAC Districts APAC Districts Percent responding 2009 Tamil Nadu Districts Risk Perception: Proportion Reporting Moderate to High Risk Figure 15: Comparison of Select BSS Indicators, APAC and non-APAC Districts, BSS, Tamil Nadu, 2009 23 APAC Evaluation Findings clearly indicate that APAC has had a significant impact on health outcomes, both for the general population as well as MARPS, in Tamil Nadu (Figure 16). The converging trends noted in 2007–2008 point to the ripple effect APAC has had throughout the state, as well as to the work of others agencies, including the BMGF and TANSACS. 0 10 20 30 40 50 60 70 80 90 100 2002 2003 2004 2005 2006 2007 2008 2009 2010 Average Number Tested in Thousands Year Non APAC Districts APAC Districts 25 districts 7 districts Figure 16: Trends in ICTC Testing, MIS, Tamil Nadu, 2002-2010 24 APAC Evaluation APAC TECHNICAL ASSISTANCE Background A key component of successful technical assistance is the willingness of partners to receive guidance and assistance from the provider. All respondents spoke proudly of APAC’s work and felt that they shared a role in its success. In meetings with government agencies, e.g., SACS, State Health Society, and NACO, Tamil Nadu’s work stood as a shining example of targeted interventions’ effectiveness in slowing the HIV epidemic. Little distinction or credit was given to APAC per se—rather, as is consistent with the spirit of shared ownership, most saw it as their own accomplishment. Approximately 53% of APAC’s Phase III expenditure went to technical assistance, with 24% of total technical assistance going directly to the government. To better understand APAC technical assistance, Figure 17 illustrates the structure and relationship of key institutions at the national, state, and district levels, which play an active role in HIV/AIDS prevention, care, and support efforts. As USAID moves toward focusing on Health Systems Strengthening (HSS), understanding potential areas for convergence and technical assistance will be crucial.34 Areas where convergence has occurred in Tamil Nadu are colored light gray (State Training Resource Center STRC, Antenatal Clinics ANC). Areas with potential for greater convergence in the future are a pale blue (ART, ICTC, and STI). As shown below, APAC/VHS, a transparent, public-private partnership, has, by virtue of its structure and work, served as a bridge between two parallel entities, SACS and the State Health Department. The transition for HIV/AIDS programming from a freestanding, vertical health-delivery system toward a more integrated service-delivery model will be an important component of HSS. Integration or convergence of services, considered both more cost effective and sustainable, will prove critical to India’s successful provision of effective, public health care to its growing population of 1.2 billion. 34 The term “convergence” derives from the verb to converge, meaning to tend to come together at one point. Within the HSS lexicon convergence refers to the “coming together” of parallel health structures, such as the HIV/AIDS prevention efforts under NACO, with the National Rural Health Mission, into a single functioning operation, rather than parallel delivery systems. Historically, family planning efforts have operated as parallel systems and recently have been incorporated or converged into “Reproductive and Child Health Programs.” 25 APAC Evaluation Figure 17: Relationship between Health Agencies/Programs at National, State and District levels Ministry of Health and Family Welfare Department of AIDS Control Department of Health Department of Family Welfare National AIDS Control Organization (NACO) National Rural Health Mission (NRHM) National AIDS Control Program (TRGs, NHCRC) Reproductive and Child Health Program (RCH) State AIDS Control Society (SACS) State Health Society (SHS) District Health Society (DHS) District AIDS Prevention and Control Unit (DAPCU) TSU SIMU TIs ART Centers STIs (counselor) ICTCs STRC ANC Clinics TNHSP (SHDRC, SHCC, SHRRC) STATE NATIONAL DISTRICT VHS APAC 26 APAC Evaluation STATE NATIONAL PRIVATE SECTOR  HELLO PLUS  NAKSHATRA CLINICS  IRT-PMCH (HIV TRAINING) NGOs  TI TRAININGS  ESRM  EXPOSURE VISITS  POSITIVE NETWORKS  DATA TRIANGULATION  FOLK MEDIA STRATEGY  NHCRC  TRG- NACP IV  CONSULTANTS - 3  EXPOSURE AND SHARING  SUPPORT TO SACS (MHC, DAPCU etc.)  TSU - TN & KER  BSS  OVC TRUST  TNHSP - SHCC, SHDRC, SHRRC etc.  BRANDING OF ART CENTERS As identified by the GH Tech APAC documentation team, APAC has utilized multiple approaches to technical assistance: 1. Evidence generation/research 2. Strategy development 3. Experts / consultants 4. Experience sharing 5. Training and mentorship 6. Systems and procedures Given the large number of APAC technical assistance activities, Figure 18 identifies graphically the most significant technical assistance endeavors of APAC Phase III (full details available in Appendix H, Table of Technical Assistance Phase III). To highlight and demonstrate the effect of the above-mentioned technical assistance approaches, Figures 19– 21 illustrate the manners in which three distinct APAC technical assistance activities—the TSU, the Tamil TNHSP and MHC—have employed numerous technical assistance approaches. The potential implications of each for NACP-IV and HSS are also discussed. Figure 18: APAC Technical Assistance Activities 27 APAC Evaluation Technical Support Unit TSU employed five TA approaches in Tamil Nadu (Figure 19).35 The partnership created by the TSU with the State Training Resource Center (STRC) is noteworthy. In other states, this relationship often has been problematic; in Tamil Nadu, convergence between NACO/SACS and the State Health Department has occurred with the STRC offering training to workers in both divisions.36 The TSU experience sharing and review meeting, as well as a similar meeting for the project directors of SACS, were highly appreciated by all participants, who mentioned how much they learned from sharing and discussing their innovations and challenges. The successful transition of all APAC TIs to the government by March 2011 is further indication of the successful TA provided by the TSU. NACO has already decided that the TSU will continue for NACP-IV, but as NGO capacity continues to grow and the technical competence of the SACS and DAPCU increases, the TSU’s long-term role will need to be determined. 35 The Kerala TSU used similar approaches but this discussion will be focused on the Tamil Nadu TSU. 36 Uttar Pradesh has faced many challenges with finding a suitable STRC. In Maharashtra the TSU— (HIV/AIDS Prevention and Control Project for Maharashtra (USAID), or AVERT) —has faced many challenges in working with the Maharashtra SACS. EVIDENCE GENERATION  LINELISTING/ VALIDATION OF TI SITES  MARPS MAPPING FOR TAMIL NADU TSU Technical Support Unit EXPERTS/ CONSULTANTS  NEED BASED STUDIES, ASSESSMENTS ETC. TRAINING/ MENTORSHIP  PARTNERSHIP WITH STRC  TI NGO PARTNERS  STATE OFFICIALS EXPERIENCE SHARING  TSU EXPERIENCE SHARING MEETING  PD SACS MEETING STRATEGY  TRANSFER OF TI EXPERIENCE APAC Figure 19: APAC Technical Support Unit 28 APAC Evaluation Tamil Nadu Health Strengthening Project (TNHSP) Technical assistance to the TNHSP offers another example of how four TA approaches were utilized. APAC deputed two consultants to TNHSP, one of whom proved instrumental in guiding the creation of the State Health Communication Resource Center. The International Conference on HSS (ICONHSS) was also noted as an excellent example of international cross sharing and learning. The Infection Control and Biomedical Waste Management Studies have resulted in standardization of procedures in hospitals throughout Tamil Nadu. The use of specially designed receptacles for waste and of gloves for all blood-related procedures was systematically observed by the evaluation team during field visits. In addition, APAC leveraged a total of US$182,000 from GOI for implementing the biomedical waste program. The TNSHP, a commendable example of a TA partnership, offers great potential for future HSS endeavors. TAMIL NADU HEALTH SYSTEMS PROJECT (TNHSP) TRAINING/ MENTORSHIP • ICONHSS • REG. TRAINING CENTERS￾INFECTION CONTROL & BIO-MEDICAL WASTE MANAGEMENT • DATA MANAGEMENT SYSTEMS & PROCEDURES • STATE HEALTH COMMUNICATION CENTRE • STATE HEALTH RESEARCH RESOURCE CENTER LEVERAGED FUNDS FROM GOI EVIDENCE GENERATION • INFECTION CONTROL & BIO WASTE MANAGEMENT STUDIES EXPERTS/ CONSULTANTS • DATA RESOURCE MANAGER (2) Figure 20: APAC Tamil Nadu Health Systems Project 29 APAC Evaluation Master Health Check —“Full Body Check Ups”—An Innovative Collaboration to Reach MARPS In Tamil Nadu, fewer than 30 percent of MARPs have accessed government STI services. Barriers include stigma and discrimination, lack of comprehensive services, distance, and timing. The MHC, which FSW and MSM referred to as “full body checks,” was designed to increase service uptake by MARPs by systematically addressing the identified barriers. The MHC, initiated at the suggestion of TANSACS, incorporated screenings for STI, HIV, and cervical cancer, in addition to basic general investigations (blood sugar, hemoglobin), within a free, general health check. TANSACS contributed a total of US$850,000 toward this initiative. Working collaboratively, Tamil Nadu’s three TI partners, TANSACS, APAC, and TAI:  Upgraded 145 STI government clinics (infrastructure, equipment, supplies, staff support)  Mobilized 110 TI NGOs in 32 districts to line list all MARPS and provided each with a unique ID  Developed operational guidelines  Conducted training for a total of 600 medical officers, nurses, lab technicians, counselors, and NGO functionaries, focusing on both technical management of STI as well as stigma and discrimination reduction  Provided clinic mentorship by 30 trained doctors to ensure quality of diagnosis and treatment TANSACS & DEPT. Of HEALTH CONCEPT of MASTER HEALTH CHECK (MHC) APAC TA SUPPORT EVIDENCE GENERATION  ASSESSMENT OF 145 FACILITIES TRAINING/ MENTORSHIP  DEVELOPED OPERATIONAL GUIDELINES  TRAINED 600 HEALTH CARE WORKERS  CLINIC MENTORSHIP BY 30 TRAINED DOCS STRATEGY DEV. REACHED OUT TO  HARD TO REACH MARPS  NEVER TESTED & LOST TO FOLLOW UP NATIONAL RURAL HEALTH MISSION (NRHM) TAMIL NADU AIDS INITIATIVE (TAI) Figure 21: APAC Master Health Check 30 APAC Evaluation The results were noteworthy:  Between September 2009 and March 2011, 47,178 MARPs (28,095 FSW and 19,083 MSM), representing approximately 49% of the total estimated MARPs population, participated in MHC.  Some TI NGOs reported 100% uptake, including MARPS who previously had never accessed services or HIV testing.  FSW and MSM interviewed by the evaluation team reported satisfaction with the care provided, singling out for special praise: - Quality of care, especially being physically touched and seen as a whole person by the doctor - Convenience of having all services (including blood testing) available at one time - Need for “full body checks” to be done every year MHC’s potential as a critical strategy for NACP IV should be considered and explored. It also offers a brilliant example of true integration between HIV/AIDS programming and HSS. These three examples describe the impact that APAC technical assistance has had at multiple levels. Further evidence of the impact can be seen in ICTC uptake (Figure 16, “Health Outcomes”), increased coverage of MARPs throughout the state, improved spending by both the Tamil Nadu and Kerala SACS, and the creation of sustainable systems and institutions at both the state and national level. In the field, the evaluation team observed strengthened capacity of many local institutions (NGOs, service delivery points) and personnel to deliver HIV prevention programs throughout Tamil Nadu. The only TA approach where the evaluation team observed that work remained to be done, was in the support of evidence building and its use for program development and/or policy change. While the generation of evidence and its use for on-going project design (e.g. BSS, other studies) has been identified as an important technical assistance area, it rated seventh out of ten in the relative ranking exercise, slightly above sustainable models and private sector involvement in perceived importance to project success and sense of shared ownership. Despite numerous successful examples of evidence building, use of data for decision making is lacking, as observed at certain service-delivery points and government institutions. APAC has provided TA to DAPCUs, TSU, TANSACS, NACO, and TNHSP in using data for decision-making. Some examples include mappings, community prevalence studies for STIs, data triangulation, etc. However, the importance of using data, though highlighted in many documents, was not reflected in the field. In a number of instances, evaluation team members observed that denominators were not reported or taken into account when presenting trends (e.g., coverage, prevalence, testing), thereby creating a false sense of achievement.37 The evaluation team did not observe the data, which had been analyzed and interpreted at a higher level (e.g., NACO, SACS), was reverted to the field for local use. In NACP-IV, technical assistance for analyzing and using data for program planning and management will be important for all levels of program personnel and service providers. 37 This was seen in Karur, Perambalur and Villupuram by the team, where HIV positivity rates among tested were presented as HIV prevalence rates which showed great decline over the years. “FOR THE FIRST TIME THE DOCTOR LOOKED AT ME ALL OVER AND NOT JUST AT MY GENITALS.” - MSM, Chennai 31 APAC Evaluation APAC Success Stories Arokya Anjalai – VOICE-BASED SMS FOR SEX WORKERS: Throughout India, sex work is moving from brothels and streets to private homes, making HIV prevention outreach increasingly difficult. APAC conducted a pilot program with 118 FSW who agreed to receive 10 telephone voice messages. The topics included HIV testing, STI symptoms, cervical cancer, condom use and regular medical check-ups. Preliminary results indicate the potential of voice-based SMS for raising awareness, increasing knowledge and providing reminders about testing and medical exams. HELLO + Toll Free Helpline: Today in India, cell phone usage is almost universal. Capitalizing on this, APAC created a partnership with TATA Business Service Solutions to provide free hotline HIV/AIDS information and counseling services. Originally designed for PLHIV, a broader utilization, especially by a younger, non-infected population, has been noted. The service offers counseling and information from 7am to 11pm seven days a week. It assures confidentiality, is available in four languages and callers can choose to speak to a male or female counselor. Launched in 2009, the service receives an average of 500 calls per month. In April, 2011 Hello + was transferred to TANSACS; APAC continues to provide technical assistance, including training of staff and monitoring for quality assurance. SELF-DEFENSE FOR SEX WORKERS: Sex workers are often subjected to violence and find themselves alone and unable to handle physical assaults. "Safe Plus" was initiated by FSWs who requested self-defense training to protect themselves from violence. Two hundred sex workers from five Tamil Nadu and two Puducherry districts were trained in Taekwondo, a form of self-defense karate. After the training sex workers felt more confident to deal with difficult clients and they also reported a more positive image of themselves. Since these new skills acquisition, there has been a reduction in reported violence. RAISING ESTEEM & INCOME: Shakthi + “An Effective Cover for Home-Based Sex Workers": Livelihood support projects for sex workers have historically been seen as a preferable alternative to the demands of the sex trade Shakthi + was designed for HIV + FSW with this objective. It further aimed to promote psycho-social well-being (self-esteem) and to provide a set of skills required for an alternative regular income (vocational training). By the end of the project, 16 of the 32 experimental group members had left the sex trade. Increases in self esteem, nutrition, CD4 count and ART adherence were also noted in the experimental group versus the control group. For Home-Based Sex Workers a livelihood project provides a credible justification for daily activities and for their incomes. A number of livelihood activities are done in groups creating a venue for discussion on health related subjects, which is important for Home-Based sex workers who are hard to reach through standard outreach interventions. With the increased income a decrease in the number of clients was also reported. The potential of livelihood support as an additional HIV prevention strategy should be further explored in India's rapidly changing sex work environment. OVC TRUST, Tamil Nadu Trust for Children Affected with AIDS, TNTCAA: By 2010, a total of 79,719 HIV+ children were registered for Pre-ART in Tamil Nadu. Children affected by HIV in India face many challenges including stigma, school drop-out, entry into child labor, becoming orphans, etc. As a step toward improving the quality of life of these children, TANSACS created the OVC Trust in 2007 with a government corpus fund of USD $1 million. A public appeal for applications placed in the newspaper invited single and double orphans to apply for support from the trust. Using interest only, funds have been distributed to 1,500 children for education, nutrition and medical support. 32 APAC Evaluation APAC Success Stories, cont. CATCH THEM YOUNG - "RUSE" - Using Older Clients as Bait: Evidence shows that young and new entrants to sex work are most at risk for HIV infection. Many APAC-targeted interventions (TI) achieved full coverage of estimated sex workers, but had little scope to find new and young entrants to the business. A pilot initiative aimed to identify older male clients known to prefer younger women. Targeted-Intervention FSW Peer Educators met with the clients and requested assistance in reaching these women. From June– December 2011, a total of 148 new FSW were identified by the Kanyakumari district NGO. 54% of whom were between 18–30 years of age (24% less than 25 years), and 126 were home- or mobile-based, thereby demonstrating the potential of recruiting older clients to identify women at risk. NACO FOLK MEDIA CAMPAIGN: Folk Media is an effective way to reach marginalized populations and vulnerable groups. Due to its high social acceptance, it is also an effective tool for community mobilization. NACO's Folk Media campaign aimed to increase and sustain HIV/AIDS awareness among MARPS and to generate demand for services. Trained folk media teams delivered standardized messages at events around the country. APAC support to the roll-out of the National Folk Media Campaign included deputation of a communications consultant to NACO, support in conceptualisation, designing, implementing, monitoring and evaluation as well as organisation of three national level workshops for planning and experience sharing. A total of 105 resource persons were trained as Trainers of Trainers in 47 folk forms. Across the country there were 41,844 performances using 166 different scripts and songs, all created for these events. Taking an additional step forward, APAC is collaborating with NACO to establish a National HIV/AIDS Communication Resource Centre as a repository for all HIV/AIDS communication material and as a center for material development and communication research. HEALTH SYSTEMS STRENGTHENING - PUSH - Project for Upgrading Safety in Healthcare: Training on Biomedical Waste Management: Infection control and the management of biomedical waste are critical ingredients of a successful HIV/AIDS prevention program. Health care workers, numbering in the millions in India, all require sound knowledge of these issues. APAC, working collaboratively with TANSACS and the Tamil Nadu Health Systems Project (TNHSP) have partnered with a private global medical technology and training company to train Tamil Nadu health care workers on biomedical waste management. A baseline assessment of 11 regional health training centers was conducted and training modules and tools developed. In 2012, the goal is to train a total of 40,000 health care workers in the state on Biomedical Waste Management. 33 APAC Evaluation CONCLUSIONS 1. The evaluation study demonstrates that, for the period 2002–2007, APAC had a measurable, positive effect on health outcomes in the seven districts of Tamil Nadu where APAC has intervened since 1995. A secondary data source confirms a similar trend from 2005–2010. 2. Declining HIV and STI/VDRL prevalence in the general population and in MARPS, increases in service uptake, and improved behavioral indicators, over time, were noted in all 32 districts of Tamil Nadu. 3. APAC’s efforts, and its technical assistance to TANSACS, caused a ripple effect that also contributed to the declining trends. Described by the APAC evaluation team as a “rising tide” with the ability to “lift all boats,” APAC’s work, combined with that of other donors and a proactive Tamil Nadu government, have resulted in declines in prevalence and uptake of services, beyond those considered part of a natural disease progression. 4. Strong leadership and effective collaboration with the Tamil Nadu government were considered critical to APAC success and sense of shared project ownership. 5. The effect of APAC technical assistance at the NGO, state, and national levels was evident in health outcomes, successful transitioning of all NGO targeted interventions to the government, and in the replication of state agencies, such as the Communication Resource Center, at the national level. 6. APAC has been successful at evidence generation, but additional work is required to improve the accurate use of data by program personnel at all levels, from SACS to DAPCU to NGO personnel. 7. APAC and HIV/AIDS programming in Tamil Nadu has also had an effect on HSS, as demonstrated by the uptake of PPTCT ANC testing and through successful initiatives, such as the master health checks, which strengthened government STI clinics. 8. APAC allocations for HIV prevention in children were limited. Collaboration and support to TANSACS initiatives like the OVC Trust is an example of a successful APAC intervention and serves as a potential model for further work with children at risk. CONTRIBUTING FACTORS Identifying the specific factors that have contributed to each of the overall findings and conclusions is not an exact science. Using both quantitative as well as extensive qualitative analysis, the evaluation team determined the following to be the most significant contributing factors: Long-term investment: APAC operated uninterrupted in Tamil Nadu from 1995–2012. Statistical analysis showed a significant interaction between time and the seven APAC districts where interventions had taken place for the duration of the project. Strong and effective collaboration with the government was mentioned by most participants as “key” to APAC success and a sense of shared ownership in Tamil Nadu. The original tripartite agreement and the creation of a project management committee, headed by the Health Secretary of Tamil Nadu, were seen as important factors to the successful working relationship. 34 APAC Evaluation Strong Leadership, for both APAC and of TANSACS, emerged as the most important element in the relative ranking. Strong leadership, clearly described by participants, included specific knowledge, competencies, and personality traits. Acceptance of this leadership by the government was also identified as an important factor. Effective Technical Assistance by APAC to all levels—national, state, NGOs, and the private sector— was noted, with particular reference to the following areas: • Generation of evidence: APAC worked at the field level throughout the project life and many generated studies were based on locally identified needs and ground realities. The BSS, begun in 1997, served as a model throughout the country. Data triangulation exercises, facilities assessments, etc., reflected the needs of health care providers and communities seeking services. • Technical Support Units (TSU) in Tamil Nadu and Kerala were credited with having a positive effect on NGO-targeted interventions in all districts of both states. • DAPCU initiated services in 2008 in all “A” category districts; APAC’s financial and administrative management systems were critical to their success in Tamil Nadu. • Health Systems Strengthening, particularly support given by APAC to the TNHSP, was considered important to strengthening many service delivery areas, particularly for infection control and biomedical waste management, two critical areas for HIV prevention. • Experience sharing and the provision of opportunities for experiential learning through demonstration projects and exposure visits were valued by all participants, from SACS project directors to MSM and FSW community members. • Innovations designed to address ground realities (e.g., reaching younger sex workers, PLHIV, etc.) were important to many government officials, who have limited opportunities for creativity and innovation. They appreciated APAC efforts to identify, implement, and evaluate new approaches and strategies and acknowledged the government’s inability to create an environment where innovation was possible. 35 APAC Evaluation RECOMMENDATIONS To USAID: Focus investments in “Health Systems Innovations” • Strengthen generation, analysis, and use of evidence at all levels—from policy makers to peer educators—and support management information systems for health The generation, analysis, and use of data for program planning and management requires continued support from USAID. Although agencies like NACO have made great strides, moving from multiple paper-based registers to computers, a large gap remains in health care and NGO workers’ capacity to effectively analyze and use data. India’s rapidly developing IT industry should be encouraged and utilized to address the needs of the health care sector. All levels of health care and NGO staff should receive basic data-analysis training. Creativity in the private sector (NGO and industry) should be encouraged to generate innovative solutions. • Encourage field-level efforts and innovations by NGOs through a challenge grant mechanism One of APAC’s great strengths has been its ongoing field-level activities. The majority of program planning has been based on observations of field-level realities and needs. Effective public health programs require this type of direct exposure and policy should not be made without a valid field perspective. In India, one of the country’s greatest assets is the work of NGOs. USAID/India, a pioneer in working with NGOs, has built NGO capacity as partners in public health.38 To continue this field-level work and innovation, the creation of a challenge grant mechanism, which would enable and capacitate local NGOs to serve as innovation labs for health, should be explored. • Support opportunities for national and international cross sharing and experiential learning Experiential learning, (i.e., learning that is on-site, hands-on, learning-by-doing, and that strives to draw meaning from a direct practice or experience) an important method of adult learning, allows experienced adults to learn from others’ experiences.39 USAID should support these activities (demonstration projects, learning sites, exposure visits, etc.) in addition to national and international thematic or cohort meetings to encourage cross sharing and innovation. Again, a challenge grant mechanism could be utilized at such meetings to encourage South-to-South studies, testing, and potential adaptation and scale-up of innovative models. • Don’t forget the children In Tamil Nadu alone there are nearly 80,000 HIV+ children registered for ART services. While APAC has done commendable work to lower HIV prevalence, relatively little has been done for children and families affected by HIV/AIDS. Children affected (and infected) by AIDS (CABA) should also be considered as a population at risk. With treatment, many of these children have the potential to live long and productive lives, yet they face innumerable challenges, including stigma and discrimination at schools and in employment. USAID, in their eminent role as champions for children’s health and rights, should encourage new approaches to the challenges faced by orphans and vulnerable children (OVC). 38 USAID began work in the late 1980s through Private Voluntary Organizations for Health (PVOH I & 2). 39 USAID/India: SAMARTH Activity - Documentation of Lessons Learned; November 2011, pg 13 36 APAC Evaluation To National AIDS Control Organization (NACO): • State AIDS Control Societies (SACS) should be given greater autonomy For the past 20 years, NACO has provided strong and commendable leadership to India’s fight against HIV/AIDS. As NACO and the epidemic have matured, early vertical, stand-alone operations have been decentralized to SACS and, in 2008, to District AIDS Prevention and Control Units (DAPCU) in high￾prevalence sites. Greater autonomy for SACS operations in their own states will be very important to the success of NACP-IV. Collaboration with the SHS will be critical for greater integration of HIV/AIDS services into the existing health structure. As the epidemic changes and varies from state to state, it will also be crucial that SACS be positioned to analyze and respond to local needs and changing epidemic environments.40 • Develop a management information system that allows for greater flow of data down to program managers NACO has developed a detailed and rich Computerized Management Information System. However, data continues primarily to flow upward; field-level workers possess only limited understanding and use of the data collected. To improve program planning and management, this same data should be available and analyzed locally by SACS, TSU, and TI NGO teams. • Scale-up Master Health Check The success of the master health check in Tamil Nadu should be replicated in other high-prevalence states and carefully analyzed to determine its relevance across the country for reaching out to MARPS as well as other marginalized populations (migrants, etc.). The potential for this single intervention to raise testing levels for HIV, STI/VDRL, diabetes, anemia, cancer, etc., has significance not only for HIV prevention, but for larger HSS efforts, as well as for the health and well-being of the population. To State AIDS Control Society (SACS): • Health Systems Research – to understand “changing nature” of disease locally Health Systems Research can be understood as program level studies designed to inform and improve health systems. These studies tend to be quasi-experimental designs with comparison groups and can be either prospective or retrospective. The best designs tend to be prospective in nature and require the same forms utilized in day-to-day work in such a way that regular systematic data collection and analysis is part of the study design. This alleviates the common criticism of research by health workers, which is the burden of new forms. Training requirements are minimal and can serve as a means for reinforcing existing data collection systems. The expected study results should be geared toward understanding a particular health system 40 Examples of potentially changing epidemic environments observed by the evaluation team included cultural and geographic differences (religion, access); a move from brothel and street based prostitution to home-based prostitution; increasing rates among MSM; differences between migrant populations; enhanced PPTCT resulting in decreased transmission to children; increased life span of PLHIV and greater risk of transmission (through blood contamination or sex); uptake of ART (discordant couples transmission rates); drugs and needle use/availability. 37 APAC Evaluation question and the results should be utilized by the same health workers to propose a solution to a problem or deficiency in health system delivery. For HIV-prevention efforts in a state like Tamil Nadu, health systems research could begin to address some of the challenges faced in the movement of sex workers from the street to private homes and to address the decreasing trend in ICTC testing observed in 2010. A similar type of study would have been ideal for master health checks. • Build capacity of all levels to analyze and use data for program planning As mentioned above, data is not analyzed or used for local program planning. This problem stems partly from limited capacity and training of DAPCU and NGO program managers for this purpose. This should be an area of focus for NACP-IV, as it should contribute to enhanced performance as well as to greater sustainability of program interventions. • Strengthen DAPCUs to address a complete prevention-to-care continuum” The DAPCU has primarily focused on the prevention of HIV, specifically on ICTC and the targeted interventions. As studies continue to demonstrate the importance of ART adherence for decreasing viral load, a broader prevention to care continuum will need to be included. Given the variability of districts in India, it will also be important for each DAPCU to fully understand their local needs and infection scenario, both from a prevention as well as a treatment perspective. • Keep focus on “hard to reach”—seek innovative approaches Sex workers and MSM/TG continue to carry higher instances of HIV prevalence than their general population counterparts do. As sex work moves from visible sites, such as the streets and brothels, to home and mobile-based operations it remains even more important to reach them with knowledge and services. Because the current strategies for targeted interventions may soon be ineffective, it will be crucial for the SACS to continue to focus on these populations at risk through creative and innovative approaches. • Encourage sharing within state to find solutions APAC-supported international, national, and state sharing meetings were reported by all participants to be invaluable learning opportunities. Exposure visits to “demonstration centers” also proved instrumental in helping participants, including SACS project directors, TSU staff and peer educators. Facilitating this type of sharing within a state can be both cost effective and useful for finding solutions to local problems. 38 APAC Evaluation APPENDIX A. SCOPE OF WORK USAID/INDIA Office of Program Support DELIVERY ORDER STATEMENT OF WORK Evaluation of AIDS Prevention and Control Project (APAC) PROGRAM PROJECT INFORMATION PROGRAM PROJECT TITLE AIDS Prevention and Control Project (APAC) START-END DATES September 30, 1992 to March 31, 2012 BUDGET $47.25 Million PROGRAM/PROJECT DESCRIPTION The AIDS Prevention and Control (APAC) project, a bilateral program launched in September 1992, implements HIV/AIDS prevention, care and treatment programs in Tamil Nadu, Puducherry and Kerala in partnership with the National AIDS Control Organization (NACO) and local State AIDS Control Societies (SACS). The project started HIV/AIDS interventions in 1995, and is currently in its third and last phase of interventions. The management of the project is by the Voluntary Health Services (VHS) – a leading NGO providing community-based health care service to rural and poor populations in selected districts of Tamil Nadu. The project implements activities under the overall direction of the Project Management Committee (PMC) with the Principal Secretary Health and Family Welfare, Government of Tamil Nadu as the Chairperson and represented by NACO, USAID, VHS, Tamil Nadu SACS and Puducherry SACS. Over the past fifteen years, this project has been implemented in three phases, each of which has distinct and specific strategies and approaches. FIRST PHASE The first phase of the project was departure from the conventional awareness generation programs. For the first time, this project introduced the concept of targeted interventions and accentuated its focus on implementing the HIV/AIDS prevention and control activities among female sex workers and long-distance truck drivers. The project’s core technical strategies included NGO capacity building, community mobilization, and behavior change communication (including peer education), condom social marketing, and STI management. Besides, the project also supported several pioneering research initiatives including mapping of Most-At-Risk-Population (MARPs), behavioral surveillance surveys and facility assessments to support evidences and for monitoring project/program outcomes. Some of its most significant activities included the development and positioning of a strong network of local non-governmental organizations, 39 APAC Evaluation developing a panel of consultants and institutions for capacity building, strengthening a critical mass of private health care providers trained in sexually transmitted infections management, and innovations in condom social marketing. An external mid-term review of the project held in May 2000 observed that the project was achieving its results and recommended the extension of the project beyond March 31, 2002. SECOND PHASE In its second phase (April 2002-March 2007), the goal of the project was to reduce sexual transmission of HIV/AIDS, improve care and support to affected populations, and expand outreach to a larger audience including MSM and migrants. Significant initiatives in the second phase included: integrating voluntary counseling and testing services in NGO programs to provide easy access to testing for high-risk groups; home-based care services; campaigns to address stigma and discrimination in health care and work place settings; and behavioral and facility assessments. The project had been successful in meeting its objectives and established effective and replicable systems for capacity building and monitoring. Besides, the project also successfully engaged the private sector and faith based organizations in HIV/AIDS prevention and care programs, institutionalized the peer education component in collaboration with SACS and other stakeholders. The February 2006 evaluation of APAC observed that the project has been successful in achieving its benchmarks. The evaluation report recommended expanding the comprehensive prevention, care and support services, and shifting the project’s focus from implementation to providing strategic technical support at the state and national level. THIRD PHASE In its third phase (April 2007-March 2012), the project supports comprehensive prevention to care continuum programs in seven high-prevalence districts of Tamil Nadu, and provides technical assistance to NACO and SACS to scale up and strengthen HIV/AIDS programs. The goal of the project during the third phase is to establish sustainable network models in HIV/AIDS prevention, care and support, and enhance the state’s response to HIV/AIDS prevention and control. The third phase focused on a phased transitioning of project activities; engaging private sector In prevention and care programs; supporting robust systems to address social, economic and health issues of PLHA’s and OVCs; and demonstrating an integrated health system strengthening approach. During this phase, the APAC project continues to establish its interventions and activities based on clear evidence and tested approaches, while continuing to support interventions that are cost-effective and replicable. The project has also developed local capabilities by building the capacity of the state AIDS societies, state health services, municipal agencies, indigenous NGOs and institutions to ensure sustainability of the programs. Above all, the project has focused on developing appropriate strategies to transition its activities in a phased manner so that the project activities can be sustained beyond March 2012. Significant Achievements The project receives assistance through both unilateral and bilateral mechanisms. This bilateral assistance is primarily focused on supporting prevention programs and providing technical assistance to SACS and NACO. The unilateral assistance is focused on care and support, strategic information, research, communication. supporting innovations and documentation of project activities. The project’s efforts have enabled the state to support evidence-based interventions, address gaps in prevention and care programs, and contain the spread of the HIV epidemic in the state. 40 APAC Evaluation The APAC project has played a critical role in engaging civil society in HIV/AIDS programs and established proven systems and strategies for behavior change, service delivery and targeted evaluations, several of which have been adopted by SACS/NACO. The project in coordination with other agencies has played a pivotal role in controlling the HIV epidemic in Tamil Nadu. The prevalence among antenatal women declined steadily from 1.13% (in 2001) to 0.25% (in 2008). The project is recognized for its cost-effective approaches, close-collaboration with government, robust systems, technical assistance, innovations, and evidence-based interventions. Knowledge, risk perception and behavioral indicators (such as condom use, STI treatment) have also improved over the years and have remained consistently high. The project has also successfully transitioned several of its activities to the state programs and provided technical assistance to the state AIDS control societies on a range of program aspects including strategic planning, mainstreaming, strategic information and roll-out of programs for orphans and vulnerable children. STATEMENT OF WORK EVALUATION PURPOSE The purpose of this final performance evaluation of the third phase of the project is to analyze rigorously the following: 1. NGO-Government partnership model: This evaluation will analyze the effects of this innovative partnership model on the overall state HIV/AIDS prevention and care programs and ascertain the specific elements, processes and systems that facilitated improved ownership of the project at the state and national level. 2. Health outcomes and contributing factors: The evaluation will assess select outcome and impact indicators (to be decided jointly by the evaluation team and the India Mission) among most-at￾risk-populations and people living with HIV/AIDS. The evaluation will focus on behavioral indicators (such as knowledge, risk-perception, service utilization), as well as assess service coverage and prevalence (HIV, STI) among the target populations. In order to have a comprehensive picture of the impact of interventions, an analysis of the overall state performance as well as select districts (both APAC as well others) in Tamil Nadu will be undertaken and the contributing factors that led to improved health outcomes and impact will be assessed. 3. Technical Assistance: Finally, the evaluation team will assess the process and effects of technical assistance support provided by the project to various stakeholders including State AIDS Control Societies, Tamil Nadu Health Systems Project, NACO and ascertain relevance of these TA areas for NACP-IV. Ideally, this evaluation will identify and assess41 the most significant technical assistance approaches and interventions that involve a target group and relevant comparison groups (that have not received USAID assistance) in order to yield substantive findings, conclusions, and recommendations. 41 SI received written documentation approving this proposed modification to the Statement of Work 41 APAC Evaluation This evaluation will build upon the findings of the documentation activity that is scheduled to start on Sept. 20, 2011 by GHTECH. This separate activity will review and document the strategies, experience and key lessons of the APAC project in: a) providing technical assistance to different stakeholders; b) scaling up prevention to care continuum services in concentrated epidemic settings; c) transitioning of project activities to the National AIDS Control Organization (NACO), Tamil Nadu State AIDS Control Society (TANSACS); and d) using strategic information for program management and policy change. This assignment will also analyze and document the key factors that contributed to the successful coordination between the APAC project and the national, state governments. It will identify, document and disseminate best practices and success stories (of both the APAC project and TANSACS) in HIV/AIDS prevention, care and treatment programs. Based on the current schedule, the SI evaluation team will be able to review this GH Tech report in order to build upon and expand their findings. INTENDED USES OR OTHER AUDIENCES FOR THE EVALUATION There are clear lessons learned that can be drawn from APAC’s evolving approach, evidenced by the strong goodwill experienced among program stakeholders at the state and national level. This evaluation will analyze the outcome of the third phase technical assistance so that 1) NACO and the State Aids Control Societies (SACS) can apply them to other states in India, and 2) USAID/India can apply them to future HIV/AIDS program designs. The primary intended users of this evaluation are NACO, TANSACS, and USAID/India, particularly the HO and Mission management. NACO will be particularly interested in findings and recommendations concerning the change management aspects of this program and how it could apply to other states in India. USAID/India will use this evaluation to inform new designs that increasingly focus on health systems strengthening, including institutional capacity building, human resources for health, and health-related demonstration models that can be widely replicated. The secondary audience of the evaluation is local institutions, other donors, and perhaps other USAID Missions worldwide. EVALUATION QUESTIONS This evaluation will answer the following questions, in priority order: • To what extent did project interventions effect health outcomes and impact for most-at-risk populations and people living with HIV/AIDS, including men, women, and children? • What lessons can be drawn for future designs from the project’s governance system especially its project management systems? • How effective and efficient has the project’s change management role been with respect to system strengthening approach and technical assistance in improving the capacities of local institutions (SACS, DAPCUs, NGOs, civil society, and private sector) to deliver AIDS prevention and control programs in Tamil Nadu? What lessons can be drawn for future designs by GOI and USAID/India? • How effective has the project been in transitioning its activities to other partners such as donors, government, NGOs and its impact on sustainability? 42 APAC Evaluation • How effective were the approaches to scale-up/replication of APAC introduced innovations at the state and national level? • How effective was the project in supporting evidence-building and its utilization for program development and/or policy change? TECHNICAL REQUIREMENTS FOR EVALUATION DATA COLLECTION AND ANALYSIS METHODS Data collection methodologies will be discussed with, and approved by, the USAID/India Health Office team prior to the start of the assignment. Considering that APAC is a complex program involving state and local partners, multiple activities, locations and target groups that spans 15 years, USAID anticipates a mixed methods approach. This approach will incorporate standard qualitative methods such as key informant interviews, group interviews, focus groups, structured observations, or illustrative case studies (that don’t replicate GH Tech’s report), as well as quantitative analysis of existing primary and secondary data sources such as the CMIS, HSS, BSS, and DLHS. If required, sample surveys could be introduced that respond to key evaluation questions. We would also like to explore introducing matched case-control evaluation design to increase the overall rigor of this performance evaluation. The evaluation will address the key questions stated above, while articulating the framework (or the “essential ingredients”) that in combination led to desired outcomes over the life of the project in various APAC￾supported districts. It is envisioned that this elaborated framework would then be used as a guide to inform future replication strategies. We anticipate that the specific methodology will be discussed at length and refined during the evaluation planning phase and the TPM, but for illustrative purposes we propose the following approach: Sample districts could be selected in a ratio of 1:4 (1 case APAC district for 4 control non-APAC districts), ~30% or 10 districts of Tamil Nadu can be selected for evaluation. Districts can be matched for size, population, sex ratio, rural-urban ratio, literacy rate, or other relevant characteristics. In addition to looking at HIV/STI prevalence rates, this multi-layered analysis should include a matrix of indicators (or proxy indicators) that best isolate and examine which elements of the intervention led to positive outcomes, specifically institutional capacity building, health systems strengthening and partnership indicators. Taking into account time/resource constraints, the depth and breadth of this matrix will depend on the availability and quality of data, relevance to the key evaluation questions stated above, and relevance to potential project replication. The evaluation team is also expected to propose measures for minimizing type I and type II errors and biases. Desk review of documents: USAID/India will provide the team with all relevant country and project specific documents including proposals, evaluation reports and other relevant documents for conducting this desk review. The evaluation team is expected to collect and collate relevant international documents, reports, and data, and all team members are expected to review these documents in preparation for the team planning meeting. This desk review will help to organize the materials for the external evaluation team analysis and review of progress to date, and facilitate their utilization during the field work, analysis and report writing stages. Data sources: Data sources that the team will be expected to utilize, review and analyze include the project design documents, project proposal, annual work plans, M&E data, state annual action plan, 43 APAC Evaluation sentinel surveillance reports, state data triangulation report, behavioral surveillance surveys, NGO evaluation reports, and other project-related documents and reports. Additional relevant documents related to HIV programming in India may be utilized as supporting documents, as well as relevant international standards. COMPOSITION, TECHNICAL QUALIFICATIONS AND EXPERIENCE REQUIREMENTS OF THE EVALUATION TEAM USAID seeks a four-member evaluation team (two international and two local members) comprised of a Team Leader/Senior Technical HIV/AIDS Expert/Health and HIV/AIDS Analyst, an Evaluation Methods Specialist, a Senior Public Health Specialist, and a Public Health Specialist/HIV/AIDS Analyst. All team members must have extensive HIV/AIDS program management, technical or implementation experience, familiarity with USAID’s objectives, approaches, and operations, and prior evaluation/assessment experience. The team will have experience in planning interventions for most-at-risk populations (MARPs) and needs to have collective expertise among them in programming for specific vulnerable populations such as female sex workers, MSM and IDU. All team members must have technical expertise including understanding of the HIV context in concentrated epidemics, with global knowledge on issues related to HIV prevention and/or care, support and treatment issues. Prior experience in India will be an asset; knowledge of the Indian National AIDS Program is desirable, though not essential. Collectively, the team must have experience in evaluating HIV/AIDS programs worldwide. In addition, individual team members should have the technical qualifications and required experience identified for the specific position below: 1. Team Leader/Senior Technical (HIV/AIDS) Expert/Health and HIV/AIDS Analyst (international): This Team Leader/Senior Technical (HIV/AIDS) Expert/Health and HIV/AIDS Analyst in the field of international HIV/AIDS prevention, care and treatment has an excellent understanding of global HIV/AIDS strategies and knowledge of the Indian epidemic and programs. Specifically, s/he should have an excellent understanding of the drivers of HIV infection in concentrated epidemics, with prior work experience in designing, monitoring and evaluating HIV/AIDS programs for specific most-at-risk populations. S/he should have knowledge and experience on technical assistance support strategies for strengthening institutions such as government, private sector, civil society and should have experience in designing, managing and evaluating large scale technical assistance projects. Additionally, s/he should have proven experience in leading and managing large-scale evaluations of various HIV/AIDS programs throughout the world. S/he should be familiar with the functioning of large donor funded programs in India. The person must have the ability to lead a diverse team of technical and management experts, and to interface with various stakeholders ranging from governmental to non-government organizations and donors, beneficiaries, etc. A minimum of 15 years of experience in the design, management and evaluation of HIV/AIDS prevention and control programs is required (LOE up to 34 days). 2. Evaluation Methods Specialist (international): This expert will have deep knowledge of evaluation methodologies and their practical applications. A minimum of ten years of experience in strategic planning, surveillance, operations research, and/or monitoring and evaluation of global and national HIV/AIDS programs is required. S/he should have strong experience in conducting comparison group analysis, understanding of secondary literature reviews and developing sampling methodologies. Experience in presenting research publication will be an added advantage (LOE up to 30 days). 3. Senior Public Health Specialist (local): This Senior Public Health Specialist should have an extensive and strong experience in designing, implementation, and evaluation of HIV/AIDS prevention, care 44 APAC Evaluation and treatment projects. The person should be considered as an expert in integrated public health programming. Additionally, a good understanding of health systems strengthening, particularly in institutional capacity development of government health systems, is desirable. Knowledge and experience including in-depth understanding of HIV prevention, care, support and treatment issues including OVC would be an added advantage. A minimum of 15 years of experience in the design and management of public health programs, including health systems strengthening, is required. Having knowledge and understanding of the Tamil Nadu State HIV/AIDS program and government systems would be an added advantage (LOE up to 30 days). 4. Public Health Specialist/Health and HIV/AIDS Analyst (local): The Public Health Specialist/Health and HIV/AIDS Analyst should be an expert in HIV/AIDS prevention programs focused at concentrated epidemics. The specialist should have experience with the country specific HIV/AIDS prevention and control strategy and its approaches, specifically with prior work experience in designing, monitoring and evaluating HIV/AIDS programs for specific most-at-risk populations. Expertise in health program management specifically in managing innovations in health sector, institutional arrangements, governance, strategy planning and organizational development will be essential. A minimum of 10 years of experience in health management is required. A minimum of seven years of experience in the design and management of HIV/AIDS prevention and control programs is required. Knowledge of health systems strengthening in the Indian context will be an asset. (LOE up to 30 days). EVALUATION MANAGEMENT ROLES AND RESPONSIBILITIES The Health Evaluation Specialist in conjunction with the Evaluation COTR, the APAC AOTR and Activity Managers, other key Health Office team members and the Contracting Officer (CO), will provide overall direction to the assessment team.  The Contractor will be responsible for obtaining visas and country clearances for travel for consultants.  The Contractor will be responsible for coordinating and facilitating assessment-related TPM, field trips, interviews, and meetings in conjunction with USAID and the APAC Project.  The Contractor will be responsible for submitting an illustrative budget for all estimated costs incurred in carrying out this review. The proposed cost may include, but not be limited to: (1) international and in-country travel; (2) lodging; (3) M&IE; (4) in-country transportation; and (5) other office supplies and logistical support services (i.e., laptop, communication costs, etc.) as needed.  The Contractor will be responsible for in-country logistics including transportation, accommodations, communications, office support, etc. 45 APAC Evaluation SCHEDULE The duration of the evaluation will be for five weeks starting from early January to mid-February 2012. The evaluation team is expected to provide a schedule (in a tabular form) defining when specific steps in the evaluation process will occur and when deliverables are due. Team Planning Meeting (TPM): A two-day team planning meeting will be held by the evaluation team at an offsite location before the evaluation begins. This will be facilitated by the evaluation team leader, and will provide the Mission with an opportunity to present the purpose, expectations, and agenda of the assignment. The evaluators shall come prepared with a draft set of tools and guidelines and a preliminary itinerary for the proposed evaluations. In addition, the TPM will also:  Clarify team members’ roles and responsibilities  Establish the timeline, share experiences and firm up the evaluation methodology  Finalize the methodology guidelines including tools and questionnaires to be used by the team.  Discuss and finalize evaluation questions based on the SOW Site Visits and Interviews: Conduct a thorough review of the Project through site visits and interviews. Interviewees will include key members from all stakeholder groups, including National AIDS Control Organization, Tamil Nadu State AIDS Control Society, Puducherry State AIDS Control Society, Kerala State AIDS Control Society, Voluntary Health Services, other donors and partners in HIV/AIDS Prevention and Control, USAID and beneficiaries. Interview questionnaire will be prepared in advance and finalized during the TPM. Site visits will be planned taking into consideration factors like geographical diversity, representation of various beneficiary groups, and scale of interventions. The team will evaluate state and district level periodic reports to verify results for the indicators. REPORTS AND DELIVERABLES Draft Work Plan and Pre-Departure Briefings: The evaluation team will develop a draft work plan prior to departure from Washington, D.C. The team will meet with USAID/India and other relevant contractor staff for at least three working days prior to departure for the field. Mid-Point Review/Briefing: The evaluation team will provide a mid-point briefing to the USAID/India team, including evaluation and technical members, to clarify any outstanding queries that may have emerged since the initiation of the evaluation process. If this is not feasible based on scheduled field work, the Team Leader will submit weekly progress reports to the COTR via email by OOB beginning of the next week. Oral Presentation: The evaluation team will provide an oral briefing on its findings and recommendations to relevant staff in the field, to GOI and state government officials, and to USAID staff at the conclusion of the visits to the various implementing partners. The evaluation team will be required to debrief the Mission Director and Deputy Mission Director separately on the observations and recommendations. Reports: The evaluation will be required to submit the following reports: 1. Draft Report: The evaluation team will present a draft report of its findings and recommendations to the USAID/India’s APAC AOTR and Activity Managers, Health Evaluation Specialist and 46 APAC Evaluation Evaluation COTR, and other key Health and Program Support Office staff one week after return to the United States. 2. Final Report: The final report, with executive summary and in electronic form, must be received by the Evaluation COTR, Health Evaluation Specialist and USAID/India APAC AOTR within seven working days after receiving the final comments on the draft evaluation report from USAID/India team. The final report should also be submitted to PPC/CDIE/DI. The final report should include an executive summary of no more than three pages, a main report with conclusions and recommendations not to exceed 20 to 30 pages, a copy of this scope of work, evaluation questionnaires used to collect information on each of the program components, and lists of persons and organizations contacted. EVALUATION LOE AND BUDGET LEVEL OF EFFORT LABOR CATEGORY LEVEL MAXIMUM LOE Senior Technical Expert (HIV) /Team Leader 1 38 Evaluation Methods Specialist 1 35 Senior Public Health Specialist 1 30 Public Health Specialist 1 30 Independent government estimate (budget) for this evaluation. 47 APAC Evaluation APPENDIX B. PERSONS CONTACTED NEW DELHI National Aids Control Organization Anant B. Sahu, Programme Officer (M&E) Aradhana Johri, Additional Secretary, IAS Priyanka Bhatt, Monitoring and Evaluation Officer Ugra Mohan Jha, Monitoring and Evaluation Officer USAID Alka Gupta, PS/FSO Amy Wielkoszewski, Program Office Anagha Khot, Project Management Specialist, HO Anand Rudra, Project Management Specialist, HO Arvind Kumar, Project Management Specialist, HO Beth A. Hain, Program Office (PPPs) Beth Haytmanek, Health Development Officer, HO Chandan Samal, Project Development Specialist, RFMO Charushila Lal, M&E Specialist, PS Ekta Saroha, Project Management Specialist, HO Elizabeth Callender, Program Officer, PS Eric Schaffer, Controller Jamie Browder, Deputy Director, HO Jamie Browder, Deputy Director, HO Jamie Browder, Deputy Director, HO Jennifer Gratez, Program Office Jyothi Krishnamoorthy, Chief Accountant, RFMO Karen Kelly, PEPFAR Coordinator Kerry Pelzman, Director, HO Manju Ranjan Seth, Program Management Assistant, HO Neeta Rao, Project Management Specialist, HO Ramesh Babu, Project Management Specialist, HO Sampath Kumar, Project Management Specialist, HO Sampath Kumar, Project Management Specialist, HO Sangeeta Kaul, Project Management Specialist, HO Sathi Anand, Lead Secretary, HO Shanti Conly, Technical Consultant, USAID/Washington Sheena Chhabra , Team Leader, Health System D, HO Sheena Chhabra , Team Leader, HO Uma Ravikumar, Program Management Assistant, HO Vandana Vats, AD A Specialist, ROAA Vijay Paulraj, Project Management Specialist, HO Walter Pakala, Financial Analyst, RFMO 48 APAC Evaluation KERALA ICTC Jyothi Rly Station L. Usha, Counselor Shiji R., Lab Technician ICTC PPTCT Women and Child Hospital V.R. Bindu, Counselor S. Sudhakaran, Lab Technician A. S. Divya, Lab Technician IRD Thiruvananthapuram 20 ORWs+PEs+HRGs C. Balachandran, Project Director P. S. Sibi, Accountant Preethi Peter, ANM Sarika, M&E Assistant Sathikumari, Project Manager U. J. Prasanth, Counselor KSACS Dr. Dennis, Jt. Dir. TI Dr. K. Shylaja, Project Director Socio Economic Unit Foundation 40 ORWs+PEs+HRGs Divya Chandran, Nurse Gopakumar, M&E Assistant John Cyril, Project Manager P. Prameela, Counselor Technical Support Unit Anantha Sivan.K, Program Officer- TI Bardot S.V, Program Officer- TI Binu George, Program Officer- TI Dr. Babu Chekku, Data Analyst Dr.M.Prasanna Kumar, Team Leader TSU & SP George Kutty.G, Finance Manager Rajeenald.T.D, Team Leader-Capacity Building Sudheer, K.B Program Officer- TI Vimal Ravi, Team Leader – TI TAMIL NADU ACCT Antony Prakash, M&E Assistant Arul Selvi, ORW 49 APAC Evaluation Community Members (25 PEs & KPs) Ganeshan, Proejct Coordinator Mallika, ORW Saravanan, ORW Sathiya, ORW Shanthi, Counsellor Xavierduai, Accountant APAC Dr. A. Edwin Sam, Program Manager, Research Dr. Beena Thomas, Consultant Dr. Bimal Charles, Project Director, APAC/VHS Ebenezer C. Luke, Program Manager, Care and Support Francis Porsingula, Program Manager Gayatri Mishra Oleti, Director, Strategic Planning and Partnership Jacob C. Varghese, Consultant J. Benjamin Franklin, Consultant Joevalan Niranjan, Program Officer, DAPCU Dr. Md. Esoof, Consultant M. Padmavathi, Consultant Nithyanandan, TNHSP Data Coordinator P.M. Rajaram, Director, Finance and Contracts Dr. M. Satchithanandavalan, Consultant Dr. N. Usman, Consultant P. Bobby, Consultant Dr. P. Elangovan, Consultant Dr. P. Mahalingam, Senior Program Manager, Care and Treatment Dr. Ravi Raj William, Consultant Saravanan, TNHSP Data Analyst S. Shoba, Health Systems Strengthening S. Lokabiraman, Program Manager, MIS Dr. T. Ilan Chezhian, Director Program Operations APAC – Link Worker Scheme S. Santhya, Senior Program Manager APAC/TANSACS Joe Valan Niranjan, Programme Officer ARM A. Bakthavatchalam, Director J.D. Bharrathipriyaa, Outreach Worker K. Arul Prakash, Outreach Worker K. Mano, Outreach Worker L. Arun Emmannuel, Monitoring and Evaluation P. Raja, Outreach Worker R. Kodeeswaran, Outreach Worker R. Rajendra Prasad, Outreach Worker S. Sridevi, Accountant 50 APAC Evaluation ART Amutha, Data Manager Dr Siva, ART MO Dr Vanita, ART MO Jayalakshmi, Staff Nurse Poonkodi, Community Care Coordinator Prema, Phamacist Sharmila, Counsellor Thomas Victor, Counsellor Uma Devi, Counsellor ART Centre, Chengalpattu Government Medical College Anitha Sagaga Mary, Pharmacist Baby, District Supervisor Christina Roseline, Lab Technician Lilly, ART Counsellor Revathy, Nurse Rajasekar, Data Entry Operator ART Centre, Karur Dr. Ananth Kumar, Medical Officer Gnasekaran, Counselor P. Merci, Counselor Karpagam, Counselor Parameshwaran, Data Manager Sangthi, Staff Nurse Sarvana Kumari, Lab Tech P. Thanmanam, Legal Aid Clinic ART Centre, Villipuram Medical College Dr. V. Karthikeyan, ART Medical Officer ART Salem Medical College Dr. Chandramohan, Medical Officer J. Thennarasi, Staff Nurse Kamalakannan, Lab Technician Kavitha, Lab Technician Raja, Counsellor Sam Osborn, Counsellor Sudha, Counsellor Thangaraj, Lab Technician Vijayarani, Counsellor Community Care Centre Chengalpattu Sumathy, Community Care Counsellor 51 APAC Evaluation Community Care Centre Salem Usha, Outreach CSR A. Pazhania Pillai, Project Coordinator DAPCU Dr. Nithya Nanthini, District Program Manager Pushpalatha, District Supervisor V Nehru, M&E Assistant A M Navin Prakash, Account Assistant K Sivakumar, Admin. Assistant K Selvaraj, Office Assistant DAPCU, Kancheepuram A. Shanmuragam, Administrative Assistant M. Baby, District Supervisor R. Komala, Finance Assistant T. Sass Priya, Monitoring and Evaluation Assistant Dr. V. Nithya Nandhini, District Program Manager Vijaya Kumar, District Manager DAPCU, Perambalur Arul Kumar, DS Asaithambi, Support Staff Dr. Elangovan, DPM Punitha, Administrative Assistant Revathi, Finance Assistant Selvaraj, RRC DM Vinothkanna, M&E Assistant DAPCU, Salem A. Tamil Arasi, Administrative Assistant Barath Dhasan, Monitoring and Evaluation Specialist Gandhimathi, Project Coordinator (CABA) Dr. L. Arunachalam, District Program Manager Poongathai, Office Assistant R. Kalai Selvi, Finance Assistant R. Selvam, District Supervisor DAPCU, Villipuram A. Elumalvi, District Manager, Red Ribbon Club Dr. N. Dayanidhi, District Program Manager K. Elango, Office Assistant/DAPCU K. Sambendan, Assistant Director M. Mahinderan, Monitoring and Evaluation P. Murugesan, Finance Assistant Prema, District Supervisor 52 APAC Evaluation Gramium – TI NGO C.Senthilkumaran, Project Manager Community Members (PEs & KPs) 19 Durai samy, ORW Ezhavarasi, ORW M.Purusothaman, Counsellor Mari Pitchai, ORW P. Narayanan, Project Director R.Marimuthu, ORW Saranya, M&E Assistant HEADS – TI NGO Community Members (18 PEs & KPs) K. Suresh, M&E M. Dinakaran , ORW M. Manikandan, ORW P. A. Kathan, ORW P. Nitya, ORW R. S. Jayakumar, Accountant Ram Priya B., Project Manager T. Karthik, Counselor Health and Family Welfare Development Girija Vaidyanathan IAS, TN Health Secretary HQ Hospital Karur Dr. P. Raghunath, Hospital Superintendent Dr. Vijaya Pushpa, DC TNHSP HQ Hospital, Perambalur Dr. Thulsiram, Dental Surgeon Dr. V. Sekar, Resident Medical Officer ICTC Centre, Kancheepuram General Hospital P. Sundarajan, ICTC Counsellor ICTC (Gen and PPCTC) Abhiraman, Counsellor (Gen) Jayanthi Lab Tech, (ICTC - Gen) Karumbaiyeram Counsellor, (PPTCT) Selvaraj, Chief Lab Technician Srinivasan, Lab Tech (ICTC - PPTCT) ICTC/PPTCT Karur A Amutha, Lab Tech A Mohana Sunderam, Counselor O Malathi, Counselor (Leave Vacancy) R Kalpana, Counselor (STI) S Abirami Thendral, Lab Tech S Revathi, Counselor (PPTCT) 53 APAC Evaluation ICTC Salem Medical College Anitha, Counsellor Jagadesh Raj, Counsellor Loganathan, Lab Technician Kamala, Counsellor Perana, Counsellor ICTC Villipuram Medical College A. Ramakrishnan, Lab Technician P. Reula Samathana Priya, ICTC Counsellor S. Tamilarasi, Lab Technician V. Ashok Kumar, Counsellor ICWO Sarvana Kumar, Project Vedhachalam, Outreach Worker IFPEC S. Esther Shanta, Outreach Worker B. Baby, Outreach Worker Indian Community Welfare Organisation Anbu, Outreach Worker Jeyamala, Outreach Worker Krishna Veni, Outreach Worker and ANM Meenalosani, Counsellor Naga Lakshmi, Outreach Worker Peer educators (23) Poulina, Outreach Worker and ANM Pushpa, Outreach Worker Thulasi, M&E Officer V. Shaleema, Director Lead N. Radha, Director Pache Trust P. Manoharan, Director PDI, Salem Community members (15) G. Indira, M&E Officer I. Ambalavanam, Chief Functionary P. Kalaimani, Staff Nurse Dr. R. Rajaveni, Medical Officer Peer Educators (9) Rebecca, Project Officer (TI) Sheela Vincent, Programme Manager T. Manimekalai, Outreach Worker 54 APAC Evaluation PPTCT Salem Medical College Manigandan, Counsellor REEDA Alagarsamy, Project Coordinator Arivuamuthan, M&E Assistant Bakyaraj, ORW Community Members (11 PEs & KPs) Jeeva, Counsellor Marimaran, ORW Sasi, ORW Valamathi, Accountant Russ Foundation Berlin Jose, Director Sahodaran, Chennai Anandaraj, M&E Officer Dhanabagyam, Counsellor Maya, Community Mobilizer Rambha, Accountant Rupakala, ORW Saikumar, ORW Shoba, Community Mobilizer Soundarya, Peer Educator Swetha, Project Manager Thangaraj, Counsellor Vinodh Kumar, ORW 2 Peer Educators 7 Community Members SATUT S. Raja, Monitoring and Evaluation Seval K. Govindaraju, Director Social Welfare Association for Men A. Kalpana, Accountant E. Bhaskar, Outreach Worker G. Reema, Counsellor M. Yashoda, M&E Officer N. Nadhagopal, Outreach Worker N. Durairaj, Outreach Worker N. Babu, Outreach Worker P. Velu, Project Manager STI Centre, Kancheepuram General Hospital S. Ashok Kumar, STI Counsellor 55 APAC Evaluation STI Centre, Villipuram Medical College G. Meenashi, STI Counsellor STI Clinic Dr. Sangeetha, STI Medical Officer Malathi, Counsellor STI SWAM, Chengalpattu P. Velu, Project Manager SWTVT B. Sudharasan, Program Manager VRDP Vimalanathan, District Resource Person - Program V. Prasanna, District Resource Person - Training TAI, Salem Jayaganesh, Project Officer TANSACS Albert, I/C Information Centre Palani, Consultant Mainstreaming Pughazh, GIPA Coordinator Dr. Valan, Regional Coordinator CST TNSACS Dhivya Ramalingam, Dy. Dir. IEC, TNSACS Dr. A S Valan, RC CST, NACO-TNSACS Dr. S Vijayalakshmi, Dy. Dir. STI, TNSACS K. Sambandam, Asst. Dir. DAPCU K. Shanmugam, Consultant Youth Affairs, TNSACS Nageshwaran, IEC CAPACS Pugazh, GIPA Coordinator, TNSACS Raghuram, AD ICTC S. Ashokan, Dy. Dir CCC, TNSACS T. Sampath, Consultant Vol Blood Donation, TNSACS Velmani, JD Finance Vijaya Priya, AD Procurement Technical Support Unit Balasubramanian V., Project Officer - TI C. Swaminathan, Project Officer - TI Eswara Murthy C., PO – Monitoring & Evaluation Indhu Sivakumar, Team Leader - CB Kabilan Annaderos., Project Officer - TI K. Pramod, Head TSU & Team Leader – TI K. Vani, Project Officer - TI 56 APAC Evaluation Rani Nallathambi, Program Officer – TI S. Janakiraman, Manager Finance S. Swaminathan, Project Officer – TI Ravi Kumar .S, Project Officer – TI Rebecca Hannah, Project Officer – TI TNHSP Dr. M. Kamatchi, Expert Advisor We Care Rani Kumar, District Resource Person 57 APAC Evaluation APPENDIX C. RESEARCH STUDIES SL. NO. TITLE PUBLISHED YEAR AGENCIES/ORGANIZATION FORMAT 1 BSS Wave -1 Tamil Nadu 1996 AIMS Research (P) Ltd. Hard & Soft 72 Study on the Quality and Availability of condoms in TN 1996 Centre for Development Research and Training Hard copy 2 BSS Wave -II Tamil Nadu 1997 AIMS Research (P) Ltd. Hard & Soft 3 BSS Wave -III Tamil Nadu 1998 AIMS Research (P) Ltd. Hard & Soft 30 Communication Impact study 1998 APAC / ORG MARG Hard copy 32 Community Prevalence of Sexuality Transmission Diseases in TN 1998 APAC Hard copy 47 Evaluation report on Social marketing training program 1998 APAC / CED Hard copy 53 Formative research on AIDS for Development of Communication strategy 1998 APAC / MODE Hard copy 66 Realities of sexual Behavior in Tamil Nadu 1998 APAC Hard copy 4 BSS Wave -IV Tamil Nadu 1999 APAC/AC Nielsen Hard & Soft 24 A study on condom usage among target audience in Karur PATH intervention site 1999 APAC / KLA Hard copy 33 Community prevalence of Sexually Transmitted Diseases in Tamil Nadu 1999 APAC Hard copy 71 STD Health Care Providers survey in TN Round II 1999 APAC Hard copy 5 BSS Wave -V Tamil Nadu 2000 APAC/AC Nielsen Hard & Soft 49 Evaluation study report on APAC communication Campaign conducted in TN 2000 APAC / Equations Hard copy 6 BSS Wave -VI Tamil Nadu 2001 APAC/SRI -IMRB Hard & Soft 17 BSS Wave - I Pondicherry 2001 ACNielsen Hard & Soft 50 Evaluation study report on APAC communication Campaign conducted in TN 2001 APAC / Equations Hard copy 68 Situational Assessment of the Industrial workforce in TN 2001 APAC / TNS Mode Pvt. Hard copy 7 BSS Wave -VII Tamil Nadu 2002 APAC/SRI -IMRB Hard & Soft 18 BSS Wave - II in Pondicherry 2002 APAC Hard & Soft 29 Communication Evaluation study 2002 APAC /ACNielsen Hard copy 8 BSS Wave -VIII Tamil Nadu 2003 APAC/SRI -IMRB Hard & Soft 13 BSS Wave-III Rural Tamil Nadu 2003 APAC Hard & Soft 25 A study on adolescent sexual behavior 2003 APAC Hard copy 34 A study on adolescent sexual behavior 2003 APAC/Seva Nilayam Society Hard copy 46 Evaluation on HIV/AIDS campaign in TN & Pondicherry 2003 APAC / IMRB Hard copy 57 Prevalence of STD in General population, Truckers & Helpers & Women in prostitution in TN 2003 APAC Hard copy 58 APAC Evaluation SL. NO. TITLE PUBLISHED YEAR AGENCIES/ORGANIZATION FORMAT 70 STD Health Care Provider Survey in TN Wave III 2003 SRI -IMRB Hard copy 9 BSS Wave -IX Tamil Nadu 2004 APAC/SRI -IMRB Hard & Soft 19 BSS Wave - III Pondicherry 2004 APAC Hard & Soft 31 Communication Needs Assessment study 2004 APAC / TNS Mode Pvt. Hard copy 56 Management of Behavior Surveillance Survey Experience of Tamil Nadu 2004 APAC Hard copy 59 Prevalence of STI & HIV among General population in TN 2004 APAC Hard copy 60 Prevalence of STI among Truckers & Helpers 2004 APAC Hard copy 64 Rapid Assessment study for planning HIV/AIDS Prevention Program in Theni District 2004 APAC Hard copy 10 BSS Wave -X Tamil Nadu 2005 APAC/SRI -IMRB Hard & Soft 20 BSS wave - IV Pondicherry 2005 APAC Hard & Soft 11 BSS wave -XI Tamil Nadu 2006 APAC/SRI -IMRB Hard & Soft 14 BSS Wave-IV Rural Tamil Nadu 2006 APAC/SRI -IMRB Hard & Soft 22 BSS wave - VI in Pondicherry 2006 SRI -IMRB Hard & Soft 51 Facility Assessment HIV Testing & Counseling centers TN & Pondicherry 2006 APAC Hard & Soft 21 BSS wave - V in Pondicherry 2007 SRI -IMRB Hard & Soft 38 Ethnographic mapping of truckers Halt Points in TN & Pondicherry 2008 APAC/SRI -IMRB Hard & Soft 54 Health care facility survey (IV round) In TN 2008 APAC / TNS Mode Pvt. Hard copy 63 Rapid Assessment of Private sector Hospitals in Tamil Nadu 2008 APAC/Mafoi Hard & Soft 69 Size Estimation and Resources mapping of Core and vulnerable populations (IMRB) 2008 APAC/IMRB Hard & Soft 12 BSS Wave- XII Tamil Nadu 2009 SRI -IMRB Hard & Soft 15 BSS in TN IID Districts Wave 1 2009 APAC/SRI -IMRB Hard & Soft 16 Risk Assessment of People living with HIV/AIDS in APAC IID Districts 2009 APAC/SRI -IMRB Hard & Soft 23 BSS Wave - VII in Pondicherry 2009 SRI -IMRB Hard & Soft 44 Evaluation of the Effectiveness of APAC Mobile Health Education Units (MHEU) in reach and usefulness to Target group and Core group 2009 APAC Hard & Soft 55 Mainstreaming HIV services in private sector hospitals in TN 2009 APAC / SAATHI Hard & Soft 65 Reaching the Un-Reached: An Exploratory Study Of Hidden and difficult to Reach sex workers in TN 2009 APAC Hard & Soft 74 Willingness to pay by People Living with HIV/AIDS (PLHAs) for private Medical services 2009 APAC Hard & Soft 26 Adult Literacy Program and Entrepreneurial Capacity Building for Specific Population sub groups-Baseline data 2010 APAC/hand in Hand Hard & Soft 59 APAC Evaluation SL. NO. TITLE PUBLISHED YEAR AGENCIES/ORGANIZATION FORMAT 28 Assessment of STI Clinic based Targeted Intervention in Kancheepuram District 2010 APAC / External Consultant Hard & Soft 35 District level analysis of HIV epidemic by using data triangulation approach (in Tamil Nadu, UK, UP and Kerala) 2010 APAC/NACO Hard & Soft 36 District level mathematical Modeling of HIV/AIDS epidemic in TN 2010 APAC / Indian Statistical Institute Hard & Soft 39 Evaluation of State Level Communication Campaign (SLCC) in Tamil Nadu 2010 APAC / GfK Mode Hard & Soft 40 Evaluation of District Level Communication Campaign (DLCC) among MSM in 7 IID APAC Districts 2010 APAC / GfK Mode Hard & Soft 52 Facility Assessment of Secondary Level public Hospitals in Tamil Nadu (272 Gov. Hospitals profiles, 30 district reports & State report) 2010 APAC / TNHSP /GIRH Hard & Soft 61 Profiling MSM in TN an Exploratory study 2010 APAC Hard & Soft 62 Quality of Life of Positive Female Sex workers in Trichy (Pre &post-intervention) 2010 APAC Soft copy 67 Reproductive, Child and Sexual Health needs of women living with HIV/AIDS 2010 APAC / PWN+ Hard & Soft 27 Assessment of Nakshatra Clinics (A - PPP initiative) in APAC Intensive Intervention Districts (IID) 2011 APAC / CARB Hard & Soft 41 Evaluation of Hello+ Helpline Service of APAC : A - PPP initiatives 2011 APAC /SRI- IMRB Hard & Soft 42 Evaluation of Institutional Care Project at IRT Perundurai: A PPP initiative 2011 APAC/STEM Hard & Soft 43 Evaluation of Task Sharing in STI Clinics in Tamil Nadu 2011 APAC/External Consultant Hard & Soft 45 Evaluation of the Effectiveness of Street Plays through Resource and Training centers 2011 APAC/SRI -IMRB Hard & Soft 48 Evaluation Study of Comprehensive Care and Livelihood Support - Sakthi Plus 2011 APAC/External Consultant Hard & Soft 58 Prevalence of STI among women in prostitution 2003 -04 APAC Hard copy 73 Various End Of project Evaluation of APAC supported projects Various Years APAC/External Consultants Hard copy 74 National Level Mental Health Study 2011 APAC and NACO 75 Effectiveness of multimedia in counseling 2012 Ongoing 76 Evaluation of OVC Trust 2011 APAC 77 Evaluation of DAPCU in program Management 2012 Ongoing 78 Effectiveness of SMS services among MARPs - A pilot Study 2012 Ongoing 37 End Of project Evaluation of APAC supported projects Hard copy 60 APAC Evaluation APPENDIX D. FOCUS GROUP INSTRUMENT Focus Group Discussion Regarding NGO-Government Partnership Model INSTRUCTIONS: Each participant should please spend 5 minutes individually ranking which element they see as most critical to the success and ownership of APAC and which elements they individually see as being less important. Most important is No. 10 and least important is No. 1. No two elements may share the same number. Following this individual assessment all participants will join together to share/discuss their individual assessments and reach a consensus on a relative ranking of the elements below. Please rank the following 10 items: 10 is most important - 1 is least important SL.NO DESCRIPTION RANKING MOST IMPORTANT: 10 LEAST IMPORTANT: 1 1 Project Management Committee, with equal partnership between government and NGO, including regular meeting of all partners (ARC, Tripartite Agreement) 2 Strong Leadership of Project 3 Adequate budget (and staff) 4 Engagement/participation of the private sector 5 Emphasis on building capacity of all participants -(individuals, NGOs, government, research centers) - with focus on needs and mentoring 6 Involvement of community being served, in project design and implementation - i.e. culture of participation 7 Strong focus on generation of “evidence” and using it for on-going project design (e.g. BSS, other studies) 8 Strong systems for monitoring and managing project’s progress (e.g. MIS for technical and financial, transparent systems) 9 Opportunities and encouragement for innovation of new ideas (e.g. sms) 10 Creating replicable and sustainable models (e.g. State Health Data Resource Centre) 61 APAC Evaluation APPENDIX E. FIELD VISIT SCHEDULE FIELD VISIT SCHEDULE SITE VISITS TI NGO MSM & FSW ICTC/PMTCT ART STI JAN 23 2012 JAN 24 2012 JAN 25 2012 JAN 26 2012 JAN 27 2012 JAN 28 2012 TEAM 1 Chennai APAC meeting Karur Site visits Trichy SI Team meeting Perambalur Site visits TEAM 2 Salem Site visits Villupuram Site visits JAN 30 2012 JAN 31 2012 FEB 01 2012 FEB 02 2012 Team 1 Michele Andina & Dipanjan Roy Team 2 Robertina Mera & Anita Rego TEAM 1 Chennai USAID Midterm brief Relative ranking Meeting with Health Secretary Kerala TSU Relative ranking & Site visits TEAM 2 Kancheepuram Site visits 62 APAC Evaluation APPENDIX F. RELATIVE RANKING RESULTS Relative Ranking Focus Group Discussion Regarding NGO-Government Partnership Model INSTRUCTIONS: Each participant should please spend 5 minutes individually ranking which element they see as most critical to the success and ownership of APAC and which elements they individually see as being less important. Most important is No. 10 and least important is No. 1. No two elements may share the same number. Following this individual assessment all participants will join together to share/discuss their individual assessments and reach a consensus on a relative ranking of the elements below. SL.NO DESCRIPTION RANKING MOST IMPORTANT: 10 LEAST IMPORTANT: 1 1 Project Management Committee, with equal partnership between government and NGO, including regular meeting of all partners (ARC, Tripartite Agreement) 2 Strong Leadership of Project 3 Adequate budget (and staff) 4 Engagement/participation of the private sector 5 Emphasis on building capacity of all participants -(individuals, NGOs, government, research centers) - with focus on needs and mentoring 6 Involvement of community being served, in project design and implementation - i.e. culture of participation 7 Strong focus on generation of “evidence” and using it for on-going project design (e.g. BSS, other studies) 8 Strong systems for monitoring and managing project’s progress (e.g. MIS for technical and financial, transparent systems) 9 Opportunities and encouragement for innovation of new ideas (e.g. SMS) 10 Creating replicable and sustainable models (e.g. State Health Data Resource Centre) 63 APAC Evaluation Please rank the following 10 items: 10 is most important - 1 is least important ITEMS DESCRIPTION TAMIL NADU KERALA SUMMATED TOTAL FINAL RANKINGS N=61 TANSACS NGO CONSULTANT APAC TSU TSU Project Management Committee Project Management committee with equal partnership between government and NGO including regular meeting of all partners (ARC, tripartite agreement) 2 1 7 9 5 3 27 4 Leadership Dynamic leadership that engages and supports government, CSOs and donors in project planning, management, implementation and monitoring 10 9 5 10 9 10 53 10 Budget Adequate, timely and flexible budget 7 7 2 8 8 7 39 7 Private sector Engagement/ participation of the private sector 1 2 1 3 1 1 9 1 Capacity building Emphasis on building capacity of various entities (individuals, NGOs, government, training resource centres) with focus on need and mentoring 9 10 8 7 10 6 50 9 Community involvement Culture of participation as seen as involvement of community being served in project design and implementation 6 6 10 2 7 8 39 6 Evidence Strong focus on generation of “evidence“ and using it for on-going project design (e.g., BSS, other studies) 5 5 4 4 2 5 25 3 Systems Strong systems for monitoring and managing project progress (e.g., MIS for technical and financial, transparent systems 8 8 6 6 6 9 43 8 Innovation Opportunities and encouragement for innovation of new ideas (e.g., SMS) 4 4 9 5 4 4 30 5 Sustainable/ replicable models Creating replicable and sustainable models (e.g., State health data resource centre) 3 3 3 1 3 2 15 2 64 APAC Evaluation APAC Evaluation 65 APPENDIX G. STATISTICAL ANALYSIS Statistical Analysis Methods The retrospective non-intervention comparison group repeated measures design was analyzed using a generalized mixed models approach. A logistic regression model using the basic exposure set (districts subject to APAC intervention throughout all the exposure period) was used to compute the propensity score. Three basic data analysis tables were constructed, two that have subjects nested within districts corresponding to the HIV Sentinel Survey with data from 2002 to 2008 and the BSS with data from 1998 to 2009 and another with rates and variances of rates by districts with data from 2005 to 2011 (PPTCT). The propensity score was computed using a separate table where the unit of analysis is the district. Variables available for the computation of the propensity score were women’s literacy for year 2001, infant mortality rate, population of each district for year 2001, human development index for year 2001, gender development index for year 2001, and estimated number of migrants. The propensity score was used as an analysis weight but only for baseline years (2002 or 2003 depending on the comparison). The computed propensity score adequately adjusted for the univariate significant difference between exposure (APAC districts) and non-exposure (non APAC districts) in a model were the dependent variable was HIV status by making the exposure non-significant after including the propensity score weight in the multivariate model. The conclusion from this exercise is that the odds of being HIV positive were significantly different by district at baseline in a non-adjusted model, but not after adjusting for the propensity score, which means that the variables included in the computation of the propensity score were indeed confounders for exposure. Mixed logit models were used to compute the odds of being HIV positive and mixed poisson models were used to compute the odds of having an STI. These odds are reported as prevalence rates per year and their corresponding confidence intervals for exposed (APAC) districts and non-exposed districts at a given point in time. Patient level covariates used in building the patient-level models were age, sex, locality (urban/rural), education, migrant status, occupation, spouse occupation and whether the subject was MARPS or not. A custom logistic model was built to compute prevalence rates when no patient-level data was available such as with PPTCT data. Rates and their respective 95% confidence intervals were weighted by the inverse of the variance and further adjusted by the total number of subjects tested in order to address a perceived powerful testing bias. Output of all models is the mean of the referenced indicator by year (margin in statistical parlance) which is usually a prevalence rate (HIV) or proportion (BSS) or an average total (number tested) and its respective confidence interval. P values are considered statistically significant when below 0.05. SPSS (Statistical Package for the Social Sciences) and STATA 11 were used in this statistical analysis. APAC Evaluation 66 APPENDIX H. TECHNICAL ASSISTANCE PROVIDED BY APAC APAC EVALUATION -- TECHNICAL ASSISTANCE PROVIDED PHASE III APPROACHES/CLIENT NATIONAL NACO STATE TNSACS - KSAC - PACS - TNHSP CSO PRIVATE SECTOR EVIDENCE GENERATION/ RESEARCH Data triangulation White paper on PPP initiatives in health sector Process documentation and Impact assessment of Folk media TANSACS& PACS BSS Data triangulation OVC Trust Facility assessments for private hospitals for establishment of ICTC STI Community prevalence study (1998,2004) STI health care providers survey (1998,) Communication need assessment (2000,2005) Assessment of ICTC in Govt & private sector (2006) TNHSP Facility assessments in public hospitals for system strengthening and integration Patient counselor training needs assessment Equipment maintenance study Assessment on Infection Control and BMW status in 11 medical colleges (Public and private) Assessment of performance of Housekeeping agencies in secondary care hospital Gender Audit Human Resources in Health care in Tamil Nadu ( Distribution of HRH in TN) Kerala BSS for KSACS Mapping of MARP in Kerala Support to Positive women’s network for Reproductive child and sexual health study STRATEGY DEVELOPMENT Developed concept and Design for addressing truckers in partnering with NHAI Concept on National HIV/AIDS Communication Resource Centre (NHCRC)country Folk Media roll out - 3 national level workshops Managing two Technical Support Units Development of State specific communication strategy Social marketing of Condoms Hello Plus - Helpline for PLHIV APAC Evaluation 67 APAC EVALUATION -- TECHNICAL ASSISTANCE PROVIDED PHASE III APPROACHES/CLIENT NATIONAL NACO STATE TNSACS - KSAC - PACS - TNHSP CSO PRIVATE SECTOR EXPERTS/CONSULTANTS Technical Resource Group- TI at National level (Phase 2, 3 and 4) Member in the Technical working group for STI in NACP IV planning Member, National Planning commission - HIV/AIDS Deployment of APAC officer for communication Positioned two consultants for providing TA on IEC, Mid media, Monitoring and evaluation Member in National working group on review and standardization of thematic IEC materials Chairperson for consortium on Communication in the state EXPERIENCE SHARING Exposure visits to Global fund team Conducted exposure visits and training programs to Project Directors and TI officers of all SACS in the country National Level SACS Project Directors meet and facilitated exposure to APAC intervention TEXSHARE: National level experience sharing meeting for TSUs NESSCOM - national level experience sharing meeting Development of target specific IEC for use in the state Branding of ART center and development of IEC package Joint initiative of state level communication campaign: Dillu Durai: On increasing risk perception & Right Ranga on ABC messages District level communication campaign for MSM and Demand Generation for ICTC ( Thozha Tho zha) Development of Design and branding of IEC van for strengthening communication to reach the unreached TNHSP International Conference on Health systems strengthening ESRM - Thematic ( Half yearly) Peer educators convention Exposure visits to other interventions FBO - Round table conference APAC Evaluation 68 APAC EVALUATION -- TECHNICAL ASSISTANCE PROVIDED PHASE III APPROACHES/CLIENT NATIONAL NACO STATE TNSACS - KSAC - PACS - TNHSP CSO PRIVATE SECTOR TRAINING/MENTORSHIP Technical assistance in developing the STI training modules for health care providers Manual development on program management for STRC Knowledge management centre in STRC Task Shifting for STI care in Govt hospitals NGO - TI related training Training of HIV positive networks - Governance Capacity building of health care providers - CETC ( Continuing Education and Training centres) Training of health care providers on HIV clinical management - IRT PMCH Training of Nakshatra doctors in partnership with IRT PMCH SYSTEMS & PROCEDURES Developed Peer Education model and shared with NACO for replication TA for developing financial guidelines for TI State Information Centre on HIV/AIDS DAPCU Knowledge treasury- cataloguing of all IEC materials produced in the state on HIV PPP - Operationalization of 110 ICTC in Private hospitals Master health check for Core groups SIMU TNHSP State Health Communication Center Regional Training Centers for Infection Control and Bio-Medical Waste Management State Health Data Resource Center State Health GIS State Health Research Resource Center Participatory site visits ( Half yearly) Nakshatra STI clinics Arogyan clinics Apollo tyres STI Clinics Social Impact, Inc. 2300 Clarendon Boulevard Suite 1000 Tel: (703) 465-1884 Fax: (703) 465-1888 www.socialimpact.com