EVALUATION EVALUATION OF COMMUNITY HIV LINK PROGRAMS SEPTEMBER 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Kerry A. Richter, Le Thi Quynh Nga and Tran Thap Long of Management Systems International, a Tetra Tech Company. EVALUATION OF COMMUNITY HIV LINK PROGRAMS Contracted under AID-440-C-14-00003 Vietnam Evaluation, Monitoring and Survey Services DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Evaluation of Community HIV Link Programs i CONTENTS Acknowledgments ...................................................................................................................iv Acronyms and Abbreviations .................................................................................................. v Executive Summary ................................................................................................................ 1 1. Evaluation Purpose ..................................................................................................................................... 1 2. Program Background ................................................................................................................................. 1 3. Evaluation Questions, Design, Methods and Limitations .................................................................. 2 4. Findings and Conclusions ......................................................................................................................... 2 4.1 Case Findings and Contributions to Provincial Cascade and 90/90/90 Targets ................ 2 4.2 Stakeholder Recognition ................................................................................................................. 4 4.3 Development of CBOs ................................................................................................................... 5 4.4 Sustainability Findings ...................................................................................................................... 5 4.5 Cross-Cutting Issues ........................................................................................................................ 5 5. Recommendations ...................................................................................................................................... 6 I. Introduction .......................................................................................................................... 8 1. Overview ...................................................................................................................................................... 8 2. The C-Link Programs ................................................................................................................................ 8 II. Evaluation Purpose ............................................................................................................. 9 1. Objectives .................................................................................................................................................... 9 2. Evaluation Questions ............................................................................................................................... 10 III. Evaluation Methodology ................................................................................................. 10 1. Evaluation Strategies ................................................................................................................................ 10 2. Field Site Selection ................................................................................................................................... 11 2.1. Criteria for Selection .................................................................................................................... 11 2.2 Site Characteristics ........................................................................................................................ 11 3. Data Collection and Analysis Methods ............................................................................................... 12 3.1 Document and Literature Review .............................................................................................. 12 3.2 Collection and Analysis of Secondary Data ............................................................................. 12 3.3 Key Informant Interviews ............................................................................................................. 12 3.4 Brief Quantitative Surveys ............................................................................................................ 13 3.5 Informal Observation .................................................................................................................... 14 4. Analysis ....................................................................................................................................................... 14 5. Limitations of the Methodology ............................................................................................................ 14 IV. Findings ............................................................................................................................ 15 1. Performance in Case Findings and Contribution to Provincial Cascade and 90/90/90 Targets ........................................................................................................................................... 15 1.1 Northern Coast Region – COHED ........................................................................................... 15 1.2 Northern Mountains Region – CCRD ...................................................................................... 22 1.3 Southern Region – LIFE Center .................................................................................................. 30 2. Performance in Linkage and Networking and Stakeholder Recognition .................................... 39 2.1 Overview of Programs’ Linkages and Networks.................................................................... 39 2.2 Stakeholder Recognition and Assessing the Enhancement of Coordination and Networking with Key Local Stakeholders .......................................................................... 40 Evaluation of Community HIV Link Programs ii 3. Development of Civil Society in the Project Area ........................................................................... 49 3.1 Overview of Program Interventions in CBOs’ Development ............................................. 49 3.2 Evaluative Findings .......................................................................................................................... 50 4. Sustainability .............................................................................................................................................. 51 4.1 Context ............................................................................................................................................. 51 4.2 Evaluative Findings .......................................................................................................................... 52 4.3 Factors That Will Influence the Effectiveness or Necessity of Continuity of Active Case Finding .................................................................................................................. 53 5. Cross-Cutting Issues ............................................................................................................................... 54 5.1 Gender .............................................................................................................................................. 54 5.2 Monitoring and Evaluation ............................................................................................................ 55 6. Best Practices and Lessons Learned .................................................................................................... 58 V. Conclusions ....................................................................................................................... 60 1. Performance .............................................................................................................................................. 60 2. Sustainability .............................................................................................................................................. 62 VI. Recommendations .......................................................................................................... 62 Annex 1: Proportions of Tested and Positive KPs ............................................................. 64 Annex 2: Results of CBO Organizational Capacity Assessments, Northern Coast....................................................................................................................................... 65 Annex 3: Results of CBO Organizational Capacity Assessments, Northern Mountains ............................................................................................................................... 66 Annex 4: Results of CBO Organizational Capacity Assessments, Southern Region ..................................................................................................................................... 67 Annex 5: Data Collection Schedule ..................................................................................... 68 Annex 6: List of Informants .................................................................................................. 80 Annex 7: Interview Guides ................................................................................................... 85 Annex 8: Beneficiary Client Satisfaction Survey .............................................................. 106 Annex 9: CBS Data Collection Form ................................................................................ 113 Annex 10: Data Collection Form, CBO/Self-Help Groups/Clubs ................................... 114 Evaluation of Community HIV Link Programs iii LIST OF TABLES Table 1: Characteristics of Field Sites ......................................................................................................................... 11 Table 2: Key Informants Interviewed .......................................................................................................................... 13 Table 3: Number of CBOs Implementing C-Link in Year 1 and Year 2 by Province and Date of Formation ................................................................................................................................................. 22 LIST OF FIGURES Figure 1: Achievements vs. Targets in Outreach Activities for C-Link Year 1 (October 2014– September 2015) in the Northern Coast region .............................................................................. 17 Figure 2: Achievement vs. Target in Outreach Activities, Q1 & Q2 of C-Link Year 2 (October 2015–March 2016), Northern Coast Region ..................................................................................... 18 Figure 3: Contribution of C-Link to Provincial Achievements on the Cascade, Year 1 (October 2014–September 2015), Nghe An, Northern Coast ........................................................................ 19 Figure 4: Estimated Contribution of C-Link to 90/90/90 Targets, Q1 & Q2 of Year 2 (October 2015–March 2016), Nghe An, Northern Coast Region .................................................................. 19 Figure 5: Percentage of Key Population Clients Who Are Satisfied with Services .......................................... 21 Figure 6: Achievement vs. Targets in Outreach Activities, C-Link Year 1 (October 2014– September 2015), Northern Mountains Region................................................................................ 24 Figure 7: Achievement vs. Targets in Outreach Activities, Q1 & Q2 of C-Link Year 2 (October 2015–March 2016), Northern Mountains Region ............................................................................. 25 Figure 8: Contribution of C-Link to Provincial Achievements on the Cascade, Year 1 (October 2014–September 2015), Dien Bien Province, Northern Mountains Region ............................... 26 Figure 9: Contribution of C-Link to 90/90/90 Targets, Q1 & Q2 of Year 2 (October 2015– March 2016), Dien Bien Province, Northern Mountains Region ................................................... 27 Figure 10: Percentage of Key Population Clients Who Are Satisfied with Services ....................................... 29 Figure 11: Achievement vs. Target in Outreach Activities, C-LINK YEAR 1 (October 2014– April 2015), Southern Region ................................................................................................................. 33 Figure 12: Achievement vs. Target in Outreach Activities, Q1 & Q2 of C-Link Year 2 (May– November 2015), Southern Region ...................................................................................................... 34 Figure 13: Contribution of C-Link to Provincial Achievements on the Cascade, Year 1 (October 2014–September 2015), Ho Chi Minh City, Southern Region .................................... 34 Figure 14: Contribution of C-Link to Provincial Achievements on the Cascade, Year 1 (October 2014–September 2015), Ho Chi Minh City, Southern Region .................................... 35 Figure 15: Percentage of Key Population Clients Who Are Satisfied with Services ...................................... 37 Figure 16: Primary Linkages and Networks in C-Link Programs ......................................................................... 39 Figure 17: Percentage Who Reported That A CBS/CSP Accompanied Them to Health Services by KP Group, Beneficiary Survey (n=194) .......................................................................................... 47 Figure 18: Number of CBO-to-CBO Collaborations Reported Before and After the C-Link Program, CBO Survey (n=8) .................................................................................................................. 49 Figure 19: Percentage of KPs Who Say They Have Health Insurance by Group, Beneficiary Survey (n=194)........................................................................................................................................... 53 Figure 20: Case Validation Process, Northern Coast ............................................................................................. 58 Evaluation of Community HIV Link Programs iv ACKNOWLEDGMENTS The evaluation team wishes to thank everyone who participated or assisted in our efforts to conduct this extensive evaluation of the Community HIV Link (C-Link) programs in Vietnam. The team talked with a number of Government of Vietnam (GVN) officials at the central level who took time to share their knowledge and insights about issues related to HIV/AIDS policies and programs and the work of C-Link programs. Special thanks go to the director general and the deputy director general at the Vietnam Administration of HIV/AIDS Control, who gave us a better understanding of how C-Link programs have contributed to the overall target of HIV eradication in Vietnam, and which adjustments should be made to achieve a higher level of outcome. The evaluation team also talked with a large number of managers and service providers at the provincial, district and site levels in Hanoi, Ho Chi Minh City, Dien Bien, Lao Cai, Hai Phong and Nghe An provinces. In these provinces, we were able to get first-hand accounts of C-Link’s support and contributions to meeting HIV/AIDS challenges in their locations. Throughout our visit, we were consistently well received. Meetings were scheduled at times that would be least disruptive to services, and the evaluation team is most grateful for the considerable time and invaluable discussions that staff shared with us. For a list of individuals interviewed during our evaluation, please see Annex 6. We are grateful for having open and critical discussions with provincial leaders of community-based organizations (CBOs), community-based supporters (CBSs) and community service providers (CSPs). We would also like to thank them for supporting us in arranging meetings with their clients. The evaluation team would like to express our sincere appreciation to experts in HIV/AIDS programming from national and international organizations, namely Do Thi Van, Director of the Vietnam Union of Science and Technology Associations; Dr. Masaya Kato of the World Health Organization (WHO); Robert Letchford of the U.S. Agency for International Development’s (USAID) Partner in Capacity Development Program; Kimberly Green, Chief of Party for the Program for Appropriate Technology in Health (PATH); and Nguyen To Nhu, Deputy Country Director for FHI 360. Special thanks go to Director Nguyen Thi Mai Huong and Deputy Chief of Party Dinh Thi Yen Nhi, both of the Center for Community Health Research and Development (CCRD), Dao Thi Mai Hoa, Director of the Center for Community Health and Development (COHED), Tran Thi Lan Anh, Associate Director of COHED, Nguyen Nguyen Nhu Trang, Director of the Center for Promotion of Quality of Life, and their management staff for spending hours with our team, making internal files and records available and never failing to be responsive to our requests for information. Importantly, our appreciation goes to the USAID Program Development Office and Office of Health for their support in meeting and assembling information. In particular, we would like to acknowledge Randolph Flay, John Eyres, Mark Breda, Nguyen Thi Minh Huong, Nguyen Thi Ha and Emily Rupp, who helped us with arranging for several policy-level meetings, accompanying us on field visits (as observers) and providing the team with invaluable guidance on USAID’s expectations and evaluation standards. This assistance was very much appreciated. Evaluation of Community HIV Link Programs v ACRONYMS AND ABBREVIATIONS AIDS Acquired Immune Deficiency Syndrome ART Antiretroviral Therapy ARV Antiretroviral C&T Care and Treatment CARE_COMM Care in the Community CBO Community-Based Organization CBS Community-Based Supporter CSP Community Service Provider CCRD Center for Community Health Research and Development C-Link Community HIV Link COHED Center for Community Health and Development COPC Continuum of Prevention to Care CSO Civil Society Organization CSP Community Service Provider DHC District Health Center FAA/FOG Fixed Amount Award/Fixed Obligation Grant FSW Female Sex Worker FY Fiscal Year GF Global Fund to Fight AIDS, Tuberculosis and Malaria GVN Government of Vietnam HCMC Ho Chi Minh City HIV Human Immunodeficiency Virus HTC HIV Testing and Counseling IBBS Integrated Biological and Behavioral Surveillance ID Identification Card IP Implementing Partner KII Key Informant Interview KP Key Population LIFE Center Center for Promotion of the Quality of Life LTFU Lost to Follow-Up M&E Monitoring and Evaluation MSI Management Systems International Evaluation of Community HIV Link Programs vi MSM Men Who Have Sex with Men OCA Organizational Capacity Assessment OCD Organizational Capacity Development OPC Outpatient Clinic PAC Provincial AIDS Center PATH Program for Appropriate Technology in Health PEPFAR President’s Emergency Plan for AIDS Relief PLHIV People Living with HIV PSP Primary Sex Partner PWID People Who Inject Drugs SMART TA Sustainable Management of the HIV/AIDS Response and Transition to Technical Assistance SOP Standard Operating Procedure SPSS Statistical Package for the Social Sciences TPM Team Planning Meeting USAID United States Agency for International Development USG U.S. Government VAAC Vietnam Administration for HIV/AIDS Control VEMSS Vietnam Evaluation, Monitoring and Survey Services VUSTA Vietnam Union of Science and Technology Associations WHO World Health Organization Evaluation of Community HIV Link Programs 1 EXECUTIVE SUMMARY 1. EVALUATION PURPOSE The purpose of the evaluation is to conduct a performance review of the three USAID/Vietnam C-Link programs to assess progress toward the achievement of objectives, recommend adjustments to current practices and generate lessons for future programs. The evaluation’s focus is on outreach activities to motivate high-risk individuals to seek testing and treatment. It explored CBO capacity-building activities conducted in Year 1 and the extent to which these activities have contributed to the effectiveness of outreach activities. 2. PROGRAM BACKGROUND In 2014, USAID launched three C-Link programs: C-Link Northern Coast – COHED, C-Link Northern Mountain – CCRD, and C-Link Southern – LIFE. The programs’ objectives include (1) improving HIV/AIDS services provided by CBOs, (2) strengthening the capacity of HIV/AIDS CBOs and (3) enhancing networking and coordination with key stakeholders. In FY2015, each program covered three provinces/cities: Quang Ninh, Hai Phong and Nghe An (implemented by COHED); Lao Cai, Dien Bien and Hanoi (implemented by CCRD); and Ho Chi Minh City (HCMC), Can Tho and An Giang (implemented by LIFE Center). In FY2016, the PEPFAR pivot1 resulted in each of the three programs being reduced from three provinces to one province, which are PEPFAR aggressive scale-up provinces, i.e. Dien Bien, Nghe An and HCMC. Program activities in Quang Ninh, Can Tho and An Giang were concluded. In this year, a substantial increase in the programs’ targets for case finding in Dien Bien and Nghe An2 sought to support these provinces to achieve 90/90/90 targets. Activities in Lao Cai, Hanoi and Hai Phong focused on continuous care and support to People Living with HIV (PLHIV). Targeted key populations (KPs) under all three programs include PLHIV, people who inject drugs (PWID), female sex workers (FSW) and primary sex partners (PSP). In urban areas like Hanoi and HCMC, men who have sex with men (MSM) are also included. The key approach to implementation of outreach activities varies across the three programs, as follows: • COHED provides support for strengthening institutional capacity of all CBOs, including forming new CBOs to coordinate CBSs (peers to KPs) to conduct outreach. • CCRD supports existing CBOs in Hanoi but works directly with Dien Bien and Lao Cai CSPs, either peers to KPs or non-peers, such as village health workers. 1 As a focus country in the initial phase of PEPFAR, the U.S. Government (USG) quickly became the largest financer of Vietnam’s HIV response. PEPFAR purchased a large share of HIV commodities, including antiretroviral therapy (ART) drugs and methadone; supported direct HIV service delivery and provided technical support in policy, planning, implementation and evaluation. In the Country Operational Plan (COP) 2016, the “PEPFAR 3.0 Pivot” approach continues the focus on sustaining epidemic control and improving efficiency through geographic and population prioritization. PEPFAR Vietnam’s goal for COP 2016 is to demonstrate significant contributions toward “90-90-90” targets in the five aggressive scale-up provinces, which contain high HIV burden, high unmet ART need, and where PEPFAR can have the greatest impact (Source: VIETNAM Country/Regional Operational Plan (COP/ROP) 2016 Strategic Direction Summary, Version May 27, 2016) 2 The targets in HCMC in the Fiscal Year 2016 were not changed in this context due to the fixed service-delivery based contract between USAID and LIFE. Evaluation of Community HIV Link Programs 2 • LIFE supports existing CBOs that manage and guide CBSs. It has prioritized eight CBOs that have potential for development and can achieve the given targets to provide support for strengthening institutional capacity. All implementing partners (IPs) used performance-based payments as an incentive for CBSs/CBOs to identify new HIV/AIDS cases and provide care and support to PLHIV. LIFE gives autonomy to CBOs in allocating incentives by themselves. 3. EVALUATION QUESTIONS, DESIGN, METHODS AND LIMITATIONS Key evaluation questions require an assessment of the following: (1) Achievement in identifying high-risk individuals from the key populations for HIV intervention services and enhancing coordination and networking with key local stakeholders; (2) Contribution to improving HIV/AIDS service cascades toward the targets of 90/90/90; (3) Stakeholder recognition of the programs’ contributions in supporting sustainable HIV/AIDS responses; (4) Effectiveness in the development of civil society organizations in the project sites; and (5) Program sustainability, as reflected through improvements in CBO capacity and continuation of HIV case finding activities. While assessing the programs’ performance toward annual targets or semi-annual targets for all program sites, analysis of the reasons for the programs’ achievements in case findings focused on three priority 90/90/90 provinces: Dien Bien, Nghe An and HCMC. 4. FINDINGS AND CONCLUSIONS 4.1 Case Findings and Contributions to Provincial Cascade and 90/90/90 Targets Overall, the programs applied a flexible and adaptable approach to structuring and implementing outreach activities in differently evolving local contexts. All IPs gained a good understanding of local areas’ situations and needs and used evidence obtained through timely situational and thematic analysis to inform the choice of the approaches in a particular area. A combination of approaches—including effective capacity building for CBSs/CSPs/CBOs in case finding; establishing and strengthening coordination and collaboration between community-based networks and the health system; and nurturing an enabling environment for outreach activities—have produced a synergistic effect on program performance. This combination of approaches builds on and includes the proven peer-driven outreach intervention model, with a performance-based incentive system for effective implementation. Findings for each program follow. Northern Coast Region – COHED  All targets across the COPC cascade were exceeded in FY2015. During this year, C-Link also greatly contributed to the provincial HIV COPC cascade in Nghe An Province with 66.9 percent of total new HIV cases reached. However, during the first two quarters of implementation in FY2016, most achievements fell below the new half-year targets.  It is unlikely that C-Link will meet Nghe An’s 90/90/90 targets in the project districts by the end of the program. Analysis of data from Q1 and Q2 of FY2016 showed that in many districts the Evaluation of Community HIV Link Programs 3 program achieved 55.5 percent of the first 90 half-year target in this period, while according to the estimated KP numbers, all the KPs have received HIV tests.  During both Year 1 and Year 2, the proportion of newly detected HIV-positive cases out of tested KPs was lower than the 5 percent benchmark of PEPFAR’s Vietnam prevention program for key populations in a concentrated epidemic—declining from 3.7 to 2.5 percent. Identified challenges that affect C-Link program achievements include:  Analysis suggests that the targets in the C-Link work plan for FY2016 were too ambitious.  In Nghe An, well-established and capable CBOs were few. Building their capacity would require significant efforts to improve technical and management aspects; the process would be time￾consuming and would take longer than the project life of three years. Most CBOs had been established only a few months before the evaluation. Northern Mountains Region – CCRD  The program greatly exceeded its indicator targets in Year 1. The same was observed after Q1 and Q2 of FY2016, and the program will likely achieve its annual targets by the end of the year.  The program greatly impacted the Dien Bien HIV COPC cascade in both 2015 and 2016, contributing about 80 percent of newly detected HIV cases and 82.3 percent of new or re￾engaged PLHIV at OPCs for the whole province.  The program has successfully targeted high-risk and hidden KPs in the project areas. Newly identified HIV-positive cases referred to OPCs increased notably, from 51.1 percent in 2015 to 72.0 percent in the first two quarters of 2016.  The program’s flexibility in adapting to local contexts, reflected through the choice to work directly with CSPs or through CBOs to do outreach, positively affected program achievements.  The provincial and district coordinators in Dien Bien Province have been an effective bridge between the Provincial AIDS Center (PAC), District Health Center (DHC), CCRD and CSPs.  Providing appropriate support for clients to obtain a valid ID at the testing stage, rather than waiting for a positive test and the need to have an ID for OPC registration, is meaningful. Having an ID also helps in the verification process of new or old HIV cases and avoids double counting. Several challenges arose during the implementation of C-Link in the Northern Mountains:  C-Link contributed an estimated 41.9 percent of the half-year target for the first 90 target (new HIV-positive cases found) and 13.5 percent for the second (HIV-positive cases receiving treatment) in Dien Bien province in 2016. Evidence indicated that the provincial 90/90/90 targets have been set too high and are potentially unrealistic.  Low enrollment in OPCs was reported as a challenge and a remaining need for Dien Bien Province to address. One of the most important reasons was limited transportation for people who live in remote areas. Difficult topography in remote districts also makes it difficult for CSPs to expand their networks from their communes to others, which can reduce effective outreach. Evaluation of Community HIV Link Programs 4 Southern Region – Life Center  In both FY2015 and FY2016, the program achieved all milestones in providing HIV prevention packages and care and treatment to PLHIV and exceeded all targets.3  Improvements along the HIV COPC cascade were significant between Year 1 and Year 2, including numbers of reached KPs tested (from 88.5 to 91.1 percent), HIV-positive KPs found (from 7.2 to 9.2 percent) and new positive OPC cases registered (from 80.9 to 91.8 percent).  The program effectively targeted the high-risk men who have sex with men (MSM) demographic.  The estimated C-Link contribution to 90/90/90 targets is similar to the estimated contributions to the HCMC cascade in FY2015 (11.5 percent of newly found HIV-positive cases and 48.2 percent of new registrations in OPCs).  Establishing a lead CBO and linking it with a HTC/OPC in each district has fostered good relationships between the health services and CBOs and increased efficiency.  For LIFE, tying a monthly incentive payment to CBO milestones and cascade targets contributed to success. Challenges to C-Link implementation in the Southern Region included the complexity of the Ho Chi Minh context, including the large number of migrants to the city who lack a residence certificate. HTCs and OPCs tended to complain significantly about the performance of CBSs at less-established CBOs. 4.2 Stakeholder Recognition  The Vietnam Administration for HIV/AIDS Control (VAAC) felt that the three IPs were very active in contributing to the COPC and were working well within the health system.  Some HIV/AIDS program implementers felt that the USAID C-Link programs should join the national efforts of civil society advocates in Vietnam, especially on revising the Law on the State Budget to allow funding for CBOs’ HIV/AIDS prevention services in annual provincial budgets.  Local HIV/AIDS authorities and health services greatly appreciate C-Link’s contributions to provincial annual targets and 90/90/90 targets. The peer-driven approach had many advantages, especially in reaching hidden KPs. The program’s contributions to reducing stigma in the communities and providing care to PLHIV were noted.  In urban areas such as HCMC, Hanoi and Vinh City, the PACs recognize the value of CBOs and work closely with them. In less urban areas, PACs had minimal interaction with the CBOs. This was particularly true on the Northern Coast, where many CBOs were newly established. Challenges include:  After donors withdraw, the health systems in rural provinces tend to return to traditional outreach approaches, such as using village health workers or community social workers for outreach to KPs. 3 The evaluation team used data that LIFE provided tracking the progress of milestones for each CBO. By definition and according to the steps in the verification mechanism set up between LIFE and the PACs in the targeted provinces, the newly found HIV-positive cases are the KPs who are referred to HIV testing and counseling (HTC) by CBSs and have an HIV-positive result, then are verified by the PAC as newly found HIV cases. Evaluation of Community HIV Link Programs 5 4.3 Development of CBOs The C-Link design included a component for strengthening the capacity of the IPs. All three IPs fulfilled the special conditions in their agreements with USAID during the first year of implementation. CBOs can be broadly categorized into three groups: 1) Those that were well-established in urban centers before C-Link and will remain strong after it ends. 2) Those in urban areas established as self-help groups or CBOs before C-Link, and for which the program’s support and training allowed them to “take off.” A few began self-sustaining financial activities, proposal writing and collaborating with other CBOs during C-Link implementation. 3) Those that were previously self-help groups or did not exist as a group before C-Link. They have built their skills through C-Link to provide outreach services. They have not embraced self￾sustaining funding opportunities. With the changes in PEPFAR funding, it is not likely that CBOs in the third group will survive without C-Link funding. Capacity building for income generation appears to be much more effective for CBOs whose members have skills and education, such as the MSM CBOs in HCMC and Hanoi; it has less of a sustainable impact for others. Detailed analysis by CBP is contained in the text of the report. 4.4 Sustainability Findings As mentioned, only CBOs in groups 1 and 2 (already established before C-Link, at least as self-help groups) have the potential to sustain their operations after the programs end. CBOs and CBSs in the “scale-down” areas, where only Care in the Community (CARE-COMM) cases received incentives, said they were not sure they could continue to do outreach, since reaching cases and bringing them to the OPC was costly and time-consuming. While PAC and DHC officials universally praised C-Link outreach for finding hidden cases and understood its value, none were funding outreach or making concrete plans to do so. Also, the need for CBOs to have legal status to enable formal relationships with PACs was seen as a barrier to sustaining the current outreach model after C-Link ends. For government stakeholders at all levels, the preoccupying concern is the transition to social health insurance. Only those with health insurance will be able to continue antiretroviral (ARV) treatments after support from international donors ends, and a large proportion of KPs do not have health insurance. Also, OPCs are beginning to move to hospitals because treatment provided at freestanding OPCs is classified as prevention, which health insurance does not cover. Stakeholders expressed concerns that clients will be lost in the transition as this process goes forward. 4.5 Cross-Cutting Issues Gender The work plans of all programs incorporate strategies and activities to promote gender equity. The Northern Mountains – CCRD and Northern Coastal – COHED programs emphasize outreach to PSPs and successfully reached this key population in all project areas. The Southern – LIFE Center included PSPs as a KP in their annual work plans and standard operating procedure (SOP), but have not implemented actions that have yielded tangible results. Evaluation of Community HIV Link Programs 6 Monitoring and Evaluation All three C-Link programs developed comprehensive activity monitoring and evaluation plans. The programs’ recording and reporting systems function well, although the new case verification and reporting process was complex and time-consuming. Local HIV prevention databases are unlinked across the HTCs, adding challenges to the implementation of Circular 09/2012/TT–BYT. 5. RECOMMENDATIONS 1) USAID/Vietnam should continue the C-Link program, with some modifications, to identify and refer HIV cases as it has effectively contributed to the 90/90/90 targets.  USAID/Vietnam should encourage provinces to reset the 90/90/90 targets based on new size and prevalence estimates for KPs. The size estimate exercise should involve district stakeholders (DHC), community-based outreach workers and other community members who are knowledgeable about KPs. USAID/Vietnam and its experienced partners should provide technical assistance on this exercise to C-Link IPs.  USAID/Vietnam and its partners should continue assistance to improve the HIV monitoring and evaluation system and HIV case verification process, as per Circular 09/2012/TT–BYT.  IPs should continue network-based peer outreach in remote areas, including collaboration with mobile testing and treatment and/or with commune health centers to provide support on ART retention. Mobile testing and treatment quality assurance monitoring should be conducted to ensure client confidentiality and privacy is maintained.  USAID/Vietnam and its IPs should prioritize a study on the effectiveness of combining CBSs’ peer-driven outreach activities and provision of lay test services among different KP groups to increase case identification efficiency.  In the Northern Coastal areas, IP should work with USAID/Vietnam, PAC and DHC to determine whether case finding is still viable, especially in districts with a consistently low HIV￾positive rate among new HTC clients. Consider shifting from case finding to support for anti￾retroviral therapy (ART) retention or lost-to-follow-up (LTFU) PLHIV in these areas.  In the Northern Coast, consider involving government HIV health officials to coordinate outreach efforts. COHED should conduct a gender analysis on the effectiveness of and preferences for same-sex versus opposite-sex outreach. LIFE should conduct a gender analysis on effective approaches to PSPs to fully execute the developed SOP. 2) USAID/Vietnam should continue its assistance in developing civil society only in urban areas and with established organizations; given the short time period of C-Link implementation, forming new CBOs to implement C-Link is not advantageous.  IPs should begin or continue supporting CBO linkages with private providers, such as commodity production enterprises to give CBOs opportunities for income-generating activities.  USAID/Vietnam and IPs should continue advocacy for civil society organizations (CSOs) to improve their institutional frameworks, in collaboration with the Global Fund. The frameworks should stipulate and plan for budget allocation at the central or provincial level. Supporting the development of CSOs should be a long-term program goal, if this is still a priority. Evaluation of Community HIV Link Programs 7 3) Prioritize preparations for the transition to social insurance by working with VAAC and PACs to fill gaps for HIV/AIDS programming.  USAID/Vietnam should continue assisting PACs to implement health insurance for ARV treatment. Include communication and dialogue with VAAC, MOH and central-level health insurance agencies.  C-Link should: o Closely track retention rates among ARV clients. o Support the purchase of health insurance and other administrative procedures required to obtain and continue treatment. o Coordinate more closely with PAC/OPC to refer newly found cases to treatment programs. 4) Document successful models of outreach, including technical recommendations for specific KP groups such as PSPs. Evaluation of Community HIV Link Programs 8 I. INTRODUCTION 1. OVERVIEW In recent years, Vietnam has had some success in controlling the spread of its HIV epidemic. The number of new cases has stabilized at about 14,000 per year since 2010, and overall population prevalence remains low at about 0.4 percent. The epidemic remains concentrated among key populations: people who inject drugs (PWID), female sex workers (FSWs) and men who have sex with men (MSM). Within these groups, prevalence has also declined. Although the results from sentinel surveillance sites have varied over the reporting periods, the VAAC Country Reports show declining prevalence between 2009 and 2013 among male PWID (from 18.4 percent to 10.3 percent), for FSWs (3.2 percent to 2.6 percent) and for MSM (from 16.7 percent to 3 percent).4 However, certain geographic areas remain a cause for concern when providing assistance for these populations, as does the unmet need for anti-retroviral therapy (ART). The percentage of those who know their HIV-positive status and seek treatment remains low. Stigma and discrimination toward people living with HIV (PLHIV), albeit having declined in recent years, remains unacceptably high and affects whether people who are at risk of transmission seek testing and treatment services.5 2. THE C-LINK PROGRAMS The Community HIV Links program (C-Link) was designed as a successor to the Pathways for Participation project, which ended in February 2014. The Pathways project originally aimed to strengthen civil society organizations (CSOs) to better enable them to support the Government of Vietnam (GVN) HIV/AIDS response. Five CSOs participated as lead organizations to implement a variety of HIV prevention, AIDS care and support, technical training and CBO-strengthening activities. Delays in approving the project led to the five CSO partners focusing on capacity strengthening and technical training instead of implementation of the planned HIV/AIDS program activities. Pathways also completed epidemic profiles and an inventory of CSOs/CBOs in its nine project provinces.6 The Pathways project ended early because of continued delays in gaining project approval and the implementing organization’s failure to obtain registration in Vietnam. USAID’s C-Link became the first program in which USAID/Vietnam made direct awards to Vietnamese CSOs; it included three of the five Pathways partners (CCRD, COHED and LIFE Center) as implementing partners. The three C-Link objectives in the original design of the program were: 1) To improve the delivery of effective community-based HIV/AIDS services along the continuum of prevention to care (COPC) provided by CBOs; 2) To strengthen the capacity of HIV/AIDS CBOs (including the implementing partners [IPs]); and 3) To enhance networking and coordination with key stakeholders. The starting implementation dates for the three C-Link partners vary:  CCRD (Northern Mountains) signed its cooperative agreement in mid-May 2014 and quickly received approval from the Prime Minister’s Office that September. 4 Vietnam Administration of AIDS Control (VAAC). 2010, 2012 and 2014 Vietnam AIDS Response Progress Reports. 5 Vietnam National Network of People Living with HIV (VNP+). (2015). Stigma Index 2014. 6 RTI International. (September 2012). Rapid assessment of CSO technical and organizational capacity. Report of phase 1: Mapping of CSOs involved in HIV community-based response. Evaluation of Community HIV Link Programs 9  COHED (Northern Coast) signed the cooperative agreement in July 2014 and held a project launch workshop in Hanoi in January 2015.  LIFE Center (Southern) received a fixed amount award/fixed obligation grant (FAA/FOG) in May 2014 and received project approval the following month. The first annual work plans were dated May 2014 – May 2015 for CCRD and LIFE Center and July 2014 – September 2015 (15 months) for COHED. In October 2015, the U.S. President’s Emergency Plan for AIDS Relief (PEPFAR) announced its Phase 3.0 initiative, or “pivot,” with the rationale that the limited resources available from international donors should be used in a more targeted and efficient way. In Vietnam, five provinces (Nghe An, Thanh Hoa, Dien Bien, Son La and Ho Chi Minh City) were named as “aggressive scale-up” areas with the aim of having a rapid impact on case finding and registering PLHIV for ART. The PEPFAR interagency team conducted needs assessments in the first four of these provinces in October 2015 in support of this redirection of funding. In addition to cutting the level of funding and focusing available funding more intensively on outreach to support the 90/90/90 targets, districts were redistributed among the USAID and Global Fund-supported outreach programs to remove any overlap.7 The PEPFAR pivot caused profound changes to the implementation areas of the Northern Mountains and Northern Coast districts of the C-Link program, as discussed in separate sections that follow. PEPFAR targets were also adjusted at the time of the pivot. C-Link’s contributions toward the provincial 90/90/90 targets in the scale-up provinces have been put into higher prominence given the changing environment—even though the programs do not provide coverage for the entire provinces. These factors must considered when evaluating the achievements and effectiveness of the program, and are noted where relevant. II. EVALUATION PURPOSE 1. OBJECTIVES The objectives of the evaluation are to: 1) Assess progress toward major objectives and achievement of the program’s purpose, which is to contribute to selected provinces attaining 90/90/90 targets (90 percent of high-risk individuals are aware of their HIV status; 90 percent of HIV-positive persons receive ARV; and 90 percent of HIV patients are retained in treatment programs). The assessment will include a review of the degree to which key intermediate results are being achieved. 2) Assess the sustainability of the program in the areas of CSO financing and human resource capacity. 3) Estimate and analyze the unit costs for activities related to finding PLHIV. 4) Identify best practices and lessons learned from CSO local partner engagement for future USAID programming. The focus of the evaluation is on outreach activities to motivate high-risk individuals to seek testing and treatment. It also explored CBO capacity-building activities conducted in Year 1 and the extent to which these activities have contributed to the effectiveness of the outreach activities. Objective 3 is being evaluated separately and will be discussed in another report. 7 The exception is Ho Chi Minh City, where some overlap remains among outreach programs in some districts; however, the programs are coordinated by the PAC and implementing agencies to avoid competition or duplication. Evaluation of Community HIV Link Programs 10 2. EVALUATION QUESTIONS The evaluation asks the following questions. Performance  Are project (activity) objectives being achieved? Specifically: o To what extent are the programs implemented by the three local partners achieving their objectives of identifying high-risk individuals from the key populations for HIV intervention services, and enhancing coordination and networking with key local stakeholders? o What are the reasons that the C-Link partners are/are not on track toward achieving their negotiated USAID targets?  What are the implementing partners’ contributions to improving HIV/AIDS service cascades toward the target of 90/90/90?  To what extent do stakeholders recognize the contributions of implementing partners in supporting sustainable HIV/AIDS responses?  How do USAID-supported programs (administered by the three local implementing partners) affect the development of CSOs within the project sites? Sustainability  Have the institutional capacities of implementing partners and CBOs increased as a result of USAID support, and if so, how? Identify the key support activities that have contributed to strengthened institutional capacity.  Will case finding activities continue at project sites after USAID-supported programs end? Do PACs have any plans to sustain the activity approach? What is the possibility of mobilizing other funding sources to maintain case finding activity? This report follows the above sequence of key questions to present the evaluation team’s findings. It first briefly introduces the program activities and the local context in which they have been operating. It then highlights important findings, followed by brief conclusions and recommendations. For each program, it presents performance results and analysis of the reasons that have possibly contributed to the observed results. It also presents aggregated findings on sustainability and cross-cutting issues such as gender and M&E. III. EVALUATION METHODOLOGY 1. EVALUATION STRATEGIES The evaluation team applied MSI’s Evaluation Standard Operating Procedures (SOPs) to the C-Link performance evaluation to ensure that the evaluation team focused on analytical issues from the outset. The SOPs define step-by-step timelines for the pre-planning work required to manage an effective evaluation process. Pre-planning included a background literature review, identification of key stakeholders and the development of an evaluation schedule. The latter was important as it permitted the distribution of notification and scheduling letters to IPs and government officials ahead of time. The final field schedule is in Annex 5. The Team Planning Meeting (TPM) process began in Hanoi in mid-March 2016. All evaluation team members, relevant Vietnam Evaluation, Monitoring and Survey Services (VEMSS) staff and Evaluation of Community HIV Link Programs 11 representatives of USAID’s program and technical offices participated. The TPM covered best evaluation practices, such as the usage of MSI’s Getting to Answers Matrix, as part of a structured process for developing the evaluation’s detailed methodology; the importance of findings, conclusions and recommendations matrices; the need to update the analysis on an ongoing basis; and a clarification of team roles and responsibilities. 2. FIELD SITE SELECTION 2.1. Criteria for Selection Six provinces were selected for field visits—Hanoi (for a limited scale as an exploratory trip), Lao Cai, Dien Bien, Hai Phong, Nghe An and Ho Chi Minh City—among the nine C-Link provinces. Within these provinces, from one to three districts were selected, depending on the stage of implementation and the diversity of the setting. The field sites were selected to cover all three implementation partners and to provide a diversified picture of program settings and key population groups (MSM, FSW, and PWID). They include districts that currently have full implementation activities (including case finding) and those where implementation has been scaled down. One district where C-Link implementation had ended was featured to gain insight into what activities continued without support and to understand the effects of ending the program. Other criteria taken into account were obtaining a mix of urban, suburban and rural areas; having a large number of key population members (KPs); having diverse program activities; and (in Ho Chi Minh City) being covered only by C-Link programs without overlap with the Global Fund and SMART TA outreach programs. Given the focus of the evaluation described in Section II.1, the evaluation team will analyze the program’s contribution to provincial HIV COPC cascades and best practices on case finding approaches mainly for Dien Bien, Nghe An Province and HCMC. The assessment on CBO development will include Hanoi (at a limited scale), Nghe An and HCMC. 2.2 Site Characteristics Table 1 summarizes the site characteristics. TABLE 1: CHARACTERISTICS OF FIELD SITES Province District Implementation Status Reason for Selection Northern Mountains Dien Bien Dien Bien Phu City ✓ Urban area, large population of PWID Dien Bien Dong ✓ Rural area, large population of PWID Lao Cai Bao Thang † Scaled-down area Northern Coastal Nghe An Vinh City ✓ Urban area, diversified program activities Hung Nguyen ✓* Suburban area, diversified program activities Que Phong ✗ Rural area, large population of PWID, ended program activities Hai Phong Ngo Quyen † Scaled-down area with strong CBOs Hai Phong City † Scaled-down urban area with strong CBOs Evaluation of Community HIV Link Programs 12 Province District Implementation Status Reason for Selection Southern Ho Chi Minh City District 7 ✓ Urban area, only C-Link working with MSM District 11 ✓ Urban area, only C-Link working with FSW, “friendly” services for KPs Go Vap ✓ Suburban area, only C-Link working with PWID ✓Full implementation; ✓* Implementation just started; † Scaled down; ✗ Ended 3. DATA COLLECTION AND ANALYSIS METHODS The evaluation uses a mixed-methods design, including (but not limited to) document and literature review, analysis of secondary data, key informant interviews, brief stakeholder surveys and site visits. 3.1 Document and Literature Review The evaluation team conducted a comprehensive review of all C-Link program documents, including the program descriptions from the cooperative agreements and modifications; pre-award survey recommendations for IPs; work plans; annual, semi-annual, quarterly and milestone reports; and M&E plans. The team also reviewed documents provided by the IPs, including organizational capacity assessments (OCAs) for CBOs and IPs; training materials and training reports; samples of monthly CBO reports; standard operating procedures (SOPs) for implementation; and various others. The team also reviewed documents from secondary sources, including studies and assessments on HIV/AIDS service delivery in Vietnam; PEPFAR’s Country Operational Plan for FY2015; GVN strategies and plans concerning HIV/AIDS; provincial HIV/AIDS action plans; and other research, assessments or studies on HIV/AIDS community outreach and key populations in Vietnam and elsewhere. 3.2 Collection and Analysis of Secondary Data The evaluation extensively uses national and sub-national data that is pertinent to the evaluation questions. These include size estimations for key populations (FSWs, PWIDs and MSM) and provincial targets for the number of KPs to reach, both those attending HTC and those with HIV who are registered and continuing treatment at OPCs. The evaluation team also collected service data at the provincial and, where possible, district levels and compared it with program monitoring statistics over time. 3.3 Key Informant Interviews In-depth interviews, mainly in the form of group discussions, were conducted with key informants at central, provincial and site levels. Key informants are listed in Table 2. The list of informants with position and province is in Annex 6 and the key informant interview guides are Annex 7. Evaluation of Community HIV Link Programs 13 TABLE 2: KEY INFORMANTS INTERVIEWED Central Implementing Partners Provincial District Local Implementers USAID: AOR/CORs and M&E specialists VAAC VUSTA WHO SMART-TA Healthy Markets Chief of Party Deputy Chief of Party Program managers Technical staff M&E staff Director of PAC and/or Deputy Director of PAC Head and/or Deputy Head of HIV Surveillance Department Head and/or Deputy Head of HIV Testing Department Head and/or Deputy Head of ART Department Head of DHC Head of OPC Head of HTC HTC/OPC Staff Local coordinator CBO heads CBS/CSPs At each CBO, the evaluation team arranged to meet and interview members of key populations for the beneficiary survey (description follows). When time permitted, the team also asked one or two beneficiaries who had been referred to HTC or OPC by each CBO to be interviewed in greater depth about their experiences with, and perspectives on, the C-Link program. Six people were interviewed in depth (two in Dien Bien and four in Ho Chi Minh City). The evaluation team hoped to meet with the Department of Home Affairs, which covers civil society registration issues at both the central and provincial levels, but arrangements were not possible. 3.4 Brief Quantitative Surveys The evaluation team also conducted brief quantitative surveys with the heads of the CBOs (or other knowledgeable people), with CBSs/CSPs and with beneficiaries. Eight8 CBOs that the team visited during the field visits completed the CBO survey. Most of the CBO surveys were completed through one-on￾one interviews, with the CBO leader usually providing some statistical information later. The CBS/CSP survey was self-administered (using paper and pencil) and was given to all available CBSs/CSPs during field visits. In total, 34 CBSs/CSPs filled out the CBS/CSP survey. For the beneficiary survey, the evaluation team asked each CBO it visited to have its CBSs recruit 10 people for the survey: five who had attended HTC and five who had attended OPC. The strategy was the same in Dien Bien for CSPs. Because the CBSs/CSPs were likely to recruit people they knew well (and who held favorable opinions of C-Link), the team also asked them to recruit five people they had reached for the first time in the past two weeks (whether or not they had visited HTC/OPC services). In practice, the organizations told the evaluation team that they were unable to easily contact such recent clients. For this reason, the survey team recruited a small number of beneficiaries in each district through the other KPs recruited for the survey (snowball sampling). Although this group was recruited independently of the CBOs/CBSs/CSPs, they were screened for eligibility before their survey participation to assure that only those who had been reached by a C-Link CBS or CSP took part. The evaluation team interviewed 194 KPs in three provinces: HCMC (94), Dien Bien (48) and Nghe An (52). The short beneficiary questionnaire was set up on handheld devices for ease of administration and immediate data entry. The VEMSS research assistant and two additional interviewers with field experience administered the survey. The survey questionnaires for beneficiaries, CBSs and CBOs are in annexes 8, 9 and 10. 8 So far, the evaluation team has only seven; it has not been able to track down Vuot Song in HCMC. Evaluation of Community HIV Link Programs 14 3.5 Informal Observation Where possible, interviews of HTC/OPC staff took place at health centers (HTC/OPC) so the team could observe interactions with clients, client numbers and approximate waiting times. All CBO interviews took place at each CBO’s office, where the team could observe the atmosphere and examine project materials. The CBS/CSP and KP surveys were also usually administered at the CBO offices. 4. ANALYSIS As mentioned, the evaluation team collected all data available on service statistics in the C-Link target provinces, from the IPs’ monitoring systems and from provincial and district offices when available. Achievements on project indicators and on the COPC cascade are presented in relation to project, provincial and PEPFAR targets, and as a proportional contribution to provincial and district-level achievements. Further details on the analysis of project monitoring data are in Section IV: Findings. Qualitative data from key informant groups and individual interviews was first recorded as handwritten interview notes and then transcribed to electronic format by members of the evaluation team. The electronic interview note files included some near-verbatim transcription of respondents’ answers, but also some summarized information according to the structure of the interview guides. These notes were then analyzed through content analysis by systematically searching for response topics, then entering respondent quotes and information into a matrix organized by location, respondent and evaluation question. Triangulation of information was achieved by confirming information given in interviews with that available from project documents; the evaluation team also asked the same question to multiple stakeholders at several levels. The qualitative data matrices were then systematically reviewed to formulate the evaluation findings. Data from the short surveys administered to CBO leaders and CBSs/CSPs was entered into a spreadsheet file for analysis. The beneficiary survey was administered using a handheld device; the evaluation team used Statistical Package for the Social Sciences (SPSS) to analyze the resulting data file with simple frequencies and cross-tabulations. 5. LIMITATIONS OF THE METHODOLOGY Implementation of the C-Link program is complex, with three implementing partners, widely diversified settings and a variety of implementation approaches. While the evaluation team was interested in assessing the effectiveness of the different adaptations that IPs made, only a rigorous and long-term research protocol could reliably evaluate these differences. Thus, the team’s assessments are based on the available data and limited by the time period of the evaluation. While the team attempted to triangulate its findings with both qualitative and quantitative information, the quantitative surveys were implemented using non-probability (convenience) sampling, and the sample sizes are small. Therefore, the team does not attempt to draw firm conclusions from these surveys and uses the data only in an illustrative way. Finally, many program sites have undergone implementation changes in the two-year period; the program has ended or been scaled down in some districts, while others just began implementation a few months ago. This factor adds another layer of complexity when trying to assess progress under the C-Link program. Evaluation of Community HIV Link Programs 15 IV. FINDINGS 1. PERFORMANCE IN CASE FINDINGS AND CONTRIBUTION TO PROVINCIAL CASCADE AND 90/90/90 TARGETS EVALUATION QUESTIONS Are project (activity) objectives being achieved? Specifically:  To what extent are the programs implemented by the three local partners achieving their objectives of identifying high-risk individuals from the key populations for HIV intervention services?  What are the reasons that the C-Link partners are/are not on track toward achieving their negotiated USAID targets?  What are the implementing partners’ contributions to improving HIV/AIDS service cascades toward the target of 90/90/90?  To what extent do stakeholders recognize the contributions of implementing partners in supporting sustainable HIV/AIDS responses? 1.1 Northern Coast Region – COHED 1.1.1 Program Overview and Local Context The Northern Coast’s original project area included three provinces: Hai Phong, Nghe An and Quang Ninh. This area contains several large urban areas (including Hai Phong and Vinh City, populations of 604,000 and 164,000 respectively) and for the most part is fairly densely populated, except for some remote mountainous districts inhabited by ethnic groups. Hai Phong province had the fourth-largest number of PLHIV in the country at the outset of program implementation, and prevalence among FSWs exceeded 10 percent.9 Nghe An Province also has a large number of PLHIV (7,131 in 2015), but fewer than half were estimated to be enrolled in treatment (3,036 or 42.6 percent).10 In addition to the large number of PWID in the three provinces, the estimated numbers of at-risk MSM in urban areas are high.11 COHED signed the project agreement with USAID/Vietnam in July 2014 and gained approval from the Prime Minister’s Office eight months later. The implementation period for Year 1 was about six months (March – September 2015). In Year 1, C-Link covered all seven districts in Hai Phong, three districts in Nghe An plus Vinh City, and two districts in Quang Ninh.12 The sudden PEPFAR pivot in fiscal year 2016 (Year 2 of C-Link) caused a drastic shift in implementation areas. One province (Quang Ninh) ended activities completely. Another (Haiphong) ended case finding and focused on providing care and support to PLHIV, including bringing people back into treatment 9 HSS, 2013 as cited in COHED’s 15-month Work Plan, July 2014 – September 2015; PEPFAR (2015), ibid. 10 PEPFAR (2015), ibid. 11 COHED, 15-month Work Plan, July 2014 – September 2015. 12 In this period, the program focused in four districts in Quang Ninh (Dong Trieu, Quang Yen, Uong Bi and Ha Long), three districts in Hai Phong (Ngo Quyen, Tien Lang and Kien Thuy) and five districts/cities/towns in Nghe An Province (Vinh, Cua Lo, Que Phong, Tuong Duong and Quy Chau). Evaluation of Community HIV Link Programs 16 (PEPFAR’s CARE_COMM indicator), accompanied by continued CBO strengthening. In Nghe An, which under the PEPFAR pivot is considered a PEPFAR aggressive scale-up province for 90/90/90 targets, districts shifted among USAID programs. C-Link began activities in 12 new districts plus Vinh City and Cua Lo. Activities ended in the three mountainous and rural districts of Tuong Duong, Que Phong and Quy Chau and shifted to the SMART-TA “mountainous model,” where hamlet health workers conduct outreach. C-Link also began working in 12 new districts in Nghe An this year. Year 2 of C-Link is the first that the USAID IPs offered support for the provincial 90/90/90 program targets. To contribute to Nghe An’s 90/90/90 goals, the program expected to meet 100 percent of the provincial targets along the COPC in 2016 and 2017. Targets for new HIV cases increased, from 437 over three provinces in the original M&E plan to 836 in Nghe An alone. The target for the CARE_COMM indicator13 for FY2016 and FY2017 increased from 642 to 1,361, including 409 for Hai Phong and 952 for Nghe An.14 This CARE_COMM target includes both HIV cases identified that were previously linked to the treatment program and new cases registered by OPC in the reporting period. 1.1.2 Progress Toward Targets on Case Findings in Each Reporting Period FY2015 Achievements:  All targets for the three project provinces (Quang Ninh, Hai Phong and Nghe An) across the COPC cascade were exceeded: achievements on the indicators were from 1.7 to 6.2 times the targets15 (Figure 1).  The percentage of new HIV-positive cases reached and registered to OPC was especially high (95 percent). Issues:  The proportion of new PLHIV reached out of reached KPs tested was 3.7 percent—lower than PEPFAR’s 5 percent benchmark for concentrated HIV epidemic area. It was much lower than the target at 10%.  While the C-Link achievements are impressive for Year 1, it is also clear that the targets set for this period were not ambitious enough. For example, the Year 1 work plan estimated the number of unreached PWIDs in Ngo Quyen, a single district of Hai Phong, to be 1,258,16 yet the target for KPs reached in the total area covered by the C-Link program was set at only 1,845. 13 Defined as “the number of PLHIV receiving care and support services outside of the health facility.” 14 This target was set to support Nghe An in increasing the number of HIV cases linked to treatment programs (C-Link targets aim to link 1,151 HIV cases to treatment programs over the remaining two project years). In FY2016, the program targets 952 cases, including 376 new cases registered in OPC and 576 HIV cases identified as previously linked to treatment programs. 15 Source for target numbers: 15-Month Work Plan (July 2014-September 2015); for achieved numbers: Annual Report (October 2014September 2015). The 15-Month Work Plan indicates starting outreach activities in October 2014. 16 This number was included in the 15-Month Work Plan (July 2014September 2015). Evaluation of Community HIV Link Programs 17 FIGURE 1: ACHIEVEMENTS VS. TARGETS IN OUTREACH ACTIVITIES FOR C-LINK YEAR 1 (OCTOBER 2014–SEPTEMBER 2015) IN THE NORTHERN COAST REGION FY2016 Achievements: To compare achievements with targets for the first two quarters of FY2016 (October 2015March 2016), the evaluation team used half-year targets (dividing the targets for the whole fiscal year in two).17 Figure 2 shows progress toward targets for the first half of FY2016. As noted, the targets for HIV testing and OPC enrollment were set only for Nghe An Province this year. C-Link began activities in 12 new districts plus Vinh City and Cua Lo. Activities were ended in the three mountainous and rural districts of Tuong Duong, Que Phong and Quy Chau. In this period:  Targets for the number of reached KPs tested and patients retained in ART for at least 12 months have been slightly exceeded (4,360 achieved vs. 4,186 target for testing and 121 achieved vs. 91 target for retention). Issues:  Most achievements in Nghe An were below the new targets for FY2016 during the first two quarters of implementation. The number of KPs reached (75.0 percent met); the number of new HIV-positive cases reached and registered to OPC (52.6 percent met); the number of HIV cases identified that were previously linked to the treatment program (36.2 percent met); and the number of new cases registered in OPC (CARE_COMM) (58.2 percent met) were not fully achieved. Therefore, the program would need to address HIV cases along the COPC at a much higher pace in the second half of FY2016 to reach its targets, particularly with regard to LTFU cases and registration at OPC. 17 The evaluation team used the indicators with terminologies matched in work plans and reports, and in definitions included in the program’s M&E plan. By definition and in the verification mechanism set up between COHED and Nghe An PAC, new HIV cases are KPs who are referred to the HTC by CBSs and have an HIV-positive result, then are verified by PAC. Evaluation of Community HIV Link Programs 18  The proportion of newly detected HIV-positive cases out of tested KPs declined from 3.7 percent in Year 1 to 2.5 percent in Year 2—still lower than the 5 percent benchmark of PEPFAR Vietnam’s prevention program for key populations in a concentrated epidemic.  An assessment of the targets for Year 2 in Nghe An (using the estimated KP size updated in March 2016 provided by COHED) suggests that the targets set for tested KPs and newly detected HIV-positive cases could be too high. Detailed elaboration of the target analysis will be provided in Section VI.1.1.4: Analysis of the Factors Influencing the Program’s Performance in Case Findings. FIGURE 2: ACHIEVEMENT VS. TARGET IN OUTREACH ACTIVITIES, Q1 & Q2 OF C-LINK YEAR 2 (OCTOBER 2015–MARCH 2016), NORTHERN COAST REGION 1.1.3 Contribution to Provincial Cascade and 90/90/90 Targets The evaluation team assessed C-Link’s contribution to the provincial cascade in Nghe An in FY2015 (Year 1) by comparing C-Link achievements to the service statistics reported by the province as a whole. To use a consistent approach across provinces, the team used provincial data from the VAAC. Data shows that:  In Year 1 the program contributed 66.9 percent (326/487) and 32.2 percent (309/960) of total newly found HIV cases reached by CBSs and registered to OPCs respectively (Figure 3). This is a reasonable contribution as in this period the program spanned only five out of 21 areas, covering approximately 56 percent of all KPs in the province. Evaluation of Community HIV Link Programs 19 FIGURE 3: CONTRIBUTION OF C-LINK TO PROVINCIAL ACHIEVEMENTS ON THE CASCADE, YEAR 1 (OCTOBER 2014–SEPTEMBER 2015), NGHE AN, NORTHERN COAST To estimate the contribution of the C-Link program toward the 90/90/90 targets for FY2016 and FY2017, the evaluation team compared achievements in the first half of FY2016 to the targets for the 14 areas currently covered by the program.18 Targets for the two-year period were divided by four and compared to achievements in the first half of FY2016. Figure 4 shows that:  C-Link is on pace to contribute 55.5 percent (116/209) of new HIV-positive cases and 69.8 percent (201/208) of cases previously linked to the treatment program.  This contribution should be weighed against the proportion of KPs in Nghe An Province covered by the program (65.5 percent) and the fact that the targets could be unrealistically high, as discussed. The achievements in the first two quarters of FY16 might not be a good estimate of future achievements, as the program was still being implemented and most CBOs and CBSs were new. The rate of new HIV cases reached by the program in these districts may decline over the next two years, per the general trend throughout the province. FIGURE 4: ESTIMATED CONTRIBUTION OF C-LINK TO 90/90/90 TARGETS, Q1 & Q2 OF YEAR 2 (OCTOBER 2015–MARCH 2016), NGHE AN, NORTHERN COAST REGION 18 Data on 90/90/90 targets in 14 districts/towns/cities in Nghe An Province are on page 16 of the work plan for FY2016. Evaluation of Community HIV Link Programs 20 1.1.4 Analysis of Influences on the Program’s Performance in Case Findings Enabling Factors COHED’s strong coordination and collaboration effort with local stakeholders, especially the government agencies in each province, from the outset of and during the implementation process of the program was the first key enabling factor. Good linkages were formed between the program and local government agencies in the first year of the program for planning and implementation. Specific examples are:  At the outset of C-Link implementation, COHED selected districts in the three provinces in consultation with provincial counterparts and using the CSO mapping conducted by Pathways.19 Selection prioritized high-burden districts, those without other service interventions and those with established CBOs.20  In the first year, targets of C-Link implementation were set through meetings with stakeholders, including consensus workshops with representatives from all involved government agencies in each province. The final targets were then negotiated and agreed upon with USAID.  The PACs participated in field visits and mentoring visits for the outreach workers and CBOs, and PACs and health staff participated in C-Link meetings.  Quarterly coordination meetings led by PAC have included participation from HIV-related KPs of CBSs/CBOs in all three provinces. These meetings allow CBOs/CBSs to receive updated HIV news at the provincial and national levels and for CBOs to share information on their project work with local government-agencies. In these meetings, representatives from health service settings (HTC/OPC) attended and provided feedback and recommendations for the referral linkage system between CBOs and public medical settings and discussed how to improve it for the better in the upcoming months. Several factors enabled the program’s success in supporting KPs for OPC enrollment and ARV therapy retention both years. These include:  Long-term relationships between a couple of HTCs/OPCs with CBSs who have conducted outreach on different programs for many years.  In each targeted province, project CBSs ensured that patients received appropriate treatment and care guidance from HTC counselors and, if appropriate, provided additional support for patients to register at an OPC at the patient’s convenience. A high percentage of the KPs the evaluation team interviewed expressed satisfaction with the services they received and the environment in which the services were provided (Figure 5).  Closed linkages in each area between CBSs and relevant OPCs facilitated by COHED’s HIV/AIDS community linkages coordinators assured the quality of services from community to OPC and clients to avoid overlap, and the successful enrollment of new HIV-positive clients. All of the HTC/OPC staff that the evaluation team interviewed saw the value of outreach in finding hidden cases, LTFU cases and dropouts from treatment. 19 USAID Pathways for Participation Project. (September 2012). Rapid Assessment of CSO Technical and Organizational Capacity. Report of Phase 1: Mapping of CSOs involved in HIV community-based response. 20 Program Description, USAID Community HIV Link: Northern Coast (Response to RFA No. SOL‐486‐14‐000001). Evaluation of Community HIV Link Programs 21 “There is a good linkage. HTC/OPC knows about CBOs in their area.” (Project coordinator for Hung Nguyen, Cua Lo, Thanh Chuong, Tan Ky and Nam Dan districts, Nghe An Province)  Dual enrollments to OPC and Methadol treatment for PWIDs contributed to a high percentage of OPC enrollments and ARV therapy retention. A couple of KPs revealed that enrolling to Methadol treatment helped them avoid going to a rehabilitation center for PWIDs. FIGURE 5: PERCENTAGE OF KEY POPULATION CLIENTS WHO ARE SATISFIED WITH SERVICES Challenges Two key reasons possibly explain the program’s shortfalls in achieving its targets for the proportion of HIV-positive out of tested KPs and re-engagement of LTFU cases; they are:  The 90/90/90 targets in a number of districts were over-ambitious: Examples of possibly over￾ambitious targets are: o The targeted number of KPs to be reached ranges from 1.1 to 2.3 times the total estimated number of KPs in 13 of 14 areas covered by C-Link. o The targeted number of KPs to be reached and tested is 1.3 to 1.6 times the total estimated number of KPs in three districts (Hoang Mai, Nam Dan and Nghia Dan). o In several new districts, a high number of KPs reached received an HIV test, but few positive cases were found. Nam Dan is an example: 355 KP individuals were tested (accounting for 79 percent of the annual target), but only three new positive cases were found (17 percent of the annual target). The likelihood of finding another 15 positive cases among the remaining untested KPs is very low. Similar observations may be made for other districts (see Annex I for district analysis). This implies that that HIV prevalence among KPs in these districts is lower than estimated or, as reported by many respondents in the field, that most positive cases have already been found. o The targets for LTFU cases reengaged in treatment are likely also too high. Achievement toward these targets is very low in the first half of the year; key informants in the districts visited said that most LTFU cases occur when a PWID is placed in a rehabilitation facility. Stakeholders noted the over-ambitious targets for districts: Evaluation of Community HIV Link Programs 22 “The target set by project for districts is difficult to achieve because it was based on provincial estimate. It is too high that is not realistic.” (Project coordinator for Hung Nguyen, Cua Lo, Thanh Chuong, Tan Ky and Nam Dan districts, Nghe An Province)  The shift in the project area in Year 2 required the program to re-establish linkages and networks and train CSPs. Most CSPs were new and were receiving training in the first quarter of FY2016. In addition, the program’s approach to forming new CBOs and spending time for capacity￾building activities for them could have dispersed the resources; hence mitigating the intensity and pace of the outreach activities. “Not all CBOs achieved their target, some months they got, some months they did not.” (Project coordinator in Vinh City, Nghe An Province) Table 3 shows the number of CBOs that have been involved in C-Link during Year 1 and Year 2 by province, and by whether or not the CBO was newly formed under the C-Link project or was previously established. Only two CBOs in Nghe An have been part of C-Link in both Year 1 and Year 2; one of these is a new CBO and the other was established a year before the program. This shows the lack of continuity in CBOs during the implementation period. TABLE 3: NUMBER OF CBOs IMPLEMENTING C-LINK IN YEAR 1 AND YEAR 2 BY PROVINCE AND DATE OF FORMATION Quang Ninh Nghe An Hai Phong Total New Previously (Total New) Established New Previously Established New Previously Established Year 1 only 2 3 6 3 0 1 15 (8) Year 2 only - - 8 3 0 0 11 (8) Both years - - 1 1 0 8 10 (1) COHED’s implementation structure for the Northern Coast includes district-level coordinators who work with the CBOs. Most coordinators cover more than one district; currently, three coordinators are in Nghe An and one is in Hai Phong. Coordinators conduct outreach training, liaise with the PAC and with the HTC/OPC and oversee M&E for their geographic area. Interviewed coordinators said their main challenge, the area that they spent the most time on, was helping the CBOs with reporting. This is a particular problem for new CBOs/CBSs, and with members who do not have much education. 1.2 Northern Mountains Region – CCRD 1.2.1 Program Overview and Local Context The original Northern Mountain project area included three provinces: Hanoi, Lao Cai and Dien Bien, which presented drastically different contexts for implementation. Hanoi is Vietnam’s second-largest city, and accordingly has the second-largest number of PLHIV and estimated key affected population groups. Lao Cai and Dien Bien have primarily ethnic populations and are two of the poorest provinces in Vietnam, with some of the highest rates of HIV prevalence in the country. These two provinces are mountainous with isolated communities, and travel is difficult, especially in the rainy season. CCRD signed the project agreement with USAID/Vietnam in May 2014 and gained approval from the Prime Minister’s Office three months after submission. After considerable planning, formative research and recruitment of outreach workers, field outreach activities started by the beginning of FY2015 Evaluation of Community HIV Link Programs 23 (October 2014 in Ha Noi). In Year 1 of implementation (ending in September 2015), the C-Link program area consisted of four districts in Hanoi (although the MSM outreach was not confined to those districts), six districts in Dien Bien and four districts in Lao Cai. The shift in PEPFAR focus at the beginning of FY2016 (starting in October 2015) considerably narrowed the program in the Northern Mountain region; only Dien Bien province continued full implementation, with the number of districts covered increasing from six to eight. These eight districts included four existing districts and four new districts. The four new districts included some of the poorest and most remote communities in the country. Also in FY2016, the programs’ targets increased substantially for case finding in Dien Bien to help the province achieve its 90/90/90 targets. Outreach activities in Lao Cai and Ha Noi were downscaled to fewer districts and maintenance services only; case finding ended by the close of 2015 with only CARE_COMM services (continuous care and support to PLHIV) continuing. 1.2.2 Progress Toward Targets on Case Findings in Each Reporting Period FY2015: Achievements:  Most of the indicator targets were greatly exceeded, with achievements ranging from 2.0 to 11.1 times the targets21 (Figure 6). The program has reached various types of KPs, including 5,179 PSPs, who made up 39.0 percent of the total KPs reached.  Statistics indicated that the C-Link program in three provinces successfully targeted and reached populations at high risk of HIV infection: o The aggregated proportion of reached KPs in the three provinces who were tested and returned to collect their results was 64.3 percent (8,532/13,269). This proportion is significantly higher than those identified in the Integrated Biological and Behavioral Surveillance (IBBS) 2009, with a range of 23.0 percent to 48 percent of PWIDs, FSWs and MSMs receiving testing and being aware of their HIV status.22 o The proportion of new HIV-positive cases reached23 out of KPs tested was 7.7 percent (653/8,532), which is higher than PEPFAR’s benchmark of 5 percent. This reflects the effectiveness of outreach activities in accessing hidden KPs. Issues:  Engaging HIV-infected people with OPCs remains a challenge in all three provinces. The aggregated percentage of new HIV-positive clients reached and registered with OPCs for all three provinces is 51.1 percent (334/653), with 55.7 percent (233/418) in Hanoi, 43.5 percent (70/161) in Dien Bien and 42.7 percent (32/75) in Lao Cai. This means around 50 percent of newly detected HIV cases did not register with OPCs. 21 The evaluation team used the indicators with terminologies matched in CCRD’s work plans and reports and included in the M&E plan. By definition and based on the steps in the verification mechanism set up between CCRD and the PACs in the targeted provinces, new HIV-positive cases are the KPs who are referred to HTC by CSPs and have HIV-positive results that PAC then verifies as new HIV cases. 22 Found in CCRD’s project agreement; no data was more recent than the 2009 IBBS. 23 By definition and based on the steps in the verification mechanism set up between CCRD and the PACs in targeted provinces, the new HIV-positive reached cases are the KPs who are referred to HTC by CSPs and have HIV-positive results that PAC then verifies as new HIV cases. Evaluation of Community HIV Link Programs 24  As with the targets for the Northern Coast, it is clear that the Year 1 targets were not ambitious enough for most indicators along the cascade. For example, the estimated number of PWIDs only in Dien Bien Province in 2013 was 3,173 as of June 15, 2014;24 the number of PLHIV in 2014 who were alive and managed by the public health system as of 9/30/2014 was 4,087.25 Comparing these figures to the target of 2,400 for reaching all KP groups, including PSPs in Hanoi and two other provinces, it is clear that the target figures were too low. FIGURE 6: ACHIEVEMENT VS. TARGETS IN OUTREACH ACTIVITIES, C-LINK YEAR 1 (OCTOBER 2014–SEPTEMBER 2015), NORTHERN MOUNTAINS REGION FY2016: As noted, the Year 2 implementation of C-Link (FY16) supports case finding, treatment referral, and care and support for PLHIV in Dien Bien Province (expanded to eight districts from six in Year 1). Case finding in Hanoi and Lao Cai ended before January 2016, and outreach continues for CARE_COMM only. In response to the 90/90/90 targets for Dien Bien Province, outreach and referral targets for C￾Link in Dien Bien have increased by approximately 300 percent compared to the total Year 1 targets for all three provinces. Figure 7 shows achievements for the first half of FY2016 (October 2015March 2016) toward targets for all of FY2016, divided by two. Achievements:  Achievement on each indicator exceeds the targets by a factor of 1.2 to 1.8. The program will likely achieve the annual targets by the end of FY2016.  The aggregated percentage of KPs who were tested and returned to get results for all three provinces increased from the previous reporting period, from 64.3 percent to 78.0 percent (5,964/7,692). In Dien Bien, this proportion increased from 73.5 percent (2,804/3,813) to 84.0 percent (3,957/4,736). The increase in the aggregated referral percentage in this province 24 Dien Bien’s Financial Sustainability Plan for HIV/AIDS activities in the period 2015-2020 25 Dien Bien’s Financial Sustainability Plan for HIV/AIDS activities in the period 2015-2020 Evaluation of Community HIV Link Programs 25 was contributed mainly by three of the new districts,26 which had a combined figure of 96.7 percent (964/997).  In all three provinces, the aggregated proportion of newly detected HIV-positive cases among tested KPs declined from 7.7 percent in the previous reporting period to 6.2 percent27 (368/5,964), largely due to a drop in Dien Bien Province from 5.7 percent (161/2,804) to 4.4 percent (174/3,957). Despite this, the figure remains higher than PEPFAR’s 5 percent benchmark. The analysis at the district level in Dien Bien during this period shows that—except for Dien Bien Dong and Dien Bien Phu, with proportions of 10.2 percent and 8.2 percent respectively—the proportions in the province were low, particularly in three new districts (ranging from 1.0 percent to 3.8 percent).  The percentage of new HIV-positive clients reached and registered with OPCs increased significantly, from 51.1 percent to 72.0 percent. This increase was especially high in Dien Bien Province, reaching 83.9 percent (146/176). Data analysis at the district level shows a significant improvement in the continuing C-Link districts, and percentages in the new districts are high. Issues:  An assessment of the targets for Year 2 (FY2016) in Dien Bien (using the estimated KP size updated in March 2016 by the Dien Bien PAC) suggests that the target set for reached KPs could be too high. The target for newly found HIV-positive cases may be achievable if the program can maintain the high percentage of KPs reached and tested from the first half of FY2016, and if the rate of new positive cases is at least 5 percent. Otherwise, given a possible decline in the rate of new positive cases (last period, it was only 4.4 percent), this target will be difficult to achieve, particularly given the difficulties of implementation in the new districts. FIGURE 7: ACHIEVEMENT VS. TARGETS IN OUTREACH ACTIVITIES, Q1 & Q2 OF C-LINK YEAR 2 (OCTOBER 2015–MARCH 2016), NORTHERN MOUNTAINS REGION 26 Muong Nhe, Nam Po and Tua Chua 27 PEPFAR’s threshold for a concentrated population on HIV is 5 percent. Evaluation of Community HIV Link Programs 26 1.2.3 Contribution to the Provincial Cascade and 90/90/90 Targets Analysis of C-Link’s contribution to the provincial cascade is presented only for Dien Bien Province in both implementation periods. An estimate of the program’s contribution to the 90/90/90 targets in Dien Bien Province was reached by comparing achievements in the first six months of FY2016 to half of the provincial targets for the calendar year 2016 (Figure 8). Data shows that the C-Link program largely contributed to Dien Bien COPC cascade’s achievements.  Figure 8 shows that in Year 1 (October 2014 to September 2015) C-Link contributed 85.6 percent (161/188) of the newly detected HIV-positive cases that CSPs reached and 82.3 percent (163/198) of PLHIV newly engaged in OPC (including both new and LTFU or dropout cases). This contribution reflects C-Link’s effective outreach to hidden high-risk KPs, as the contribution of the reached KPs during this period was only 64.4 percent (3,813/5,971).  In the first six months of FY2016, C-Link’s results accounted for 79 percent of new HIV-positive cases (174/221) and about 80 percent of total OPC enrolment in Dien Bien. Although the rate of newly found HIV-positive cases of tested KPs is low in a few new remote districts, the program continued to outreach effectively to hidden KPs in other districts. FIGURE 8: CONTRIBUTION OF C-LINK TO PROVINCIAL ACHIEVEMENTS ON THE CASCADE, YEAR 1 (OCTOBER 2014–SEPTEMBER 2015), DIEN BIEN PROVINCE, NORTHERN MOUNTAINS REGION  While C-Link made substantial contributions to the provincial cascade, it contributed less than Dien Bien’s 90/90/90 program targets. Specifically, C-Link’s contributions accounted for 41.9 percent of the half-year target for the number of high-risk KPs to reach and counsel, 25.6 percent of new HIV cases and 13.5 percent of ARV treatment clients (Figure 9).  The estimated contributions from C-Link to the Dien Bien 90/90/90 program results are far below the actual contribution to the provincial cascade during FY2015 and the first six months of FY2016, further indicating that the provincial targets have been set too high and are potentially unrealistic. Moreover, the inclusion of four of the poorest and most remote districts in FY2016, and the fact that outreach activities did not begin until December 2015, affected the program’s optimal effectiveness in case findings in quarters 1 and 2 of FY2016. Evaluation of Community HIV Link Programs 27 FIGURE 9: CONTRIBUTION OF C-LINK TO 90/90/90 TARGETS, Q1 & Q2 OF YEAR 2 (OCTOBER 2015–MARCH 2016), DIEN BIEN PROVINCE, NORTHERN MOUNTAINS REGION 1.2.4 Analysis of Influences on the Program’s Performance in Case Findings Enabling Factors: Implementation of C-Link programs in the Northern Mountains Region used a multi-pronged strategy, which could be considered a key enabling factor. In addition to a flexible and local context￾adapted outreach approach, advocacy and stigma reduction efforts were factors that effectively supplemented the program’s achievements.  CCRD used a flexible approach to align program activities with different local contexts. In Hanoi, CCRD supported CBO to provide outreach. Here, CCRD’s coach for HIV service delivery has built up all CBOs’ capacity to target high-risk KPs and achieve case finding along the HIV COPC cascade according to targets. In 2015, C-Link CBOs identified more than half of newly detected cases in Hanoi. The program applied an innovative initiative to encourage CBOs’ creativity and proactivity. Three CBOs whose proposals were accepted by CCRD received intensive support from the program to implement their ideas.  In Lao Cai and Dien Bien, the program worked directly with CSPs, but also greatly involved local health authorities/services. (In all provinces, PAC and health service providers were involved and coordinated closely with the C-Link program, but at different levels.) Specifically, in Dien Bien and Lao Cai, PACs noted that the program’s provincial coordinator played a useful role as a bridge between the PAC, DHC and CCRD. At the district level, DHC staff collaborate with the program and are responsible for managing the CSPs and providing close monitoring and support. This structure aligns well with management roles of the health system while strengthening and making use of the advantages of the community-based network to support the system to achieve its HIV/AIDS prevention and care targets. Their (CCRD/C-Link staff) approach to work with local partners is appropriate. (District health center staff in Dien Bien) What was different between C-Link and other USAID-supported projects before was that CCRD managed CSP directly. However, PAC was still involved as a body who oversees all HIV/AIDS-related activities in the province. This model became more important since the concept of CBO/CSO was relatively new for us, so we needed to get involved. (Lao Cai PAC) Evaluation of Community HIV Link Programs 28  The program achieved remarkable results in its advocacy efforts to promote an enabling environment for HIV prevention activities. These were done through a series of advocacy meetings and communication campaigns at both the provincial and community levels. Although no concrete data resulted, the objective to garner “understanding and support among local leaders” is, to an extent, achieved. This effectively created an enabling environment for CSPs, PLHIV and the C-Link program as a whole. “In the first year, many advocacy meetings were conducted by C-Link and PAC. I would say they were essential start-up. I could not imagine if the program can achieve anything without political support and commitment from all-level leaders, especially when the concept of having community-based groups involved in health service delivery.” (Lao Cai PAC) “When the program planned to organize an advocacy workshop in a district, I called district leaders in District People’s Committee and DHCs to tell them the importance of the workshop.” (Dien Bien Provincial Coordinator) Communication campaigns were also found effectively mobilize community support, as well as reduce the stigma against people living with HIV/AIDS. “As a result, there has been an improvement in social attitude toward PLHIV. People gradually consider HIV a ‘normal’ disease like other diseases.” (Dien Bien PAC)  The program has developed effective linkages/networking to the community as well as the health care system. As a result, the gap between HIV-infected people and HIV services is being filled. The network of relationships has a synergistic effect contributing to the program’s performance. A close relationship between CCRD and PAC was established early when C-Link got started and helped to form strong linkages between C-Link-specific activities and the current service system, including HTCs and OPCs. The close relationship between CCRD and PAC can be evidenced by how local government partners understand and position C-Link in the provincial HIV/AIDS program. “CSPs are important players. … Before C-Link, CSPs and HTC were not interacting often. The relationship was established during C-Link.” (HTC staff, Dien Bien Dong District, Dien Bien) “C-Link activities were well-aligned with the national targeted program. … The C-Link program provides a comprehensive support from outreach to care and treatment. The support of CSPs to ARV patients is an outstanding difference of the program as compared with other programs…” (District Health Center director, Dien Bien) Smooth networking between CSPs and health service providers benefited KP individuals from the community. Most of the KPs the evaluation team interviewed expressed satisfaction with the services they received and the environment in which the services were provided (Figure 10). Moreover, the success of the program in finding hard-to-reach and LTFU cases is valued by the health systems, further improving the relationships. We had very effective coordination and connection with CSPs from C-Link program. “They effectively helped us to find more clients, and we tried our best to make all procedures more convenient for clients… I think our collaboration mutually benefited both sides.” (Bao Thang OPC/HTC staff, Lao Cai) Evaluation of Community HIV Link Programs 29 “Staff of HTC and OPC knew CSPs. Their attitude toward CSPs is good. The commune health staff [who was involved in ARV treatment] was very friendly.” (A CSP, Dien Bien Province) FIGURE 10: PERCENTAGE OF KEY POPULATION CLIENTS WHO ARE SATISFIED WITH SERVICES CCRD used an effective outreach approach with an enhanced referral mechanism. CCRD used an effective approach in case finding activities, employing KP network-based outreach by CSPs and village health workers. The performance-based incentive policy highly motivated CSPs. Outreach activities were well connected with the service system by a referral mechanism, which were developed and mutually agreed upon with other stakeholders. “C-Link has provided high[ly] effective and systematic outreach activities with performance-based incentives. The program has a high coverage. Before C-Link, many projects carried out interventions in capital areas. C-Link, right at the beginning, had a policy to implement intervention in remote areas…” (PAC Dien Bien) “Involvement of community health centers [and community health staff] further increased the effectiveness of C-Link. CSPs knew the network of KPs, and health workers had medical expertise and [were] well respected by community.” (District OPC Chief, Dien Bien Dong, Dien Bien) “CSPs were very helpful in finding both new HIV cases [newly detected] and patients who dropped out their treatments. For dropped out cases, we just informed CSPs and they took care of everything very effectively in a short time.” (District OPC staff, Bao Thang, Lao Cai) In addition to HIV services, C-Link provided other helpful support to KP clients. CCRD decided at the beginning of implementation that all clients should have a valid ID or residential registration certificate, even at the testing stage. For those lacking these papers, the program provided support in obtaining them through the district coordinators or CSPs, who contacted local authorities. The rationale behind this strategic decision was that anonymous testing could actually increase self￾stigmatization. However, it also had the advantage that the process of helping clients who did not have an ID obtain one, which done at the outset and eased the transition from HTC to OPC. In addition to easing the verification process of new HIV cases and avoiding double counting, this policy likely will contribute to the rate of OPC registration for CCRD. Evaluation of Community HIV Link Programs 30 Challenges: Despite the positive achievements in the Northern Mountains, the difficult topography in very remote districts continues to create challenges. Low enrollment in OPCs is a remaining need for the province to address. One of the most important reasons for this low enrollment is the limited means of transportation for those who live in remote areas. The difficulty in transportation also affects CSPs’ ability to expand their networks across communes, thus impacting their ability to do outreach effectively. CSPs noted that, during the rainy season, the limitations extend beyond transportation allowances, and it is almost impossible to travel to certain areas of a district. On the other hand, availability of services is also a concern in mountain areas. “HTC locates very far away, and majority of KPs did not have a transportation vehicle. Even we had no difficulty in reaching and talking with KPs, it was still uncertain that they would come to HTC because they could go out of their village…” (A CSP in Dien BienDong District, Dien Bien) The weak capacity of CSPs also affected the program; nearly 60 percent of the CSPs were minority members and did not have adequate skills and experiences in community-based HIV/AIDS care and support. At the start of C-Link, no CBO or community service networks were in Dien Bien or Lao Cai. “Some CSOs’ education level capacity, and their capacity in communication was very poor. That’s why training was required, but it was enough but might not be sufficient. … It required time for actual practice.” (District Health Center staff, Dien Bien Dong, Dien Bien) The national M&E system for HIV/AIDS programs has not been standardized (as of the end of FY2015). Along with that, limited capacity of provincial M&E systems greatly affected the process of case verification and case finding efforts for C-Link. 1.3 Southern Region – LIFE Center 1.3.1 Program Overview and Local Context The C-Link agreement between USAID and LIFE Center is a fixed amount award/fixed obligation grant (FAA/FOG) rather than a cooperative agreement. For this reason, the Southern C-Link program is managed through milestones. LIFE Center signed the project agreement with USAID in May 2014, and GVN gave its approval in June 2014. The LIFE Center submitted reports on milestones as well as annual and semi-annual performance reports to USAID/Vietnam. Targets along the cascade were the same both project years:28  Reached KPs: 4,000 high-risk29 KPs (PWID, FSW and MSM)  Tested KPs: 50 percent  Newly detected HIV positivity rate: Not committed to in annual work plan, but 5 percent of tested KPs was verbally advised by USAID  Rate of newly detected HIV positive KPs enrolled to treatment: 80 percent 28 Source: Information provided by Ms. Nguyen Nguyen Nhu Trang, Director of LIFE Center in the document “Comment on Evaluation Findings” following the Validation Workshop 29 As defined by the technical SOP. Evaluation of Community HIV Link Programs 31 In Year 1, the Southern region encompassed three provinces: Ho Chi Minh City (HCMC), Can Tho City and An Giang. HCMC is the largest city in Vietnam and has the highest number of HIV cases (more than 26,000 in treatment during 2013), the biggest KP population (estimated 54,388 at the outset of the program) and the highest number of CBOs. While the city is estimated to have the largest number of KPs in each sub-group (PWID, FSWs and MSM), the number of MSM seems particularly large given the group’s invisibility in many other locations in Vietnam (MSM population size was estimated at 23,817 in 2013). In 2013, HIV prevalence in HCMC was measured at about 14 percent for MSM, 18 percent among PWID and 5 percent overall for FSWs (but 12 percent for street-based FSWs).30 An Giang is a small province in the Mekong Delta, but has a high number of KPs (about 4,000 in 2013) and PLHIV (4,492 in 2013). Can Tho City is the center of economic life in the Delta region; the number of KPs was estimated at about 10,000 in 2014, including 5,500 MSM.31 Can Tho has one of the highest numbers of FSWs among the Vietnamese provinces; HIV prevalence exceeds 10 percent for this group.32 In Year 2 of the Southern C-Link program, An Giang and Can Tho provinces were scaled down. Can Tho was reduced to CARE_COMM activities only from October 1, 2015, until May 1, 2016, while case finding continued in An Giang until December 2015 and scaled down thereafter. Full implementation continued in HCMC throughout this period. Year 2, starting in May 2015 for LIFE, overlaps with the first year of PEPFAR’s Country Operational Plan 2015 – COP 15 strategy, which focuses on achieving 90/90/90. However, LIFE’s target setting was not affected due to the fixed obligation contract. Milestones and cascade targets were the same as the previous year (including 4,000 people provided with HIV prevention services and 2,000 PLHIV provided with care and support with treatment adherence). 1.3.2 Progress Toward Targets on Case Findings in Each Reporting Period FY2015: Achievements: Figure 11 shows the level of target achievement for outreach activities in Year 1, from the beginning of implementation until April 2015.33 It shows that:  All milestones in providing HIV prevention packages and care and treatment to PLHIV were achieved; moreover, all targets were exceeded with achievements at 1.1 to 2.8 times the goals.34  Along the cascade, the program achieved the figure of 88.5 percent (3,782/4,273) for KPs who received HTC and returned to get test results (vs. 50 percent target); 7.2 percent (272/3,782) of 30 Center for Promotion of Quality of Life (LIFE). (September 2014). Review of COPC service coverage, needs and gaps in service use by key populations in Ho Chi Minh City and service delivery plan for Community HIV Links Project. Prevalence figures from IBBS, 2013 and HSS+, 2013. 31 Center for Promotion of Quality of Life (LIFE). (September 2014). Review of COPC service coverage, needs and gaps in service use by key populations in Can Tho City and service delivery plan for Community HIV Links Project. 32 Country Operational Plan 2015 – COP15. 33 The M&E Plan includes a timeline for all milestones. The last milestone for service delivery of reaching 4,000 KPs in Year 1 was set on April 15 (11 months after contract signing). In addition to the milestones in absolute values (providing prevention services to 4,000 KPs and care and treatment to 2,000 PLHIV), the contract includes targets along the cascade: 50 percent of reached KPs to receive HTC and return to get test results; 5 percent of newly found HIV-positive cases out of tested KPs; and 80 percent of newly found HIV-positive cases register to OPC. 34 The evaluation team used data from LIFE that tracked the progress of milestones for each CBO. By definition and based on the steps in the verification mechanism set up between LIFE and the PACs in targeted provinces, the newly found HIV-positive cases are the KPs who are referred to HTC by CBSs and have an HIV-positive result, then PAC verifies them as newly found HIV cases. Evaluation of Community HIV Link Programs 32 newly found HIV-positive cases among tested KPs (vs. 5 percent target); and 80.9 percent (220/272) of newly found HIV-positive cases registered to OPC (vs. 80.0 percent target).  The number of KPs who received HTC and returned to get test results in HCMC was even higher at 91.7 percent (2,517/2,744). During 20102013, the rate of PWID having been tested for HIV, or having tested and received results in the past 12 months, had always accounted for less than 40 percent.  The aggregated percentage of newly found HIV-positive cases among tested KPs was 7.2 percent (272/3,782), higher than the PEPFAR benchmark of 5 percent. This percentage in HCMC was 8.6 percent (216/2517), significantly higher than the provincial averages in 2014 and 2015 of 5.2 percent and 5.3 percent respectively.35 This reflects the effectiveness of outreach activities in reaching hidden KPs.  The total percentage of newly found HIV-positive cases registered to OPC was also high, 80.9 percent (220/270). In HCMC, this percentage was 88.0 percent (190/216).  The reached KPs consisted of 25.4 percent (1,085/4,273) PWIDs, 22.3 percent (951/4,273) FSWs and 52.4 percent (2,237/4,273) MSM. KPs in HCMC accounted for 64.2 percent (2,744/4,273) of the total in three project areas, of whom 59.9 percent (1,645/2,744) were MSM. Issues:  Though the program performed very well at the aggregated level, analysis of CBOs’ performance reveals that a few CBOs in HCMC had a very low performance. The percentage of newly found HIV-positive cases among tested KPs who were referred by the CBOs, namely Hoa Co May, Tinh Ban 1, Niem Tin (the CBOs of PWIDs and FSWs) and Song That, FGG, Sac Mau Cuoc Song, Aloboy (the CBOs of MSMs), ranges from zero to 4 percent. This is lower than the program target and PEPFAR benchmark of 5 percent. No HIV-positive KPs referred by Nu Cuoi and FGG were enrolled to OPC. Among these CBOs, Hoa Co May, Tinh Ban 1, Song That and Sac Mau Cuoc Song did not reach any KPs in this reporting period.  The aggregated percentage of newly found HIV-positive cases registered to OPC who were reached by the CBOs of PWIDs and FSWs (74.7 percent, or 62/83) was considerably lower than the number reached by the CBOs of MSMs (96.2 percent, or 128/133).  None of the reached KPs are PSPs. 35 Data was provided by HCMC PAC and includes the number of KPs tested and the number of newly found HIV-positive cases throughout the city. The PAC responded to the evaluation team’s data request saying that it was not possible to get data on newly found HIV-positive results only for KPs; thus the evaluation team estimated the provincial percentage of KPs using the PAC-provided data. Given the method used, the actual percentage on KPs must be lower than 5.2 percent (in 2014) and 5.3 percent (in 2015), as the nominators include the newly found HIV-positive cases in other sub-populations. Evaluation of Community HIV Link Programs 33 FIGURE 11: ACHIEVEMENT VS. TARGET IN OUTREACH ACTIVITIES, C-LINK YEAR 1 (OCTOBER 2014–APRIL 2015), SOUTHERN REGION FY2016: Achievements:  Targets for all indicators in the period MayNov 2015 were either met or exceeded. Achievement of each indicator was 1.0 to 3.8 times the target (Figure 12).  This period has much higher achievements along the cascade than the previous period. All percentages increased, including 91.1 percent (2,916/3,200) of reached KPs tested; 9.2 percent (268/2,916) of new HIV-positive cases found among tested KPs; and 91.8 percent (246/268) of newly found HIV-positive cases registered to OPC.  The amount of newly found HIV-positive cases registered with OPCs in HCMC reached 93.6 percent (162/173). PWID registrations increased significantly to 92.1 percent (58/63). Issues:  Similar to the previous period, a few CBOs in HCMC had low performance. The percentage of newly found HIV-positive cases among tested KPs who were referred by the CBOs, namely Tinh Ban 1, BGB (CBOs of PWIDs and FSWs) and Song That, FGG and Sac Mau Cuoc Song (CBOs of MSMs), ranges from zero to 4 percent. This is lower than the program target and PEPFAR benchmark of 5 percent. BGB and FGG detected no HIV-positive KPs. Sac Mau Cuoc Song was able to refer only one of six HIV-positive KP to OPC. None of these CBOs are well established.  The aggregated percentage of newly found HIV-positive cases registered to OPC that CBOs referred for PWIDs and FSWs (92.1 percent, or 58/63) was still slightly lower than for MSMs (94.5 percent, or 104/110).  None of the reached KPs are PSPs. Evaluation of Community HIV Link Programs 34 FIGURE 12: ACHIEVEMENT VS. TARGET IN OUTREACH ACTIVITIES, Q1 & Q2 OF C-LINK YEAR 2 (MAY–NOVEMBER 2015), SOUTHERN REGION 1.3.3 Contribution to Provincial Cascade and 90/90/90 Targets Using data provided by the VAAC for FY2015 and milestone data from LIFE, the evaluation team estimated the contribution of C-Link to HCMC’s cascade achievement. The milestones covered October 2014 to August 2015, so the program’s service data for September 2015 to March 2016 was not included. The reports did not include numbers of OPC registrations for both new and old cases, so the team estimated this figure by adding the number of newly found HIV-positive cases to the number of previously diagnosed cases and ART dropout cases for which the program provided care and support.  C-Link’s contributions to the target for numbers of tested KPs and newly found HIV-positive cases were quite low at 6.5 percent (3,559/54,904) and 17.3 percent (306/1,764) respectively (Figure 13).  The contribution of new registration cases for OPCs is in line with expectations at 56.1 percent (1,001/1,785). FIGURE 13: CONTRIBUTION OF C-LINK TO PROVINCIAL ACHIEVEMENTS ON THE CASCADE, YEAR 1 (OCTOBER 2014–SEPTEMBER 2015), HO CHI MINH CITY, SOUTHERN REGION Evaluation of Community HIV Link Programs 35 To estimate the program’s contribution to 90/90/90 targets in HCMC, the evaluation team compared achievements over the 11-month period from January 2015 to November 2015 against provincial targets for the calendar years 2016 and 2017 in HCMC (reduced by a factor of 11/24 to account for the difference in time periods), as Figure 14 shows. The estimated C-Link contribution to 90/90/90 targets is 11.5 percent of newly found HIV-positive cases and 48.2 percent of new registrations at OPCs. These proportions are similar to the estimated contributions to the HCMC cascade in FY2015. However, limited data availability and mismatching between the program’s reporting system and governmental planning and M&E systems mean that this estimate should be interpreted with caution. FIGURE 14: CONTRIBUTION OF C-LINK TO PROVINCIAL ACHIEVEMENTS ON THE CASCADE, YEAR 1 (OCTOBER 2014–SEPTEMBER 2015), HO CHI MINH CITY, SOUTHERN REGION 1.3.4 Analysis of Influences on the Program’s Performance in Case Findings Enabling Factors: In the difficult and complex environment of Ho Chi Minh City, LIFE has made systematic progress toward achieving outreach targets, contributing to provincial 90/90/90 targets and building CBO capacity. It uses several implementation approaches that contribute to its successful performance:  LIFE Center effectively applied a standard approach to support CBOs in provision of HIV services while prioritizing CBOs for institutional capacity strengthening. For HIV services provision, the center followed standard interventions. It systematically developed the annual work plans and SOP, formalized cooperation between LIFE and PAC and CBOs and HTC/OPC through a model “CBO and strategic districts” and met monthly with among stakeholders or need-based coaching and mentoring to CBOs. For institutional capacity building, the center selected eight CBOs with the greatest potential for further development and sustainability, according to the results of the CBO organizational capacity assessments (OCAs). This approach enabled the program to work on both program objectives but did not disperse resources so much to endanger the program’s efficiency and effectiveness. Evaluation of Community HIV Link Programs 36  LIFE Center effectively engaged the PAC in three project areas to formalize the linkages among the Center, PAC, health services and CBOs. The program’s work plan was incorporated into the provincial plan.  The establishment of the lead CBOs and formalization of “lead CBO and strategic districts” has fostered good relationships and effective coordination between health services and CBOs and made the program run more efficiently. For any problems with the CBSs of the lead CBOs, health staff contacted the lead CBO leaders to solve the problems. “Collaboration [that] follows a standard procedure will be effective. The formalized linkage between a CBO and a strategic district is effective. Only (lead) CBOs should refer clients to the friendly OPC.” (HTC and OPC in District 7, HCMC)  The lead CBOs use specific eligibility criteria to enforce and monitor careful screening of eligible KPs to provide prevention services. This has contributed to the high aggregated percentage of reached KPs receiving HTC and returning to get test results in Year 1, at 91.7 percent (2,517/2,744) in HCMC.  Lead CBOs’ ability to assist PLHIV without residence certificates through being a guarantor for these clients contributes to the percentage enrolled at OPCs. HCMC contains a large number of migrants, who often lack a residence certificate for the city. This affects their ability to register for treatment within the city. The lead CBOs in strategic districts were able to be a guarantor for these clients using a standardized guarantee form (as long as the client has an ID card). “[To enroll to an OPC] is difficult as it requires health insurance; and the documents such as a residence registration logbook or a temporary residency registration card. With the mechanism of friendly HTC/OPC connection, the friendly HTC/OPC is willing to receive the KPs who do not have sufficient required documents. A friendly HTC/OPC has a list of CBSs with their photos attached with an official letter from PAC.” (Leaders of G3VN CBO)  CBOs were motivated to achieve the given targets when payment was linked to targets along the COPC. According to LIFE Center, each CBO was assigned milestone and cascade targets; if they did not meet targets for that month, the CBO was not reimbursed. LIFE strictly implemented this mechanism, withholding payments from CBOs that fell short. “An advantage of using CBOs to do outreach is that referred clients are followed up after referral. If only peers are used, follow-ups of clients may not be carried out after referral.” (Leaders of G3VN CBO)  The mechanism empowered CBOs instead of paying individual CBSs, giving CBOs the autonomy to distribute the total amount to their different activities within guidelines and with the approval of LIFE. This reinforced the CBO structure and enhanced accountability of the CBO leaders for the given targets. The four CBOs that the evaluation team visited applied the following distribution structure for the received incentive: o 50 percent is spent on travel costs for the CBSs. o 35 percent is spent for group meetings on special topics. Each session is attended by about 10 KPs. Each KP receives VND 30,000 for travel costs. o 15 percent is spent on the operation and administration of the CBO.  Effective linkages between KPs and CBSs and health services were crucial. The interviews with KPs identified that a high percentage of interviewed KPs were satisfied with services provided by CBSs, HTC and OPC (Figure 15). Evaluation of Community HIV Link Programs 37 FIGURE 15: PERCENTAGE OF KEY POPULATION CLIENTS WHO ARE SATISFIED WITH SERVICES  Effective institutional capacity building for well-established CBOs that LIFE Center had a previous relationship leveraged effective provision of HIV services of these CBOs from the beginning of the program. A few strong MSM CBOs such as The Boy, M for M, G-Link and G3VN particularly seem to have benefited from the capacity-building activities and performed well in consistently providing HIV services both years.  Using CBOs that run multiple HIV programs at the same time was considered an advantage. It allowed CBOs and CBSs to increase income and maintain outreach activities continuously. A combination of CBSs’ outreach and lay testing could have contributed to an increase in the HIV-positive rate of tested KPs. “Lay tests are useful for busy people. Only people with positive results to lay tests are referred to HTC for a confirmative test. The positive rate among reached people has increased from 4 percent to 7 percent.” (Leaders of G3VN CBO) “G3VN is implementing lay tests. Hence, the KPs referred to the HTC are very likely to be HIV-positive.” (HTC in District 7)  The pressure of having all HIV-positive KPs enrolled to OPC before a complete withdrawal of international support for ARV treatment contributed to the observed high rate of OPC enrollment in Year 2. According to the LIFE Center, all CBOs reinforced the slogan “Now or never” to all CBSs.  The targets set in the original project agreement staying stable from the beginning to the end, without a sharp increase in FY2016, was a favorable element. Challenges:  Challenges in working within the complex urban environment still affect C-Link’s performance in HCMC. The large number of migrants from outside HCMC and a mobile population who may seek services outside their district of residence creates a difficult situation for outreach workers. While the lead CBO system has greatly improved complaints about CBOs that have CBSs who try to manipulate the system by bringing in clients for multiple tests, these are difficult to solve completely. Evaluation of Community HIV Link Programs 38 “Phap Bao, Vuot Song, Alo Boy, Hoa Co May and Cuoc Song Moi CBOs referred many old KPs who were not required to have a repeated HIV test because of the target pressure. The CBSs brought many KPs at the same time when the HTC was about to have a lunch break.” (HTC in District 7) “We may not receive KPs referred by Vuot Song CBO or CBO numbered 62, as most of the referred KPs do not have a required papers.” (HTC in District 11)  Weaker CBOs tend to have a higher turnover of CBSs, resulting in the observed underperformance of a few CBOs listed in the performance section. “Tinh Ban, Vuot Song CBOs and the CBO numbered 62 have a high CBS turnover. CBSs of G-Link and M for M CBOs are more stable.” (HTC in District 11)  CBOs of PWIDs and FSWs tend to face more challenges in establishing trust and linkages with health services and local authorities, partly due to the existing community’s resistance. This affected CBOs’ ability to provide effective support to CBSs, especially when they need a temporary residential certificate from the local authority. “We have been able to build trust with HTC/OPC but have not gained credit from the Ward People’s Committee. We have not been able to guarantee for the drug users to the People’s Committee when they need to get a Committee’s certificate.” (Leader of CBO Vuot Song) “Police may catch me together with other FSWs. They may think that I have informed the police to come so they lose credit for me.” (CBS of CBO Nu Cuoi)  PWIDs and FSWs tend to have a lower socio-economic status and awareness of the necessity of HIV testing and ARV treatment. CBS reported that PWIDs and FSWs faced a special challenge in continuing ARV treatment during the campaign “Three zero.” This contributed to the lower percentage of KPs tested out of those reached and the lower percentage of OPC enrollment out of HIV-positive KPs. “A KP, who was using Methadol and on the way to an OPC for taking ARV medicine, was caught. The CBO Vuot Song had to give a say.” (Head of the CBO Vuot Song) “Clients (FSWs) do not have money, an ID card, a residence place. They do not have any identification paper after returning from a rehabilitation center.” (CBS of CBO Nu Cuoi) “A drug user who was craving for drug was hard to reach.” (CBS of CBO Vuot Song) “For CBSs who do not have enough required papers, it’s very difficult. CBSs can’t go to receive medicine on behalf of them forever.” (CBS of CBO Vuot Song)  CBSs reported increasing difficulty for KPs to enroll to OPCs in the transition period. They were concerned that many KPs would drop out of ARV treatment. “A KP who is temporarily residing in HCMC cannot buy health insurance. The cost for lab test of liver enzyme is three hundred thousand dongs. The cost of viral load test is 1.5 million dongs. Many KPs may drop out from treatment.” (CBS of CBO Vuot Song) “To buy health insurance is difficult as it requires a person to buy with his/her family in the same residence registration logbook. Second, clients usually do not want to have ARV treatment in their living area.” (CBO G3VN) Evaluation of Community HIV Link Programs 39  Outreach to PSPs does not appear to be a priority. Interviews with CBSs and CBOs indicated that they did not know what approach would be effective to reach PSPs. They reported that KPs changed sex partners despite a requirement prohibiting this. The program’s annual work plans both years included sex partners of the PWIDs as a KP and the SOP includes a guidance to encourage PSPs to take HIV tests. “KPs change their sex partners frequently so CBSs do not prioritize to reach this group.” (CBSs of Vuot Song CBO) “MSMs’ sex partners are not a priority group because MSMs do not have a permanent sex partner. For the MSM who are infected with HIV, CBSs encourage them to bring their current sex partners to have an HIV test.” (CBO G3VN) 2. PERFORMANCE IN LINKAGE AND NETWORKING AND STAKEHOLDER RECOGNITION EVALUATION QUESTIONS To what extent are the programs implemented by the three local partners achieving their objectives of and enhancing coordination and networking with key local stakeholders? To what extent do stakeholders recognize the contributions of implementing partners in supporting sustainable HIV/AIDS responses? 2.1 Overview of Programs’ Linkages and Networks Figure 16 illustrates a compilation of all types of primary linkages and networks in three C-Link programs: between government and IPs, between IPs and CBOs and, most critically, between outreach workers and their clients. FIGURE 16: PRIMARY LINKAGES AND NETWORKS IN C-LINK PROGRAMS As mentioned, three IPs have implemented different approaches to implementation and management that shaped different linkages and networks. Coordination across various stakeholders was organized according to the selected approach. Evaluation of Community HIV Link Programs 40  CCRD chose to work with existing CBOs in Hanoi while recruiting CSPs individually in Lao Cai and Dien Bien provinces. In these two provinces, CCRD engaged PAC and DHC staff to provide support for managing, coordinating, coaching and supervising CSPs. CSPs have monthly meetings with district coordinators that representatives from HTC and OPC attend. The provincial coordinator and CCRD staff join in this meeting occasionally when necessary.  COHED supported existing CBOs in urban areas such as Vinh and Hai Phong City, but it established new CBOs in rural districts in Nghe An Province based on the existing networks of KPs. Nevertheless, COHED paid the CBSs directly. A quarterly meeting of all CBOs, HTCs, OPCs, PAC and COHED takes place in Vinh City.  LIFE primarily supported existing CBOs in all project areas. It established two new CBOs to cover two large districts that one CBO could not cover. In HCMC, each CBO was formally linked with a strategic OPC, following guidance in an official letter from the HCMC PAC. A monthly meeting takes place for CBOs, HTCs, OPCs, LIFE and PAC. In all provinces/cities, PAC is the only authorized agency that can provide a certificate for KPs to be peer educators. Local authorities are engaged in the activities for creating an enabling environment, such as community cultural events, advocacy workshops (for CCRD and COHED) and application for legalization of CBOs (for LIFE). Leaders of many CBOs join regional or national networks of KPs and participate in regular activities or access information provided through these networks. 2.2 Stakeholder Recognition and Assessing the Enhancement of Coordination and Networking with Key Local Stakeholders Overall, all visited PACs were positive about the C-Link programs. The PACs acknowledged that the design of the program—using an incentive-based system to motivate peers to reach high-risk key populations and link them to testing and treatment—was the best way to find hidden cases and achieve the 90/90/90 targets. They appreciated that C-Link fills gaps in the government system and meets the needs of vulnerable populations. “C-Link is part of our total strategy to reach the 90/90/90 targets.” (Provincial coordinator, Nghe An PAC, Northern Coast) One reason that local stakeholders and agencies appreciate C-Link is that its programs are highly relevant to national and local needs and plans. C-Link programs’ achievements and objectives contribute to the objectives of the national HIV/AIDS prevention and treatment strategy (planned through 2020 with a vision toward 2030) and provincial plans of all visited provinces. The overall programs’ community-based approach is congruent with the National Guideline for Management, Treatment and Care of HIV/AIDS 2015. The interviews of various stakeholders at the district and provincial levels highlighted the need for outreach activities; during the period of intensive efforts for achieving 90/90/90 targets, active case finding is especially required. The peer-driven approach was acknowledged as having many advantages, especially in reaching hidden KPs. The programs’ contributions to reducing stigma in the communities and providing care and support to PLHIV were especially noted. As the program name suggests, linkages are the foundation of C-Link. This section analyzes each linkage that is critical to the effectiveness of the program, as well as the networks that form its basic strategy. Due to the lack of a baseline on the status of the described linkages and networks prior to the C-Link programs, this evaluation could not apply social network analysis methods in a robust manner. Informant interview and desk review were the main sources of evidence to assess the strength of coordination and networking. Stakeholder recognition is used to reflect the strength of each the linkages. Description of specific joint activities illustrates actual implementation of a linkage. Evaluation of Community HIV Link Programs 41 2.2.1 C-Link Linkages: PACs/VAACs and IPs Achievements:  The Vietnam Administration for HIV/AIDS Control (VAAC) acknowledged that the three IPs are competent, very active in contributing to the COPC and working well within the health system. All three IPs have cooperated well with the PACs, and the PACs appreciated the IPs’ contribution. According to CCRD, Hanoi PAC even took the initiative to approach them and came to meet with them to learn about their experience in working with CBOs in case finding, care and support. “CCRD, COHED, LIFE are very active in COPC within the health system; HIV/AIDS specialization is not established in the communities and the CBOs thus contribute to changing attitudes. These three agencies are active in the CCM of GF. There are doing lay testing with VUSTA & CBOs in parallel—now collaborating; providing needles and condoms.” (VAAC’s Deputy Director)  The following good practices of the IPs resulted in the described effective collaboration between IPs and PACs and helped the IPs gain credit: o Worked with the PACs in the project areas from the beginning of the program to create consensus and joint ownership. This was essential, as the PAC coordinates all HIV/AIDS interventions in the province. o Conducted pre-implementation meetings and consultations on formative research. o Developed the referral system and validation process for the program, through a close consultation with the PAC. o Carried out regular monthly meetings among all key stakeholders that included PAC’s leadership. o Incorporated the project work plan with the provincial HIV/AIDS annual work plan in the project areas through engagement of the provincial coordinator (for the Northern Mountainous-CCRD program) or focal provincial staff in charge of providing oversight of the program at the PAC. o Worked flexibly, responsively and professionally: CCRD has done community based outreach very well. It has produced results. Interventions are systematic. The program coverage is large. They apply result-based management… (Dien Bien PAC) I have learned a lot from CCRD’s professional working style. I have learned about data synthesis and interpretation. My planning skills have been enhanced. CCRD is flexible and listened to the local feedbacks. (Provincial Coordinator/Specialist in IEC department of Dien Bien PAC) Issues:  For the Northern Coastal – COHED program, the unexpected end of the project activities in the mountainous districts at the end of the first year put this IP in a difficult situation in communicating with HTCs/OPCs in these districts. The situation definitely was unpleasant for both sides. Evaluation of Community HIV Link Programs 42 2.2.2 C-Link Linkages: CBOs and PAC Achievements:  VAAC’s recognition of the contribution and roles of the CSOs in HIV/AIDS prevention, care and treatment has been concretely stated in Vietnam AIDS Response Progress Report 2014 and National Guideline for Management, Treatment and Care of HIV/AIDS 2015 issued with the Health Minister’s Decision 3047/QD/BYT dated July 22, 2015. In this guideline, peer groups and self-help groups are listed as a provider of care of PLHIV. Self-help groups or CBOs can carry out outreach to high-risk KPs and provide counseling and testing for HIV in difficult topographical areas (such as mountainous or remote areas) or when KPs do not come to a health facility for counseling and testing due to stigma and discrimination. Civil society organizations (CSOs) play a critical role in implementing prevention interventions at community level, such as; Information, Education, and Communication (IEC) of Behavior Change Communication (BCC), peer education, Voluntary Counseling and Testing (VCT), condoms, needle & syringe distribution, legal counseling and support. (Vietnam AIDS Response Progress Report 2014)  PACs have recognized the roles of CBOs in HIV prevention, care and support activities, especially the well-established ones in urban areas. PACs in HCMC and Hanoi acknowledge a significant contribution to detecting hidden positive HIV cases by CBOs. The PACs in HCMC, Hanoi, Vinh and Hai Phong cities have been collaborating with CBOs, particularly in information￾education-counseling activities, and sought support from these CBOs on their visits to HIV￾affected families, especially on the occasion of World AIDS Day. In return, PACs provide support to CBOs. For example, Sun Flower CBO in Dien Bien Province is provided a room in the Dien Bien PAC building to be used for their office.  PACs’ recognition of the value of CBOs has been cemented through including CBOs as a service provider in the provincial plan. In HCMC, the 90/90/90 strategy for 2017 includes CBOs as a key player to promote outreach, pilots for lay testing, self-testing and care and support for LTFU cases. In Dien Bien Province, the provincial plan for 2016 recognized the 2015 achievements and contributions of 11 clubs for HIV/AIDS prevention and three Sun Flower groups and included plans to continue these clubs’ and groups’ activities in outreach, IEC, care and support.  In large cities such as HCMC and Hanoi, CBOs participate in regular PAC meetings on HIV prevention and treatment. It is not possible to attribute the observed recognition to C-Link programs or any other programs individually. However, as the C-Link programs have supported CBOs in doing outreach to KPs and providing care and support to PLHIV, it is reasonable to believe the programs must have contributed to the national recognition and a more enabling environment for CSOs. Issues:  In less urban areas, such as Lao Cai Province or the mountainous and rural districts in Nghe An Province, PACs had minimal experience/interaction with the CBOs. This was particularly true in Nghe An Province where many of the CBOs were newly established.  A lack of legal status of many CBOs was seen as a barrier to their being able to sign a contract to provide services to local agencies, both public and private. This will likely prevent the current outreach model from moving forward once C-Link ends. Evaluation of Community HIV Link Programs 43  In all visited provinces and cities, CBOs were not consulted or invited to participate in the development process of a provincial plan for HIV prevention, care and support.  Concrete evidence of the link between CBOs and PAC is not reflected in provincial budgeting, even with the provincial plans that included activities supposed to involve CBOs. PACs tend to expect CBOs to contribute voluntarily to HIV prevention activities. 2.2.3 C-Link Linkages: CBOs and HTC/OPC Achievements:  Well-established CBOs—particularly MSM CBOs and a couple CBOs of PWIDs and FSWs in urban areas such as HCMC, Hanoi and Vinh City and a CBO of PLHIV “Sun Flower” in Dien Bien Phu City—have gained credit and thus maintained good relationships and effective collaboration with HTC/OPC. Leaders of the visited HTC/OPC know heads or deputy heads of these CBOs and contacted them whenever there were outreach issues with CBSs or when they needed to engage CBOs in HIV prevention and treatment activities. Most of the health providers interviewed at the HTCs/OPCs visited could give names of the leaders of the well￾established CBOs that have collaborated with them closely on outreach activities.  Well-established CBOs demonstrated their competence, commitment and enthusiasm in outreach activities. The turnover rate of CBSs of these CBOs is low, so the link with HTC/OPC has been stable. They have been very responsive to HTC/OPC requests and requirements. For instance, the head of a CBO that has registered as a social enterprise who is a lawyer came to meet the head of an HTC soon after receiving a call from the HTC requesting a meeting to discuss issues about client referral. The responsiveness and close collaboration helped to maximize the rate of tested KPs who returned for receiving HIV test result. Number of KPs who did not return to get test result has been very low. If there is a KP who does not return to get test result, we will call the head of the CBO. If there were many KPs referred by a CBO who did not return to get test result, we would not let that CBO to refer more KPs to our facility. This requirement has helped to ensure KPs to return to get test results. (HTC in District 11, HCMC)  In HCMC, OPCs shared the benefits of having the formal arrangement of a strategic district or a “friendly OPC” for a CBO to refer KPs for treatment and guarantee for KPs who did not have permanent residence certificates in the district where the friendly OPC is located but desired to have ART there. HCMC PAC sent a letter to OPCs to inform them if they were a friendly OPC for a specific CBO. This arrangement facilitated effective communication and collaboration between friendly OPC and CBO, resulting to a strong link between them. With G3VN [name of a CBO], there has been a formal collaboration with a signed letter. It referred MSMs and most of them are new. They did not refer repeatedly old MSMs. The management committee of G3VN has managed the CBO well, based on targets. They are young and more competent than CBOs of PWIDs and FSWs. (HTC/OPC in District 7, HCMC)  IP-organized monthly or quarterly meetings with participation of representatives from CBOs, HTC/OPC and IPs were useful for all sides to discuss issues that emerged in the last month or quarter. The meetings were the opportunities for all sides to interact and strengthen their link. Evaluation of Community HIV Link Programs 44 Issues:  A few HTCs/OPCs in HCMC complained about performance of the CBOs of PWIDs and FSWs for which the HTC/OPC was not designated as a friendly HTC/OPC. CBSs of these CBOs referred old KPs repeatedly, referred KPs to many HTCs and/or referred KPs that were not legitimate as required. They brought many KPs at once around the end of the month and demanded HTC’s verification right away. The CBSs did not follow the SOP because of target pressure. The interviewed health providers listed a few names, such as Phap Bao, Vuot Song, AloBoy, Hoa Co May and Cuoc Song Moi.  CBOs in Nghe An province have fewer direct regular interactions with their respective HTC and OPC. The C-Link program’s coordination system includes a field coordinator who acts as an intermediary between CBOs and HTCs/OPCs for data verification and management issues. CBOs mainly communicate with her. She has interacted directly with CBOs, HTCs/OPCs and COHED. The standardized management procedure of the program does not require HTC/OPC representatives to attend monthly meetings of CBOs. There is a quarterly meeting for CBOs and HTC/OPC representatives in Vinh City, but due to a large number of attendees, the chance for an in-depth interaction between heads of CBOs and HTCs/OPCs in the same district could have been limited.  A few CBOs of PWIDs and FSWs have had a high turnover rate of CBSs, and that has affected the link with their respective HTC/OPC. The CBOs of MSM have a low turnover rate, so the link between them and HTC/OPC has been more stable. Tinh Ban and Vuot Song CBOs in HCMC were cited as examples. 2.2.4 C-Link Linkages: HTC/OPC and CBS/CSP Achievements:  Interviewed health providers in HTCs/OPCs visited and health officials in general in local health facilities highlighted the advantages of CBSs and CSPs as peers of KPs in outreach, provision of care and support to KPs and PLHIV. They acknowledged and appreciated contributions and support of CBSs/CSPs in finding cases and bringing back LTFU cases. Before having CSPs, HTC/OPC did not have many clients every month. So we highly value CSPs’ roles. (Dien Bien Dong DHC, Dien Bien Province) In the remote villages, the CBS could bring in cases that wouldn’t go to the village health workers. (Nghe An, HTC) If the CBOs/CBS did not bring them in, we wouldn’t have any clients; our location is difficult to find. (HCMC, HTC staff)  CBSs/CSPs reported HTC/OPC staff having been very supportive and not stigmatizing or discriminating against them.  Many HTCs and OPCs have long-term relationships with CBSs and CSPs who have done outreach on different programs for many years. Issues:  Many HTCs and OPCs that the evaluation team visited reported problems with CBSs and CSPs who try to manipulate the incentive system by bringing in low-risk clients for testing and Evaluation of Community HIV Link Programs 45 bringing the same clients in multiple times. The IPs have come up with solutions to this, such as changing the incentive system to place quotas on the proportion of negative cases (e.g., one in 20 cases must be positive). While the problem appears to be improving with time, it is still an issue that can be a barrier for optimizing the programs’ effectiveness. Health providers whom the evaluation team interviewed at the HTCs and OPCs that they visited shared that CBSs and CSPs knew how to play around in client referral depending on whether the incentive is paid based on the proportion of positive or negative cases. If the incentive is paid without giving a target for the positive rate, CBSs and CSPs will bring many non-high-risk cases in. If the incentive is paid with a target for the positive rate, they will bring KPs to multiple HTCs, which increases the burden for verification of newly detected cases. They [CBS] brought even taxi motorbike drivers or tea sellers to HTC. They brought a KP to HTC several times. They did not bring a KP if they knew the KP’s test would be positive as the positive cases would require verification by an OPC. The program fails its objectives due to this malpractice. (HTC and OPC in District 7, HCMC)  In HCMC, during the transition period to move forwards to using social insurance for HIV services, the emerging paper requirements and stricter screening of clients for enrollment with OPCs has created a tension between CBSs/CBOs and OPCs, as OPCs could not enroll the KPs without a resident certificate for the district in which OPC is located. The OPC in District xxx required KPs to show ID cards until then they provide counseling and HIV test. They are unpleasant sometimes. We have to accompany with KPs. In case we could not persuade them, we had to contact LIFE to solve problem… I still keep a record of our argument (A CBS of Vuot Song CBO)  The link between CBSs/CSPs and HTCs/OPCs or public health systems has by no means been institutionalized, so CBSs/CSPs will not be continuously used for future outreach activities after the programs end. While highlighting the outstanding advantages of CBSs/CSPs as peers to KPs who can reach hidden, high-risk cases, health providers and officials were very straightforward that village health workers would be the choice for future outreach activities once the programs end. The new role to carry out HIV outreach activities can be added to village health workers’ job descriptions without any added cost. This is aligned with the HIV/AIDS law that stipulates that only specialists can access information of PLHIV. We will use village health workers for outreaching to KPs and PLHIV. The provincial health department is managing them so we only need to add a task to their job description. They have been receiving monthly allowance. (Dien Bien PAC) 2.2.5 C-Link Linkages: CBOs and Communities Achievements:  There is some evidence that communities give credit to CBOs and establish official linkages with them. In the Southern region, recognition from local authorities was noted in the OCAs. Types of relationships are diverse. For example, in HCMC, the Head of Nu Cuoi CBO has established a good personal relationship with an official from the ward people’s committee that enabled this CBO to have office space in the community center. G3VN CBO obtained a membership certificate from the HCMC AIDS Association. Vuot Song CBO was included in the decision letter from the Commune People’s Committee to provide social support for PWIDs after rehabilitation. Members of Vuot Song CBO also join the Empathy Club organized by the Ward’s Women’s Union. Evaluation of Community HIV Link Programs 46  With the described links established with communities, CBOs have gotten more supportive attitudes from the ward authorities and police for their activities.  The support and credit gained from the community have resulted from various activities and factors, including the C-Link programs’ community events, advocacy activities that help to reduce stigma in communities and form relationships. Issues:  Interviewed CBOs reported occasional resistance from community leaders. Stigma toward MSM, PWID and FSWs still creates tension for CBOs. One CBO that the team visited was located in the CBO leader’s home in a residential area; they have had issues with their neighbors who do not like KPs in their neighborhood and have called the police on several occasions. 2.2.6 C-Link Linkages: CBS/CSP and KPS The Peer Driven Intervention model bases its strategy on the assumption that peers have networks with others in their population group. Peer outreach workers therefore can access members of their group who would otherwise not be linked to the health system and can reach out to a more diverse group of KP members.36 Achievements:  This linkage between CBSs/CSPs and KPs was found to be strong from both the qualitative and quantitative information collected. It has increased through the programs. The KP members that the evaluation team met spoke of the support that they received from the CBSs/CSPs and of the bond that was formed. The KP survey identified that CBSs/CSPs accompanied approximately half of KPs to HTC/OPC (Figure 17). Without his (CBS’s) support, I would not have known where to go for an HIV test. (Male PWID, HCMC) Without the CBS’ encouragement and support, I think I would not have returned to get treatment. I felt I was healthy without any troubles. (Female PSP, Dien Bien)  All interviewed CBSs and CSPs demonstrated their high motivation in conducting outreach, as the incentives that they receive often do not even cover their own transportation costs. The interviewed CBSs/CSPs reported that they often have to visit new PWID clients many times before they can convince them to be tested or, with even more difficulty, to register for treatment. They reported non-monetary benefits for themselves: Since I have worked as a CBS, my knowledge about HIV has improved so I can convey more correct messages to KPs. I am getting more confident and happier in my life. I have learned how to protect myself. I feel I am helpful to my friends. I feel more peaceful in my life. (A CBS of Nu Cuoi CBO, HCMC)  CBSs/CSPs have formed strong bonds with KPs. They have not only provided KPs with care and support in HIV/AIDS, but they have also referred KPs to non-HIV services. Through their network, CBSs/CSPs got information about the clinics that provide gynecological examination and treatment of reproductive tract infections free of charge, so they referred KPs there. A CBS 36 Broadhead, R. S., Heckathorn, D. D., Weakliem, D. L., Anthony, D. L., Madray, H., Mills, R. J., & Hughes, J. (1998). Harnessing peer networks as an instrument for AIDS prevention: results from a peer-driven intervention. Public Health Reports, 113(Suppl 1), 42. Evaluation of Community HIV Link Programs 47 even helped a KP to settle down in a charity house. Interviewed KPs reported that whenever they were depressed, they contacted a CBS/CSP to talk and they felt better. CBSs shared that they visited KPs frequently. I called Thach Da clinic to refer KPs there. I know a few clinics that conduct examination and treatment free of charge for poor people. (a CBS of Nu Cuoi CBO, HCMC) In the early days after I restarted ART, I had many side effects such as constipation and weight loss. I lost four kilograms so far. I called Ms. Chung to feel consoled and shared…whenever I felt difficult, I called her to gain her support. (LTFU KP who returned to get ART after being outreached and supported by a CSP, Dien Bien Province) FIGURE 17: PERCENTAGE WHO REPORTED THAT A CBS/CSP ACCOMPANIED THEM TO HEALTH SERVICES BY KP GROUP, BENEFICIARY SURVEY (N=194) Issues:  In HCMC, CBSs reported that they faced difficulties in outreach and establishing links with sex partners of PWIDs, FSWs and MSMs though the program’s annual work plans in sex partners as KPs. They seemed not to give importance to reaching out to these KPs or less capable of doing it. MSMs do not have long term partners. It is hard to know and outreach. (G3VN CBO, HCMC) No. We will outreach to a sex partner of a client if only s/he agrees. We do not invite sex partners to attend small group discussions as C-Link prioritize other high risk KPs. (CBSs of Nu Cuoi CBO, HCMC)  A gender discrepancy between a KP and CBS can be an issue in establishing an effective link between them. In Vinh city, a male IDU shared his wish to be supported by a male CBS so that he would be able to share his concern and gain support more easily. Evaluation of Community HIV Link Programs 48  In mountainous areas, turnover of CSPs, especially in the first year, must be a challenge for continuous CSP-KP linkage and care and support to KPs. The list of CSPs in five districts (Tuan Giao, Muong Ang, Dien Bien, Dien Bien Dong and Dien Bien Phu) in Dien Bien province updated as of January 2016 shows eight CSPs left and eight successors of the left CSPs out of 48 CSPs. The more remote districts have more CSPs who left. Topographic difficulty could be a barrier for CSPs who did not have sufficient motivation to overcome them. Since October 2014, there have been turnover of five CSPs [in Dien Bien Phu City]. This is a barrier. (District Coordinator in Dien Bien Phu city, Dien Bien Province) 2.2.7 C-Link Networking: Among CBOs Achievements  Networking from CBO to CBO has increased and strengthened during C-Link. Opportunities for CBOs to meet and collaborate with other CBOs have increased. Peer-driven outreach is particularly effective in urban areas where there are many newcomers and where efforts are made to “snowball” using previous contacts and online social networks. These networks also assist other programs in implementing mobile and lay testing.  Through the programs, CBOs interact with each other frequently through various activities such as trainings and monthly meetings. They exchange knowledge, share challenges with outreach and then find solutions together. Meeting other CBOs also increases the peer group network; in HCMC, this is particularly helpful as many KPs get tested in other districts.  In both HCMC and urban areas in the Northern Coast, there is CBO-to-CBO mentoring between stronger, more established CBOs and newer ones.  Leaders of CBOs join national and/or regional networks of KPs, such as networks of PWIDs, MSM or PLHIV, which provide opportunities for them to engage in many other activities of the networks. They can access information on the available services. Issues: The weaker or less established CBOs tend to be less able to take advantage of networking opportunities. In other word, the stronger, more established CBOs are better able to take advantage of networking opportunities. In the CBO survey, the evaluation team asked about collaboration between CBOs. As seen in Figure 18, CBOs that had more collaboration before C-Link were also more likely to have increased collaborations now. (Each line represents one CBO in the graph; the vertical axis shows the number of organizations that each CBO connected with before and after C-Link.) Evaluation of Community HIV Link Programs 49 FIGURE 18: NUMBER OF CBO-TO-CBO COLLABORATIONS REPORTED BEFORE AND AFTER THE C-LINK PROGRAM, CBO SURVEY (N=8) 3. DEVELOPMENT OF CIVIL SOCIETY IN THE PROJECT AREA EVALUATION QUESTION How do USAID-supported programs (administered by the three local implementing partners) affect the development of civil society organizations in the project sites? 3.1 Overview of Program Interventions in CBOs’ Development The C-Link design included a component to strengthen the capacity of the implementing partners, as well as the CBOs they worked with. All three IPs had special conditions within the C-Link agreements to be fulfilled during Year 1 of implementation. These special award conditions largely concerned governance criteria (such as instituting a board of directors), putting financial controls in place and establishing human resource policies. Following OCA in 2014, each IP developed management responses or an action plan to strengthen management, governance, strategic information and technical capacity. A follow-up assessment on institutional capacity for IPs took place in 2015. In 2014, baseline OCAs took place for all potential CBOs that IPs considered for collaboration or to strengthen their capacity (Annex 2, 3 and 4). Following these OCAs, each IP applied different approach:  COHED developed a roadmap for organizational capacity strengthening for all existing CBOs. It established new CBOs in the areas that had none, then developed a roadmap for conducting interventions to strengthen these CBOs. In fact, more than half of the CBOs working with C￾Link in Nghe An Province are newly formed for the program, either from an existing self-help group or a group of individuals. The interventions include training on leadership, planning, PR, HIV knowledge and outreach skills and M&E. The new CBOs included those that represent key populations (KPs) and have joined a KP network. However, heads of the CBOs were not necessarily members of a KP (for example, a CBO in Hung Nguyen District in Nghe An Province). They might have a strong interest in community based activities or have a large existing network in their community. The CBOs that were part of the program in Year 1 all received follow-up assessments in addition to the initial OCA. Because of the change in the Evaluation of Community HIV Link Programs 50 geographic area of the program, the CBOs that joined the program in Year 2 had not received a follow-up OCA by the time of the evaluation.  CCRD chose to build institutional capacity for eight existing CBOs in Hanoi that had previous experience in HIV services and were ready for further coaching, mentoring and capacity building to provide HIV services. CCRD developed an action plan to build the capacity of these CBOs in leadership, planning, finance management, communication, advocacy and technical capacity in HIV services. The program applied an innovative initiative to encourage CBOs’ creativity and proactivity by inviting CBOs to submit a proposal for innovative prevention and capacity building. In 2015, a follow-up OCA took place.  The LIFE Center applied an “empowerment” and “results-based management” approach to capacity building to only eight CBOs that have shown potential for further development. These CBOs received previous support for organizational capacity strengthening from the Global Fund Program. The Global Fund program had supported these CBOs to develop a financial management system. LIFE set targets along the HIV COPC and required these CBOs to fulfill them; conducted interventions for organizational capacity strengthening such as mentoring, coaching and leadership training; provided vision and mission development assistance; and guided the CBOs on M&E system establishment and engagement of private providers. The center empowered these CBOs by giving them the autonomy to allocate the incentives they received for HIV services after they achieved targets along the HIV COPC. Further screening of potential CBOs allowed prioritization of which organizations would receive support. CBOs that were not able to achieve the targets continuously were omitted. As a result, at the time of the C-Link program evaluation, only five CBOs were receiving LIFE’s support for organizational capacity building. 3.2 Evaluative Findings It is noted that following USAID/Vietnam guidance, evaluation of effectiveness of the case finding activities was the focus. Evaluation on CBO development therefore was carried out in a brief manner. Implementing Partners All IPs have demonstrated strengthened capacity. This is reflected through either follow-up assessment or the ability to carry out C-Link programs effectively, or recognition and appreciation of various stakeholders, as described. All three implementing partners fulfilled their special conditions in the first year of implementation. IP staff completed human resource training and financial training where appropriate, strengthened their financial systems and systemized their staff development plans. CBOs CBOs fall broadly into three groups: 1) CBOs that were well-established before C-Link and would remain strong without C-Link. There are only a few of these, located in large urban centers. A couple of CBOs are in HCMC and Hai Phong City. 2) CBOs that were established as a self-help group or CBO before C-Link, and the program provided support and training that allowed them to “take off.” Most notably, a few CBOs began self-sustaining financial activities during C-Link implementation, as well as proposal writing and collaborations with other CBOs. Again, only a few CBOs are in this group; they are located in HCMC, Vinh City and Hanoi. The MSM CBOs in all regions appeared to undergo the greatest development, becoming increasingly active in running their business activities and using their MSM networks. In the Evaluation of Community HIV Link Programs 51 HIV/AIDS area, Hai Dang CBO in Hanoi, with support from CCRD, had its abstract accepted and was invited to join a panel discussion at the recent International AIDS Conference in South Africa. 3) CBOs that were previously self-help groups or did not exist as a group before C-Link. Their outreach skills have been built through C-Link and they remain a source of support for their members, but they have not embraced self-sustaining funding opportunities. Also, many former CBOs received funding from previous projects that are now dissolved. Issues  With the changes in PEPFAR funding, it is not likely that CBOs in the third group will survive without C-Link funding. Capacity building for income generation appears to be much more effective for CBOs with members who have skills and education, such as the MSM CBOs in HCMC, but it has less of a sustainable impact for others.  During the short implementation period for C-Link, most newly established CBOs in Northern Coastal did not build extensive organizational capacity though CBSs’ skills and knowledge in HIV/AIDS and outreach along the COPC were improved. Significant efforts to build their capacity would be required to improve technical and management aspects; the process would be time-consuming and run beyond the three-year project life, while most CBOs had been established only a few months before the evaluation.  The PWID/FSW CBOs also faced greater barriers in moving forward, including community resistance to their gatherings, police inquiries into their activities and some members’ continued drug use. This pattern reflects known challenges faced by PWIDs and FSWs, such as lower educational attainment, more difficult living conditions, less stable jobs and a lack of residency registration certificates.  Many CBOs in HCMC face challenges in applying for legal status. As mentioned, Ward People’s Committees do not seem to pay attention to this matter or are not aware of the legal framework or required procedure for licensing a CBO to be a cooperative (hop tac xa). 4. SUSTAINABILITY EVALUATION QUESTIONS Will case finding activity continue at project sites after USAID-supported programs end? Do PACs have any plan to sustain the activity approach? What is the possibility of mobilizing other funding sources to maintain case finding activity? 4.1 Context  International donors have contributed a large proportion of the HIV/AIDS response in Vietnam, but have made it clear that this funding will be discontinued in a few years.  For government stakeholders at all levels, the preoccupying concern is the transition to social health insurance. Only those with health insurance will be able to continue ARV treatment after support from international donors ends. This transition is beginning now, as some OPC centers have already moved to hospitals. Full application of this scheme is planned for 2017. Moving toward this goal, the GVN is working on issues such as promotion of health insurance and reorganization of HIV/AIDS services. Basically, these services will be integrated with other health examination and treatment services at hospitals. Each province is working on developing a list of HIV services covered by health insurance. In addition, freestanding OPCs that are Evaluation of Community HIV Link Programs 52 classified as prevention sites and are being maintained need to be upgraded to meet treatment facility standards. In HCMC, the health system is trying to enroll PLHIV to an OPC where they register for regular residency. The large number of migrants, including those who do not have residence registrations and health insurance, is pushing the health system to consider numerous options. For example, migrants can buy health insurance in the province where they have a regular residence registration, and they will be allowed to sign up for a health clinic in the city.  An original objective of the C-Link program was to strengthen civil society organizations, in part to prepare for an eventual loss in international funding. Besides planning for CSOs to be self￾sustaining and to legitimize their status through linkages with government health services, outreach was hoped to be incorporated into district-level services and funded by the GVN. This objective was de-emphasized in Year 2 with the PEPFAR emphasis on outreach only, although actual implementation strategies with regard to CBOs varied by region. 4.2 Evaluative Findings Achievements  IPs such as CCRD and LIFE continue collaborating with PACs in the City’s HIV agenda. CCRD has shared their experience in strengthening CBOs’ capacity to provide HIV services with Hanoi PAC. They continue advocating with Hanoi PAC to allocate budget for outreach activities by the CBOs. LIFE continues working with HCMC PAC to provide HIV services to MSMs.  CBOs and CBSs express strong desire to be involved continuously in HIV activities. A few well￾established CBOs, such as G3VN, have started exploring a few options for their related business, such as being a condom vendor for a private company and selling condoms through their MSM network. They were also exploring the possibility of being a provider of quick HIV testing in the long term. They expressed their enthusiasm to continue collaborating with the HCMC AIDS Association on HIV activities.  Interviewed CBSs shared that they would continue helping their peers by sharing information and knowledge about HIV and HIV services, as this is possible without receiving incentives. Issues  Despite their willingness and enthusiasm, all interviewed CBOs and CBSs foresaw huge challenges in continuing case finding activities at the same scale, without financial support. CBSs in mountainous areas were straightforward in saying it would be hard for them to continue. The discontinuity of case finding activities was observed in the “scale-down” areas where only CARE_COMM cases received incentives. The CBSs/CSPs there said they were not sure if they could continue to do outreach, since reaching cases and bringing them to the OPC was time￾consuming and they needed to generate income. Like in the area the team visited where outreach activities stopped completely, the scale-down and reduction in incentives had come suddenly and unexpectedly. The team’s visit to a district previously covered by the Northern Coastal-COHED program identified that former CBSs only continued their support to KPs at the level of sharing information or encouragement. If the project started up again, I would love to do the work. All of the CBO members would come back, they are enthusiastic. Now I do some construction work and gather forest products for money...I still see my clients, and we keep our word with them to help them go to the OPC center. (Former CBS in an old district, Nghe An province) Evaluation of Community HIV Link Programs 53  None of the visited PACs had a plan for funding CBSs/CSPs to do outreach activities once the international support ceases. As mentioned in the previous section, PACs consider using village health workers as they are in the health system and receive monthly allowance. One PAC stated that it planned to fund CBSs, but its plans were vague. “We want to continue activities when C-Link ends. … C-Link has set a firm foundation for us to build on in the future. … We want to continue the model of working with the CBOs, but due to our limited local resources, the incentives will be reduced and the workload will also be reduced. … We will probably contract individually with the CBSs, depending on the annual budget.” (PAC, Northern Coastal)  Some PACs expressed the need for CBOs to have legal status to enable a formal relationship. This was seen as a barrier to moving forward with the current outreach model if C-Link ends. Otherwise, PACs hope CBOs can contribute their time voluntarily. 4.3 Factors That Will Influence the Effectiveness or Necessity of Continuity of Active Case Finding  Requirement of health insurance: A large proportion of surveyed KPs do not currently have health insurance, particularly FSWs and PWID (Figure 19). CBSs shared many difficulties for KPs as migrants to buy health insurance with the requirement of having a regular residence registration or buying health insurance with other family members. This will affect CBSs/CSPs’ motivation to do case finding, as KPs will not be able to receive or continue ART without health insurance. In that case, there is no point in finding hidden HIV cases. FIGURE 19: PERCENTAGE OF KPs WHO SAY THEY HAVE HEALTH INSURANCE BY GROUP, BENEFICIARY SURVEY (N=194)  Movement of HIV services to general examination and treatment department in a general hospital: CBSs/CSPs, KPs and health providers at the current HTCs/OPCs expressed concerns regarding the OPC moves to the provincial hospitals. The privacy and confidentiality of PLHIV can easily be compromised during hospital registration and locating the OPC. New general health staff may not be sensitized to the needs of HIV-positive clients and could engender a sense of stigma and discrimination. Combined with the need for treatment clients to Evaluation of Community HIV Link Programs 54 have a valid ID and residency papers, as well as health insurance, many OPC clients could be lost to follow-up in the transition. “If HIV examination and ARV treatment applies the same procedure for examination of a disease with health insurance, clients’ confidentiality will be affected. A client will have to present at many desks and declare (his/her) HIV positive status. I am very concerned. Many KPs must be concerned too so they may not come to hospital for ARV treatment” – CSP in Dien Bien Phu city, Dien Bien province  Decline in HIV prevalence or proportion of newly detected HIV cases out of tested KPs: The HIV prevalence rate in many provinces, including Nghe An Province – one of the priority provinces for the 90/90/90 plan, has been low, remaining around 2 to 3 percent in the last few years. This is lower than the 5 percent prevalence defined by PEPFAR for a concentrated epidemic area. The above mentioned analysis suggests that the first 90 targets were too high in Dien Bien and Nghe An in the recent context of epidemics. Data shows that the proportion of newly detected HIV cases in many districts has been very low while all estimated KPs have been tested. With this analysis, the evaluation suggests that active case findings be continued for only the KPs that have an emerging epidemic or still have many hidden cases, such as MSMs or in districts that still have a large of KPs that have not been targeted for outreach. For the districts where most KPs have been tested, the program should shift resources to provide CARE_COMM services. After 2017, a suggested focus is finding and supporting lost-to-follow-up cases to bring them back to ART and promoting accessibility of HIV/AIDS services in remote areas. 5. CROSS-CUTTING ISSUES 5.1 Gender Achievements:  All three implementing partners have incorporated gender into their strategies to various degrees. In the Northern Mountains, work plans call for incorporating strategies to promote gender equity.  All three IPs included primary sex partners (PSP) as a target group for the intervention in the annual work plans, including actively seeking partners of PWID and PLHIV. The C-Link programs for the Northern Mountains and Northern Coastal regions successfully reached out to PSPs, as reflected in service statistics.  LIFE included gender identity and gender sensitivity in all of its CBS trainings, including training on the different needs of men, women and transgender people. It also included gender sensitivity and gender mainstreaming in its leadership trainings and incorporated gender integration in its organizational capacity development plans. Issues:  Effectiveness and preferences for same-sex versus opposite-sex outreach: In its planning documents, COHED recognizes the gender issues inherent in the C-Link program. One is that men are usually responsible for safe sex and that couples’ counseling may be beneficial when a male KP tests positive. They also include PSPs as a key target group. A second issue is that outreach workers are more often women than men, and it is easier to recruit PLHIV women to be outreach workers than PWID or ex-PWID men. In this regard, COHED recognizes that “CBOs have acknowledged a Evaluation of Community HIV Link Programs 55 gender bias with regard to the provision of prevention and support services, and efforts are being made to encourage men to play a more active role.” The evaluation team explored this issue in several interviews with KPs and CBS/CSPs. One young man, a former PWID who was waiting for his HIV test results when the evaluation team spoke to him, had received counseling from a young woman who was not a PWID. He said he would have preferred to talk to another man about his fears of contracting HIV. Several female CBSs the team talked to said they had some issues with their husbands or male partners related to their outreach work, including jealousy over their outreach to men or worry about their late hours. One male CBS working in a mountainous area commented that it was more difficult for him to outreach to women, since he could not take them on his motorcycle for HIV testing and counseling because their husband or partner might get jealous or suspicious. All of the CBSs the team talked to had strategies to deal with these issues, such as developing trust with partners as they continued outreach over time. Nevertheless, training materials and CBO training and mentoring should include lessons learned and strategies for addressing these issues, and efforts to include more men as outreach workers for PWID should continue.  Outreach to PSPs in Southern Region: The developed annual work plan and SOP both include PSPs as a KP and prevention of HIV transmission to sex partners and relatives. The SOP guides CBSs to encourage sex partners to take the HIV test. Nevertheless, the evaluation team found that the plan and guidelines to reach sex partners were not operationalized to yield tangible achievements. The service statistics did not include PSPs. The interviews to CBSs identified that they did not give sufficient attention to this population or faced challenges in reaching PSPs. A systematic assessment or study on PSPs of the KPs to gain a good understanding about this population and effective approaches to outreach PSPs is recommended. 5.2 Monitoring and Evaluation The M&E system for C-Link centers on two main components:  The recording and reporting system for all levels of C-Link itself: USAID, the IPs, the CBOs and the CBSs/CSPs. Field coordinators play a crucial role in this reporting chain, providing technical assistance to CBOs and CBSs/CSPs to prepare reports for the IPs.  Validation of outreach cases by district and provincial health offices, following the Circular 09.2012.TT.BYT on surveillance of HIV and sexually transmitted infections. This validation is critical for several reasons, determining whether the CBSs/CSPs are given the incentive for the case; whether the case counts toward C-Link achievements and targets; and the contribution of C-Link to the total provincial and district cases. Field/program coordinators also assist in the validation process; further discussion follows. The Programs’ Recording and Reporting System Achievements: All three C-Link programs developed comprehensive activity monitoring and evaluation plans. Key features of the C-Link M&E system include:  Involvement at all C-Link levels: CBSs or CSPs, program coordinators and implementing partners’ M&E specialists.  Clear data flows: from CBS to implementing partners and USAID. Evaluation of Community HIV Link Programs 56  Comprehensive M&E frameworks and indicators.  Data collection forms adapted from similar programs that have been previously tested and used by other partners (USAID/SMART-TA). The evaluation team found that all of the data collection and reporting systems function well. As one key informant said, “The completed forms and reported numbers are accurate to a centimeter.” The strength of the system results from:  Rigorous and comprehensive M&E capacity-building plans: systematic training and refresher sessions to build the M&E capacity of CBSs/CSPs from the beginning of implementation, followed by coaching and mentoring. Capacity-building efforts by implementing partners were significant achievements, since in remote and rural areas the majority of CBSs/CBOs were ethnic minorities or had limited educational background. “Since we have been trained well by COHED, we have become capable and have conducted field trips to CBOs in other districts to show them how to fill out the forms.” (CBO, Nghe An) “Project staff do monthly coaching for CBSs and CBOs, so now everyone can use that properly.” (CBO leader, Nghe An)  Effective quality assurance, including the CBO leaders data-checking the CBO logbooks, field/program coordinators checking CBO reports and the M&E officer checking the district and provincial reporting.  Adaptations and simplifications as the staff gained experience. “In the training on project data collection forms, representatives from CBOs discussed and gave comments to change forms to be more practical. Some comments were adopted and revisions were made in M&E forms.” (CBS, Nghe An) Issues:  While C-Link’s M&E system appeared to function effectively, the M&E workload caused some burdens for CBS/CSPs who collected primary data. In the Northern Coast, CBOs and CBSs spent a great deal of time on preparing reports and checking M&E data because most of the CBOs and CBSs are new. “There are too many forms for M&E and reporting. Members of CBOs who are taking ARV sometimes lose their memory, so they make mistakes in the report.” (Local Coordinator) “[There is] too much paperwork for CBOs. They have to fill many forms, especially for people like the CBSs; they are peers and some of them have little education.” (Local Coordinator) “Every month, CBO leaders have to fill in six forms (client form, referral form, outreach and referral logbook, follow-up client logbook, summary of clients, payment order). It takes two or three days to finish and revise these forms in total.” (CBO Leader) The field coordinators interviewed by the evaluation team said they spent most of their time on M&E and helping CBOs with reporting. Evaluation of Community HIV Link Programs 57  The evaluation team notes that indicators were not consistently presented across project documents such as project agreement, the M&E plan and annual work plans. Some key C-Link activities seem to lack outcome-level indicators. An example is the support to retain ARV patients in the care and treatment programs. It is unclear how C-Link programs measure it, though the programs are believed to have contributed significantly to retaining ARV patients. Case Validation Process It is noted that within the scope of this evaluation, the evaluation team did not examine thoroughly and systematically the government’s surveillance system. In this regard, the team used the data of newly detected HIV cases that each IP reported to USAID/Vietnam as they had been validated by this system and the team did not re-validate these validated cases. The following findings on case validation focus on the aspects of coordination and smoothness of the validation process of the cases for C-Link program, instead of validity of the validated cases. Achievements: C-Link implementation partners have put great effort into working with the PACs and DHCs on the verification process of the program’s outreach data. The evaluation team found:  Effective coordination/collaboration with PACs and health facilities (HTC/OPC), with clear roles and responsibilities for each party. In addition to what other sections of the report have addressed, effective coordination with PACs and HTC/OPCs benefits M&E work in C-Link programs. Integrating M&E data from the C-Link program into the provincial database in HCMC is one example.  Close involvement of PACs and health facilities in data sharing, verifying newly found HIV cases and reviewing programs’ achievements. “We always prioritize the groups (CBOs). We are trying our best to verify cases for CBSs when they turn in the list (of new HIV cases found) as soon as possible. From what we observed, we think they were very organized in terms of M&E forms.” (HTC manager in HCMC) “After LIFE got the agreement with HTC/OPC, they made clear the responsibilities of CBOs. Monthly, HTC/OPC meets with G3VN key staff to review the activity [using finalized M&E data] and to solve problem/issues.” Issues:  The new case verification and reporting process is complex and time-consuming. Particularly, the local HIV prevention databases are unlinked across the HTCs and local agencies are trying to synchronize the systems across the entire province or city. KPs’ right to anonymous testing creates additional challenges.  The complexity of the process is a particular concern as it has caused severe delays to the CBSs receiving their incentives. Figure 20 shows the six-part process required to validate cases in the Northern Coast.  In the Northern Mountains Region, the CSPs also reported severe delays in payment of incentives; though the system has improved recently, delays continue as a result of long distances between districts and a backlog at the PAC. In Ho Chi Minh City, none of the CBSs or CBOs the team interviewed complained about delays in payment of incentives. However, the set-up of the incentive Evaluation of Community HIV Link Programs 58 system in the Southern Region, and the fact that CBOs have their own income-generating activities, allows CBOs to sometimes advance payment of incentives using CBO funds. FIGURE 20: CASE VALIDATION PROCESS, NORTHERN COAST 6. BEST PRACTICES AND LESSONS LEARNED When this evaluation was taking place, several emerging issues were dominating the concerns of both local-level and central-level stakeholders. These include:  The need for PLHIV to have health insurance to receive treatment in the near future, and for OPC centers to move inside hospitals so that treatment is covered by health insurance. The evaluation team’s Beneficiary Survey showed that only 58 percent of KP members have health insurance; for FSWs and PWIDs, the figure is much lower. This emerging issue will complicate the current situation, where a bottleneck exists at the OPCs because so many PLHIV do not have an official ID or residency papers to obtain treatment.  The increasing difficulty in some areas of making continual achievements toward 90/90/90 targets. Many district officials, CBOs and CBSs in mountainous and coastal areas said most HIV￾positive cases had already been identified, and it was doubtful that they could continue to find new cases at the current rate. Some areas also had few LTFU or dropout cases, so few cases could benefit from outreach. Also, for CARE_COMM areas, the amount of incentives may be inappropriate given the difficulty of reaching these cases and convincing them to enter or re￾enter treatment; many LTFU and drop-outs have reasons for not wanting to be in treatment.  Reduction of international funding, particularly for treatment, but also for harm reduction commodities. The government does not have enough funding to meet this gap before the announced date of the end of funding for PEPFAR and the Global Fund. Evaluation of Community HIV Link Programs 59  Greater recognition of civil society: The Law on Associations, under consideration in the Vietnamese Legislature, would purportedly give citizens the right to form civil organizations. However, some CSOs feel the bill will actually create greater barriers to establishing and legitimizing CSOs, and tighten government control over CBOs. Within this context, the best practice the team observed in its evaluation was adopting a flexible and adaptable approach to structuring and implementing outreach activities in differently evolving local contexts. It is essential to gain a good understanding of local areas’ situations and needs and use evidence obtained through timely situational and thematic analysis to inform the choice of approaches in a particular area. Specific examples include:  Engaging government staff at the provincial level and district level as C-Link coordinators. This practice guaranteed government involvement with C-Link, generating a good understanding of the outreach challenges. It also ensured that a government staff member oversaw the C-Link monitoring data, as oversight of HIV activities and epidemics in the local area is their regular mandate.  Capacity building for CBOs that have a foundation of skills and the motivation to continue as an organization. In Ho Chi Minh City, LIFE will continue the C-Link program with only eight MSM CBOs. The MSM CBOs that the team met in HCMC had clearly expanded and developed under C-Link, while the PWID/FSW CBOs had mainly improved their outreach skills and a strengthened ability for self-help and support to their members. In the Northern Coastal region, it is uncertain if any newly formed CBOs will continue after C-Link ends.  Finding flexible solutions for PLHIV without a long-term residence certificate in HCMC, with friendly OPCs. In HCMC, some “friendly” OPCs allow CBOs to provide guarantees for PLHIV who do not have an official residence certificate. This solution may be possible only in HCMC, but is worth highlighting as an example of the IP, CBO and OPC working together to improve treatment coverage.  An IP that has been carrying out different HIV programs/projects can leverage the partnership with PACs as these programs complement each other. For example, CCRD also supports Dien Bien and Nghe An provinces to strengthen the province M&E systems for HIV/AIDS. LIFE has carried out programs for both C-Link and Global Fund. The evaluation team would also like to highlight the following lessons learned:  The planning challenges created by the PEPFAR pivot and the reorganization of districts among USAID programs caused a waste of monetary and human resources in the districts that began implementation and then were cut.  The short implementation period for new districts creates a delay in implementation, as CBS/CSPs must be trained and relationships built and, in some areas, CBOs created. It also damages PEPFAR’s image and the relationships that were built in the former program areas. These decisions were made at the central level and were outside PEPFAR-Vietnam’s control, but the team presents information about their consequences here in the hopes that these issues be weighed in the future when PEPFAR makes decisions. Evaluation of Community HIV Link Programs 60 V. CONCLUSIONS This section summarizes the conclusions by evaluation question. 1. PERFORMANCE To what extent are the programs implemented by the three local partners achieving their objectives of identifying high-risk individuals from the key populations for HIV intervention services, and enhancing coordination and networking with key local stakeholders? The evaluation team concludes that all three implementing partners are achieving their objectives of identifying high-risk individuals for testing and linking PLHIV to treatment services in fiscal year 2015. In fiscal year 2016, two programs are well on track, but the Northern Coastal Region program had shortfalls in meeting its targets. Coordination and networking with local stakeholders has been effective in all three regions and has enhanced the service environment for CSOs. What are the reasons that the C-Link partners are/are not on track for achieving their negotiated USAID targets? Enabling factors leading to the achievement of the tangible results include:  A multi-pronged strategy could be considered as a key enabling factor. In addition to a flexible and local context-adapted outreach approach, advocacy and stigma-reduction efforts were factors that effectively supplemented the program’s achievements.  Strong linkages between IPs and PACs, between IPs and CBOs and between health services, CBOs and CBSs/CSPs are essential to the program’s successful functioning. The foundation of the linkages is the strong relationships between outreach workers and their clients, and the high motivation and dedication of the CBSs/CSPs.  Continuity in geographic areas enabled the programs to continue building capacity for the trained CBOs and CBSs/CSPs and strengthening established linkages. Start-up and building relationships takes time.  Government involvement in program coordination in the field in the Northern Mountains contributed to a smooth operation of field activities.  Systematic and focused capacity development for CBOs that have a foundation of skills and organizational strength helped avoid dispersing resources.  Appropriate performance-based incentives that were tightened with targets along the continuum of care cascade contributed to the accountability and commitment of CBOs and CBSs/CSPs.  Effective recruitment of and capacity building for experienced and highly motivated CBSs/CSPs with continuous coaching through monthly meetings of CBSs/CSPs, CBOs, HTCs/OPCs and IPs. Factors leading to under-performance toward targets include:  Unrealistic targets for fiscal year 2016 in Northern Coastal areas.  Changes in the geographic areas and focus of the C-Link program, which disrupted the ability of the implementing partners to make continuous progress toward project targets.  High turnover of CBSs/CSPs in mountainous areas and a few weak CBOs for PWIDs and FSWs. Evaluation of Community HIV Link Programs 61  Weak CBOs, especially newly formed CBOs in the Northern Coastal Region and a few CBOs of PWIDs and FSWs in HCMC.  Changes in the incentive system that have outreach workers doing the same work for less money than they received in the past.  Complicated and lengthy M&E and validation systems, and delays in paying incentives caused by this and other factors at the PAC or DHC level.  Difficult topographical conditions and KPs’ lack of vehicles for transportation in mountainous areas, affecting the level of achievement in CARE_COMM and OPC registration targets. What are the implementing partners' contributions to improving HIV/AIDS service cascades toward the target of 90/90/90 (90 percent of high-risk individuals are aware of their HIV status, 90 percent of HIV-positive persons receive ARV and 90 percent of HIV patients are retained in treatment programs)? Contributions to the provincial cascade in FY2015 were in line with the proportion of the province covered and KP size estimates in all three program areas. Progress in the first half of FY2016 is largely on track with the previous year’s achievements. However, overly ambitious 90/90/90 targets were based on outdated estimates. In the Northern Coast, the proportion of new HIV cases found is lower than 5 percent in both reporting periods. To what extent do stakeholders recognize the contributions of implementing partners in supporting sustainable HIV/AIDS responses? Stakeholders clearly recognize C-Link contributions and feel that the program complements other efforts that make up the country response. They particularly value how C-Link finds hidden HIV cases, lost-to-follow-up cases and dropouts from treatment. Local health officials and services staff feel that this outreach is essential to reaching the 90/90/90 targets. Other programs feel that coordination with C-Link has gone well. Governmental stakeholders strongly acknowledged the contribution and advantages of CBOs and CBSs/CSPs in community-based HIV prevention, care and support activities. Nevertheless, the recognition has not been systematized through budget allocation in the provincial/city work plans or a formal engagement of CBOs and CBSs/CSPs in a local work plan development process. How do USAID-supported programs (implemented by the three local implementing partners) affect the development of civil society organizations within the project sites? Two implementing partners have a long history of developing community-based organizations; the third, CCRD, did not have such experience at the outset of C-Link, but it has developed successful strategies in that area. Evidence of civil society strengthening was seen in the two regions working with CBOs, though it was difficult to evaluate progress for the CBOs in the Northern Coast since most of them are new. In the Southern Region (HCMC), the MSM CBOs have the capacity to “take off” with successful income-generating activities while the FSW/PWID CBOs will likely not survive without external funding. Finally, since organizational development takes time and there is a lack of follow-up OCAs for many CBOs, it is difficult to measure progress in the short period of implementation. Evaluation of Community HIV Link Programs 62 2. SUSTAINABILITY Have the institutional capacities of implementing partners and CBOs increased as a result of USAID support and, if so, how? Identify the key support activities that have contributed to strengthened institutional capacity. The three IPs have strengthened their financial and governance systems in response to the special conditions placed on their agreements with USAID for C-Link. Outreach skills and the ability to fulfill M&E requirements have been clearly strengthened through C-Link. CBOs can be divided to three groups: 1) CBOs that were well-established before C-Link and would remain strong without C-Link; 2) CBOs that were established as a self-help group or CBO before C￾Link, and the program provided support and training that allowed them to “take off.”; and 3) CBOs that were previously self-help groups or did not exist as a group before C-Link. With the changes in PEPFAR funding, it is not likely that CBOs in the third group will survive without C-Link funding. Will case finding activity continue at project sites after USAID-supported programs end? Do PACs have any plan to sustain the activity approach? What is the possibility of mobilizing other funding sources to maintain case finding activity? While most of the PACs say they would like to continue outreach activities, and to some extent the CBO activities, they did not have concrete work plans and accompanying budgets to support outreach. None had mobilized other funding sources to continue outreach. The evaluation team did not see evidence of CSPs or CBSs being able to continue outreach after funding ends; the minimal amount that the incentives provide is essential for transportation costs, building a relationship between CSPs/CBSs and their clients and some support for CSPs/CBSs. Effectiveness of active case finding activities can be affected by the emerging context. The requirement of health insurance for KPs to be able to enroll for OPC can be a huge challenge. Approximately half of KPs do not have health insurance and KPs face difficulty in buying health insurance. Without being able to help KPs enroll for treatment, CBSs/CSPs will be less motivated to find cases. The movement of HIV services to general examination and treatment department in a general hospital concerned KPs in terms of a possibility of stigma and the loss of confidentiality. A shift of active case finding to care and support to PLHIV and LTFUs cases should be considered in the areas with a continuous decline in HIV prevalence or where there is a high proportion of newly detected HIV cases among tested KPs. VI. RECOMMENDATIONS 1) USAID/Vietnam should continue the C-Link program for identifying and referring HIV cases, as it has been effective in contributing to the 90/90/90 targets. The following modifications should be considered:  USAID/Vietnam should encourage provinces to reset the 90/90/90 targets based on new size and prevalence estimates for KPs. It is recommended that the size estimate exercise should involve district stakeholders (DHC), community-based outreach workers and other community members who are knowledgeable about KPs. USAID/Vietnam, together with its experienced partners, should provide technical assistance on this exercise to C-Link IPs. Evaluation of Community HIV Link Programs 63  USAID/Vietnam and its partners should keep providing assistance to improve the HIV M&E system and HIV case verification process, as per Circular 09/2012/TT–BYT.  IPs should continue additional approaches to network-based peer outreach in remote areas, including collaboration with mobile testing and treatment and/or with commune health centers to provide ART retention support. Quality assurance of the mobile testing and treatment campaigns should be given attention to ensure clients’ confidentiality and privacy.  USAID/Vietnam and its IPs should prioritize a study on the effectiveness of combining CBSs’ peer-driven outreach activities and provision of lay test services among different KP groups to increase case identification efficiency.  In the Northern Coastal areas, IP should work with USAID/Vietnam, PAC and DHC to determine whether case finding is still viable especially in the districts where the HIV-positive rate among new HTC clients is low constantly. Consideration should be made to shift from case finding to support for ART retention or lost-to-follow-up PLHIV in these areas.  In the Northern Coast, consider involving government HIV health officials to coordinate outreach efforts.  COHED should carry out a gender analysis on the effectiveness and preferences for same-sex versus opposite-sex outreach. LIFE should carry out a gender analysis on effective approaches to PSPs to fully execute the developed SOP. 2) USAID/Vietnam should continue its assistance in developing civil society only in urban areas and with established organizations; given the short period of C-Link implementation, forming new CBOs to implement C-Link is not advantageous.  IPs should begin or continue providing support CBO linkages with private providers, such as commodity production enterprises. This will give CBOs opportunities for income-generating activities.  USAID/Vietnam and IPs should continue advocacy for civil society organizations (CSOs) to improve their institutional frameworks, in collaboration with the Global Fund. The frameworks should stipulate and plan for budget allocation at the central and/or provincial level. Supporting the development of CSOs should be a long-term program goal if this is still a priority. 3) Prioritize preparations for the transition to social insurance by working with VAAC and PACs to fill gaps for HIV/AIDS programming.  USAID/Vietnam should continue assisting PACs to implement the health insurance policy for ARV treatment. This also should include communication and dialogue with VAAC, MOH and health insurance agencies at the central level.  C-Link should: o Closely track retention rates among ARV clients. o Support the purchase of health insurance and other administrative procedures required to obtain and continue treatment. o Coordinate more closely with PAC/OPC in referring newly found cases to treatment programs. 4) Document successful models of outreach, including technical recommendations for specific KP groups such as PSP. Evaluation of Community HIV Link Programs 64 ANNEX 1: PROPORTIONS OF TESTED AND POSITIVE KPs PROPORTION OF TESTED KPs OUT OF ESTIMATED NUMBER OF KPs AND PROPORTION OF NEW HIV-POSITIVES REACHED AMONG TARGETS IN NGHE AN, NORTHERN COASTAL Project sites Proportion of tested KPs out of estimated KPs number Proportion of new HIV-positive reached out of target Cửa Lò 74% 17% Diễn Châu 74% 22% Hưng Nguyên 89% 31% Nghi Lộc 78% 36% Qùy Hợp 52% 14% Quỳnh Lưu 66% 56% Tân Kỳ 97% 15% Thái Hòa 59% 42% Thanh Chương 69% 33% Vinh 33% 59% Yên Thành 42% 0% Evaluation of Community HIV Link Programs 65 ANNEX 2: RESULTS OF CBO ORGANIZATIONAL CAPACITY ASSESSMENTS, NORTHERN COAST RESULTS OF INITIAL AND FOLLOW-UP ORGANIZATIONAL CAPACITY ASSESSMENT OF C-LINK CBO, NORTHERN COAST Number of CBOs with OCA Quang Ninh (5 CBOs) Hai Phong (9 CBOs) Nghe An (22 CBOs) 5 5 9 9 16 5 1st OCA 2nd OCA 1st OCA 2nd OCA 1st OCA 2nd OCA Rating 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1. Governance 5 2 7 1a. Legal status 1 4 3 1 5 2. Administration 3. Human Resources 4. Financial Management 4 3 5. Organizational Management 3 5 3 8 10 5 6. Program Management 2 3 2 1 11 7. HIV/AIDS Knowledge 2 3 4 1 7 1 8 11 14 5 8. Outreach Skills 3 2 2 1 7 1 8 4 12 5 9. Sustainability/Fund-raising 2 1 3 3 1 10. Networking/mentoring other CBOs 1 1 2 Rating: 3=strong capability 2=some capability 1=weak; blank=not rated. For 1a, 3=has legal status, 2=has recognition from health services and local authorities. Not all organizational characteristics were updated in the follow-up OCA; for example, leadership skills were sometimes mentioned in the initial OCA, but no assessment for them was made in the follow-up. Evaluation of Community HIV Link Programs 66 ANNEX 3: RESULTS OF CBO ORGANIZATIONAL CAPACITY ASSESSMENTS, NORTHERN MOUNTAINS RESULTS OF INITIAL ORGANIZATIONAL CAPACITY ASSESSMENTS OF C-LINK CBOS, NORTHERN MOUNTAINS Rating Hanoi (9 CBOs) Dien Bien (4 CBOs) 1 2 3 1 2 3 1. Governance 1 4 1 1 3 1a. Legal status 2 2. Administration 1 3 3. Human Resources 1 4 1 3 4. Financial Management 6 2 2 2 5. Organizational Management 6 1 1 3 6. Program Management 1 5 2 2 2 7. HIV/AIDS Knowledge 4 4 2 8. Outreach Skills 9. Sustainability/Fund-raising 1 2 4 2 10. Networking/mentoring other CBOs 4 4 2 Rating: 3=strong capability 2=some capability 1=weak; blank=not rated. For 1a, 3=has legal status, 2=has recognition from health services and local authorities. Evaluation of Community HIV Link Programs 67 ANNEX 4: RESULTS OF CBO ORGANIZATIONAL CAPACITY ASSESSMENTS, SOUTHERN REGION RESULTS OF INITIAL ORGANIZATIONAL CAPACITY ASSESSMENTS OF C-LINK CBOs AND AREAS OF SUBSEQUENT TRAINING, SOUTHERN REGION Number of CBOs by Rating An Giang (2 CBOs) Ho Chi Minh City (6 CBOs) 1st OCA Training/ mentoring 1st OCA Training/ mentoring 1 2 3 1 2 3 1. Governance 2 3 †1 1a. Legal status 2 2 3 1 †1✓5 2. Administration 1 1 ✓2 6 ✓6 3. Human Resources 1 1 2 4 4. Financial Management 1 1 2 4 5. Organizational Management 2 †2✓2 3 3 †6✓6 6. Program Management 1 1 †2✓2 2 4 †6✓6 7. HIV/AIDS Knowledge 2 ✓2 2 4 ✓6 8. Outreach Skills 2 1 2 3 †4 9. Sustainability/Fund-raising 10. Networking/mentoring other CBOs Rating: 3=strong capability 2=some capability 1=weak; blank=not rated. ✓=training †=coaching/mentoring Evaluation of Community HIV Link Programs 68 ANNEX 5: DATA COLLECTION SCHEDULE PILOT IN HANOI (24-25 March, 2016) Date Time Institutions Stakeholders/ Informants Note TEAM SPLIT TEAM 1: WORK IN HOANG MAI DISTRICT THU 24 MAR 9:00 – 10:00 Interview CBOs’ leaders – Lighthouse group - 2 leaders of Community￾based Organizations (CBOs) 10:00 – 11:00 Interview CBSs – Lighthouse group - 5 Community-based Supporters (CBSs) 13:30 – 14:30 Hoang Mai District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 15:00 – 16:00 Hoang Mai District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 16:00 – 17:00 Hoang Mai District HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 17:00 – 19:00 Interview KPs – Lighthouse group - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases TEAM 2: WORK IN TAY HO DISTRICT THU 24 MAR 8:30 – 11:00 Hanoi HIV/AIDS Center (Hanoi PAC) - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) Evaluation of Community HIV Link Programs 69 Date Time Institutions Stakeholders/ Informants Note 13:30 – 14:30 Interview CBOs’ leaders – SHP Clinic - 2 leaders of Community￾based Organizations (CBOs) 14:30 -15:30 Interview CBSs – SHP Clinic - 5 Community-based Supporters (CBSs) 16:00 – 18:00 Interview KPs – SHP Clinic - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases FRI 25 MAR 8:30 – 9:30 Tay Ho District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 10:00 – 11:00 Tay Ho District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 11:00 – 12:00 Tay Ho District HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 13:30 – 17:30 Evaluation team meeting Evaluation of Community HIV Link Programs 70 HO CHI MINH CITY (28-31 March, 2016) Date Time Institutions Stakeholders/ Informants Note SUN 27 MAR Hanoi - HCMC MON 28 MAR 8:30 – 10:00 (venue at HCMC PAC) - Go Vap District Outpatient Clinic (OPC) - Go Vap District HIV Testing & Counseling Clinic (HTC) - OPC manager - HTC manager - Relevant officers Kerry Hanh Dang 10:30 – 12:00 USAID AOR Ms. Trang Le Kerry 8:30 – 11:30 District 7 Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program District 7 HIV Testing & Counseling Clinic (HTC) - Technical staff (who work/collaborate with CL program) - Clinic staff in-charge of database and documentation District 7 Outpatient Clinic (OPC) - Technical staff (who work/collaborate with CL program) - Clinic staff in-charge of database and documentation 13:30 – 14:30 Interview CBOs’ leaders: G3VN - Leader of Community￾based Organizations (CBOs) All team 14:30 – 15:30 Interview CBSs – G3VN - 5 Community-based Supporters (CBSs) 16:00 – 18:00 Interview with KPs – G3VN - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases Evaluation of Community HIV Link Programs 71 WORK IN GO VAP DISTRICT TUE 29 MAR 8:30 – 9:30 HCMC HIV/AIDS Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) Hoang Tran Kerry Hanh Dang 10:00 – 11:30 (venue at HCMC PAC) - District 7 Outpatient Clinic (OPC) - District 7 HIV Testing & Counseling Clinic (HTC) - OPC manager - HTC manager - Relevant officers 8:30 – 11:30 Go Vap Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program Go Vap Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) Go Vap HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 13:30 – 14:30 Interview CBOs’ leaders - Smile group - Leader of Community￾based Organizations (CBOs) All team 14:30 – 15:30 Interview CBSs – Smile Group - 5 Community-based Supporters (CBSs) 16:00 – 18:00 Interview with KPs – Smile Group - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases WORK IN DISTRICT 11 WED 30 MAR 8:00 – 9:30 (venue at HCMC PAC) - District 11Outpatient Clinic (OPC) - District 11 HIV Testing & Counseling Clinic (HTC) - OPC manager - HTC manager - Relevant officers Evaluation of Community HIV Link Programs 72 SPLIT TEAM TEAM 1: WORKING WITH CBO 14:00 – 15:00 Interview CBOs’ leaders - GLink - Leader of Community￾based Organizations (CBOs) Kerry Hanh Dang Chang Le 15:00– 16:30 Interview CBSs - GLink - 5 Community-based Supporters (CBSs) 17:00 – 19:00 Interview with KPs - GLink - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases TEAM 2: WORKING WITH HEALTH OFFICERS 13:30 – 17:00 District 11 Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program Nga Le Long Tran District 11 Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) District 11 HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers THU 31 MAR 10:30 – 12:00 LIFE Center - Director/Deputy Director - CL program manager - Accountant All team 13:30 – 14:30 Interview CBOs’ leaders – Vuot Song Group - Leader of Community￾based Organizations (CBOs) 14:30 – 15:30 Interview CBSs – Vuot Song Group 5 Community-based Supporters (CBSs) 16:00 – 18:00 Interview with KPs – Vuot song Group - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached cases PM HCMC – Hanoi Evaluation of Community HIV Link Programs 73 DIEN BIEN (TEAM 1) (04-07 April, 2016) Date Time Institutions Stakeholders/ Informants Note SUN 3 APR PM Hanoi – Dien Bien WORK IN DIEN BIEN PHU CITY MON 4 APR 8:00 – 9:00 Dien Bien Phu City Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 9:30 – 11:00 Dien Bien Phu City HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 13:30 – 15:00 Dien Bien Phu City Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 15:30 -17:30 Group discussion with CBSs in Dien Bien Phu City - 4 Community-based Supporters (CBSs) TBD Interview KPs in Dien Bien Phu City - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached KPs - other KPs WORK IN DIEN BIEN DONG DISTRICT TUE 5 APR 8:00 – 9:00 Dien Bien Dong District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 9:30 – 11:00 Dien Bien Dong District HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 13:30 – 15:00 Dien Bien Dong District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 15:30 – 17:30 Group discussion with CBSs in Dien Bien Dong District - 5 Community-based Supporters (CBSs) Evaluation of Community HIV Link Programs 74 Date Time Institutions Stakeholders/ Informants Note 17:30 – 19:00 Interview KPs in Dien Bien Dong District - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached KPs - other KPs WED 6 APR 8:00 – 11:00 Dien Bien Provincial HIV/AIDS Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) 14:00 -15:30 Group discussion with CBSs in Dien Bien Dong District - 5 Community-based Supporters (CBSs) TBD Interview KPs in Dien Bien Dong District - 5 KPs received service at HTC - 5 KPs received service at OPC - 5 recently-outreached KPs - other KPs THU 7 APR AM Interview with KPs in Dien Bien Dong District PM Dien Bien - Hanoi Evaluation of Community HIV Link Programs 75 NGHE AN (TEAM 2) (05-08 April, 2016) Date Time Institutions Stakeholders/ Informants Note MON 4 APR AM Hanoi - Vinh PM Vinh – Que Phong WORK IN QUE PHONG DISTRICT TUE 5 APR 8:00 – 9:00 Que Phong District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 9:30 – 10:30 Que Phong Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 11:00 – 12:00 Que Phong HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers 14:00 – 16:00 Interview CBSs – Huong Que Group - 1 Community-based Supporter (CBS) Que Phong - Vinh WORK IN HUNG NGUYEN DISTRICT WED 6 APR AM Team meeting 13:30 – 14:30 Hung Nguyen District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 15:00 – 16:00 Hung Nguyen District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 16:00 – 17:00 Hung Nguyen District HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers THU 7 APR 8:30 – 11:00 COHED Quarterly Meeting Phuong Dong Hotel – Vinh City 13:30 – 14:30 Interview CBO s’ leaders – Hung Nguyen Group - 2 leaders of Community￾based Organizations (CBOs) Evaluation of Community HIV Link Programs 76 Date Time Institutions Stakeholders/ Informants Note 14:30 – 15:30 Interview CBSs – Hung Nguyen Group - 5 Community-based Supporters (CBSs) 16:00 – 18:00 In-depth interview with KPs – Hung Nguyen Group - 1 KPs received service at HTC - 1 KPs received service at OPC - 1 recently-outreached cases FRI 8 APR 8:30 – 11:30 Nghe An HIV/AIDS Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) 13:15 – 14:15 COHED Office in Nghe An - Project Coordinator Long Tran 14:00 – 15:30 Vinh City Health Center - Director/Deputy Director - Head of HIV Testing and Counseling Center Kerry Hanh Dang 14:15 – 16:30 Interview CBO leaders - Leaders of Song Lam Xanh and Vuot song group Long Tran Vinh - Hanoi Evaluation of Community HIV Link Programs 77 LAO CAI (HOANG TRAN) (14-15 April, 2016) Date Time Institutions Stakeholders/ Informants Note WORK IN BAO THANG DISTRICT THU 14 APR 8:00 – 9:00 Bao Thang District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 9:30 – 11:00 Bao Thang District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) 13:30 – 15:00 Group interview with CBSs - 2 Community-based Supporters (CSBs) 15:00 – 17:00 Interview KPs - 2 clients using services in OPC, HTC WORK IN LAO CAI CITY FRI 15 APR 8:00 – 11:00 Lao Cai HIV/AIDS Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) PM Lao Cai - Hanoi Evaluation of Community HIV Link Programs 78 HAI PHONG (TEAM 2) (21-22 April, 2016) Date Time Institutions Stakeholders/ Informants Note THU 21 APR 13:30 – 15:00 Ngo Quyen District Health Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program 15:00 – 16:30 Ngo Quyen District HIV Testing & Counseling Clinic (HTC) - Clinic manager - Relevant officers Ngo Quyen District Outpatient Clinic (OPC) - Clinic manager - Clinic staff (who work/collaborate with CL program) FRI 22 APR 8:00 – 11:00 Hai Phong HIV/AIDS Center - Director/Deputy Director - Officer(s) responsible for HIV/AIDS program - Officer(s) in charge of OPC, HTC, Methadone - CL Provincial coordinator (separate meeting) 13:30 – 15:30 Interview CBO leader in Thuy Nguyen district - Hoa Hai Duong group - 1 leaders of Community￾based Organization (CBO) Interview CBSs – Hoa Hai Duong group - 5 Community-based Supporters (CBSs) 16:00 – 18:00 Interview CBO leader in Hai Phong City - Song Tich Cuc group - 1 leaders of Community￾based Organization (CBO) Interview CBSs – Song Tich Cuc group - 5 Community-based Supporters (CBSs) Evaluation of Community HIV Link Programs 79 INTERVIEWS WITH OTHER STAKEHOLDERS IN HANOI Date Time Institutions Stakeholders/ Informants Note FRI 01 APR 10:00 – 12:00 Center for Community Health Research and Development (CCRD) - Director/Deputy Director - CL program manager/ staff - Finance staff 14:00 – 16:00 The Center for Community Health and Development (COHED) - Director/Deputy Director - CL program manager - Finance staff MON 11 APR 14:00-15:00 FHI 360 Ms. Nguyen To Nhu Hoang Tran Kerry Nga Le TUE 12 APR 15:30-17:00 Vietnam Administration of HIV/AIDS Control Mr. Bui Duc Duong Kerry Nga Le WED 13 APR 8:30 – 17:30 USAID meeting Hoang Tran Nga Le Long Tran 10:00 – 11:00 Healthy Markets Ms. Kimberly Green Kerry FRI 15 APR 10:00-11:30 Vietnam Union of Science and Technology Associations (VUSTA) Mrs. Do Thi Van Kerry Nga Le Chang Le WED 20 APR 11:00-12:00 WHO Dr. Masaya Kato TUE APR 26 14:00-16:00 CCRD Skype call THU APR 28 10:00-12:00 COHED Skype call 14:00-15:30 USAID Partner Capacity Development Program Robert Letchford Project Director 16:00-18:00 LIFE Center Skype call Evaluation of Community HIV Link Programs 80 ANNEX 6: LIST OF INFORMANTS ** Information about Community-Based Supporters’ (CBSs’) identity is kept confidential and protected in compliance with the USAID’s regulation. Only the number of CBSs participating in the interviews and the name of their organization are stated in this Annex. No. Full Name Title Organization U.S Government (USG) Agencies 1 Joakim Parker Mission Director United States Agency for International Development in Vietnam (USAID) 2 Randolph Flay Assistant Director United States Agency for International Development in Vietnam (USAID) 3 Emily Rupp Deputy Director - Program Development Office United States for International Development in Vietnam (USAID) 4 Mark Breda Senior HIV/AIDS Technical Adviser - Office of Health United States Agency for International Development in Vietnam (USAID) 5 Nguyen Thi Minh Huong HIV/AIDS Drug Rehabilitation Specialist - Office of Health United States Agency for International Development in Vietnam (USAID) 6 Nguyen Thi Ha M&E Specialist/Gender Advisor United States Agency for International Development in Vietnam (USAID) National and International Stakeholders 7 Bui Duc Duong Deputy General Director Vietnam Administration of HIV/AIDS Control 8 Do Thi Van Director VUSTA Component Project - Global Fund Supported Project on HIV/AIDS 9 Nguyen To Nhu Deputy Country Director FHI 360, USAID SMART TA Project in Vietnam 10 Masaya Kato Doctor World Health Organization (WHO) 11 Kimberly Green Chief of Party PATH, USAID Healthy Markets in Vietnam 12 Robert Letchford Project Director USAID Partner - Capacity Development Program Implementing Partners 13 Nguyen Thi Mai Huong Deputy Chairwoman Center for Community Health Research and Development (CCRD) 14 Dinh Thi Yen Nhi Deputy Chief of Party Center for Community Health Research and Development (CCRD) Evaluation of Community HIV Link Programs 81 No. Full Name Title Organization 15 Dao Thi Mai Hoa Director Center for Community Health and Development (COHED) 16 Nguyen Nguyen Nhu Trang Founder and Director Center for Promotion of Quality of Life (LIFE-Center) Hanoi Provincial Level 17 Tran Thi Bich Hau Head of HIV Testing and Counselling Center Hanoi HIV/AIDS Center 18 Ta Thi Hong Hanh Head of Communication and Harm Reduction Intervention Department Hanoi HIV/AIDS Center 19 Duong Lam Tuan Officer Hanoi HIV/AIDS Center District Level 20 Nguyen Thi Minh Director Hoang Mai District Health Center 21 Nguyen Van Toi Officer in charge of HIV/AIDS program Hoang Mai District Health Center 22 Nguyen Thi Ha Doctor in charge of Counselling Linh Dam Outpatient Clinic and HIV Testing & Counselling Center 23 Nguyen Thi Ngoc Oanh Doctor in charge of Care and Treatment Linh Dam Outpatient Clinic and HIV Testing & Counselling Center 24 Dang Vu Huong Officer Linh Dam Outpatient Clinic and HIV Testing & Counselling Center 25 Nguyen Kim Dung Director Tay Ho District Health Center 26 Le Thi Hong Loan Doctor Tay Ho Outpatient Clinic and HIV Testing & Counselling Center Community-based Supporters (CBSs) 27 5 CBSs SHP Clinic Ho Chi Minh City Provincial Level 28 Tieu Thi Thu Van Director HCMC Provincial HIV/AIDS Center 29 Van Hung Health Officer of Department of Care and Treatment HCMC Provincial HIV/AIDS Center 30 Mai Thi Hoai Son Health Officer of Department of Methadone Treatment HCMC Provincial HIV/AIDS Center District Level 31 Nguyen Thi Ngoc Dung Head of Clinic Go Vap District Outpatient Clinic 32 Thu Vu Hoang Truc Head of Center Go Vap District HIV Testing & Counseling Center Evaluation of Community HIV Link Programs 82 No. Full Name Title Organization 33 Nguyen Trong Minh Tan Officer in charge of Testing & Counselling District 7 District Health Center 34 Nguyen Anh Tuyet Officer in charge of Care & Treatment District 7 Outpatient Clinic and HIV Testing & Counselling Center 35 Kim Chi Na Head of Department District 11 Care and Treatment Department 36 Bui Thi Tu Anh Head of Department District 11 HIV Testing and Counseling Department Community-based Supporters 37 6 CBSs G3VN 38 4 CBSs G-Link 39 4 CBSs Vuot Song Group Dien Bien Provincial Level 40 Hoang Xuan Chien Director Dien Bien Provincial AIDS Center 41 Vu Hai Hung Deputy Director Dien Bien Provincial AIDS Center 42 Dang Thi Thanh Head of Department of Monitoring and Surveilance Dien Bien Provincial AIDS Center 43 Pham Xuan Sang Deputy Head of Department of Monitoring and Surveilance Dien Bien Provincial AIDS Center 44 Lo To Khuyen Head of ARV Treatment Deparment Dien Bien Provincial AIDS Center 45 Nguyen Kim Hoa Head of HIV Testing and Counselling Center Dien Bien Provinicial AIDS Center 46 Hoang Thi Chuong Head of Outpatient Clinic Dien Bien Provincial Hospital 47 Ms. Nguyen Thi Thuy C-Link provincial coordinator Dien Bien Provincial AIDS Center District Level 48 Vu A Su Director Dien Bien Phu City Health Center 49 Tran Dinh Dai Head of Department of Planning Dien Bien Phu City Health Center 50 Mao Thi Tai HIV/AIDS specialist Dien Bien Phu City Health Center 51 Vu A Cau Deputy Director Dien Bien Dong District Health Center 52 Vu Van Quan Health officer in charge of HIV/AIDS program Dien Bien Dong District Health Center 53 Lo Van Vinh Health officer Dien Bien Dong Outpatient Clinic Evaluation of Community HIV Link Programs 83 No. Full Name Title Organization 54 Lo Thi Thoa Health officer Dien Bien Dong Outpatient Clinic 55 Luong Thanh Nghi Health officer Dien Bien Dong HIV Testing and Counselling Center Community-based Supporters 56 8 CBSs Nghe An Provincial Level 57 Nguyen Van Dinh Director Nghe An Provincial HIV/AIDS Center 58 Duong Tien Hung Health Officer Nghe An Provincial HIV/AIDS Center District Level 59 Le Quang Trung Deputy Director Que Phong District Health Center 60 Vi Van Hai Head of Clinic Que Phong District Outpatient Clinic 61 Nguyen Bich Hau Health Officer Que Phong District HIV Testing and Counseling Center 62 Nguyen Thi Thu Ha Head of Clinic Hung Nguyen District Outpatient Clinic 63 Nguyen Thi Mui Health Officer Care and Treatment Department, Hung Nguyen District Outpatient Clinic 64 Nguyen Thi Hai Ly Health Officer Hung Nguyen District Outpatient Clinic 65 Le Duy Sy Deputy Director Vinh City Health Center 66 Luu Van Hung Head of HIV Testing and Counseling Center Vinh City Health Center 67 Dang Huu Cuong Head of Epidemic Control Department Vinh City Health Center Community-based Supporters 68 1 CBS Huong Que Group 69 7 CBSs Hung Nguyen Group 70 1 CBS Suc Song Moi Group 71 1 CBS Song Lam Xanh Group Lao Cai Provincial level 72 Trần Minh Hiếu Director Lao Cai Provincial AIDS Center 73 Ngo Thi Thanh Quyen Head of HIV Testing and Counseling Department Lao Cai Provincial AIDS Center 74 Nguyen Van Khai C- Link provincial coordinator Lao Cai Provincial AIDS Center District Level 77 Tran Xuan Hung Director, Bao Thang District Health Center Evaluation of Community HIV Link Programs 84 No. Full Name Title Organization 78 Tran Van Khanh HIV/AIDS Coordinator Bao Thang District Health Center 79 Dang Quang Sinh Head Bao Thang Outpatient Clinic Community-based Supporters 82 2 CBS 83 1 patients Hai Phong Provincial level 84 Doan Thi Thu Director Hai Phong Provincial HIV/AIDS Center 85 Dao Viet Tuan Deputy Director Hai Phong Provincial HIV/AIDS Center 86 Le Thi Thu Huong Head of Care and Treatment Department Hai Phong Provincial HIV/AIDS Center 87 Nguyen Thi Lan Huong Head of Finance and Planning Department Hai Phong Provincial HIV/AIDS Center District Level 88 Vu Thi Luong Director Ngo Quyen District Health Center 89 Dao Huong Tra Health Officer Ngo Quyen District Outpatient Clinic 90 Pham Thi Hanh Health Officer Ngo Quyen District HIV Testing and Counseling Center 91 Dao Thi Luong Health Officer Ngo Quyen District HIV Testing and Counseling Center Community-based Supporters 92 4 CBSs Hoa Hai Duong Group 93 7 CBSs Song Tich Cuc Group Evaluation of Community HIV Link Programs 85 ANNEX 7: INTERVIEW GUIDES USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview to Provincial AIDS Center Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about PAC and your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. What are the provincial plans to address 90/90/90 targets? What are key interventions and approaches to achieve these targets? What improvements in HIV cascades along the continuum from prevention to care in last two years? 2. What do you think about the C-Link program? If compared with other programs that also use outreach activities for case finding, what are the distinguished contributions, strengths or weaknesses of the C-link program? How does it fill or does not fill in the gaps/leaks in current HIV responses? 3. What are your views about using “CBSs - cộngtácviêncộngđồng” for outreach and provision of HIV prevention, support and care to KPs? What are the advantages and disadvantages of using “CBSs - cộngtácviêncộngđồng” with diversified backgrounds vs. peers? 4. Are there examples of CBOs/IPs in this CL Program involving in improving enabling environment for HIV responses? For example, coordinating with authorities and local police to improve penalization practices toward KPs? 5. How is annual planning process in HIV response carried out? What agencies participate in the planning process? What changes in the planning process in HIV responses at each level: from province to commune in last two years? Are there any new stakeholders involved in this process? 6. In general, what are advantages/disadvantages for involving CBOs in HIV/AIDS responses? What are your views about having CBOs and HTC/OPC to collaborate and coordinate in case finding activities? 7. Are there plans to keep involving CBOs and their members in HIV/AIDS responses? If yes, in what stage: needs assessment, planning, implementing, or evaluation? 8. What are your views about CBOs’ involvement in planning process? Are there any examples on the contributions of the IPs/CBOs to the process of developing local HIV/AIDS work plans? Or policy? Evaluation of Community HIV Link Programs 86 Sustainability 1. Do you think the outreach activity for case finding continue at project sites after USAID- supported programs end? By who? Where they get the funding for that? 2. What conditions needed to maintain the case finding activity at project sites? 3. What stakeholders involve in case finding at project sites? How will they work together/collaborate in this activity? 4. Does PAC have any plan for case finding? Does PAC have plans to mobilize other funding sources to maintain case finding in project site? What are they? 5. Does HTC/OPC include targets for case finding in their annual plan? 6. Do PACs have plan in increase HTC/OTC availability in remote area? 7. Do PACs have plans to maintain, support or continue using the CBS/CBOs in case finding? How will PAC continue strengthening capacity for CBSs/CBOs in case findings? 8. Do you have plan to scale up/ maintain C-Link program in your province when program finish? Are there any examples of replication/scaling up the approach in working with CBOs and CBSs in C-link Programs to other Programs or Activities? Networks 1. How often are you in contact with the CBOs that work with CLink? (can add names) 2. How many people from the CBOs do you know personally? 3. How many people from the CBOs do you know well enough to call with a question? How often do you talk to the CBO to give information? 4. Before the CLink program, did you know these people from the CBOs? How many did you know, how frequently did you have contact with them, did you call them with questions? Evaluation of Community HIV Link Programs 87 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview to District Health Center (DHC) Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about DHC and your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. What are the district plans to address 90/90/90 targets? What are key interventions and approaches to achieve these targets? What improvements in HIV cascades along the continuum from prevention to care in last two years? 2. What do you think about the C-Links program? If compared with other programs that also use outreach activities for case finding, what are the distinguished contributions, strengths or weaknesses of the C-link program? How does it fill or does not fill in the gaps/leaks in current HIV responses? 3. What are your views about using “CBSs - cộngtácviêncộngđồng” for outreach and provision of HIV prevention, support and care to KPs? What are the advantages and disadvantages of using “CBSs￾cộngtácviêncộngđồng” with diversified backgrounds vs. peers? 4. Are there examples of CBOs/IPs in this CL Program involving in improving enabling environment for HIV responses? For example, coordinating with authorities and local police to improve penalization practices toward KPs? 5. How is annual planning process in HIV response carried out? What agencies participate in the planning process? What changes in the planning process in HIV responses at each level: from province to commune in last two years? Are there any new stakeholders involved in this process? 6. In general, what are advantages/disadvantages for involving CBOs in HIV/AIDS responses? What are your views about having CBOs and HTC/OPC to collaborate and coordinate in case finding activities? 7. Are there plans to keep involving CBOs and their members in HIV/AIDS responses? If yes, in what stage: needs assessment, planning, implementing, or evaluation? 8. What are your views about CBOs’ involvement in planning process? Are there any examples on the contributions of the IPs/CBOs to the process of developing local HIV/AIDS work plans? Or policy? Evaluation of Community HIV Link Programs 88 HTC/OPC 1. What is the workload of the current HTC/OPC? What is the designated number of clients at HTC/OPC? What are implications for the increase number of clients coming to HTC/OPC? 2. How is accessibility of HTC/OPC in remote communes? 3. How is quality of HTC/OPC? What are strengths and weaknesses? What implications for quality of HTC/OPC if there are more clients? 4. What is the DHC’s plan to address the overload issues (if there are)? Sustainability 1. Do you think the outreach activity for case finding continue at project sites after USAID- supported programs end? By who? Where they get the funding for that? 2. What conditions needed to maintain the case finding activity at project sites? 3. What stakeholders involve in case finding at project sites? How will they work together/collaborate in this activity? 4. Does DHC have any plan for case finding? Does DHC have plans to mobilize other funding sources to maintain case finding in project site? What are they? 5. Does HTC/OPC include targets for case finding in their annual plan? 6. Do DHC have plan in increase HTC/OTC availability in remote area? 7. Do DHC have plans to maintain, support or continue using the CBS/CBOs in case finding? How will DHC continue strengthening capacity for CBSs/CBOs in case findings? 8. Does DHC have plan to scale up/ maintain C-Links program in the district when program finish? Are there any examples of replication/scaling up the approach in working with CBOs and CBSs in C-link Programs to other Programs or Activities? Networks 1. How often are you in contact with the CBOs that work with CLink? (can add names) 2. How many people from the CBOs do you know personally? 3. How many people from the CBOs do you know well enough to call with a question? How often do you talk to the CBO to give information? 4. Before the CLink program, did you know these people from the CBOs? How many did you know, how frequently did you have contact with them, did you call them with questions? Evaluation of Community HIV Link Programs 89 USAID/Vietnam C–Link Programs Evaluation Guideline for qualitative interview to HIV Testing and Counseling Center (HTC) Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about HTC and your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. How many clinical staffs at the HTC? What is designated number of clients per month when HTC was established? 2. What is the average number of clients receiving HIV counseling and testing a month? What percentage of those clients referred by the CBSs/CBOs? 3. Have you observed any changes of clients coming to take HIV tests since C-Link program started? Are there more or less high-risk KPs coming to test? Are there more or less regular HIV tests among KPs? Are there more KPs living far away to come for HIV tests? Are there more newly founded HIV positive cases? 4. What factors make these changes? Is C-link program contributing to these changes? How often do CBSs accompany with the KPs? 5. How does this HTC collaborate with CBSs/CBOs to do outreach to high risk KPs? In addition to the C-links program, does this HTC work with other programs to do outreach? What are the distinguished contributions, strengths or weaknesses of the C-link Program’s outreach approach? What do you think about outreach activities carried out by the CBSs/CBOs (with diversified background) in the C-Link program? As compared with other programs using peer educators? 6. What do you think about referral mechanism created by the C-link program? 7. What do you think about payment of incentives based on number of clients referred by CBSs? How effective was it? 8. What are the existing mechanisms or interventions for quality assurance of the services provided at the HTC? 9. What advantages or challenges is HTC having as the number of clients coming to HIV test increases? What measures will HTC take to address the challenges? Evaluation of Community HIV Link Programs 90 10. Once C-link program ends, how will outreach activities to high risk KPs be continued? Networks 1. How often are you in contact with the CBOs that work with CLink? (can add names) 2. How many people from the CBOs do you know personally? 3. How many people from the CBOs do you know well enough to call with a question? How often do you talk to the CBO to give information? 4. Before the CLink program, did you know these people from the CBOs? How many did you know, how frequently did you have contact with them, did you call them with questions? Evaluation of Community HIV Link Programs 91 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview to Out-Patient Clinic (OPC) Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about OPC and your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. How many clinical staffs at the OPC? What is designated number of clients per month when HTC was established? 2. What is the average number of clients enrolled to ARV treatment monthly? What percentage of those clients referred by the CBSs/CBOs? 3. What percentage of clients retains ARV treatment? What are major groups of clients who dropped out or do not follow ARV protocol? 4. Does this OPC set targets for percentage of ARV clients retaining? What is your plan to achieve these targets? 5. Since C-link program started, are there more or less the KPs who are lost to follow come back for ARV treatment? What contributions from C-link program to help lost-follow ARV clients to come back? How often do CBSs accompany with the KPs to OPC? 6. How does this OPC collaborate with CBSs/CBOs to provide continuous support and care to ARV clients? What is the mechanism for collaboration to follow-up ARV clients effectively? 7. In addition to the C-links program, does this OPC work with other programs to do outreach? What are the differences between the outreach methods of these programs? What are the distinguished contributions, strengths or weaknesses of the C-link Program’s outreach approach? What do you think about outreach activities carried out by the CBSs/CBOs (with diversified background) in the C-Link program? As compared with other programs using peer educators? 8. What do you think about referral mechanism created by the C-link program? 9. What do you think about payment of incentives based on number of clients referred by CBSs? How effective was it? 10. What are the existing mechanisms or interventions for quality assurance of the services provided at the HTC? 11. What challenges is OPC facing as the number of clients registering to OPC increases? 12. Once C-link program ends, how will outreach activities to high risk KPs be continued? Evaluation of Community HIV Link Programs 92 Networks 1. How often are you in contact with the CBOs that work with CLink? (can add names) 2. How many people from the CBOs do you know personally? 3. How many people from the CBOs do you know well enough to call with a question? How often do you talk to the CBO to give information? 4. Before the CLink program, did you know these people from the CBOs? How many did you know, how frequently did you have contact with them, did you call them with questions? Evaluation of Community HIV Link Programs 93 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview to Community-Based Organization (CBO) Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your CBO and your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. 1. CBOs’ institutional capacity building 1.1. How long has your CBO existed? Did you form at the time that C-Link began activities, or did you exist before that? 1.2. What is your CBO’s mission? 1.2.1. Has your mission changed since joining C-Link? 1.2.2. What is your Vision toward 2020? 1.3. To what extent has the initial plan on CBOs development been accomplished (in the first year of C-Link) been accomplished? 1.3.1. What are the adaptations of the initial plan? 1.3.2. What are the current focuses for strengthening CBOs’/IPs institutional capacity? 1.4. What key measures on strengthening institutional capacity of CBOs? How have you tracked the improvements of the institutional capacity of CBOs’/IPs’? 1.5. What changes in the institutional capacity of IPs/CBOs as results of USAID support in terms of leadership, management, planning, implementing, monitoring, human resource’s competence, management, fund raising, networking, communication and advocacy? Can you give key milestones that reflect the improvements? 1.6. What key supporting activities that have resulted in these changes? 1.7. Have you received support and capacity building from the IP in seeking and applying for other funding sources? Evaluation of Community HIV Link Programs 94 1.7.1. Have you applied for other funding besides that from C-Link? Where and to do what? 1.7.2. Have you received other funding besides that from C-Link? Where and to do what? 1.8. How has IPs’/CBOs’ networking changed as results of the C-link Program’s support? What are the new partners that IPs/CBOs started to network with? With what partners have IPs/CBOs contacted more frequently? What has the program contributed to these changes? 1.9. How is the collaboration and coordination between CBOs and governmental agencies? Is the mechanism for collaboration created in C-link Program effective and sustainable? What are the enabling conditions and challenges for collaboration between CBOs and governmental agencies? 1.9.1. How often are you in contact with the government service providers (HTC/OPC)? 1.9.2. How many people from the government service providers do you know personally? 1.9.3. How many people from the HTC/OPC do you know well enough to call with a question? How often do you talk to the HTC/OPC to give information? 1.9.4. Before the CLink program, did you know these people from the HTC/OPC? How many did you know, how frequently did you have contact with them, did you call them with questions? 1.10. Do you have any collaboration with private organizations and business owners? How important of this collaboration? What purposes? What plan for further engagements? 2. Outreach Activities 2.1. Target setting 2.1.1. What are the annual and monthly targets in reaching high risk KPs of your CBO? How did your CBO set these targets? 2.1.2. What was your CBO’s plan to achieve this target? How have IPs supported you in implementing this plan and achieving your CBO’s targets? 2.1.3. Did you set target on the number of high risk KPs for each CBS? How did CBOs support/work with CBSs to achieve this target? 2.2. Outreach approach 2.2.1. What are the criteria to select CBS? Who do you consider to be an “active” CBS? 2.2.2. What are your views about using CBSs with diverse backgrounds to reach high risk KPs? What pros and cons as compared with using only peers? If there were no outreach activities by CBSs, would many high risk KPs come to HTC and/or OPC? (to drop? Nearly all CBS are peers) Evaluation of Community HIV Link Programs 95 2.2.3. Are there different approaches for outreach to each KP group? Are outreach activities targeting primary sexual partners of the IDUs, FSWs, and MSMs? 2.2.4. If the Program is effective in the first year, from the second year, are remaining KPs who refused to go for test/treatment getting harder to reach and bring them to HTC or OPC? What are the changes in approaches to reach these KPs? 2.2.5. How have your CBO collaborated with other CBOs/OPC/HTC/PAC and others in outreach activities to high risk KPs? Give an example of good collaboration and networking in outreach activities. Is there competition between CBOs or between CBSs in reaching and referring high risk KPs? 2.2.6. What advantages and challenges for CBOs in this program? 3. Sustainability 3.1. Do you think CBOs are going to maintain the case finding activity, outreach activities and provision of packages in HIV prevention, care and support in project sites after USAID- supported programs end? Where you get the funding for that? 3.2. What conditions are needed to maintain the case finding activity in project sites? 3.3. After C-Link Program finishes, how will CBOs continue to invest for institutional capacity building? Evaluation of Community HIV Link Programs 96 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview Community Based Supporter Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your views about the Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information PWID……MSM…..FSW…… Other………………………… Gender: Male…… Female…….. 1. Outreach 1.1. What are the high risk KPs do you reach? How do you reach them? Do you have different ways to reach each group of high risk KPs? Do you reach primary sexual partners of IDUs, FSWs, MSMs? Can you describe specifically the process you went through to do outreach; then provide packages of HIV prevention or support and care? Did this relationship exist before C￾Link Program? 1.2. How did KPs react to your outreach? How were KPs’ attitude about your provision of HIV prevention, care and support? What percentage of KPs who you successfully referred to HTC or OPC? 1.3. Can you give an example of successful cases and failure cases (e.g. KPs agree to go to HTC and test; or go to OPC for ARV treatment) 1.4. What factors do you think make those cases successful or failed? What challenges of outreach activities? What did you do to overcome those challenges? Are KPs who refused to go for test/treatment getting harder to reach and bring them to HTC or OPC? What are the changes in approaches to reach these KPs? 1.5. Were CBSs trained approaches for outreach to each KP group? How useful were the training contents for your outreach activities? Did you apply what you learned in the training courses? What advantages or challenges for application? How effective other supports from the Program for you to do outreach activities effectively? 1.6. Do you have targets in the number of reached high risk KPs? Referred KPs? Tested KPs? KPs registered to OPC? KPs retaining ARV treatment? 1.7. Are there other people in your community who also do outreach activities to high risk KPs? What are the advantages and disadvantages of CBSs who are not in the same group with KP (if applicable)? Evaluation of Community HIV Link Programs 97 1.8. What factors motivated you to participate in this program as CBSs? What do you think about the payment of incentives in this program? What other supports did you receive to help you do outreach activities? 1.9. How do you follow up the referred KPs? Are KPs motivated to use referral card? What advantages or disadvantages of the current referral mechanism? 1.9.1.Which is more difficult: motivating KPs to get tested or to go for treatment if they test positive for HIV? Why or why not? 1.9.2. Is it difficult to motivate KPs to continue in treatment once they start? Why or why not? Have you ever convinced someone to re-start treatment after they had started? How did you motivate them? 1.10.How is collaboration between CBSs and CBOs? How important of this collaboration to make outreach activities successful? What the strengths and weaknesses of the collaboration? What changes has C-Link Program created in this collaboration? Did this collaboration exist before C-Link Program? 1.11.Networks 1.11.1. How often are you in contact with the government service providers (HTC/OPC)? 1.11.2. How many people from the government service providers do you know personally? 1.11.3. How many people from the HTC/OPC do you know well enough to call with a question? How often do you talk to the HTC/OPC to give information? 1.11.4. Before the CLink program, did you know these people from the HTC/OPC? How many did you know, how frequently did you have contact with them, did you call them with questions? 1.12.How is collaboration between CBSs and HTC/OPC? How important of this collaboration to make outreach activities successful? What the strengths and weaknesses of the collaboration? What changes has C-Link Program created in this collaboration? Did this collaboration exist before C-Link Program? 2. Sustainability 2.1. Do you think you are going to maintain the case finding activity, outreach activities and provision of packages in HIV prevention, care and support to KPs in project sites after USAID￾supported programs end? If yes, what motivations for your continuation? If no, what are barriers? 2.2. What condition needed to maintain the case finding activity in project sites? Evaluation of Community HIV Link Programs 98 USAID/Vietnam C–Link Programs Evaluation Guideline for qualitative interview for Key Population member Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your views. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Group: Recruitment for survey: FSW….. MSM….. PWID…… HTC…….OPC……. CBS met recently….. Outreach 1. Could you describe how did CBS meet you? What information and/or support she/he gave you? How often does she/he contact you? By what means? (refresher from quantitative—to get more detail) 2. How do you think about CBSs reaching you to do counseling and advise you to go for HIV test or ARV treatment? o What did you do when CBS reach you? Did you think about what he/she said? o When the CBS first talked to you, did he/she mention about getting some money for getting tested? o Before the CBS, did anyone else ever talk to you about HIV and testing? 3. Did you take the HIV test? Did you know the test results? o How long after the CBS first talked to you did you go for the test? o Did CBSs accompany with you to the HTC/OPC? What/who convinced you to do that? What support did CBSs provide you after you got the test results? 4. Did the CBS talk to you about getting your sex partners tested too? Have CBSs approached your primary sexual partners and persuade them to go for HIV test? Did they take HIV tests in last six months? 5. Did you bring a CBS’s referral card when you go to HTC for HIV test? What benefits for yourself of using referral card? Evaluation of Community HIV Link Programs 99 6. How did you feel about the staff at the HTC center? Did you feel that they treated you well? Did they give you the information that you needed? Did they answer your questions? o Did you have any feeling that you were treated differently because you are a PWID/MSM/FSW? In what way? (details) o Would you recommend that your friends who are PWID/MSM/FSW go to get tested too? Why or why not? 7. What are the barriers for you to go to take HIV tests? How did the CBSs persuade you? (for the KPs who did not have HIV tests in last six months) 8. Have you been to the OPC for treatment (depends on group) 9. How did you feel about the staff at the OPC center? Did you feel that they treated you well? Did they give you the information that you needed? Did they answer your questions? o Did you have any feeling that you were treated differently because you are a PWID/MSM/FSW? In what way? (details) o Would you recommend that your friends who are PWID/MSM/FSW go to get treated too? Why or why not? 10. What are the reasons why you did not follow ARV treatment regime? What will help you to follow ARV treatment better, in addition to the CBSs’ care and support? (for the KPs who do not comply with the regime) 11. What changes of yourself after CBSs met and provided counseling, care and support on HIV? What differences of you now and yourself before? Evaluation of Community HIV Link Programs 100 USAID/Vietnam C–Link Programs Evaluation Guideline for qualitative interview to IPs Introduction Hello, Thank you for receiving us today. The ongoing independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your views. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. 1. Outreach activities 1.1. What are your views on effectiveness of using CBSs for reaching hidden high risk KPs (as compared with other programs)? What do they think about idea of CL program? Especially the ideas of using CBSs with diversified backgrounds? 1.2. If the Program is effective in the first year, from the second year, are remaining KPs who refused to go for test/treatment getting harder to reach and bring them to HTC or OPC? What are the changes in approaches to reach these KPs? 1.3. Training: 1.3.1.What are the components of the training program for CBOs/CBS? 1.3.2.What knowledge and skills do you provide the CBOs/CBS with in the training? 1.3.3.Does everyone who enters the training program “pass”/ Do you ever discontinue a CBS who does not do well in the training? 1.4. CBS/CBO management: 1.4.1.What is the turnover of CBS who are trained? About what percentage are still involved in the work 6 months after training? 1.4.2.What is the definition of an “active” CBS? 1.4.3.Are there other ways that you try to motivate the CBOs and CBS besides incentives? 1.5. Target setting for CBOs and CBS: 1.5.1.How were the program and annual targets set? 1.5.2.Have program targets been amended and what were reasons for amendments? 1.5.3. Is there any risk that KPs repeatedly go for testing/enroll in treatment based on the incentive system? 1.6. What is the status of annual target achievements against the cascade? How does IP monitor contribution of C-Link Program’s cascade to 90/90/90 targets? 1.6.1.What is likelihood to achieve targets by the end of the program? Evaluation of Community HIV Link Programs 101 1.6.2.Among the targets, what targets are the most focused? What are the most likely to achieve? What are the enabling factors and barriers? 1.7. HTC and OPC: 1.7.1.Do you do any capacity building for HTC/OPC, such as stigma & discrimination training, working with FSW, MSM or PWID? 1.7.2.Have there been any problems at the HTC or OPC with the increased demand created by CLink? 1.8. What are your views on added values of CL programs to the continuum of care? 1.9. What are changes of IPs’ networking as compared with before the C-link program? Are there new partners? Are there partners with whom IP has stronger links and more frequent contacts? 1.10.What are your views on collaboration and coordination among stakeholders in the C-link Program? Especially the collaboration and coordination between CBOs/CSOs with governmental system? Across CBOs? Between CBOs/CSOs with private sector? 1.10.1. What activities do you do to stimulate and facilitate collaboration and coordination between CBOs and the government system? Across CBOs? Between CBOs and the private sector? 1.11.How do you perceive about roles and added values of IPs/CBOs in sustainable HIV responses? What are their added values to the existing system? Please give examples on the contributions of the IPs/CBOs to the process of developing local HIV/AIDS work plans? Or policy? Or facilitating enabling environment for HIV/AIDS responses through their coordination, collaboration and advocacy efforts? 1.12.After the donor supported programs end, are there spaces for IPs/CBOs to engage continuously and systematically in sustainable HIV responses? In what stage: needs assessment, planning, implementing, and evaluation? What are enabling conditions and challenges? 2. Contribution to CSOs development 1. In general, how is the development of CBOs in project site since the Programs started? What changes in CBOs have you observed? What changes in CBOs network in last two years in term of expansion and intensity of partnerships? What are the advantages and challenges, especially in CBOs/CSOs registration? What conditions of CBOs/CSOs to be able to register as a formal institution? 2. What about evolving enabling environment, for example after the issue of the Law on Association; or after the National Election of National Assembly which allows self-nominees to participate? How have IPs/CBOs contributed to these evolvements? 3. What issues have IPs’ or CBOs’ advocacy activities have addressed? How effective were advocacy activities 4. Are there any examples of replication/scaling up the approach in working with CBOs and CBSs in C￾link Programs to other Programs? Activities? 5. Are there examples of CBOs/IPs involving in improving enabling environment for HIV responses such as coordinating with authorities and local police to improve penalization practices toward KPs? 6. What are CL program contributions to the development of others CSOs in project site? 7. What examples of policies and national/provincial plans that IPs/CBOs contribute to? Evaluation of Community HIV Link Programs 102 3. Sustainability/IPs development 1. To what extent, has the initial plan on IPs development been accomplished? What are the adaptations of the initial plan? What are the current focuses for strengthening IPs institutional capacity? Who is carrying out capacity building for IPs? 2. What key measures on strengthening institutional capacity of IPs? How have you tracked the improvements of the institutional capacity of IPs’? 3. What changes in the institutional capacity of IPs as results of USAID support in terms of leadership, management, planning, implementing, monitoring, human resource’s competence, management, fund raising, networking, communication and advocacy? Can you give key achieved milestones that reflect the improvements? 4. What key supporting activities that have resulted in these changes? 5. How has IPs’ networking changed as results of the C-link Program’s support? What are the new partners that IPs started to network with? With what partners have IPs contacted more frequently? 6. How has IPs engaged with business owners and private organizations? 7. What are success/failure stories on strengthened institutional capacity of IPs? 8. What are advantages and challenges in institutional capacity building for IPs? What approach will be effective? 4. Sustainability/Outreach 1. Do you think the outreach activity for case finding continue at project sites after USAID- supported programs end? By who? Where they get the funding for that? 2. What conditions needed to maintain the case finding activity at project sites? 3. What stakeholders involve in case finding at project sites? How will they work together/collaborate in this activity? Evaluation of Community HIV Link Programs 103 USAID/Vietnam C–Link Programs Evaluation Guideline for qualitative interview International Stakeholders Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your Program and your views about the C-Links Program. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. In overall can you highlight key HIV/AIDS achievements in Vietnam in recently? 2. What are challenges for Vietnam in responses in HIV/AIDS to achieve 90/90/90? 3. What are the good model for HIV/AIDS responses you recognize recently? 4. What are your views about using “cộngtácviêncộngđồng” for outreach and provision of HIV prevention, support and care to KPs? What are the advantages and disadvantages of using “cộngtácviêncộngđồng” with diversified backgrounds vs. peers? 5. What is your view of CSO in HIV/AID responses in Viet Nam? 6. What are challenges of CSO in Vietnam in term of more involvement in HIV/AID responses? 7. What is your view of developing collaboration between government and CSO in HIV/AIDS responses? 8. What contribution of C-Link Programs in HIV/AIDS responses/continuum of prevention and care in your view? How have C-Links Programs complemented with your program? How different program leveraged each others? 9. How about the opportunity for C-Link Program to scale up their model? 10. What are your views about sustainability of the established CBOs and community based out-reach activities in HIV responses after withdrawal of USAID support? What are required to ensure sustainability of the CBOs and their outreach activities? Evaluation of Community HIV Link Programs 104 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview Vietnam Administration AIDS Control Introduction Hello, Thank you for receiving us today. As you know, since 2014, the USAID/Vietnam has supported your province the Community HIV Link Program. This Program aim to (1) Improving HIV/AIDS services provided by community-based organizations (CBOs); (2) Strengthening the capacity of HIV/AIDS CBOs; and (3) Enhancing networking and coordination with key stakeholders. From March to May 2016, USAID is conducting an independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your views about HIV/AIDS responses in Vietnam. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Gender: Male…… Female…….. Interview Question 1. In overall can you highlight key HIV/AIDS achievements in Vietnam in recently? 2. What are advantages and challenges for Vietnam in responses in HIV/AIDS to achieve 90/90/90? 3. What are the long-term plans of Vietnam in HIV/AIDS responses? 4. How VAAC coordinator all external funding for HIV/AIDS responses? 5. What are the good models for HIV/AIDS responses you recognize recently? 6. What are your views about using “cộngtácviêncộngđồng” for outreach and provision of HIV prevention, support and care to KPs? What are the advantages and disadvantages of using “cộngtácviêncộngđồng” with diversified backgrounds vs. peers? 7. What is your view of CBOs’ roles and added values in HIV/AID responses in Viet Nam? 8. What are challenges of CBOs in Vietnam in term of more involvement in HIV/AID responses? 9. What is your view of developing collaboration between government and CBOs in HIV/AIDS responses? 10. What contribution of C-Link programs in HIV/AIDS responses/continuum of prevention and care in your view? 11. What is your view about sustainability of C-link Program and CBOs working in HIV/AIDS area once the Program is completed? What are the possible measures to improve sustainability of the Program’s achievements and activities? 12. How about the opportunity for C-Link program to scale up their model? Evaluation of Community HIV Link Programs 105 USAID/Vietnam C–Link Programs Evaluation Guideline for key informant interview to AOR/COR Introduction Hello, Thank you for receiving us today. The ongoing independent evaluation to assess the Program’s progress and achievements to date; document best practices as well as lessons learnt through the implementation; and evaluate the Program’s sustainability. The evaluation team includes…We would like to meet with you to learn about your views. We commit that all the collected information will be used for only this evaluation and no names will be included in the evaluation report. Informant information Full name Cell phone Current position Sex: Male…… Female…….. Interview Question 1. How was Theory of Change (strategy) for Objective 1 and 2 developed and incorporated in the Design of the Program? 2. What provisions were made to alleviate stigma and discrimination in the program sites? 3. What provisions were made for increasing the workload for HTC and OPC from the C-link Programs? 4. What provisions were made for enabling environment such as penalization practices, livelihoods etc. 5. How was the Program’s Design developed following the Theory of Change? 6. How to set program and annual targets; adjustment of program targets and reasons for amendment; among the targets, what targets are the most focused? 7. USAID’s views on effectiveness of using CBOs and CBSs with diversified backgrounds for reaching hidden high risk KPs by years (as compared with other programs)? What evidences for using CBSs with diversified backgrounds? 8. USAID’s Expectations on the issues of coordination and collaboration between CBOs/CBSs and governmental system for the evaluation to explore? 9. USAID’s expectations on the level of C-Links Program’s investment to tackle gender issues in the Program? How was gender sensitive approach defined in C-Links Program? 10. USAID’s expectations on the issues of CBOs’ institutional capacity strengthening, including application for legal status and C-Links Program’s contribution to Civil Society Organization development? Given the current progress. What are the known challenges in this area? 11. USAID’s expectations on the areas to focus for making recommendations and developing actions following the Evaluation? What kinds of management actions does USAID expect to pursue following the Evaluation? 12. USAID’s expectations on mobilizing other funding sources? 13. What else you would like us to focus in this evaluation? Evaluation of Community HIV Link Programs 106 ANNEX 8: BENEFICIARY CLIENT SATISFACTION SURVEY Questions 1-4 are filled in by the interviewer: 1. Province____________________ 2. Gender: Male 1 Female 2 Trans 3 3. Population group FSW 1 MSM 2 IDUs 3 PSP 4 4. Sampling sub-group: Attended HTC 1 Attended OPC 2 Met recently and not use services by CBS 3 Begin the interview: 5. How old are you? ___ 6. How many other PWID/FSWs/MSM do you know (their name, their face) in this community___ 7. How many of these PWIDs/FSWs/MSM (altogether, not just from the CBO) did you meet in the past two weeks? ___ 8. How many of these PWIDs/FSWs/MSM (altogether, not just from the CBO) can you meet next week?__ Evaluation of Community HIV Link Programs 107 9. Any of the people you know in your community is member of the CBO? Yes 1 No 2  to Q11 I don’t know 3  to Q11 10. How many of them are member of CBO?___ 11. Name/nickname of CBS who recruited this respondent (for reference purpose—can use to refer to CBS in questions below) ___________ 12. How long have you known (CBS)? Just met in the past month 1 1-5 months 2 6-12 months 3 1-2 years 4 3 years + 5 13. Would you say that CBS is: (only one choice—read the options aloud) A friend 1 A close friend 2 An acquaintance 3 Someone you met through the CBO only 4 14. How did you meet CBS? (summarize in these categories) Through KP-related context (because we are both PWID, FSW, MSM) 1 Through CBO 2 Other reasons 3 Evaluation of Community HIV Link Programs 108 15. When did you first talk to CBS about HIV/AIDS? Just met in the past month 1 1-5 months 2 6-12 months 3 1-2 years 4 3 years + 5 16. Could you please tell me what topics you have talked to CBS about? Anything else? (Do not read the answers. Choose the answers that are closest to what the respondent says. Keep asking “Anything else?” until the respondent has nothing else to say.) You can prevent HIV 1 Condoms can prevent contracting HIV through sex 2 Not sharing needles can prevent contracting HIV through needles 3 You are at high risk of contracting HIV 4 The advantages of getting tested for HIV 5 I should get tested every 6 months 6 Where to get tested for HIV 7 There is treatment available for AIDS 8 Where to get treatment if I test positive for HIV 9 If I test positive for HIV I need to get treatment for the rest of my life 10 I should continue treatment once I start 11 17. I’m going to list some topics that you may have talked to CBS about. Please tell me whether you have discussed this topic with CBS (read each one – multiple choice): You can prevent HIV 1 Condoms can prevent contracting HIV through sex 2 Not sharing needles can prevent contracting HIV through needles 3 Evaluation of Community HIV Link Programs 109 You are at high risk of contracting HIV 4 The advantages of getting tested for HIV 5 I should get tested every 6 months 6 Where to get tested for HIV 7 There is treatment available for AIDS 8 Where to get treatment if I test positive for HIV 9 If I test positive for HIV I need to get treatment for the rest of my life 10 I should continue treatment once I start 11 18. How are you satisfied with the outreach that you received from the CBS? Very satisfied 1 Satisfied 2 It’s OK 3 Not really satisfied 4 Very unsatisfied 5 19. Since you began talking to the CBS about HIV, I would like to know whether you have changed any of your habits or behavior in using condoms when you doing sex. I will read some statements and please tell me which one is the most true (Read all of the responses and then have the respondent choose one) I always used condoms before with every partner and I still do now 1 I use condoms more frequently than I did before 2 I use condoms less frequently than I did before 3 No change in my condom use 4 Very unsatisfied 5 20. [skip pattern for FSW & MSM in Q3: PWID only] Since you began talking to the CBS about HIV, I would like to know whether you have changed any of your habits or behavior in using Evaluation of Community HIV Link Programs 110 Needles/injecting equipment when you inject drug. I will read some statements and please tell me which one is the most true I did not share needles when I injected drug before talking to the CBS and I still never share needles 1 I used to share needles and I don’t share them any more 2 I share needles less frequently than I used to do 3 No change in how I share needles 4 21. Have you been tested for HIV in the past 6 months? Yes 1 No 2  to Q25 I already took the test in the last 6 months and currently in treatment for HIV 3  to Q25 22. Who went with you the last time that you went for testing? (can check more than one) CLink CBS 1 CBO member 2 CBS of other program 3 Friend 4 Family member 5 No one 6 Other (please specify) 7 23. How satisfied are you with the services that you received at the HTC? Very satisfied 1 Satisfied 2 It’s OK 3 Evaluation of Community HIV Link Programs 111 Not really satisfied 4 Very unsatisfied 5 24. How satisfied are you with the attitude of staff at the HTC? Very satisfied 1 Satisfied 2 It’s OK 3 Not really satisfied 4 Very unsatisfied 5 25. Have you registered to the treatment at OPC? Yes 1 No 2  to Q30 26. Who went with you when you go to register at OPC? (can check more than one) CLink CBS 1 CBO member 2 CBS of other program 3 Friend 4 Family member 5 No one 6 Other (please specify) 7 27. How satisfied are you with the attitude of staff at the OPC? Very satisfied 1 Satisfied 2 It’s OK 3 Not really satisfied 4 Evaluation of Community HIV Link Programs 112 Very unsatisfied 5 28. Did you ever feel like the staff at the OPC treated you negatively because you are a (PWID, MSM, FSW)? Always 1 Sometimes 2 Not at all 3 29. How satisfied are you with the service at the OPC? Very satisfied 1 Satisfied 2 It’s OK 3 Not really satisfied 4 Very unsatisfied 5 30. Do you have active health insurance card? Yes 1 No 2 I don’t know 3 Evaluation of Community HIV Link Programs 113 ANNEX 9: CBS DATA COLLECTION FORM CONTENT ANSWER 1. What year did you start to work as an outreach worker or peer educator for HIV/AIDS programs? Year 2. What year did you start to work as a CBS in C￾Link program? Year 3. How many HIV/AID program are you participated as outreach worker/peer educator/CBS now? 4. How many KPs you reached in C-Link in total so far? MSM…….FSW……..IDU………Sex Partner….. 5. How many KPs you reached monthly in average in C-Link? MSM…….FSW……..IDU………Sex Partner….. 6. How many new founded HIV cases successful referral by you in C-Link program in total so far? MSM…….FSW……..IDU………Sex Partner….. 7. How many new founded HIV cases successful referral by you in C-Link program monthly? MSM…….FSW……..IDU………Sex Partner….. 8. How many Positive HIV cases successful referral to OPC by you in C-Link program in total so far? MSM…….FSW……..IDU………Sex Partner….. 9. How many new founded HIV cases successful referral to OPC you in C-Link program monthly? MSM…….FSW……..IDU………Sex Partner….. 10. Please list all channels you are using to reach KPs? 1. . 2. . 3. . 4. 11. What channels do you use most frequently? 12. What channel do you consider most effectively? 13. On average, how much incentive do you receive from C-Link program each month? 14. On average, how much do you spend for your life monthly? 15. What is the main source of your income to support your living? 16. How many peers did you connect with before working as CBS for all HIV programs in general (all channels)? 17. Before working as CBS for C-Link program, how many peers did you connect with (all channels)? 18. At the moment, how many peers do you connect with (all channels) 19. Before working as CBS for C-Link program, how many peers did you connect with? 20. At the moment, how many peers do you connect with? 21. Before working as CBS for C-Link program, how many staff at the HTC/OPC did you have a friendly connection/contact with? HTC OPC 22. At the moment, how many staff at the HTC/OPC do you have a friendly connection and contact? HTC OPC Evaluation of Community HIV Link Programs 114 ANNEX 10: DATA COLLECTION FORM, CBO/SELF-HELP GROUPS/CLUBS Name of Organization: CONTENT ANSWER 1. Year of establishment 2. How long has your CBO/Self-help Group/Club worked together (can be longer than the “year of establishment” for CBO) Year 3. How many paid staff does this CBO/Self-help Group/Club have? 4. How many members does this CBO/Self-help group/Club have? 5. How many core members does this CBO/Self-help group/Club have? 6. How many outreach worker/CBSs/peer educators do you have altogether, working for all of your programs? 7. How many CBSs in Clink program in your CBO/Self-help Group/Club? 8. How many KPs your CBO reached for C-Link in total MSM…….FSW……..IDU………Sex Partner……Total:….….. 9. How many KPs your CBO reached monthly in average in C-Link MSM…….FSW……..IDU………Sex Partner……Total:….….. 10. How many new HIV cases resulted from your referrals to HTC by your CBS in C-Link program in total? MSM…….FSW……..IDU………Sex Partner……Total:….….. 11. How many new HIV cases are referred to HTC by your CBS in C￾Link program monthly on average? MSM…….FSW……..IDU………Sex Partner……Total:….….. Evaluation of Community HIV Link Programs 115 CONTENT ANSWER 12. How many Positive HIV cases were successfully referral to OPC by your CBS in C-Link program in total? MSM…….FSW……..IDU………Sex Partner……Total:….….. 13. How many Positive HIV cases were successfully referral to OPC by your CBS in C-Link program monthly? MSM…….FSW……..IDU………Sex Partner……Total:….….. 14. How many funding sources does your CBO currently have? 15. Please listing all funding sources your CBO is receiving? 1. 2. 3. 4. 16. How many groups/clubs/CBOs/social enterprises did your organization collaborate before C-Link program started? 17. Among them, how many groups/groups/clubs/CBOs/social enterprises did your CBO have joint business activities? 18. How many groups/groups/clubs/CBOs/social enterprises is your CBO collaborating now? 19. Among them, How many groups/groups/clubs/CBOs/social enterprises does your CBO have joint business activities? 20. How many new groups or agencies does your CBO have newly connected since the start of C-Link Program? 21. Has your CBO ever contracted by the government agencies to provide any service? 1. Yes 2. No Evaluation of Community HIV Link Programs 116 CONTENT ANSWER 22. If yes, in what year were your CBO contracted for the first time? 23. What type of services was your CBO providing under that first contract? 24. Does your organization continue providing any service to a government agency? 1. Yes 2. No 25. What type of services is your CBO providing to that government agency? 26. Is your organization register as legal organization 27. If your organization registered as legal organization, what year did you register?