i HIV/AIDS Program Evaluation USAID/Central Asia Republics Final Report: May 28 2014 David Lowe, Independent Consultant & Team Leader Azizbek Boltaev, Independent Consultant Chris McDermott, Independent Consultant Britt Herstad, USAID/Washington Billy Pick, USAID/Washington ii Contents ACRONYMS....................................................................................................................................................................IV EXECUTIVE SUMMARY............................................................................................................................................VI 1. INTRODUCTION .............................................................................................................................................8 1.1 PURPOSE OF THE EVALUATION AND SCOPE OF WORK ...................................................................................................... 8 1.2 SUMMARY OF METHODOLOGY .................................................................................................................................. 8 2. THE CONTEXT .................................................................................................................................................9 2.1 THE HIV/AIDS EPIDEMIC IN THE CENTRAL ASIAN REPUBLICS AND NATIONAL RESPONSES ....................................................... 9 2.2 OVERVIEW OF THE USAID CENTRAL ASIAN REPUBLICS HIV/AIDS PROGRAM................................................................... 10 3. PREVENTION................................................................................................................................................. 12 3.1 OUTLINE OF PREVENTION PROGRAMING.................................................................................................................... 12 3.2 KEY ACHIEVEMENTS:............................................................................................................................................. 13 3.3 STRENGTHS ........................................................................................................................................................ 16 3.4 WEAKNESSES ...................................................................................................................................................... 17 3.5 CONSTRAINTS ..................................................................................................................................................... 20 3.6 SUSTAINABILITY ................................................................................................................................................... 21 4. CARE AND TREATMENT................................................................................................................................. 22 4.1 OUTLINE OF THE CARE AND TREATMENT PROGRAMMING .............................................................................................. 22 4.2 KEY ACHIEVEMENTS.............................................................................................................................................. 22 4.3 STRENGTHS ........................................................................................................................................................ 24 4.4 WEAKNESSES ...................................................................................................................................................... 25 4.5 CONSTRAINTS ..................................................................................................................................................... 26 4.6 SUSTAINABILITY ................................................................................................................................................... 26 5. HEALTH SYSTEM STRENGTHENING................................................................................................................ 26 5.1 OUTLINE OF HEALTH SYSTEM STRENGTHENING PROGRAMMING ...................................................................................... 26 5.2 KEY ACHIEVEMENTS.............................................................................................................................................. 26 5.3 STRENGTHS ........................................................................................................................................................ 29 5.4 WEAKNESSES ...................................................................................................................................................... 30 5.5 CONSTRAINTS ..................................................................................................................................................... 31 5.6 SUSTAINABILITY ................................................................................................................................................... 32 6. PROGRAM MANAGEMENT............................................................................................................................ 33 ANNEX 1: SCOPE OF WORK ............................................................................................................................. 36 ANNEX 2: BACKGROUND DOCUMENTS............................................................................................................ 40 ANNEX 3: INTERVIEW GUIDES ......................................................................................................................... 43 ANNEX 4: EVALUATION SCHEDULE .................................................................................................................. 48 ANNEX 5: ORGANIZATIONS CONSULTED AND SITE VISITS ................................................................................ 49 ANNEX 6: SITE MAPS....................................................................................................................................... 51 ANNEX 7: NGO SUB‐AWARDEES FOR THE DIALOGUE PROJECT......................................................................... 56 ANNEX 8: DIALOGUE PROJECT PERFORMANCE DATA....................................................................................... 58 ANNEX 9: QUALITY PROJECT PERFORMANCE DATA ......................................................................................... 64 ANNEX 10: RECOMMENDATIONS ..................................................................... ERROR! BOOKMARK NOT DEFINED. ANNEX 11. FUTURE PROGRAMMATIC DIRECTIONS...........................................ERROR! BOOKMARK NOT DEFINED. ANNEX 12: ENDNOTES ...................................................................................................................................... 68 iii Tables TABLE 1: EVALUATION SCHEDULE................................................................................................................................. 48 TABLE 2: ORGANIZATIONS CONSULTED AND SITE VISITS .............................................................................................. 49 TABLE 3: DIALOGUE PROJECT NGO SUB‐AWARDEES..................................................................................................... 56 TABLE 4: USAID QUALITY HEALTH CARE PROJECT’S AMEP INDICATORS YEARS 1‐5 ....................................................... 64 TABLE 5: METHOD OF MEASUREMENT FOR QUALITY PROJECT INDICATOR 5 ............................................................... 65 Figures FIGURE 1. NUMBER OF INDIVIDUALS REACHED WITH DIALOGUE SMALL GROUP OR INDIVIDUAL HIV PREVENTION INTERVENTIONS BY KEY POPULATIONS, PLWH AND PRISONERS, REGIONAL COVERAGE (CAR): TARGETS VS CUMULATIVE ACHIEVED YEARS 1‐4. SOURCE: MIS 14 FIGURE 2. PERCENT OF MSM REPORTING CONDOM USE AT LAST ANAL SEX WITH A MALE PARTNER FOR 3 CAR COUNTRIES BY YEAR OF SURVEY. SOURCE: TRAC AND MIS 15 FIGURE 3: TRENDS IN HIV TESTING OF DIALOGUE CLIENTS BY KEY POPULATIONS AND PRISONERS, REGIONAL TESTING RATES (CAR) BY YEAR FOR YEARS 1‐4. SOURCE MIS. 19 FIGURE 4: NUMBER OF PWID REFERRED BY DIALOGUE FOR DRUG TREATMENT IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 23 FIGURE 5: NUMBER OF INDIVIDUALS REACHED WITH DIALOGUE SMALL GROUP OR INDIVIDUAL HIV PREVENTION INTERVENTIONS BY KEY POPULATIONS, REGIONAL COVERAGE (CAR): CUMULATIVE YEARS 1 – 4. SOURCE: MIS 58 FIGURE 6: NUMBER OF INDIVIDUALS REACHED WITH DIALOGUE SMALL GROUP OR INDIVIDUAL HIV PREVENTION INTERVENTIONS BY YEAR AND BY KEY POPULATIONS, PLWH AND PRISONERS, REGIONAL COVERAGE (CAR): YEARS 1‐4. SOURCE: DIALOGUE ANNUAL REPORT, YEAR 4. 58 FIGURE 7: NUMBER OF PWID REFERRED BY DIALOGUE FOR HIV COUNSELLING AND TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 59 FIGURE 8: NUMBER OF FSW REFERRED BY DIALOGUE FOR HIV COUNSELLING AND TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 59 FIGURE 9: NUMBER OF MSM REFERRED BY DIALOGUE FOR HIV COUNSELLING AND TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 60 FIGURE 10: NUMBER OF PRISONERS REFERRED BY DIALOGUE FOR HIV COUNSELLING AND TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 60 FIGURE 11. NUMBER OF PWID REFERRED BY DIALOGUE FOR TB TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐4. SOURCE: MIS 61 FIGURE 12. NUMBER OF PLHIV REFERRED BY DIALOGUE FOR TB TESTING IN CAR: REFERRALS VS REDEEMED, YEARS 1‐ 4. SOURCE: MIS 61 FIGURE 13. NUMBER OF PRISONERS REFERRED BY DIALOGUE FOR TB TESTING IN CAR: REFERRALS VS REDEEMED YEARS 3 AND 4 ONLY (NO PROGRAM OR DATA YEARS 1‐2). SOURCE: MIS 62 FIGURE 14. PERCENT OF PWIDS WHO BOTH CORRECTLY IDENTIFY WAYS OF PREVENTING THE SEXUAL TRANSMISSION OF HIV AND WHO REJECT MAJOR MISCONCEPTIONS ABOUT HIV TRANSMISSION: TURKMENISTAN AND UZBEKISTAN. SOURCE: TRAC 62 FIGURE 15. PERCENT OF PLHIV IN CAR WHO BOTH CORRECTLY IDENTIFY WAYS OF PREVENTING THE SEXUAL TRANSMISSION OF HIV AND WHO REJECT MAJOR MISCONCEPTIONS ABOUT HIV. SOURCE: TRAC. 63 FIGURE 16. PERCENT OF PRISONERS WHO BOTH CORRECTLY IDENTIFY WAYS OF PREVENTING THE SEXUAL TRANSMISSION OF HIV AND WHO REJECT MAJOR MISCONCEPTIONS ABOUT HIV: KAZAKHSTAN, KYRGYZ REPUBLIC AND TAJIKISTAN. SOURCE: TRAC 63 iv Acronyms AAD Activity Approval Document AIDS Acquired Immune Deficiency Syndrome AOR Agreement Office’s Representative APMG AIDS Project Management Group ART Antiretroviral Therapy BCC Behavior Change Communication CA Cooperative Agency CAB Community Advisory Boards CAR Central Asian Republics CCM Country Coordinating Mechanism CDC GAP Centers for Disease Control and Prevention Global AIDS Program CoC Continuum of Care CoP Chief of Party CoPCT Continuum of Prevention to Care and Treatment COR Contracting Office’s Representative CPS Comprehensive Package of Services FMC Family Medical Centre FSN Foreign Service National FSW Female Sex Workers FY Financial Year GBV Gender Based Violence GF Global Fund to fight AIDS, Tuberculosis and Malaria GMS Grant Management Solutions HCT HIV counselling and testing HCW Health Care Workers HEO Health and Education Office HIV Human Immunodeficiency Virus HPP Health Policy Project HSS Health System Strengthening IEC Information, Education and Communication IPC Inter-Personal Communicator IR Intermediate Result KP Key Populations LCC Locality Coordination Council v LOP Life of Project MAT Medication Assisted Therapy MoH Ministry of Health MSM Men who have Sex with Men NGO Non-Government Organization OGAC Office of the Global AIDS Coordinator ORW Outreach Worker PEPFAR President’s Emergency Plan for AIDS Relief PHC Primary Health Care PLHIV People Living with HIV PMP Performance Monitoring Plan PSI Population Services International PWID People who inject drugs RAC Republican AIDS Center S&D Stigma and Discrimination STI Sexually Transmitted Infection TA Technical Assistance TB Tuberculosis TRaC Tracking Results Continuously TWG Technical Working Group UIC Unique Identifier Code UNAIDS Joint United Nations Program on HIV/AIDS USAID United States Agency for International Development USG United States Government WHO World Health Organization vi Executive summary This study was commissioned by the USAID/Central Asia Republics (CAR) Mission to evaluate the performance of its five year regional HIV/AIDS program in Kazakhstan, Kyrgyzstan, Tajikistan and Uzbekistan, and to inform future USAID HIV programming. The two major program components have been the USAID Dialogue on HIV and TB Project and the USAID Quality Health Care Project. The primary focus of Dialogue has been implementation of evidence-based HIV prevention interventions targeting the key populations (KPs) of people who inject drugs (PWID), female sex workers (FSWs) and men who have sex with men (MSM), with the aim of reducing HIV risk behaviors. Dialogue has also focused on creating demand among KPs for HIV counseling and testing (HCT) as an entry point to HIV care and treatment services and improving tuberculosis (TB) case detection and treatment adherence among KPs. The Quality Project works primarily on the supply side of health care for HIV and TB, enhancing the health sector’s capacity to plan, deliver, manage, and monitor enhanced programs and services. Quality also supports country partners in strengthening collection, analysis, and use of data for public health program planning and policy development. Coverage of HIV prevention programs reaching PWID, FSW and MSM has been impressive. In the four years since project commencement, the program has reached 105,135 individual KPs with evidence-based prevention interventions. The program has continued to extend its coverage each year by reaching large numbers of KP members not previously reached. Prevention interventions have reached more PWID than other KPs, which is appropriate given the epidemic is primarily driven by this group. Survey data collected by the project indicates an improvement in HIV-related knowledge and behaviors by KPs, indicating that the program is having an impact on behavior. For example, condom use among MSM clients of the program in Kazakhstan, Kyrgyzstan and Tajikistan increased respectively from 52%, 42% and 33% in 2009 to 60%, 71% and 61% in 2011. In years 1-4, a total of 19,721 KP program clients were referred to HIV testing and received their results. While this represents less than 20% of the total program clients, despite the lack of baseline data, it is highly likely that this represents a significant increase in HIV testing rates since program commencement and indicates some success (with room for improvement) in overcoming the substantial barriers commonly encountered in HIV programming in increasing HIV testing rates. To December 2013, 12,277 program clients were tested for TB, with 604 cases detected. The TB incidence rate among program clients was five times higher than the regional incidence rate, validating the program’s focus on those most in need of services. A total of 2,321 PWID clients received drug rehabilitation services to December 2013, accounting for more than 50% of those referred, representing a high uptake rate. The program has worked successfully with health services in target sites to establish Multi￾Disciplinary Teams (MDTs) that provide integrated HIV, TB and drug rehabilitation treatment and case management, including treatment adherence support. Program clients are referred to MDTs for free care by use of a program voucher system which ensures anonymity but allows for individual patient tracking and case management. MDTs have been supported by extensive training to provide comprehensive HIV-related service to KPs. Considerable progress has been made by the program in supporting closer collaboration between NGOs and government health facilities, reflecting the program focus on creation of demand for health services by KPs and improving the capacity of health services to supply quality HIV-related services. The health sector has been supportive of the MDT model, but it has not yet been formally adopted, perhaps because the program has not documented its standard operating procedures. In Kazakhstan, however, the Republican AIDS Center plans to adopt and replicate the MDT model across the country. There has been considerable support for one-off trainings of HCWs, with insufficient follow-up support through workplace mentorship. The quality of training appears to be variable, with room for vii improvement. A greater emphasis needs to be placed on adoption of curricula by educational institutions, including in-service training, although there has been some progress in this respect. The program has provided technical assistance to Ministries of Health in the development of evidence-based comprehensive packages of HIV-related services for each of the KPs. Plans for adoption of the packages by CAR governments are well advanced. Further work is needed to mobilize government funding for implementation. The program has provided technical support in establishing Community Advisory Boards and Locality Coordination Councils across CAR. These bodies consist of health sector and KP representatives and aim to improve the quality and coordination of services. There are numerous examples across the region of positive changes in service provision flowing from the work of these bodies. Another benefit has been the empowerment of KPs in advocacy for change. The USAID program, while focused on HIV prevention behavior change, has taken a broader focus in recognition that issues such as homelessness, drug addiction, gender-based violence and legal problems can contribute to increased vulnerability for HIV acquisition. Given the complexity of these broader social issues and limited referral and support options, the work of implementing partners in this area has been challenged and constrained. Collaboration with Global Fund supported HIV programming has generally been good, with leveraging of GF supported commodities and services to complement USAID funded services. USAID and CDC have identified some areas where they can rationalize their work and have committed to improved coordination to address overlap and duplication. Constraints in the enabling environment have negatively impacted coverage of prevention programs, particularly police harassment of PWID, FSW and MSM. Stigma and discrimination continues to be a significant barrier to accessing health services, although program activities have resulted in a significant improvement in HCW attitudes in some sites, with a need for continuing work. USAID programming has combined targeted technical assistance for NGOs and the health sector and support for direct service delivery by NGOs. The HIV response in CAR remains heavily donor dependent, especially in relation to NGO funding. There is, however, as a result of the USAID program, an increasing recognition by the health sector of the comparative advantages of NGOs in delivering primary HIV prevention services to KPs. With USAID support, 16 HIV-focused NGOs in Kazakhstan have now received government social funding. This is an important first step in building government commitment to greater NGO participation, which can be used regionally. USAID’s investments have contributed to establishing an engaged, informed and confident NGO sector and improved health sector service delivery which provides a good foundation for the sustainability of HIV programs. There is a need to strengthen the organizational capacity of NGOs to improve their prospects for sustainability. Weak points in health sector sustainability are the need for increased government expenditure and the turnover of HCWs, resulting in the need for more capacity development. There is also a need for refresher training for both NGOs and the health sector to improve quality. The program has also been supported by Field Support mechanisms which have included NGO capacity assessments, HIV-related gender assessments, a HIV-related policy assessment, and assessments of Global Fund Country Coordinating Mechanisms. All of these assessments were of good quality and have contributed to building a regional evidence base for program improvement. As all the assessments were conducted at a relatively advanced stage of implementation of USAID’s regional HIV/AIDS program, opportunities to incorporate findings within existing programming have been limited. A notable exception has been the incorporation of gender-based violence prevention work into programming in Years 4 and 5. There are opportunities to make good use of the assessments in the design and implementation of the follow-on HIV/AIDS program. 8 1. Introduction 1.1 Purpose of the evaluation and scope of work In line with USAID/Washington’s Program and Policy learning agenda, USAID/Central Asian Republics (CAR) and USAID/Kyrgyz Republic1 Health and Education Offices (HEO) commissioned this study to evaluate the performance of their regional HIV/AIDS investments and to make recommendations for investments over the next five years. These recommendations are based on epidemiological priorities, previous assessments by HIV field support activities, and evidence based practices and interventions to address the HIV epidemics in Central Asia. USAID/CAR is currently developing its next five year regional strategy and this evaluation will inform investments in HIV during the upcoming period. This evaluation examines programs funded by the President’s Emergency Plan for AIDS Relief (PEPFAR), through USAID/CAR.2 The scope of work specified five overarching questions: 1. To what extent were expected results achieved, and why or why not? 2. To what extent has the program addressed the most relevant HIV/AIDS issues in each country? 3. To what extent has the program enhanced local partner capacity in HIV/AIDS? 4. What key lessons have we learned from the Mission’s current HIV/AIDS investments and activities? 5. Given the answers to questions 1-4 above, how should the Mission invest resources in HIV/AIDS over the next 5 years? A series of more detailed questions relating to each of these overarching questions are set out in the Scope of Work (Annex 1.) This evaluation’s findings and strategic analysis recommend how USAID can best support an evolving regional response to HIV and shape the design of a new USAID/CAR HIV/AIDS Flagship program for five years from April 2015 - March 2020. 1.2 Summary of methodology This performance evaluation took place over a five-week period in March-April 2014, covering the first four and a half years of portfolio performance, with an emphasis on the last two years. The five member evaluation team followed a methodology consistent with USAID’s January 2011 Evaluation Policy, focusing on descriptive and normative questions, including: what the program had achieved; how it was being implemented; how it was perceived and valued; whether expected results were occurring; and other questions pertinent to program design, management, and operational decision making. The evaluation team began with a review of key background documents (see Annex 2) and key performance indicator and surveillance data. Interview guides, (based on the evaluation questions,) were developed by the evaluation team for different categories of stakeholders to ensure a consistent approach (see Annex 3). This was particularly important as two sub teams were formed and conducted separate site visits, with one visiting Kazakhstan and the Kyrgyz Republic and the other going to Uzbekistan and Tajikistan. The teams conducted a series of key informant interviews in multiple sites in each of these four countries. Site selection was based on representativeness (urban/ rural, government/non-government, etc.) and feasibility within the limited time available. (See Annexes 4 and 5 for the evaluation schedule and a list of organizations consulted and site visits.) Ongoing analysis of data by individual team members fed into regular group analysis, which allowed for emerging issues to be identified and explored as the evaluation progressed. A series of comprehensive analysis sessions were conducted by the whole team, following the conclusion of the key informant interviews, to develop preliminary key findings. These were presented to USAID/CAR in Almaty (with participation via teleconference from USAID personnel stationed in the other countries) for the purpose of receiving feedback, validation and further input. This 9 feedback was then incorporated into the draft evaluation report. Preliminary findings were also presented to the Dialogue and Quality Projects for the same purpose. 1.2.1 Limitations Several factors limited the evaluation’s findings, conclusions and recommendations. For the most part, these stem from the high level, strategic nature of the evaluation, lack of baseline information for some of the key indicators, and the limited time in which to conduct the evaluation. In practical terms, this meant that the evaluation team took a regional focus rather than conducting a detailed country by country evaluation. Nonetheless, the evaluation was significantly informed by interviews and site visits in each of the countries visited. Other constraints were the limited availability of data to assess the quality of services, the selective nature of site visits, and language barriers. These types of limitations are commonly encountered in evaluations of this type. Participation in interviews by USAID/CAR and USAID/Kyrgyz Republic staff with responsibility for program management may have biased responses by limiting criticisms of the program by implementing partners and their sub-grantees. Nonetheless, the USAID country staff made valuable contributions to the evaluation by sharing useful background information and insights. The evaluation has primarily focused on the Dialogue and Quality Projects as these are the two major investments for USAID. Accordingly, the evaluation placed a lesser emphasis on the field support mechanisms used by the program. 1.2.2 Evaluation team The five-person evaluation team was made up of three independent consultants and two USAID/Washington experts from the Office of Global Health, Office of HIV/AIDS. Collectively, the team had considerable expertise across all relevant aspects of HIV technical knowledge and programming, from prevention to care and treatment, and health systems strengthening. Most team members had extensive knowledge of the CAR’s response to HIV/AIDS from previous work. 2. The context 2.1 The HIV/AIDS epidemic in the Central Asian Republics and national responses After the collapse of the Soviet Union, all five CAR countries retained a vertical system for delivering health services. At this time, the health systems in all CAR countries underwent significant cuts in health expenditures, resulting in a deteriorating health infrastructure. As a result, the population experienced decreasing life expectancy, increasing infant mortality rates, and increasing rates of communicable diseases, including high rates of tuberculosis (TB), sexually transmitted infections (STI), and HIV infections. The HIV epidemic in Central Asia is a low level, concentrated epidemic, largely driven by people who inject drugs (PWID) located in urban centers and along drug transportation corridors from Afghanistan through Tajikistan, Turkmenistan, Uzbekistan, the Kyrgyz Republic and Kazakhstan. Estimated HIV prevalence among adults aged 15-49 in 2010 ranged from 0.17 per cent in Kazakhstan to 0.06 per cent, 0.05 per cent and 0.03 per cent respectively in Uzbekistan, the Kyrgyz Republic and Tajikistan.3 Central Asia is, however, one of only two regions in the world where the number of people newly infected with HIV is increasing. HIV prevalence among PWID is high and growing throughout the region, ranging from 3.8 -18 per cent in Kazakhstan, 16.3 per cent in Tajikistan, 14.6 per cent in the Kyrgyz Republic and 11 per cent in Uzbekistan.4 Other key populations (KP) affected by HIV in CAR are female sex workers (FSW) and men who have sex with men (MSM). Though recent surveillance data indicate a potential decline in HIV incidence in Kazakhstan and the Kyrgyz Republic, further analysis suggests that in reality, the trend might 10 simply reflect a disproportionate trend of increased testing among low-risk groups and possibly decreased testing among KPs. Recent evidence suggests that that the incidence of sexual transmission is on the rise. In 2011, the proportion of registered HIV cases in Kazakhstan acquired through sexual transmission (50.7 per cent) exceeded, for the first time, the proportion of cases acquired through sharing of injection drug use equipment (43.7 per cent).5 It is, however, thought that the increase in sexual transmission is primarily among the sexual partners of PWID and that the risk of a generalized epidemic remains low. The risk of acquiring HIV among KPs is increased by unsafe sexual practices: less than 50 per cent of PWID in Tajikistan report using a condom with FSWs, while in the Kyrgyz Republic, the syphilis prevalence in 2013 was 21 per cent among FSWs, 14 per cent among prisoners and 8 per cent among MSM.6 HIV in CAR is also particularly high among prisoners (many of whom are also PWID). Substantial evidence indicates that a combination of core interventions in the structural, biological and behavioral domains, with high programmatic coverage, linked with a supportive social and political environment, can decrease HIV risk and vulnerability among KPs.7 In Central Asia the proportion of persons who access and receive these core interventions (the coverage rate), particularly among PWID, is generally very low. Political will, the policy environment, financial resources (including donor dependency) and human capacity remain major constraints for HIV investments. Limited retention in care after HIV is diagnosed and prior to commencing antiretroviral therapy (ART) and limited adherence support for ART results in high rates of lost to follow up among people living with HIV/AIDS (PLHIV). These remain key gaps to be addressed to ensure the delivery of effective services and reduction of high PLHIV mortality rates. Linking sufficient numbers of KPs with diagnostic and treatment services is challenging due to high rates of HIV￾related stigma and discrimination by health care workers (HCWs). The availability of medication assisted treatment (MAT, i.e., opioid substitution therapy), an evidence-based best practice HIV prevention method among PWID, is limited. In some countries, policy and political constraints exist in implementing or scaling up MAT. Access to rapid HIV tests for KPs is also limited by regulations preventing non-medical organizations, including NGOs, from providing HIV screening. This limits access to HIV testing which in turn limits treatment uptake. In addition, gender based violence (GBV) remains a major issue in the region – it is both widespread and especially prevalent and accepted among KPs, who can be both victims and perpetrators. While the HIV epidemic in CAR is concentrated primarily among male PWID, infections among women, primarily due to sexual transmission, may be on the rise. For example, in Kazakhstan, the proportion of newly diagnosed cases of HIV among women increased from 26 per cent in 2006 to 37 per cent in 2010. This, however, may be due to increased HIV testing of women, following the introduction of universal testing in pregnancy, although inadequacies in the methods of data collection do not allow this to be confirmed.8 2.2 Overview of the USAID Central Asian Republics HIV/AIDS Program Current USAID/CAR support for the regional response to HIV represents an evolution from HIV programs which USAID began nearly twenty years ago. Annual funding from PEPFAR is allocated primarily to two five year projects, the Dialogue for HIV and TB Project (PSI) and the Quality Health Care Project (Abt Associates). Dialogue ends on March 31, 2015 and Quality ends on September 30, 2015. Maps showing the sites for implementation of the USAID HIV/AIDS program and the CDC program are in Annex 6. Adjunct technical support to the USAID program is provided largely through field support mechanisms, including AIDSTAR One9 and Two10, Grant Management Solutions11 (GMS) and the 11 Health Policy Project12 (HPP). Specialized field support partners have been used for focused research to inform programming and learning. An outline of implementation under field support mechanisms is provided in Section 5: Health System Strengthening. 2.2.1 The USAID Dialogue on HIV and TB Project13 Dialogue is a five-year project aimed at reducing the spread of the HIV and TB epidemics in Central Asia through improving health behaviors among KPs. Implemented by a consortium of partner organizations, Dialogue is led by Population Services International (PSI) and includes Project HOPE, the AIDS Foundation East-West, and the Kazakh Association for People Living with HIV. Dialogue works with 38 sub-awardee non-government organizations (NGOs). (See Annex 7.) The program purpose level goals include: 1. Reduction in risk behaviors associated with HIV transmission. 2. Increased use of evidence-based HIV prevention and TB treatment services by KPs. 3. Improved TB case detection among selected KPs. 4. Improved adherence to and decreased default rate from TB treatment among KPs. Implemented in Kazakhstan, the Kyrgyz Republic, Tajikistan, Uzbekistan and Turkmenistan, the program targets PLHIV and the following populations most at risk of contracting HIV and TB: · People who inject drugs · Female sex workers · Men who have sex with men · Prisoners (Years 1-3 of the program only, discontinued in year 4, except in Tajikistan14) · Migrants (Years 1-3 of the program only, discontinued in year 4) With its emphasis on prevention, Dialogue uses outreach prevention models, proven effective under previous regional projects and deemed best practices. Each model is tailored to specific program needs for a specific target population and then expanded to additional sites across the region over the course of the program. All models include a basic outreach package of services, including: prevention information-education activities based on peer education; a referral system to medical treatment and social services; and case management of program clients, including treatment adherence support. To support regional coordination and long-term institutionalization of program activities, and ensure activities are in line with National HIV/AIDS and TB strategies, Dialogue’s Regional Oversight Committee and Country Working Groups provide input for program implementation. These groups review program progress, provide expert advice, share results and lessons learned across the region, and work to strengthen the region’s strategic response to HIV and TB. A mid-term evaluation of the Dialogue Project conducted by AIDSTAR-Two in November of 2012 (after Year 3 of the project) determined the program was reaching most of its objectives.15 Notably, the HIV outreach and care targets were being met with respect to quality and coverage, while there was less success in meeting the TB targets for treatment adherence and default, possibly due to the vertical nature of the health system with HIV, MAT and TB services being provided through separate services. Following the mid-term 2012 evaluation, USAID and PSI agreed that for Years 4 and 5, the project would focus on PWID, FSW, MSM and PLHIV, and to remove prisoners and migrants as key target populations.16 More information on the Dialogue Project’s results can be found in Annex 8. 12 2.2.2 The Quality Health Care Project17 Implemented by a consortium of partner organizations, Quality is led by Abt Associates, and the lead implementing agency for its HIV component is the AIDS Project Management Group (APMG). Quality Project experts work with partners in all five CAR countries to introduce evidence-based international standards, build the capacity of public sector and non-government service providers, and institutionalize interventions. The Quality Project works primarily on the supply side of health care for HIV and TB, enhancing the health sector’s capacity to plan, deliver, manage, and monitor enhanced programs and services. Project specialists also support country partners in strengthening collection, analysis, and use of data for public health program planning and policy development. Quality works with 12sub-awardee non-government organizations (NGOs). (See Annex 7.) The intermediate results for the Quality Project are:  IR1: Increased access by vulnerable groups to enhanced services o IR1.1: Improved quality of health services and systems for key populations o IR1.2: Improved policies facilitate access to services  IR2: Improved capacity to plan, manage and monitor enhanced programs and services  IR3: Strengthen collection, analysis and use of data for use in program and policy planning, implementation and monitoring. o IR3.1: Health information and surveillance systems improved o IR3.2: Strengthened use of data for decision making. The purpose of the Quality Project is to provide technical assistance, training, equipment and commodities to assist the CARs to improve the quality, scope and coordination of health services by incorporating modern quality improvement techniques and evidence-based international standards into ongoing reforms of health systems. USAID investments in the program have assisted CARs to improve their management, financing, and implementation of medical services provided for TB, HIV/AIDS, family planning, maternal and child health, and primary health care. In addition, Quality also provides training and technical support for introduction of quality assurance methods in laboratories and public health clinics. This evaluation was focused only on the HIV component of the Quality Project’s work. Quality activities in each country are implemented simultaneously at two levels: policy assistance at the national level; and capacity building of national institutions for supervision and planning. An important goal of the Project is to assist national governments to scale up interventions successfully piloted by previous USAID projects. More information on Quality Project’s results can be found in Annex 8. 3. Prevention 3.1 Outline of prevention programing The USAID prevention program falls under the USAID/CAR Regional Development Cooperation Strategy FY2014-2018 Development Objective 3 – More Effective and Inclusive Governance Institutions that Serve the Public Good, IR3.3 Increased Use of Vital Health and Education Services. 18 USAID’s PEPFAR funds support HIV prevention programming tailored to the needs of the low-level, concentrated epidemic that disproportionately affects KPs in the region. Population targets are aligned with the three KPs as defined by the PEPFAR Country Operational Plan Guidance: PWID, FSW, and MSM.19 In addition, USAID has also supported prevention efforts for prisoners, on the basis that the majority of this vulnerable population consists of current or former PWID. 13 The core six components of USAID’s HIV prevention program are: 1. Information, education and communication (IEC) 2. Behavior change communication (BCC) through peer educators and IEC materials 3. Commodity distribution and access (condoms, needles and syringes) 4. Harm reduction for PWID 5. Referrals and social escorting of KPs to HIV Counseling and Testing (HCT) and Sexually Transmitted Infections (STI) diagnosis and treatment 6. Referrals and support to access HIV care and treatment for those who test positive for HIV. There is a strong emphasis on outreach and drop-in centers. In addition, USAID and partners have integrated gender-based violence programming into the overall efforts aimed at FSW, MSM and female PWID. USAID supports training efforts aimed at increasing KPs’ knowledge of HIV prevention, including harm reduction, and building capacity to effectively work with KPs at both NGO level (including outreach workers and case managers), and with HCWs at polyclinics, AIDS Centers, Narcology clinics and TB centers. In order to further facilitate an enabling environment, USAID programs have also worked to educate police and community members on HIV prevention for KPs in order to reduce harassment of NGO staff and clients and ensure optimal service delivery. Direct outreach activities were implemented by Dialogue consortium partners and 38 local NGOs. Furthermore, it is acknowledged by informants that these activities help to fill gaps in HIV prevention coverage due to budget cuts to Global Fund (GF) funded projects in countries, notably Tajikistan. Prevention activities are conducted in Dushanbe, Vahdat and Khujand in Tajikistan; Jalal-Abad, Osh/Karasuu, Bishkek and Karakol in the Kyrgyz Republic; and Ust-Kamenogorsk, Temirtau and Almaty in Kazakhstan; all of which are hot spots for KPs. In FY 2013, USAID allocated $4,443,656 million in PEPFAR funding to HIV prevention (through three budget codes – IDUP, HVOP and HVCT). This represented 64 per cent of USAID’s total budget for the region and 31.7 per cent of overall PEPFAR funding.20 While the GF provides the majority of funding for HIV prevention in Central Asia, USAID prevention programming is providing significant coverage as well as improving the quality of prevention interventions. The USAID HIV program provides both additional coverage beyond that of GF-funded programming and overlapping programming with additional services, particularly outreach. 3.2 Key achievements: The USAID Program has demonstrated impact on the quality, coverage and efficiency of HIV services in the region through: Increased coverage of and access to prevention interventions for KPs in target areas: Coverage of HIV prevention programs reaching MSM, FSW and PWID has been impressive. In four and a half years, Dialogue has reached 105,135 KPs with prevention interventions and 19,721 KPs were tested for HIV and know their results.21 While it is impossible to reliably estimate the percentage of KPs reached by USAID prevention programs (because size estimations are generally regarded by key informants as unreliable or at least contestable), it is clear that significantly more KPs are accessing services. Dialogue has extended its coverage each year by reaching large numbers of KPs not previously reached, using its unique identifier code (UIC) to avoid double counting. This is a significant indicator which shows the project is not solely focused on repeat contacts with KPs already reached (important as that is), but also actively makes contact with new people. Figure 1 (below) shows the number of individuals for each KP Dialogue reached with prevention interventions (vs targets) for Years 1-4 across all countries. While targets were generally not met, the number of KPs reached was very high, 14 especially considering that the KPs are largely ‘hidden’ populations because of stigma. Additional graphs on Dialogue’s prevention coverage are in Annex 8 (Figures 5 – 6). Figure 1. Number of individuals reached with Dialogue small group or individual HIV prevention interventions by key populations, PLWH and prisoners, regional coverage (CAR): Targets vs cumulative achieved Years 1-4. Source: MIS Behavioral data from TRaC surveys show increased positive behavior change: USAID’s Dialogue Project monitors behavior change among KP by periodically conducting TRaC surveys. TRaC is a multi-round survey PSI uses to gather information from a representative sample of the project’s target populations. The data collected from TRaC surveys deepens understanding of the characteristics of the KP groups such as socio-demographics, health behaviors, knowledge and access to services.22 Monitoring studies were conducted in the first year of the project (2009/2010) to measure baseline indicators for all KPs, and have been conducted periodically during the project to measure progress against those indicators. Given that there are several KP groups and multiple countries, Dialogue has staggered the surveys, some of which will be conducted this year to measure change over time. The surveys may provide a baseline for the follow-on program. An example of behavior change captured by TRaC is the percentage of MSM reporting use of a condom the last time they had anal sex. At baseline, conducted in 2010, 52 per cent, 42 per cent and 33 per cent of MSM in Kazakhstan, the Kyrgyz Republic and Tajikistan, respectively, reported that they had used a condom at last anal sex. A repeated TRaC survey conducted after two years of program interventions demonstrated an increased reported condom usage in 60 per cent, 71 per cent and 61 per cent of MSM in the respective countries (see Figure 2, below).23 Data on trends in HIV risk behavior for PWID and FSWs will become available later in 2014, following completion of TRaC surveys among these populations. Figures 14-16 in Annex 8 show trends in HIV knowledge for PWID, PLHIV and prisoners. Comprehensive package of prevention services (CPS) for KPs being implemented by providers and adopted by national governments: The Program has made a significant contribution to Ministry of Health (MoH) national HIV and health strategies in prevention through the adoption of evidence-based Comprehensive Packages of Services for each of the KPs. CPS were developed with the assistance of Quality and will be adopted in three of the target countries. In Kazakhstan, the Kyrgyz Republic and Tajikistan, final versions of the CPS for different KPs are awaiting endorsement by MoHs after which they will be incorporated in national HIV plans and strategies. Annex 9 provides country specific details of CPS progress to date. 15 Figure 2. Percent of MSM reporting condom use at last anal sex with a male partner for 3 CAR countries by Year of Survey. Source: TRaC and MIS The implementation of the CPS for PWID, MSM and FSW by USAID funded NGOs bolstered the MOHs acceptance of the essential role that NGOs play in the national HIV prevention response and laid the groundwork for adoption of the CPS. Capacity building of local NGOs and HCWs to address KP HIV prevention issues: Training, mentoring and supervision have enabled local NGOs to plan, deliver, monitor and report on robust prevention programs. As a result of capacity-building efforts, NGOs have increased demand for and access to services, reflected in the utilization of primary care services by FSW and PWID; increased coverage of female PWID through outreach; and advocacy conducted for new services. (e.g., HCWs from polyclinics providing mobile services at NGOs – Dushanbe, Vakhdat and Khujand, Tajikistan; and medication assisted therapy-tuberculosis (MAT-TB) services at the same site – Republican Narcology Center, Dushanbe, Tajikistan). USAID employed local experts to carry out a series of phased training sessions on ways to reach KP for some 4,440 HCWs. The sessions focused on: 1) basic HIV knowledge; 2) interpersonal communication (primarily HIV pre- and post-test counseling); and 3) harm reduction. Combined trainings of HCWs and NGO staff have increased ‘socialization’ of HCWs with KPs, resulting in better outcomes reported by both NGOs and clients. For example, most trained HCWs reported using their new knowledge in providing services to KPs. Development, acceptance and use of the UIC referral voucher system: One of the major achievements has been the acceptance by health services of the Dialogue referral vouchers provided by outreach workers (ORW) to clients, using the client UIC instead of their name. The UIC on the referral voucher was designed to ensure client anonymity so that KPs are more comfortable with presenting for services at government clinics. Over time, vouchers have become accepted by HCWs because they allow the ORW to track clients, and the ORWs have been effective in counseling their clients on how to interact with HCWs. Copies of vouchers are entered into the Dialogue Management Information System (MIS) database to track utilization and these data are analyzed to determine where adjustments are needed. Government acceptance of the voucher system has increased in the last two years of the project. In Uzbekistan, Dialogue successfully advocated for revising the MOH prikaz #232 to include a provision stating that HCWs nation-wide should accept the USAID-developed referral vouchers. While implementation is limited to Tashkent, more than 80 per cent of the vouchers are redeemed, demonstrating client and clinic acceptance of the voucher system. 0% 10% 20% 30% 40% 50% 60% 70% 80% 2010 2012 KZ KG TJ MSM Use of Condom at Last Sex with a Male Partner 16 Integration of gender-based violence into existing HIV prevention programs: In 2013, AIDSTAR One/Two conducted gender assessments in Kazakhstan, the Kyrgyz Republic and Tajikistan. The assessments found that while the epidemic is concentrated in male PWID, infections among women, due to sexual transmission from PWID, are on the rise.24 Through the Gender Challenge Fund, Dialogue was able to expand the HIV prevention models in Tajikistan through a new component for prevention of gender-based violence (GBV) among sex workers, MSM, and women who inject drugs. KPs learned about GBV prevention through outreach mini-sessions and trainings. Outreach workers were trained to educate, refer (and escort if needed) and motivate clients who were victims of GBV to seek support (VCT, STI testing, psychological support, legal counseling, referrals to crisis or drop-in centers). Trainings on GBV awareness and prevention were also conducted for law enforcement officials. Dialogue expanded the pilot beyond Tajikistan to the Kyrgyz Republic and Kazakhstan in Year 4 and reached 1,602 female KPs and MSM with interventions on gender-based violence prevention.25 Another example of gender integration is Dialogue’s support in Year 4 for two women drop-in centers for female PWID and FSWs in Tajikistan, which aim to address gender specific barriers to women’s health seeking behavior and lead to increased coverage. 3.3 Strengths Evidence-based models are being used to implement CPS for each KP: USAID prevention efforts support delivery of specific, evidence-based models developed for each KP. These include the Adara model for FSW, LaSky-Trusting Each Other model for MSM, and the Break the Cycle prevention model for PWID. The LaSky model focuses on sexual prevention of HIV and creating an environment where the social norm is to adopt safer sexual behaviors. NGO staff are trained to provide the relevant prevention services to clients. The models also support activities to reduce stigma and discrimination, interpersonal communication skills for working with KPs, and the use of the client UIC and voucher referral system. The models rely upon peer outreach workers who continually engage clients through promotion of testing, treatment and adherence to treatment. Front line ORW and HCW believe that the peer outreach models introduced by Dialogue has resulted in more clients contacted and retained due to trust and personal relationships. USAID supported programs focus on delivery of quality behavior change interventions. Many stakeholders positively compared this approach with non-USAID funded programs, which emphasize the number of people contacted, often to the detriment of the quality of service. NGO implementation has been supported by on-going training, mentoring, and supervision with flexibility and focused problem solving: Using a tailored technical assistance (TA) approach, USAID supports the professional development of NGO staff. Dialogue staff conduct monitoring visits with NGO workers and review reports on a monthly basis to provide continuous measurement of performance. Dialogue staff also provide feedback and seek clarification when performance problems arise. Refresher training is based on needs identified from monitoring visits. For example, monitoring visits identified a problem in communicating with KP. The training made clear to the ORWs that short and concise information related to key messages would be more effective than going into too much detail with clients. NGO outreach coordinators regularly monitor ORWs and at the end of each year, conduct a formal quality assessment of all ORW’s performance. NGO staff also participate in development and implementation of TRaC surveys and receive the results. Dialogue technical advisors guide NGO staff on use of survey results for program improvement. The project’s flexibility is another important strength. In some cases, local regulations have led to differences in implementation. In Uzbekistan, Project HOPE directly implements the models 17 because of legal constraints against foreign donors providing direct grants to NGOs. While this does not negatively affect service delivery, it does prevent the project from undertaking capacity building of NGOs, thereby negatively affecting sustainability. The models used by Dialogue have also led to NGOs and KPs feeling increased openness to engage with other organizations. An MSM NGO in Tajikistan, Legal Support, for instance, stated that the project now operates more freely in public. NGOs from other regions are contacting Legal Support for assistance with HIV prevention for MSM. In an example of diffusion of USAID-supported activities, Legal Support is developing booklets on MSM health and is providing assistance in training non-USAID funded NGO ORWs. Focus of prevention programming is behavior change, but recognizes multiple risk factors: USAID partners have recognized and responded to the additional needs of KPs which put them at risk for acquiring HIV. Many FSW find it easier to work ‘under the influence’ and drug and alcohol use is encouraged by clients. For FSW who use drugs, the Adara model focuses on building understanding of drug and alcohol addiction and related risk behavior. Activities include delivery of messages to avoid using drugs during sex work to reduce risky behaviors that lead to contracting HIV, not sharing injecting equipment and referrals to drug treatment services, including MAT (if available). GBV activities promote understanding of gender norms that have direct links to HIV, including decision-making on condom use, violence and confrontation of violence, and seeking health care services. Drop in Centers provide clients with a location where they can also deal with other issues such as drug addiction, homelessness and legal problems. Clients appreciated the drop-in centers as a place to access not only HIV prevention services, but also for other facilities such as kitchens, showers and laundry, and to socialize. 3.4 Weaknesses While collaboration between USAID partners has improved, more needs to be done with CDC, other donors, GF and governments: Initially, there was some confusion over which activities the Dialogue and Quality Projects would conduct and how they would collaborate to create a coherent and synergistic response to HIV prevention issues. After several meetings these issues were resolved and clear, non-duplicative activity plans were developed, delineating what each partner would do. From Year 2, there has been improved communication and a clearer division of labor between Dialogue and Quality. The USAID HIV prevention program and CDC/ICAP have committed to improved coordination to address overlap and duplication. The main overlap between USAID and CDC is in support provided to Multi-Disciplinary Teams (MDTs) in some sites. Following discussions between USAID and CDC it was decided that CDC would focus on TA to improve laboratory services and clinical care by MDTs and that USAID would focus on prevention programming, demand creation among KPs for health services, referral to MDTs and social support for KPs. There is an ongoing need for USAID and CDC and their partners to be in regular communication on joint support for MDTs in order to review their on-going effectiveness, address challenges which arise, share observations/problem solving and to ensure there is no duplication in TA provided to MDTs. Some NGOs are funded both by USAID and the GF. Generally, this appears to be done in a planned way with a view to promoting complementarity. For example, the GF provides KPs with access to commodities such as sterile needles and syringes and condoms and also funds NGO access to HIV rapid tests in some countries. There is however, overlap in USAID and GF outreach prevention activities at some sites. This has been justified on the basis that repeated exposure to messges and 18 behavior reinforcement is desirable and helps to bring about needed behavior change. While there is evidence to support this, repeat exposure could also be achieved by one single funding source which avoids expensive overheads and the possibility of duplication. The issue of overlap between the GF and USAID programs should be addressed by continued discussion and joint planning to ensure rationalization of funding from USAID/CAR and the GF. The new PEPFAR Global Fund Liaison position should be valuable in addressing these funding overlaps. Voucher systems need harmonization and on-going reassessment as to utility: In Tajikistan, GF and USAID partners use different referral voucher systems for referrals from NGOs to HIV testing and clinical services. The different voucher systems should be harmonized to avoid confusion on the part of clients, NGOs, governments, and HCWs and to ensure better monitoring and evaluation of the referral process and outcomes Some NGOs report that clients do not use vouchers when they access health services which would result in under-reporting of referrals where the clients’ use of the health service resulted from an ORW intervention. For example, sometimes MSM do not use GF vouchers because the voucher identifies them as MSM. As they do not want HCWs to know they have sex with other men, they may throw away the voucher and may go to the clinic as a ‘normal’ man. In contrast, the Dialogue voucher does not indicate the KP of the client, which may be a best practice and could be recommended to the GF. There may also be a need to assess the utility of vouchers for HIV testing. The redemption rate for Dialogue referrals for HIV testing is lower than the redemption rate for TB testing, reflecting the barriers that exist to HIV testing. For example, in Year 4, 80 per cent of PWID referred to TB testing in CAR redeemed their voucher, whereas only 66 per cent of PWID referred to HIV counselling and testing in the same year redeemed their voucher.26 In Almaty, Kazakhstan, ORWs reported that it was necessary to accompany KPs to HIV testing sites rather than rely on clients going independently, with a voucher. As HIV testing is generally provided for free, vouchers are not needed for the purpose of avoiding user charges. They do, however, provide a useful mechanism for tracking the number of referrals that are redeemed. Another issue that needs to be addressed in all countries is the lack of referral forms to AIDS Centers for patients who test HIV￾positive at primary health clinics. This is an important gap as there is a need to ensure that a confirmatory test is undertaken and a clinical assessment if the client is confirmed to be HIV￾positive. Service access & utilization need improvement: There are several issues related to increasing access to services. Chief among these is that HIV testing is not targeted to KPs in Central Asian countries. For example, only 1.6 per cent of the of 2 million tests conducted in Kazakhstan in 2013 were conducted on KPs.27 As knowledge of HIV status by KPs can have both benefits as a prevention strategy and is the entry point into the care and treatment cascade, USAID funded projects have actively promoted HCT to KPs, with some success. Over years 1-4, a total of 19,721 members of KP redeemed Dialogue referral vouchers for HCT and received their test result.28 This, however, represents less than 20 per cent of the total number of clients reached by Dialogue with prevention interventions. While there is no baseline data on HIV testing among KPs, the number of Dialogue clients tested for HIV in all likelihood represents a significant increase in testing rates over the last four years. Figure 3, below, shows the number of Dialogue clients tested for HIV, by KP and by year. See Annex 8, Figures 7 – 10 for more data on HIV testing rates by KPs for Dialogue clients. While the number of Dialogue PWID clients having an HIV test increased each year (see Figure 3), this was not the case for any of the other KPs. Barriers to HIV testing include: the stigma associated with HIV; fears regarding confidentiality, especially at government health facilities; and limited 19 knowledge of the benefits of early treatment. In Tajikistan, FSWs claim to be open to HIV testing, but explained that, in order to avoid police harassment, they need to have a Sanitation Certificate which shows their HIV status. They reported that this made FSWs reluctant to get tested for fear of a positive test result on their Sanitation Certificate. Conducting rapid HIV testing by NGOs is one option to increase HIV testing rates, but has its limitations. The Kyrgyz Republic was the first country in CAR to initiate a pilot of NGO-based rapid testing among KP. To date, 4,480 members of KP have been tested in non-clinical settings, with 210 positive results. However, only 64 of those 210 were followed-up with a confirmatory test at AIDS Centers.29 Figure 3: Trends in HIV testing of Dialogue clients by key populations and prisoners, Regional testing rates (CAR) by Year for Years 1-4. Source MIS. Another access related issue is the cost effectiveness of having ORWs accompany clients to health facilities rather than the client going to the facility themselves with a Dialogue voucher. Accompanying clients on their first few visits is no doubt necessary in most instances to overcome fears relating to stigma and discrimination. Interviewees believe it results in a higher uptake rate of health services and may be particularly needed for HIV testing. It also facilities ongoing NGO participation in client case management. For example, Dialogue ORWs in Bishkek, the Kyrgyz Republic reported that FSWs received better treatment by clinic staff when they were accompanied than when they went by themselves. It was, however, the practice to accompany FSW for a monthly HIV test and STI check-up, which is too frequent, especially for HIV testing. Drawbacks of accompanying clients to health facilities are firstly, that it can foster dependency and secondly, it diverts ORWs from other work such as reaching new clients. In the long term, accompanying clients is more costly and not sustainable. Quality of IEC materials: Much of the IEC material used by USAID partners were developed through GF supported projects and are of questionable effectiveness. Informants complained the content of messages were developed through a largely top-down process, and the resultant messages were not responsiveness to the needs of KPs and left little room for the local implementing partners and KPs to be involved in the development of creative HIV messages. In Tajikistan, outreach materials on HIV and TB are printed in both Russian and Tajik. Clients report that the newer batch of materials are more complicated than before. FSWs also noted that materials were translated directly from Russian into literary Tajik, which is hard for them to understand. They felt that if they 0 500 1000 1500 2000 2500 3000 3500 2010 2011 2012 2013 PWID in CAR SW in CAR MSM in CAR Prisoners in CAR HIV Testing for Key Populations in CAR, 20 were more engaged in the development of the materials in Tajik, they would be more comprehensible. According to some MSM NGO staff, clients are no longer interested in receiving the same messages they already know and just throw away leaflets. While reinforcement of messages is needed, programs need to take a creative approach by keeping messages fresh and relevant. Dialogue reported that GF-funded MSM leaflets contained explicit pictures and MSM with family problems did not want to keep the leaflets. 3.5 Constraints Health service constraints: The lack of good case management and social support for PLHIV results in a high drop-out rate from health services. In CAR, diagnosis and treatment services for hepatitis C and STI are expensive and represent a significant cost burden on clients. Even Polyclinics working with USAID supported Locality Coordination Councils, where many of the services are free, do not offer free hepatitis C and STI diagnostic or treatment services to KPs. Limited financing to support ancillary services: HIV prevention efforts for KPs are based on the ability to engage the target populations. To do this effectively, it is important to consider clients’ broader health and non-health related needs. Important ancillary services are not widely available or are inadequate. Critical services, such as drug rehabilitation, legal assistance with passports and registration, employment counseling, methadone detoxification services, housing and psychological assistance, are especially important for persons with drug and alcohol problems. Without access to these support services, continued engagement with KPs may become more difficult and KPs may be unable to stay committed to HIV prevention behaviors. Access to GF commodities is irregular: In most CAR countries, collaboration with GF projects affords USAID funded activities access to commodities such as condoms, needles and syringes, as well as ancillary services essential for KPs (such as drop-in centers, crisis centers, mobile HIV testing, NGO based HIV rapid testing, counseling by friendly doctors, etc.). USAID’s BCC programming is dependent on KPs having ready access to sterile needles and syringes and condoms so they can practice safe behaviors. However, reports from across CAR indicate supply of these GF commodities is inconsistent, and clients frequently complained they have to buy syringes when there are shortages. KPs also commented on the irregular availability of condoms, as well as their perception that they are usually of low quality. Sometimes shortages in prevention commodities occur in GF-funded programs due to delays in disbursements from the GF side, but most often this is caused by inadequate quantification and forecasting by NGOs and government. There have also been problems with adequate product specifications; for example, condoms for MSM not corresponding to users’ preferences. There may be a need for further TA to local entities on the forecasting, procurement and specification of supplies. Enabling environment/policies: Constraints in the enabling environment have negatively impacted coverage of prevention programs. Examples cited include police harassment and arrest of KPs and outreach workers, and police raids on hotspots. ORWs and clients also described situations in which police officers tried to blackmail MSM by threatening to reveal their sexuality to families, universities and employers if they did not pay bribes and refusal to accept reports of crimes against MSM once they became aware of their sexual orientation. Civil society advocacy networks for KPs are limited. The slow progress in the advancement of civil society organizations and their advocacy work has consequences for KPs. The MSM community in the region is quite disconnected. Organizational capacity building and advocacy training would be helpful. On the other hand, the network of FSWs organizations is somewhat better organized in the Kyrgyz Republic and Tajikistan. No strong network for advocacy by FSWs exists in Kazakhstan. 21 In some countries, government policies inhibit HIV prevention efforts. In Uzbekistan, ORWs are not allowed to distribute needles and syringes or condoms. These commodities are controlled by the government and only sanctioned Trust Points decide who can engage in different types of outreach activities. The result is a muted and suboptimal response. In Kazakhstan, HIV rapid tests can only be conducted by medical personnel. Lack of GBV referral options: While USAID has been at the forefront of delivering GBV interventions through HIV prevention programs, there are few referral centers (safe houses or shelters) available to FSWs and MSM victims of GBV. In the few localities where shelters or programs exist, they are often operated by religious centers or anti-trafficking organizations which exclude some KPs, especially FSWs. 3.6 Sustainability The Dialogue Project is in the process of developing a sustainability plan in order to transfer the ownership of best practices developed within the project to the government of each country. The project’s sustainability plan will be introduced at the Regional Oversight Committee in September, 2014. A mixed picture: a more informed, trained and confident KP NGO sector, but limited financial sustainability: USAID programs have significantly developed NGO technical capacity, but have placed less emphasis on organizational capacity. There have also been significant investments in building the technical capacity of HCWs. USAID’s investments have contributed to establishing an engaged, informed and confident NGO sector and improved government sector service delivery. The progress made to increase effective demand and improve the quality of supply provide a good foundation for the sustainability of HIV prevention programs. A major limitation to sustainability, however, is that financing of NGOs remains largely donor dependent. In all CAR countries except Kazakhstan, external donors account for at least 60 per cent of spending on HIV prevention programs for PWID.30 With USAID assistance, HIV NGOs in Kazakhstan have obtained government funding for the first time (see Section 5: Health system strengthening). While there are opportunities to foster greater government financing of NGO prevention programs, for the foreseeable future, NGO funding will be largely donor dependent. In the absence of donor funding there would be a significant reduction of NGO prevention programming and demand creation among KPs for essential health services. Nonetheless, in the absence of NGO funding, some committed NGO workers might choose to work on a voluntary basis as they are often dedicated community members of KPs. Poor systemic sustainability – lack of Governments’ willingness to prioritize HIV prevention funding for KP and revise policies and state social orders to support KP NGOs: Government funding for HIV prevention programs in CAR has been limited. Policy initiatives such as the provision of CPS for KPS are not backed up by budget planning to ensure adequate funding. There has been little discussion between HIV specialists in ministries of health and budget officials in the finance ministries to determine a plan and appropriate level of funding for HIV prevention efforts. Opportunities through the Orders of Social Protection, which funds NGOs working in other health areas (e.g., maternal and child health) have not been widely pursued for HIV programs targeting KPs, possibly due to the marginalized nature of such groups. Governments frequently want to attach higher priority to addressing populations other than KPs, despite epidemiologic evidence to the contrary. Throughout the region, governments consider labor migrants to be at risk. USAID initially supported labor migrants as a group in need of HIV prevention, but ceased migrant-specific targeting when it became apparent that KPs, and not migrants were the main risk groups for HIV transmission. Nonetheless, governments and some 22 development partners remain committed to putting scarce prevention resources into migrants at the cost to more effective KP interventions. With declining HIV financial resources for GF supported programming, governments will face difficulties in reaching treatment targets and maintaining current levels of HIV prevention coverage among KPs. Allowing service coverage levels to decrease would threaten much of the gains achieved to date. Priority, therefore, needs to be accorded to leveraging increased government funding for HIV. USAID/PEPFAR programming and funding uncertainty: PEPFAR funding is determined annually. This results in a lack of clarity from USAID on expected funding obligations for HIV and lack of work plan or budget approval makes it extremely difficult for partners to plan activities. USG partners would like to have a longer-term funding horizon in order to be able to more effectively plan activities. As of the close of the second quarter in FY 14, APMG, Quality’s sub-contractor responsible for implementing the project’s HIV component, is on a limited-work status. Quality programming in Kazakhstan, the Kyrgyz Republic, and Tajikistan, as well as USAID’s HIV activity planning and cooperation with local and international partners have been tabled until more definitive answers about USG programming decisions are available. The Evaluation Team’s recommendations concerning HIV prevention are in Annex 10. (Note, these recommendations have been removed from the public release version of this report as they contain procurement sensitive material.) 4. Care and treatment 4.1 Outline of the care and treatment programming Dialogue focuses on linking KPs contacted through prevention outreach to Multi-Disciplinary Teams (MDTs) who provide HIV-related care and treatment services and case management to clients for adherence and support. Dialogue was able to increase the use of evidence-based TB treatment services by KPs; improve TB case detection; and improve adherence to and decreased default rates from TB treatment for KPs. Quality Project interventions are reinforcing the continuum of HIV care and treatment to improve outcomes and quality of life for PLHIV. Quality has been successful in achieving greater access to high quality health services for KPs and in strengthening national capacity to plan, manage and monitor effective KP programs and services. Quality also works in targeted geographic areas with high concentrations of KPs to improve the capacity of services and improve coordination. It also supports development and strengthening of Community Advisory Boards (CABs) at MAT and ART sites to increase client involvement and empower communities to define priorities in care and improve treatment adherence rates. In addition, both the Dialogue and the Quality Projects strengthen collection, analysis and use of HIV data in policy and program planning, implementation and monitoring. 4.2 Key achievements 4.2.1 USAID Dialogue on HIV and TB Project USAID's investments in HIV care and treatment in the Central Asian region resulted in significant increased access to and use of these services by KPs. In the 4.25 years to December 2013, a total of 12,277 Dialogue clients were tested for TB. Over the same period, 2,321 PWID received drug treatment services, which accounts for more than 50 per cent of those PWID who had been 23 referred.31 This represents a high treatment uptake rate. These results have been achieved through building effective partnerships between government and NGO services and a well-coordinated voucher system. Figure 4, below shows the number of PWID referred to drug treatment against the number of redemptions. Figures 11 – 13 in Annex 8 show the number of referrals and redemptions for TB testing by KPs. Dialogue’s focus on case management is an integral part of its outreach activities, ensuring active identification of KPs, motivating them to get tested for HIV and TB, linking KPs with a range of HIV-related services through the voucher referral system, and supporting their enrollment in and adherence to treatment. NGOs and HCWs reported that those KPs whose engagement in care is supported by case management activities, including needs assessment and escorting to health services by ORWs, are more likely to successfully progress through the continuum of care and have the best outcomes compared to those who receive health services without outreach and social worker support. Clear evidence of this effect could be further documented through a focused data analysis and in the future, through operations research. Dialogue supported several existing MDTs and established new MDTs at selected sites. The MDTs bring together staff from local AIDS Centers, Polyclinics and NGOs to provide a continuum of prevention to care and support services for KPs. The rate at which PWID referrals were translated into redeemed vouchers for treatment is shown in Figure 4 below. Figure 4: Number of PWID referred by Dialogue for drug treatment in CAR: Referrals vs redeemed, Years 1-4. Source: MIS Dialogue’s ‘UNISON’ model for PLHIV is a patient-centered approach implemented by MDTs – doctors, nurses, psychologists, social workers, peer consultants, and narcologists (if MAT is available at the site) – providing medical and psycho-social support services. Families of PLHIV are also brought into the team, where possible, for additional support and to build a stable home environment.32 MDTs provide integrated HIV and TB care to KPs through seeking and treating TB infected patients among PLHIV as well as providing HIV testing for patients with TB and enrolling them in HIV care as needed, including ART. Case management is used by MDTs to ensure comprehensive care and treatment. Interviews and focus group discussions among KPs, NGOs and MDTs further confirmed that the MDT model makes a significant contribution to early detection of HIV and TB and timely enrollment in and adherence to treatment. For example, Dialogue supported 0 200 400 600 800 1000 1200 1400 1600 1800 Year 1 Year 2 Year 3 Year 4 Referred Redeemed PWID Referred vs Redeemed for Drug Treatment 24 NGOs supported referrals which resulted in 604 cases of TB being detected among KPs. The TB incidence rate among Dialogue’s clients was five times higher than national TB incidence rates in CAR,33 which indicates that the Project’s activities have been focused on the most at risk populations In other words, if Dialogue had been reaching people less at risk for TB, the case detection rate would have been lower. Dialogue-funded NGOs also supported self-help groups for PLHIV on ART to enhance adherence. Dialogue’s voucher system is an effective tool for ensuring KPs free access to health services. Dialogue creates demand for health services among KPs who would be unlikely to use these services due to low motivation, stigma and discrimination and other environmental barriers such as police harassment. The voucher referral system has strengthened partnerships between NGOs and clinics. In the Kyrgyz Republic, Dialogue successfully promoted the acceptance of its vouchers, with clients’ UIC used as a substitute for their personal identification. The overdose prevention program led by Dialogue makes Naloxone available wherever PWID may experience life-threatening intoxication. Naloxone is a low cost medication, (US$0.10 per ampule), used to prevent opioid overdose deaths. 4.2.2 USAID Quality Health Care Project Since inception, Quality has trained over 4,400 HCWs across CAR to provide comprehensive HIV services to KPs. In the Kyrgyz Republic, Quality provided the MoH with TA to develop guidelines for rapid HIV tests. These guidelines, and the procurement of rapid HIV tests by the GF HIV project, enabled NGOs to introduce rapid HIV tests for KPs, making testing much more accessible. In Tajikistan, Quality has successfully promoted the use of rapid HIV tests through primary health care facilities. Quality actively promoted integrated care for KPs, (particularly PWID), through the one-stop-shop model. For example, with Quality’s support, the Dushanbe MAT site successfully integrated TB diagnosis, treatment (directly observed treatment short-course - DOTS) and counseling and became the first MAT site in CAR to provide integrated opioid dependence and TB treatment services.34 International evidence indicates that the one-stop-shop model of care significantly improves TB case identification and TB treatment completion rates among PWID.35 One of Quality’s most significant achievements is its support for establishment of Community Advisory Boards (CABs) at local AIDS centers and Narcology clinics in six cities in Kazakhstan, the Kyrgyz Republic, and Tajikistan. CABs are comprised of patients that receive services at these facilities. They increase patient involvement by empowering the community to define priorities in care and drive change. The mandate of CABs includes collection of patient feedback on services to improve the quality of services and retention in treatment. Changes resulting from the work of CABs include a significant reduction in police harassment of MAT patients (multiple sites); successful advocacy for reallocation of the local budget to purchase laboratory equipment for lipid testing, (a key clinical monitoring service for those on ART (Almaty AIDS Center, Kazakhstan); and increased use of MAT by PWID through correcting their misconceptions about treatment (Ust￾Kamenogorsk, Kazakhstan).36 4.3 Strengths Empowerment of key populations: One of the key strengths of the USAID’s HIV programs in CAR is involving KPs in service planning, implementation, assessment, and decision making. Through the introduction of the CABs, Quality has enabled PWID and PLHIV to have active roles in improving health services. CABs also play an important role in re-socializing PWID by helping them re-gain key life-skills such as planning, negotiating and priority setting. Emphasis on evidence-based and low cost interventions: Examples of low cost interventions 25 include Dialogue’s voucher referral system; overdose prevention programming, which uses a very low cost medication; and CABs as their members serve as volunteers. Leverage of other resources: Through its voucher system, Dialogue ensured free access to clinical services provided by government health services across most project sites (i.e., not paid for by USAID or by clients). In the absence of the voucher system, these services would have required a user payment at many sites. However, challenges remain, such as the unwillingness of local health authorities in Almaty to accept Dialogue vouchers. Increased gender sensitiveness of interventions: MDT teams supported by USAID were staffed by male and female social workers who are able to meet the needs of clients of both genders. An example of the program’s attention to gender issues is Quality’s capacity building efforts for MAT at sites in Tajikistan, which included training on reproductive health. This was the first-ever training in CAR on this subject at a MAT site. Supporting collaboration between Governmental institutions and NGOs: USAID implementing partners have emphasized the importance of close collaboration between NGOs and government entities, including health services and MoHs. Considerable progress has been made in promoting greater private-public collaboration, especially when challenges such as low motivation of governmental HCWs to collaborate with NGOs, and NGO perceptions of governmental clinics as being overly-bureaucratic and hostile to KPs, . The MDT model, which brings together NGOs and clinics in case management, has been a significant contributor to improved NGO-government health services collaboration. Dialogue’s and Quality’s joint trainings and seminars for NGOs and MoH staff fostered closer relationships between the two, based upon an approach that identified shared interests. For example, in Tajikistan, the NGO Khayoti Nav, that helps former prisoners in their re-integration to civic life and with access to HIV and other health services, successfully partnered with Khujand’s City Police Department’s Passport Division. Since the intervention, the Passport Division has been issuing passports to former prisoners much more quickly than before, as they rely upon the referring NGO to handle all required paper work and to provide temporary registration for its clients at its shelter. 4.4 Weaknesses Lack of guidelines for MDTs: One of the key challenges experienced by MDTs is the lack of guidelines or protocols for their operations. As a result, different MDTs take different approaches to their work. For example, some MDTs demonstrate a coherent and coordinated team approach to case management, while this is less evident in others. Although it was not possible to examine the reasons for this difference within the limited time for field visits, contributing factors appear to be the personal motivation and beliefs of HCWs; competency based factors such as experience and training; and convenience, such as proximity to other services, which make coordination easier. Training weaknesses: While both Dialogue and Quality conduct a lot of useful trainings for HCWs, in most cases there is limited on-the-job follow up technical support. Key informants noted the need for support through ongoing mentorship to members of MDTs, particularly in case management, managing patients with dual and triple diagnoses, and burn-out prevention. Key informants also noted that trainings organized by Dialogue are often delivered by individuals with rich clinical and field work expertise, but who lack training skills in adult learning, presentation, group discussion and time management. Limited regional exchange opportunities: The limited opportunities for regional exchange among USAID implementing partners were considered by respondents as one of the major weaknesses of the program. ART adherence monitoring: Monitoring ART adherence by MDTs does not appear to be as 26 systemized as TB treatment adherence monitoring. While some MDTs ask patients about their adherence to ART, they do not document the results. This may occur because ART adherence monitoring is not listed among the indicators they report to USAID. As a consequence, ART adherence appears to be given a lesser priority than TB adherence. 4.5 Constraints Psychosocial counselling: The low capacity of staff in providing psychosocial counseling to KPs and their family members needs to be addressed. A shortage of psychologists trained to work with KPs puts most of the burden on social workers who lack necessary skills and expertise. During site visits, psychosocial counseling was named as one of the most important services provided by MDTs and was recommended as a priority area for USAID’s future technical assistance. Structural constraints: Most key informants indicated that the major constraints in implementing HIV programs lie at the structural level. These include active police harassment of PWID, FSWs and MSM, and general lack of confidentiality about a person’s HIV status and sexual and drug using behavior among service providers. Another significant constraint to HIV programming is limited access to prisons where many PWID are concentrated. The continuity and quality of HIV care in prisons are major concerns. 4.6 Sustainability MoHs have not yet formally adopted the MDT model. The lack of documentation of MDT protocols and standard operating procedures may have contributed to this situation. In Kazakhstan, however, the RAC plans to adopt and replicate the MDT model across the country. Low wages for HCWs contribute to high staff turnover in MDTs. As a result, health authorities constantly seek donor resources for new workforce development. The high level of HCW staff turnover, raises the recurrent costs of the program and, as a result, limits the sustainability of HCW capacity building efforts. Dialogue has provided incentive funding to MDT members through top-ups to HCW salaries and has paid full salaries for social workers. This may limit the sustainability of MDTs in the absence of ongoing donor support. The CABs are probably the most sustainable intervention as they harnesses patients’ time and labor voluntarily. Recommendations by the evaluation team relating to care and treatment are in Annex 10. (Note, these recommendations have been removed from the public release version of this report as they contain procurement sensitive material.) 5. Health system strengthening 5.1 Outline of health system strengthening programming USAID/CAR health system strengthening efforts have primarily been undertaken by Dialogue and Quality, with key additional contributions through field support mechanisms. These efforts have focused on strengthening linkages between NGOs and MoH facilities to improve KP access to quality health and social services. 5.2 Key achievements 5.2.1 USAID Dialogue on HIV and TB Project HIV-TB integration: Dialogue has made significant contributions to strengthening HIV-TB integration in project sites and more broadly in the health sector through its participation in HIV and 27 TB Country Working Groups/Technical Working Groups in each country. The MDT approach also has contributed significantly to improved integration of HIV and TB clinical services which has resulted in higher TB testing rates and effective case management, including community based treatment adherence and support. This has been supported by extensive training on HIV-TB integration. HIV-TB integration has also been enabled by Dialogue’s integrated approach to HIV and TB behavior change and education for KPs, coupled with demand creation for TB testing and supported referral to health services. Dialogue Management Information System (MIS): Dialogue has established an impressive MIS to monitor results against all performance indicators, including the work of its NGO grantees. The MIS includes each client’s UIC and tracks contacts and referrals, including referral uptake, and has case management components for PLHIV and TB clients. For TB cases, the MIS identifies and tracks an individual through TB case detection, TB treatment, and those who have successfully completed treatment or those who did not successfully complete (and the major reason). Stigma and discrimination (S&D) reduction: In recognition that S&D by HCWs and police constitute a major barrier to accessing health services, Dialogue has given high priority to creating a more favourable environment through trainings and advocacy in roundtables. In Years 1-4, Dialogue trained a total of 1,049 service providers in S&D reduction and communication skills with KPs.37 Dialogue’s daily work with NGOs in MDT sites facilitates access to services by KPs and sensitizes HCWs to the needs of KPs. This has played an important role in breaking down stigma. Dialogue reports that its work with police through roundtables and trainings has resulted in decreased cases of stigma and discrimination towards KPs in some sites, but still more work is needed. For example, in Tashkent, Uzbekistan, initial police harassment of ORWs was reduced by HIV training which included S&D – now police are now more supportive of the project. The number of health services where KPs can receive non-discriminatory treatment has clearly increased over the life of the project, especially in MDT sites. NGO organisational capacity development: Dialogue partners work closely with over 30 sub￾awarded NGOs to strengthen their capacity in financial, administrative, and reporting requirements, with annual assessments to ensure compliance with USAID requirements. 5.2.2 USAID Quality Health Care Project Government funding of NGOs in Kazakhstan: In CAR, health officials are increasingly recognizing the comparative advantages of NGOs in delivering primary HIV and TB prevention services to KPs. As a result, they are , increasingly in favor of supporting NGO/CSOs for work with KP to increase their willingness and ability to access health services. Demonstrated results have also increased support for joint case management by NGOs and health facilities. In Kazakhstan, NGOs are able to seek direct government support for health activities through social contracts, but many would benefit from TA in understanding the process and applying for contracts. Over the past three years, Quality conducted oblast-level trainings and roundtables with NGOs to promote their understanding of social contracting. This investment paid off as 16 HIV-focused NGOs from four regions received funding, (10 for the first time), from the government through mayor’s offices (Akimats) and AIDS centers. The government-funded projects address the following priority issues: supporting MDTs; assisting KPs to restore lost government identification cards (enabling them access to services within the State Guaranteed Benefits Package); improving social services for female PWID; implementing BCC campaigns; and supporting PLHIV, including HIV-infected children.38 Locality Coordination Councils (LCC): Forming LCCs has been a key achievement of Quality. These fora allow both NGOs and governmental partners to identify problems experienced by KPs in 28 accessing health services and to collaboratively develop solutions. Two LCCs have been established in each of Kazakhstan, the Kyrgyz Republic and Tajikistan. Closer links between NGOs and health care facilities have been established through the LCCs, resulting in increased access for KPs through integrated services. For example, in Dushanbe, Tajikistan, HIV and TB representatives on the LCC decided that a TB consultant would work in the HIV Department at the Infectious Diseases Hospital. In the Kyrgyz Republic, an LCC has enabled a relationship between an NGO and the Health Insurance Fund (HIF), through which outreach workers can communicate problems. Through this link, the Plus Center, an NGO in Osh, reported the illegal denial of services for a KP. The HIF was able to rectify the situation and disciplined responsible health staff. KPs have fewer difficulties accessing TB diagnosis and treatment in two TB hospitals in the Kyrgyz Republic.39 In Tajikistan, in response to requests by the Dushanbe and Vakhdat LCCs, mobile teams of specialists, including STI, TB, reproductive health, infection specialists and family doctors, now provide services to the clients of Dialogue supported NGOs, including rapid HIV testing.40 Registration of key populations: In three sites in Kazakhstan, Quality worked with local NGOs to facilitate access of KPs to primary health care (PHC) services through annual population enrollment campaigns to register KPs with specific PHC providers. Registration entitles enrollees to a range of free PHC services through the State Guaranteed Benefits Package, which includes most of the basic services needed by KPs. These populations are often reticent to seek services at facilities other than AIDS Centers, (i.e., PHC sites), due to fear of stigma and discrimination from HCWs. Quality’s work included promotion of the benefits of enrollment to create demand and assistance to KPs with enrollment. Within one month, over 100 individuals from KPs in three sites enrolled with PHC facilities.41 Reducing S&D by HCWs: Quality strengthened the capacity of HCWs to treat KPs in a non￾stigmatizing manner. An NGO representative from a CAB in the Kyrgyz Republic reported that following the training, Family Medical Centers (FMC) staff no longer refuse to provide services to KPs and that more positive relationships were forming between HCWs and KP clients. In another example, LCC members in Tajikistan who were trained on the health needs of MSM reported that their attitudes about working with MSM changed markedly as a result of the training. Prior to the training, they were reticent to treat or even communicate with MSM. After the training, they said they had a better understanding of MSM and were working within their health facility to improve services for MSM. Organizational capacity building for NGOs: Quality has played a central role in strengthening the capacity of NGOs who are funded by Dialogue and the GF, Interventions have improved NGOs’ abilities to manage programs. For example, NGO staff were given training on human resource management and program management. In the Kyrgyz Republic, Quality provided TA to the Alternativa NGO to finalize their strategic plan. Alternativa staff remarked on how the assistance from Quality helped them to more realistically assess their strengths/weaknesses and plan for the future. Training of HCWs and curriculum development: Quality provided technical assistance to revise all training modules on inter-personal communication and counseling for HCT and harm reduction, according to international guidelines. In the Kyrgyz Republic, these updated modules were approved and incorporated into the curriculum of the Kyrgyz State Medical Institute for Retraining and Continuing Education and are used for in-service training.42 Improved supply chain management: A key component of a functioning health system is an effective supply chain management system. In Tajikistan, Quality assisted GF grant implementation by introducing ARV forecasting software to national and oblast level AIDS Centers, which resulted 29 in improved ARV quantification and stock management. . In addition, Quality provided TA to the GF programs on procurement and supply chain management and has trained key AIDS Center staff on the system. 5.2.3 Field support mechanisms and other initiatives AIDSTAR NGO capacity assessments: In FY13, AIDSTAR-Two conducted rapid, structured, diagnostic assessments of selected USAID funded NGOs undertaking HIV work in Kazakhstan, the Kyrgyz Republic and Tajikistan. The purpose was to inform the development of capacity development strategies to enable NGOs to better contribute to national HIV efforts. This work has broadened understanding of the capacity development needs of NGOs, especially in the area of organizational development. Following the NGO capacity assessments, AIDSTAR-Two developed seven Capacity Developer’s Guides (tools) in: the areas of 1) NGO governance; 2) NGO business planning; 3) partnership and coordination; 4) human resource management; 5) advocacy; 6) quality service delivery for KPs; and 7) strengthening involvement of PLHIV and s other affected communities. Quality then used the guides to conduct pilot trainings for the NGOs. Gender assessments: In FY13, AIDSTAR-One/Two conducted HIV-related gender assessments in Kazakhstan, the Kyrgyz Republic and Tajikistan to inform a PEPFAR/CAR gender strategy which was subsequently developed. This work filled an important gap in strategic thinking for gender￾related HIV programming. Policy assessment: In FY13, the Health Policy Project (HPP) conducted a desk review of existing policy assessments in Kazakhstan, the Kyrgyz Republic and Tajikistan; a participatory assessment to identify HIV-related policy priorities in CAR; and developed an options paper for policy work in CAR. Key policy issues were identified in each country, including S&D, MAT, and government contracting of NGOs. CCM assessments: In FY13, Grant Management Solutions (GMS), at the request of USAID/CAR, undertook assessments of the Global Fund Country Coordination Mechanisms (CCM) in Kazakhstan, the Kyrgyz Republic and Tajikistan. Each assessment contains an overview of CCM achievements, a prioritized list of current gaps and focussed recommendations to address key governance and coordination issues. GMS also developed a CCM capacity building plan and strategy for USAID/CAR. Next steps for further USAID support are currently under consideration by the Mission and the CARs, in collaboration with the GF. Investment case: USAID/CAR is funding UNAIDS to undertake investment case studies in Kazakhstan, the Kyrgyz Republic and Tajikistan. These studies will provide data on the level of investment needed by each country to make to make an impact on the HIV epidemic in CAR and the program areas that will result in the greatest return on investment in averting new infections and saving lives. 5.3 Strengths MIS: Dialogue effectively uses MIS data as a quality assurance and improvement tool by regularly monitoring the performance of its NGO grantees and working with them to identify and rectify problem areas (e.g., a decline in referrals). Also, Dialogue assists NGOs in analyzing their MIS data for the purpose of program improvement. Issues identified from MIS data analysis requiring follow￾up are included in TRaC surveys. Government funding of NGOs in Kazakhstan: While social order funding is modest, this is an important first step in government support for HIV work by NGOs and one that could be replicated in the region to reduce donor dependency. Government funding for NGOs in Kazakhstan is particularly important given that the country is no longer eligible for new GF HIV grants. 30 Investment case work: Findings from the UNAIDS investment case can be used to guide prioritization of the comprehensive package of HIV services that have been developed with Quality’s assistance. Both of these tools will provide a solid foundation of evidence for advocacy by USAID under the follow-on program to leverage government resources and mobilize other funding in an effort to reduce donor dependency. CCM strengthening: Although it was beyond the scope and capacity of this evaluation to determine the extent to which each CCM has implemented GMS’s recommendations, the GMS assessments were a strategic activity with the potential of maximizing the effectiveness of GF grants. This is of direct interest to USAID given the synergies between USAID and GF-funded activities in the CARs and the USG’s significant level of investment in the GF. Strengthening the CCM in Kazakhstan is of particular importance as the government recognizes there is a need to continue with a similar national coordination mechanism, despite the county’s ineligibility for new GF HIV grants. Stigma: Dialogue has demonstrated flexibility in its approach to overcoming S&D barriers to service access. For example, in Almaty, Kazakhstan, it became known that FSW were being refused follow-up services at city clinics despite holding referral vouchers from Dialogue-supported NGOs. ( . Dialogue worked around this by referring FSWs to the Student Polyclinic where staff attitudes were better. Pervasive and persistently negative attitudes of HCWs towards PWID in Kazakhstan have also served as a barrier to referrals and resulted in additional activities (trainings and round tables) by Dialogue to promote tolerance towards PWID. LCCs: The main strength of LCCs is that they bring together staff from various health facilities (AIDS Centers, Polyclinics, Narcology Centers, TB sites, etc.) and NGOs to solve problems and improve coordination. LCC members in Tajikistan compared their organization to the CCM. For instance, NGOs advocate to government and donors, as does the CCM. Also, through the LCC, NGOs have another means of advocacy on behalf of their clients (e.g., free services, higher quality care from providers, or reducing facility and interpersonal barriers to KPs seeking to access services). Some respondents reported that it getting the government to work with KP NGOs has been challenging. To address this challenge, the project fostered activities aimed at ‘socialization’. For example, where possible, joint LCC trainings were conducted for government representatives and NGOs. The project also identified and encouraged key government leaders who were willing to advocate for the model with their peers. A respondent in Tajikistan said these efforts resulted in changes in the client-provider relationship, in the “hearts and minds” of HCWs, and an increased willingness to use the LCC as a forum for resolving problems. In some cases, LCC meetings are held at NGO offices so that the government is exposed to their services. Collaboration with other donors: Quality has seconded a specialist to the National AIDS Coordinating Mechanism in Tajikistan to facilitate partner information exchanges and assist in the preparation of the country application to the Transitional Funding Mechanism on HIV, TB and Malaria. This role provides critical support to ensuring a coordinated HIV response in Tajikistan. 5.4 Weaknesses NGO capacity development A key finding of the AIDSTAR-Two NGO capacity assessments was that the capacity development work of Dialogue and Quality had concentrated largely on technical trainings and mentoring of NGOs rather than organizational capacity. The assessments recommended that a “more strategic, systematic and coherent approach to capacity development … is required … to focus not only on specific technical skill areas but also on the organizational capacity needs that underpin the NGOs ability to implement programs.”43 While this finding is valid, the program design did not require USAID’s implementing partners to take a broader approach to 31 NGO capacity development. The AIDSTAR-Two capacity assessments were, unfortunately, commissioned at a relatively advanced stage of program implementation which made it difficult to make significant changes in the orientation of capacity development work. In addition, the core competencies of both Dialogue and Quality lie more in technical and program capacity development rather than in organizational development. Delays in releasing the AIDSTAR assessments by USAID/CAR were also a factor in why there has not been a significant reorientation in NGO capacity development work. Nonetheless, the assessments provide useful findings on which to base the approach to capacity development in the follow-on project and the capacity development guides can be used in that project. While Dialogue’s NGO organizational capacity development work was largely confined to ensuring compliance with USAID/PSI financial management, administrative and reporting requirements, (consistent with their program description), a somewhat broader focus was planned for Year 5 through creating opportunities for sharing of information and strategic sustainability planning during NGO visits, the regional NGO coordination forum, and the regional forum on case management for HIV and TB. Quality will also undertake more NGO organizational capacity development work in Year 5, although due to an overall project re-scoping and budget limitations, this work will be less than initially planned. Applying the regional evidence base: While USAID made a significant investment in gathering data through its field support mechanisms to inform programming, most partners have not had adequate time to adopt findings into their activities. Most of the assessments took place after Dialogue and Quality were well established. Both Dialogue and Quality noted that they were not able to undertake activities related to policy or strengthening Global Fund activities through CCMs. Where findings were available, such as the gender assessments, it is not clear if current gender￾specific activities were informed by them. Furthermore while respondents referred to the gender assessments, they did not refer to the PEPFAR gender strategy. As such, there is potential to draw upon this evidence base to improve current programs. Conversely, there has been little progress in addressing policy issues identified by HPP’s CCM assessments. LCCs: In some places, LCCs may duplicate existing coordination structures at a higher level. Respondents in Ust-Kamenogorsk, Kazakhstan, noted they disbanded their LCC when it was found to be duplicative and less able to coordinate functions than a city-level structure which was already in place. The lesson to take from this is that existing structures need to be considered before setting up LCCs. HCW training: While Quality had some success in getting MoHs to adopt training curricula, several respondents noted that there is a need to emphasize the integration of training modules into the curriculum of government institutions rather than one-off trainings. The Tajikistan AIDS Center underscored the importance of shifting the current focus from primarily one-off, short trainings for HCWs to strengthening pre-diploma and continuing medical education as this will have the most sustainable impact and be more comprehensive. USAID should encourage MoHs and RACs to invest in HIV-TB pre- and in-service HCW training to promote sustainability. 5.5 Constraints Stigma: S&D continues to be a significant barrier to service access by KPs in some sites. Stigmatising attitudes and discriminatory behaviour by police and HCWs have been a particular problem in Tajikistan. Police and health providers continue to exclude, harass or refuse services to PWID, effectively keeping them from accessing drug rehabilitation (such as MAT). In Tajikistan, Dialogue recently stepped up its efforts in this area. While Quality has succeeded in reducing stigma among some HCWs (see 5.2.2 Achievements, above), other providers remain harder to change. For instance, HCWs interviewed at some at Quality sites said they would prefer not to serve 32 MSM, while, at the same time, asserting S&D did not exist at their polyclinic. Other respondents noted that S&D remains a problem and that step-by-step, provider trainings and reinforcement should continue. NGO contracting mechanisms: To ensure continued engagement of the NGO sector in health services, governments need to create and support mechanisms to fund NGOs. While a social contracting system exists in Kazakhstan, procedures are complicated and there is limited awareness of the mechanism. Furthermore, Kazakh NGOs are required to forward fund costs and then seek reimbursement, which is beyond the financial capacity of many NGOs. Other countries in the region have not extended social orders or similar mechanisms, where they exist, to provide support for health or HIV-specific NGOs. The PEPFAR program has demonstrated results from collaboration on government and civil society collaboration. There is a need to build on these experiences through further development of workable government mechanisms for funding NGOs. In addition, opportunities exist to work with USAID’s Democracy and Governance teams on extending government financing of NGOs in CAR. Voucher financing- an unfunded mandate: A key constraint of the voucher system is that it is not linked to the financing of health services from a sustainable, indigenous source. Government funding of health facilities is linked to patient enrollment in particular clinics. Also, facilities can be reluctant to accept vouchers from patients who are not enrolled there, as they will not be covered by funding. In addition, for patients who do not have a government identity card, it is not possible to enroll in a facility, which means services provided are not going to be reimbursed. Thus, KPs typically cannot receive free services without a current government ID. Quality reports that about 5- 10 per cent of KPs do not have a valid ID. Some NGOs, like Sotsium in Bishkek, the Kyrgyz Republic, have, however, developed good relationships with facilities who will take patients without government ID. A polyclinic in Ust-Kamenogorsk, Kazakhstan, reported that while there is a Prikaze allowing patients with vouchers to receive free services, as the clinics do not receive funding for these clients unless they are registered. Exclusion of KP from funding and public facilities has not been resolved and respondents said it is only manageable because it does not involve a significant number of patients. 5.6 Sustainability LCCs can be sustained with minimal external financial support. While LCCs require a significant time commitment, their financial support needs are minimal (meeting costs and a nominal remuneration for the LCC coordinator). A LCC in Tajikistan noted that a challenge to sustainability was low motivation among HCWs, in large part due to low salaries. Participating in the project lent some prestige to the HCWs, but in the absence of an international project, LCC members thought it may be difficult to sustain interest and motivation among staff. Assisting KPs to enroll at health facilities can help foster sustainability given government financing mechanisms are based on patient enrollments and not linked to Dialogue’s voucher system. Taking into account that the existing Basic Benefits Package provides a fairly wide range of PHC and outpatient specialty consulting and laboratory services at low cost or free of charge to the population, and the PHC money-follows-patient financing mechanisms, one of the best ways to improve access by key populations to health care is to advocate and support enrollment of KPs with PHC providers.44 Furthermore, where the political will exists and the voucher system is in place, it can be sustained with the continued engagement of outreach worker referrals. To ensure sustainability, financing of the voucher system, or at least linking it to financing mechanisms, needs to be thoroughly explored. In Kazakhstan, government orders have been issued for integration of the Dialogue referral voucher system into the health care systems of Almaty, Karaganda and East Kazakhstan Oblasts, although 33 health services in Almaty still harbor resistance to accepting vouchers, since they are not linked to funding. The orders regulate improved access to free HIV-related services by KPs, including those who do not have identification and registration. In the Kyrgyz Republic, the Dialogue voucher system has been endorsed by the MoH, ensuring sustainability beyond the life of the program. In Tajikistan, Dialogue’s voucher referral system has been adopted by three regional governments. In Uzbekistan, MoH prikaz #232 provides that HCWs nation-wide should accept the Dialogue vouchers, however, implementation is currently limited to Tashkent. Looking across the region, health facilities’ acceptance of Dialogue vouchers is widespread, (with the exception of Almaty), and there is evidence voucher systems for KP (and other groups) are gradually becoming more common and accepted. The sustainability of Dialogue’s work benefits from government adoption of a number of key program components which has occurred. Dialogue is developing a sustainability plan to transfer ownership of project best practices to CAR governments. In particular, the Dialogue MIS and UIC have been adopted by RACs in Kazakhstan, the Kyrgyz Republic and Tajikistan. In Kazakhstan, the RAC has established a unique national data base to track prevention program coverage of KPs. This can be used for national level coordination, leverage of activities and avoidance of duplication. Across all countries, state institutions have expressed interest in using the MIS as an M&E tool. Additional work is needed to promote government adoption of the MIS. NGO organizational development by Dialogue has been largely confined to ensuring compliance with USAID/PSI financial management, administrative and reporting requirements. This has contributed to strengthening the organizational capacity of NGOs and their ability to attract to funds from other sources, including government. Organizational development across a broader range of areas would have been beneficial in building the sustainability of NGOs, although this was not within Dialogue’s program description. Recommendations by the evaluation team in relation to health system strengthening are in Annex 10. (Note, these recommendations have been removed from the public release version of this report as they contain procurement sensitive material.) 6. Program management Management Structure: The USAID/CAR HIV program is managed by the regional mission in Almaty, Kazakhstan with assistance and oversight from the country teams. The HIV team in the regional mission currently consists of four staff members. The agreement office’s representative (AOR) and contracting officer’s representative (COR) for bilateral projects are drawn from this team and the bilateral projects are managed from Almaty. In Tajikistan, Turkmenistan and Uzbekistan, program activities are managed by local staff in USAID country offices and in the Kyrgyz Republic by USAID/Kyrgyz Republic. Both Dialogue and Quality have central offices with Chiefs of Party (COP) in Almaty and field offices in Bishkek, Dushanbe, and Tashkent. Field office staff work closely with USAID activity managers and the AOR/CORs work closely with the COPs in Almaty. Field support activities are also managed by USAID/CAR. While the USAID management structure can cause tension due to the reliance on USAID/CAR, the evaluation team found that USAID/CAR has been largely responsive to issues raised by the country managers, within the constraints of over-stretched staff. The management structure appears to be effective, provided that clear lines of communication between the AOR/CORs and country activity managers are maintained. Similarly, bilateral projects need the same level of communication between a COP and field office staff. The field staff also need to be empowered to manage their country activities independently of the COP and central office. 34 Staffing: The USAID/CAR HIV team has had two vacant positions for two to three years, placing a burden on current staff. In addition, the frequent turnover of USAID/CAR staff has affected project management. Interview respondents stated that consistency in management is important, noting that there have been three different health directors (and deputies) within the span of three years. For Quality, this has directly affected management of the project, as these USAID management staff were Quality’s CORs. This has not affected Dialogue, as their AOR, a foreign service national (FSN), has remained the same over the life of project. The current duration of the Foreign Service Officer posts at USAID/CAR is only two years. This short amount of time has impacted the management not only of Quality, but of the whole program, as each health director has had different priorities and has left post before ideas were fully implemented. This changeover in staff has led to disjointed and inconsistent management for some of the projects. At the same time, staffing levels in CAR country offices and missions have increased. The Kyrgyz Republic recently became its own mission and as a result, the health team there has expanded from one person to four. This increase in staffing has facilitated timely management of HIV activities. Since the mission was established, however, Kyrgyz partners noted that monitoring has increased. While this might be considered an added burden on NGOs, the partners appreciate the heightened interest in their work. In terms of partner staffing, a few respondents noted there was a perception that Quality relied on an overseas partner to manage the bulk of its HIV technical work. This increased costs and put a strain on the budget. PEPFAR Coordination: The Office of the Global AIDS Coordinator (OGAC) manages PEPFAR funding and determines the overall program budget levels, the level of investment in each country and which projects and activities will be approved for funding. OGAC has not supported a high level of effort in Uzbekistan, which has the highest HIV prevalence rate in CAR and a significant TA deficit. USAID/CAR has successfully managed its program within OGAC guidance, which has encompassed the different approaches of both piloting direct service delivery models and serving as a TA model. As such, Quality and Dialogue’s activities include both models. In the overall PEPFAR context, USAID and CDC have created a PEPFAR program in which each agency programs to its comparative advantage. USAID partners reported having met with CDC’s implementing partners to delineate roles and then on a monthly basis to review work plans to foster collaboration and to ensure there is no duplication. Partners felt these efforts have been successful, noting that Quality works in different geographical areas and their focus on clinical guidelines has been in different areas from CDC’s main partner, ICAP. The evaluation team found that current partners value both USAID partner meetings and PEPFAR partner meetings, finding them helpful for communication and collaboration. Partners were uniform in welcoming the return of these meetings and expressed hope they will continue on a regular basis. Some respondents felt the work of USAID and CDC could be better coordinated through more joint meetings between USAID and CDC and their partners and joint site visits. Donor partners singled out the lack of recent meetings in Kazakhstan, and would like to see USAID revive these inter￾donor meetings. Interview respondents pointed to partner meetings as a positive aspect of the USAID program, contrasting it with Global Fund-supported organizations where coordination is less transparent. USAID partners also undertake combined monitoring visits with each other and with CDC partners, where possible. In Kazakhstan, partner staff reported the usefulness of joint monitoring visits with officials from the Global Fund. Bilateral Projects: A few respondents expressed the view that implementation would be easier if USAID had one project and one implementer. They noted that Dialogue and Quality were designed 35 to complement each other, but there were delays in starting Quality and as a result, its activities were not initially synchronized with those of Dialogue. Following a review in 2012 of the mandates and work plans of Dialogue and Quality, adjustments were made to avoid duplication. Partners were consistent in stating that this resulted in both projects complementing each other’s activities and areas of competence. Perhaps due to changes in the CORs and Quality staffing (e.g., COP), progress in implementing the Quality Project has at times been delayed. Most notably, USAID did not approve drafts submitted in Year 1 of Quality’s performance monitoring plan (PMP), with the document only being finalized in the fourth year of the project. This delay severely affected both USAID’s and Quality’s ability to monitor activities. Furthermore, since the project began, USAID’s priorities have shifted and Quality’s original broad focus was narrowed down. Quality staff contend that a number of planned activities were not completed due to changing USAID priorities. In addition, external factors, including other partners completing activities and mission reports not being available in a timely manner, affected Quality’s ability to plan and undertake activities. Field support: It appears that staff vacancies and frequent changes in USAID/CAR staffing have contributed to uneven oversight and guidance for a number of field support activities. Field support activities are often the lowest priority for the USAID/CAR staff, and as such, faced delays and discontinuities. On several occasions, Mission staff who programmed the funds into field support mechanisms were no longer in the mission when activities needed to start. As a consequence, the vision and purpose for the funding changed numerous times. In some cases, as with the Health Policy Project, funding was committed to the mechanism for as long as two years before the partner was able to proceed. AIDSTAR activities did not start until there was pressure from Washington because the mechanisms were ending. While the AIDSTAR activities offered a contribution to the overall HIV program, they were somewhat rushed and did not leave sufficient time for the same partner to undertake adequate follow-up actions. Given the AIDSAR assessments completion and approval in year 4 (of five) of the Dialogue and Quality projects, the two bilateral programs were not able to sufficiently incorporate many of their key findings . Representatives from the bilateral programs stated they were often left waiting for information from the field support activities and in some cases, never received it. They felt this impeded their ability to integrate the findings into their work plans. Recommendations by the evaluation team in relation to program management are in Annex 10. (Note, these recommendations have been removed from the public release version of this report as they contain procurement sensitive material.) 36 Annex 1: Scope of work This Annex contains extracts of the key elements of the scope of work for the evaluation. Evaluation Purpose USAID/Central Asia Republics and USAID/Kyrgyz Republic seek an expert team to evaluate the performance of the Missions’ current HIV/AIDS investments, and to make recommendations for investments over the next five years based on epidemiological priorities, good and promising practices, and service delivery gaps. The Missions are preparing to plan investments in HIV over the next several years, and recommendations from the evaluation will directly inform the development of a USAID Project Appraisal Document (PAD). The PAD lays out the rationale and justification for such assistance, and documents how USAID has conducted due assessment and planning to maximize the impact of its funding. Background on HIV/AIDS Investments in the Central Asian Republics USAID programs are designed to improve access to services by key populations through comprehensive outreach activities. During FY 2013, the USG will also intensify efforts to improve service quality and comprehensiveness by expanding public sector – NGO collaboration to increase access to services; improving referral linkages and systems; and, as feasible, continuing to promote country uptake and institutionalization of evidence-based approaches. In support of Objective Two, the USG will continue to provide targeted technical assistance, training and mentoring to strengthen the capacities of public and NGO providers to provide and manage improved services, improve HIV diagnostics and contain nosocomial transmission. Finally, through Objective Three, the USG will continue to support activities to improve the quality, availability and utilization of strategic information for program management and policymaking. During FY 2013, PEPFAR CAR will give greater focus to the expansion of outreach services that promote sexual prevention, counseling and testing and HIV-TB interventions among greater numbers of key populations. The USG will eliminate HIV work with migrants (instead using TB funds to identify and refer for testing any migrants at high risk of HIV) and will increase essential prevention work with PWID, MSM, SW and prisoners to contain growth of the epidemic. In addition, building on foundational policy work and capacity development initiated during FY 2012, PEPFAR CAR will blend support for pilot interventions, systems strengthening and capacity building activities to expand access to rapid testing as a potential tool to reach increased numbers of key populations and link individuals with services. FY 2013 programs will build on limited prior year support for pilot activities by strategically expanding the implementation of models that demonstrate improved access by key populations to more comprehensive, higher quality services. The USG will promote country ownership and institutionalization of successful models through more effective policymaking and increased NGO engagement in advocacy and service quality improvement. Refocusing the Investment Approach: The USG will address priority investment approaches highlighted in the FY 2013 funding level letter by: a) Expanding outreach prevention activities to reach greater numbers of beneficiaries; b) Continuing to support pilot and demonstration activities and related targeted policy advocacy to enhance the service continuum and expand access by key populations to comprehensive services such as rapid testing; and c) Promoting institutionalization of best practices into indigenous and donor-funded services and systems. 37 The USG will review progress to date and expand the use of multidisciplinary teams to additional sites; assess the effectiveness and consider scale up of pilot entry points (targeted at areas of high concentrations of MARPs), as mechanisms for linking and referring key populations to appropriate services and care; and undertake other approaches to build partnerships between public and NGO institutions to enhance access to services. PEPFAR CAR will also strengthen the capacity of AIDS centers to provide quality services to key populations by: a) Defining a minimum package of services to be offered at trust points, MAT dispensaries, drop-in centers and mobile units; b) Improving linkages and referrals to other vertical systems that provide care to key populations, including TB, Narcology, STI and reproductive health systems; c) Building data-sharing capacity to ensure tracking of referrals and allow follow-up of missed referrals; d) Promoting the engagement of NGOs that represent key populations in the design, delivery and improvement of preventive, diagnostic and treatment services for greater uptake and adherence. The USAID funded projects have a great complimentary collaboration in providing targeted technical assistance to CAR countries to reduce the spread of HIV among KPs, advance USG’s priority of an AIDS Free Generation and support the CAR PEPFAR strategy. Projects help to improve the capacity of the public sector and NGOs, increase access to quality HIV services and improve the quality and use of data. Target Areas and Groups  Dialogue Project increases access to HIV and TB prevention and treatment services among PWID, SW, PLHIV and MSM. The project provides TTA to reach these KP through client referrals to a comprehensive package of services in all 107 project target sites in KZ, KG, TJ and UZ.  QHCP: in KZ, KG and TJ QHCP works at the national level to create an enabling policy environment for key populations (KP)-PWID, MSM and SW-to access HIV services for prevention, care and treatment. QHCP also targets the geographic areas in each country where KP concentrations are highest.  GMS Project supports to better meet the health needs of their citizens, particularly vulnerable populations, by deepening its technical support to Country Coordinating Mechanisms (CCMs) and Secretariats in KZ, KG and TJ.  The Health Policy Project (HPP) supports policy change that fosters sustainable and equitable health services and works to build the capacity of civil society organizations (CSOs) to advocate for lasting improvements in health in KZ, KG and TJ. Development Hypothesis By improving access in public and private sectors to quality HIV prevention, care and treatment services to reduce the transmission and impact of the HIV epidemic; strengthening the capacity of the health system to deliver improved, expanded, equitable and sustainable HIV/AIDS services for Key Population, PLHIV and their families, and other affected populations; strengthening the capacity of public and private sectors to collect, analyze, manage and utilize data for evidence￾based planning and policymaking at all levels, the result will be a reduced burden of HIV/AIDS epidemics among Key Population in Central Asian countries. 38 Audience and Intended Use The audience of the evaluation report will be the USAID/CAR Mission, specifically the Health Team and M&E Unit, and implementing partners. USAID/CAR will use the report to improve its current strategy of providing support to Key Population and to share lessons learned with other stakeholders. Key Evaluation Questions In particular, the mission anticipates that the evaluation team will focus its attentions on addressing the following priority questions, relevant to 1) past program performance, and 2) strategic priorities for the future: Question 1: To what extent were expected results achieved, and why or why not?  Activity-level results (outputs)  To what extent has the USAID-funded HIV/AIDS program (including the Quality Project, the Dialogue Project, and field support mechanisms) achieved its objectives as specified in the mission’s Activity Approval Document and in the existing partner agreements?  Health system-level results (outcomes)  What evidence do we have of impacts on health systems in the region?  What, if any, additional impact has the HIV/AIDS program had on the quality, coverage, and/or efficiency of HIV services in Central Asia?  How well are existing service linkage and referral systems working?  Policy-level results (outcomes)  To what extent do the activities make a significant contribution to the MoHs national HIV and health strategies?  To what extent has the HIV/AIDS program produced the data needed to inform strategic planning?  How have existing projects contributed towards priority policy reforms?  Operational issues related to results  If expected results were not achieved, what have been the program’s major constraints with respect to these objectives?  How well have partners collaborated to improve the quality, coverage, and impacts of HIV/AIDS programming?  Are there any noteworthy areas of synergy or duplication across USG-funded and other donor-funded HIV/AIDS activities?  To what extent has the HIV/AIDS program employed gender-sensitive strategies in its activities (i.e., how have both men and women been reached and have these approaches been effective in engaging men and women?)? Question 2: To what extent has our program addressed the most relevant HIV/AIDS issues in each country?  To what extent have existing investments been strategic in terms of addressing key local epidemiological priorities and service gaps and avoiding duplication of effort? 39  To what extent are partners addressing felt or actual beneficiary needs? Question 3: To what extent has our program enhanced local partner capacity in HIV/AIDS?  To what extent are existing partners serving as technical leaders and advancing good and promising practices in HIV prevention, care and treatment? Question 4: What key lessons have we learned from current mission HIV/AIDS investments and activities? Question 5: Given the answers to Questions 1-4 above, how should the mission invest resources in HIV/AIDS during the next five years?  Priority Areas  What should be the scope and priority focus areas for USAID investments in HIV/AIDS in Central Asia over the next 5 years? What should the mission consider doing: 1) more of? 2) less of? 3) differently?  What policy priorities in the area of HIV need to be addressed in the new activity or activities to better support Central Asian national strategies in HIV prevention?  To what extent should USAID consider restructuring its investments and procurements to focus in the following four key areas: 1) prevention programming for key populations, 2) improving HIV care 3) improving access to treatment, and 4) systems strengthening, including NGO capacity building and quality of strategic information.  Implementation Strategies  What strategies should USAID adopt to improve the quality and coverage of services to address the comprehensive health needs of clients, particularly the need for access to HIV counseling and testing?  How can USAID investments help Central Asia to develop more cost-effective intervention models and reduce its dependency on donor support?  Which gender-specific activities should be incorporated in the new design?  How can or should USAID complement and leverage the support and activities of other donors/governments?  What is the potential for sustainability of USAID programs and what strategies should USAID adopt to promote sustainable HIV prevention for key populations?  What would be the ideal implementation and management plan and level of investment in each country in the region, given the size of the regional program?  Procurement Planning  Given the project end-dates of current HIV programs in March 2015 and September of 2015, what is the most suitable course of action for follow-on design/s to prevent potential gaps in service delivery and program implementation?  What kinds of surveys or special studies need to be done to better align programming to beneficiary needs in the future? 40 Annex 2: Background documents This annex lists all the documents provided to the Evaluation Team by USAID/CAR. AIDSTAR-One, Gender Assessment: Access to HIV Services by Key Populations in Kyrgyzstan. 2013. AIDSTAR-Two, Gender Assessment Report: Access to HIV Services by Key Populations in Kazakhstan. 2013. AIDSTAR-Two, Gender Assessment Report: Access to HIV Services by Key Populations in Tajikistan. 2013. AIDSTAR-Two, PEPFAR Central Asia Region Gender Strategy: Strategic Opportunities 2013- 2020. 2013. AIDSTAR-Two, Central Asian Capacity Building Strategy Project. Kazakhstan. 2013. AIDSTAR-Two, Central Asian Capacity Building Strategy Project. Kyrgyzstan. 2013. AIDSTAR-Two, Central Asian Capacity Building Strategy Project. Tajikistan. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Advocacy for HIV and AIDS Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Business Planning for HIV and AIDS Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Strengthening the Involvement of People Living with HIV and others from Affected Communities within HIV and AIDS Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Human Resource Management for HIV and AIDS Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Governance in Non-Governmental Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Partnership and Coordination for HIV and AIDS Organizations. 2013. AIDSTAR-Two, The Capacity Developer’s Guide to Quality Service Delivery for Key Populations. 2013. AIDSTAR-Two, Mid-Term Evaluation of the Dialogue Project. Central Asian Republics. 2012. Grant Management Solutions, Technical Assessment 803SC (Central Asian Republics Assessment). Republic of Kazakhstan. 2013. Grant Management Solutions, Technical Assessment 803SC (Central Asian Republics Assessment). Kyrgyz Republic. 2013. Grant Management Solutions, Technical Assessment 803SC (Central Asian Republics Assessment). Republic of Tajikistan. 2013. Health Policy Project, Desk Review of HIV-related Policy Recommendations for Kazakhstan, Kyrgyzstan and Tajikistan (2007-2012). Draft. 2013. Health Policy Project, Participatory Assessment Findings Report. Draft. 2013. 41 Health Policy Project, Proposed options paper for activities with USAID/Central Asia Region. Draft Final. 2013. ICAP, 2011 Epidemiological Profile for HIV/AIDS. Kazakhstan. 2011. ICAP, Country HIV Epidemiological Profile. Kyrgyzstan. 2011. ICAP, Country HIV Epidemiological Profile. Tajikistan. 2011. ICAP, Country HIV Epidemiological Profile. Uzbekistan. 2011. Population Services International, TRaC Summary Report. Central Asian Republics (2010): HIV and TB TRaC Study Evaluating Risk Behaviors Associated with HIV Transmission and Utilization of HIV Prevention and HIV/TB Co-Infection Prevention among IDUs. Round One. 2010. Population Services International, TRaC Summary Report. Kazakhstan, Kyrgyzstan and Tajikistan (2010): HIV and TB TRaC Study among Men who have Sex with Men in Almaty, Bishkek and Dushanbe. Round 1. 2010. Population Services International, TRaC Summary Report. Central Asian Republics (2010): HIV and TB TRaC Study Evaluating Risk Behaviors Associated with HIV Transmission and Utilization of HIV Prevention and HIV/TB Co-Infection Prevention among Female Sex Workers. Round One. 2010. Population Services International, TRaC Summary Report. Kazakhstan, Kyrgyzstan and Tajikistan (2012): HIV and TB TRaC Study among Men who have Sex with Men in Almaty, Bishkek and Dushanbe. Round 2. 2012. Population Services International, Sex Workers ADARA Model. Undated. Population Services International, MSM LaSky – Trusting each other Model. Undated. Population Services International, Prisoners START Plus Model. Undated. Population Services International, PLWH UNISON Model. Undated. Population Services International, IDUs Break the Cycle Model. Undated. Population Services International, Using data stored in Management Information System to improve tuberculosis treatment adherence among people living with HIV in Central Asia. 2013. (PowerPoint presentation to AEA Conference, 2013.) USAID/Central Asian Republics Mission, Contract between USAID/CAR and Abt Associates. 2010. USAID/Central Asian Republics Mission, Modification of Contract between USAID/CAR and Abt Associates. 2013. USAID/Central Asian Republics Mission, Cooperative Agreement between USAID/CAR and Population Services International. 2009. USAID Dialogue on HIV and TB Project, Life of Project Results (September 30, 2009 – December 31, 2013), March 11, 2013. (PowerPoint presentation to the Evaluation Team.) USAID Dialogue on HIV and TB Project, Annual Performance Report: Year 3. 2012 USAID Dialogue on HIV and TB Project, Annual Performance Report: Year 4. 2013 USAID Dialogue on HIV and TB Project, Consolidated Work Plan, Year 5. 2013. USAID Dialogue on HIV and TB Project, Performance Monitoring Plan. 2013. USAID Quality Health Care Project, Annual Report: Year 3. 2013. 42 USAID Quality Health Care Project, Quarterly Report: October-December 2013. 2014. USAID Quality Health Care Project, Work Plan: Turkmenistan Year 4. 2014. USAID Quality Health Care Project, Work Plan: Kazakhstan Year 4. 2014. USAID Quality Health Care Project, Work Plan: Kyrgyzstan Year 4. 2014. USAID Quality Health Care Project, Work Plan: Tajikistan Year 4. 2014. USAID Quality Health Care Project, Work Plan: Uzbekistan Year 4. 2014. USAID Quality Health Care Project, Kazakhstan Overview of HIV/AIDS Activities. Undated. USAID Quality Health Care Project, Work Plan: Year 4. 2014. USAID Quality Health Care Project, ‘Kazakhstan HIV Component’. March 11, 2014. (PowerPoint presentation to the Evaluation Team.) USAID Quality Health Care Project, ‘Kyrgyzstan HIV Component’. March 17, 2014. (PowerPoint presentation to the Evaluation Team.) USAID Quality Health Care Project, Whole of Project Presentation. March 11, 2014. (PowerPoint presentation to the Evaluation Team.) USAID Quality Health Care Project, The HIV Component of Quality Health Care Project in Tajikistan. March 12, 2014. (PowerPoint presentation to the Evaluation Team.) USAID Quality Health Care Project, AMEP Indicators Table. March, 2014. USAID Quality Health Care Project, Activity Monitoring and Evaluation Plan. 2013. USAID Quality Health Care Project, Achievements in HIV/AIDS 2011-2014. 2014. 43 Annex 3: Interview guides Interview guides were developed for each category of key informant to ensure a consistency of approach in interviews across the sub-teams. The questions were based on the questions specified in the Scope of Work which are listed in Annex 5 and tailored to the categories of key informants. The questions in bold typeface were the priority questions as some of the information asked in other questions was also available from other sources such as Project annual reports. USAID contractors – Dialogue and Quality 1. To what extent has your project achieved its expected HIV results? Key areas of achievement? Key areas of under-achievement? Checklist for prompting: a) Contribution to policy reforms? b) MoH’s national HIV and health strategies? c) HIV services: Quality, coverage and efficiency? d) Impact on health systems? e) Linkage and referral systems? 2. Where HIV results have been achieved, what’s has facilitated this? What have the enabling factors been? (Explore effective approaches to programming in CAR; what approaches have worked well?) 3. Where HIV results have been less than expected, what have been the major systems challenges, barriers and constraints? How have these impacted on achievement of results and how have you responded? 4. How do you go about promoting and measuring quality in your work and the work of your sub-contractors? How do you respond when there are concerns regarding quality? 5. How has your project incorporated gender strategies into its HIV activities and what have been the results? Did you find the Gender Assessments conducted by AIDSTAR to be helpful and if so, how did these Assessments influence your work? 6. Can you outline the approach your project has taken to capacity building of HIV partners and the key areas where your project has enhanced local partner capacity? (Check for technical, program and organizational capacity). Did you find the NGO capacity assessments conducted by AIDSTAR to be helpful and if so, how did these Assessments influence your work? 7. How has data produced by your project informed your project’s HIV programming and the HIV programming of partners? 8. What evidence is there of effective partner collaboration which has improved the quality and, coverage and impact of HIV programming? How could collaboration be improved? What is your perspective on the division of technical/geographic areas among partners? (Within USAID; between USG; among GF, GIZ?) 9. Are there any noteworthy areas of synergy and duplication across USG-funded and other donor funded HIV activities? 10. How does your project go about determining and prioritizing beneficiary needs? To what extent are you effectively addressing beneficiary needs? (Key gaps?) 11. What have been the key lessons learned from this project and more broadly USAID CAR’s HIV investments? (What would you do differently if you had your time over again?) 44 12. To what extent has the USAID program been strategic in addressing the key HIV epidemiological priorities and key programming needs, including important gaps? 13. What is the potential for sustainability of current USAID HIV programming and what strategies should be adopted to promote sustainable HIV programs for key populations? 14. What should be the scope and priority focus areas for USAID investment in Central Asia over the next 5 year? (More of? Less of? Differently?) Checklist for prompting: a) Prevention programming for KAPs? b) Improving HIV care? c) Improving access to treatment? Technology or innovative approaches? d) Systems strengthening: including NGO capacity building and strategic information? 15. Checklist for specific areas of future programming: a) Policy priorities? b) Improvements to coverage and quality of services, especially HCT? c) More cost effective/lower unit cost interventions? d) Gender specific activities? e) Surveys or special studies needed to guide strategic programming? f) Collaboration with the private sector? 16. How effective has USAID’s management of its HIV portfolio been? What improvements could be made to how USAID manages the program? Have there been any missed opportunities? NGOs funded/receiving TA from Dialogue and Quality 1. Thinking about the HIV work you have been doing with USAID funding, what have been your major achievements over the last 3-4 years? 2. What have been the major challenges, barriers and constraints encountered over the last 5 years. How have these effected your work and what you have been able to achieve? How have you responded to these challenges? (Barriers to HCT?) 3. How well are existing referral systems and service linkages working? Areas for improvement? 4. What other HIV projects do you work with? How do you work with these projects and is there effective collaboration? How could you improve your work with these projects? 5. What kind of funding do you have in addition to USAID’s funding to support your activities? 6. What unique services does your NGO offer? How does it contribute to overall HIV programming in this area? 7. How do you go about improving the quality of your work? What assistance do you get from Quality/Dialogue on measuring quality and how to improve quality? 8. What types of technical assistance and capacity building do you receive from Dialogue/Quality? How effectively is this support? Are there areas where their support 45 could be improved? Are there types of TA and capacity building support you would like to get but which are not currently available? 9. Do you see gender-related challenges in your HIV programming? If yes, how has your project addressed gender issues in your HIV activities and what have been the results? 10. What do you do with the data you collect? (probe for analysis and use in program improvement) 11. Thinking of the needs of your target groups, are there any important unmet needs or gaps in services? These gaps or unmet needs might be HIV-related or for broader health and social service needs. 12. In addition to evaluating USAID’s HIV program we need to make recommendations about USAID’s future programming. What should be USAID’s priorities in HIV programming over the next 5 years? Supplementary: What should USAID be doing more of, less of and differently? Checklist for prompting: a) Prevention programming for KAPs? b) Improving HIV care? c) Improving access to treatment? d) Systems strengthening: including NGO capacity building and strategic information? e) Policy priorities? f) Improvements to coverage and quality of services, especially HCT? g) More cost effective/lower unit cost interventions? h) Gender specific activities? i) Surveys or special studies needed to guide strategic programming? j) Collaboration with the private sector? Additional questions for NGOs working in democracy and governance 1. How do you collaborate with HIV service organizations? 2. What challenges do you see in providing services to KPs? 3. How could you contribute to improvement of HIV service delivery? What would enable you to do this work? Government agencies, multilaterals and other donors 1. (Both Government and donors) -What have been the major achievements over the last 3-4 years of the USAID funded HIV projects? (Quality, Dialogue, GMS, AIDSTAR re Gender & NGO Capacity assessments & HPP). Checklist for prompting: a) Contribution to policy reforms/implementation? b) MoH’s national HIV and health strategies? c) HIV services: Quality, coverage and efficiency? d) Impact on health systems? e) Linkage and referral systems? f) Capacity building with NGOs/CCMs? 46 2. (Both Government and donors) - What have been the major challenges, barriers and constraints encountered over the last 5 years by USAID funded HIV programs and how has this effected results? How effectively has USAID and its implementing partners responded to these challenges, barriers and constraints? 3. (Both Government and donors) - Are there any noteworthy areas of duplication across USAID funded and other donor funded HIV activities? 4. (Both Government and donors) - In what ways does USAID and its implementing partners collaborate with your agency and others? How effective is this collaboration – what’s been achieved? Are there ways collaboration could be improved? 5. (Both Government and donors) - What are the key areas where strategic information needs to be improved? How would this improved strategic information be used? 6. (Both Government and donors) - What have been the key lessons learned from this project and more broadly USAID CAR’s HIV investments? 7. (Both Government and donors) - To what extent has the USAID program been strategic in addressing the key HIV epidemiological priorities and key programming needs, including important gaps? 8. Government - How is HIV funding tied to your national strategy? 9. Donors - What should be the scope and priority focus areas for USAID investment in Central Asia over the next 5 years? (More of? Less of? Differently?) Checklist for prompting: a) Prevention programming for KAPs? b) Improving HIV care? c) Improving access to treatment? d) Systems strengthening: including NGO capacity building and strategic information? e) Policy priorities? f) Improvements to coverage and quality of services, especially HCT? g) More cost effective/lower unit cost interventions? h) Gender specific activities? Or gender approaches in programs? i) Surveys or special studies needed to guide strategic programming? 10. Donors - What are your plans over the next 5 years and how do you see USAID complementing or leveraging your programs? 11. Donors - How sustainable do you think USAID funded HIV prevention programs for KAPs are? What could be done to make these programs more sustainable? Key populations 1. How do you know the organization? Can you tell us a bit about your experiences with the outreach and other services (i.e., which services of this organization do you use?)? 2. How often do you use these services (i.e., see outreach workers or receive referrals)? 3. How do you find the staff and volunteers? 47 4. Would you say you are satisfied or dissatisfied with the services here? Why? 5. How could the services be improved? 6. Are there any other HIV, STI and other health and social services you use? Does this service refer you to the other services? How does that work? Do you think the referral system is effective? Are you satisfied with those services? How do they compare to this service? (For PLHIV, refer to HIV treatment services) 7. What kinds of barriers do you or others face in accessing HIV, STI, and other health and social services (e.g., S&D)? Do you have ideas of how these barriers could be addressed? 8. Why do you think people may not be referred for services or do not use referrals? Do you think the right people are being reached with these services? 9. Are there any services you need that are not provided here and which are not available anywhere? What are the type of services you need that are not available? 10. Explore issues around HIV testing: where testing is available; time taken to get results; attitudes of staff at HCT sites; barriers to testing; suggestions for how to promote HIV testing. 11. Is there anything else you’d like to share with us? Community Advisory Boards and Locality Coordination Councils 1. How have CABs and CCs made a difference? Specific achievements? 2. Clarifying the mandate/role and membership of CABs and CCs and meeting frequency. 3. Finding out what role of USAID funded CAs had in establishment and support (initial and ongoing) of the CAB and CC. 4. Do CAB and CC members (especially KAP/community members) feel they have been adequately trained/supported in how to make effective contributions? Any areas of additional training/support needed? 5. Are there any operational challenges related to how these bodies function? Is there adequate secretarial support? 6. Do decisions/recommendations of the CAB and CC get implemented? What’s the process for this? 7. Strengths and weaknesses of the CABs and CCs. How could they be improved? 8. Sustainability of CABs and CCs without ongoing USAID support? 9. How have the CC and CAB models of civil society and government working together been accepted by the stakeholders involved? 48 Annex 4: Evaluation schedule Table 1: Evaluation schedule Date Task Deliverable March 1-9  Team members review background and program documents: desk review  Develop list of interviewees and schedule for interviews  Design of interview guides for key informant interviews Interview guides March 10 Team meets in Almaty: briefing and planning meeting Finalized interview guides March 11-19 Key informant interviews and site visits: Team 1: Kazakhstan: March 11-14 Kyrgyzstan: March 17-19 Team 2: Tajikistan: March 11-14 Uzbekistan: March 17-19 March 20: both Teams returned to Almaty and completed writing up interview notes Interview notes March 21-24 Team meeting to review all data and develop preliminary findings as a basis for writing the draft evaluation report March 25-27 Team members writing the evaluation report March 28  Debriefing of evaluation preliminary findings for USAID/CAR, USAID Country Offices and USAID/Kyrgyzstan  Final team meeting to consider feedback PowerPoint presentation March 29-April 6  Further writing of the evaluation report, revisions and editing  Submission of draft Evaluation Report to USAID/CAR Draft evaluation report due April 6 April 7-9 Review of draft evaluation report by USAID/CAR, Country Offices and USAID/Kyrgyzstan April 9 USAID feedback on 1st draft of evaluation report April 10-17 Revisions to draft evaluation report in response to USAID feedback April 17 Second draft of evaluation report submitted to USAID/CAR 2nd draft of evaluation report due April 17 April 24 USAID feedback on 2nd draft of evaluation report April 29 Final evaluation report submitted to USAID/CAR Final evaluation report due April 29 49 Annex 5: Organizations consulted and site visits Table 2: Organizations consulted and site visits Regional level interviews Janel Wright, Acting Director, CDC/CAR Anna Deryabina, Director, ICAP/CAR Nedim Jaganjac, Senior Health Specialist, World Bank, Kazakhstan David Hoffman and Christopher Miller, Democracy and Governance Office, USAID/CAR Kazakhstan Health care workers at Polyclinic #2, Ust-Kamenogorst Government Meeting: Republican AIDS Center, Almaty City AIDS Center, National TB Center, Interdistrict TB Dispensary, Almaty City Narcological Dispensary, Department of Penitentiary, Almaty USAID Dialogue on HIV and TB Project, Almaty USAID Quality Health Care Project, Almaty ADALI: NGO sub-grantee of the Dialogue Project working with MSM, Almaty Development partners meeting: GFATM, UNDP, UNFPA, UNICEF, UN Women, Almaty Coordinator and members of the Community Advisory Board, MAT patients and health care workers, Ust-Kamenogorst KUAT: NGO sub-grantee of the Dialogue Project working with PWID and PLHIV, Ust-Kamenogorst Members of the Community Advisory Board, Almaty Site visit: Focus group discussion with female sex workers and outreach workers, Almaty Kyrgyzstan Government meeting: Bishkek City AIDS Center, Republican Narcology Center, Republican AIDS Center, Ministry of Health, and Center of Venereology and Dermatology, Bishkek Multilateral partners meeting: UNICEF, Soros Foundation, UN Women, UNFPA, UNAIDS, UNODC, UNDP, Bishkek CDC and ICAP, Kyrgyzstan, Bishkek USAID Dialogue on HIV and TB Project, Bishkek USAID Quality Health Care Project, Bishkek Focus group meeting with female sex workers, Sokuluk Meeting with outreach workers employed directly by Dialogue, Sokuluk Antisped: NGO sub-grantee of the Dialogue Project working with MSM, Bishkek Sotsium: NGO sub-grantee of the Dialogue Project working with PWID, Bishkek Antistigma: NGO sub-grantee of the Dialogue Project working with PWID and PLHIV, Kant Family Medical Centre HCWs #16, Bishkek MDT Team HCWs, Polyclinic, Kant AIDS Center HCWs, Osh Plus Center, NGO sub-grantee of the Dialogue Project working with PWID and PLHIV, Osh Alterniva: NGO sub-grantee of the Quality Project working with PWID and PLHIV and through the CAB, Osh Tajikistan Republican AIDS Center, Dushanbe Centers for Disease Control/ICAP, Dushanbe USAID Dialogue on HIV and TB Project, Dushanbe USAID Quality Health Care Project, Dushanbe Legal Support: NGO sub-grantee of the Dialogue Project working with MSM, Dushanbe SPIN Plus: NGO sub-grantee of the Dialogue Project working with PWID and PLHIV, Dushanbe AntiSPID: NGO sub-grantee of the Dialogue Project working with FSW, Khujant 50 Hayety Nav: NGO sub-grantee of the Dialogue Project working with prisoners, Khujant PLHIV members of the Community Advisory Board, Dushanbe Members of the Locality Coordination Council, Dushanbe Institute of Migration, a sub-grantee of the Quality Project, Dushanbe Multilateral partners meeting: UNAIDS and WHO, Dushanbe Site visit: MSM outreach at the Dushanbe Zoo Uzbekistan Government meeting: Republican AIDS Center, Ministry of Foreign Affairs, Ministry of Health, National Network of NGOs of Uzbekistan, Tashkent USAID Dialogue on HIV and TB Project, Tashkent Development partners meeting: CDC, MSF, UNDP, UNODC, WHO, Tashkent Global Fund Portfolio Manage, Uzbekistan and Tajikistan Note: meetings in Uzbekistan were limited due a lack of permission from the Government to meet with stakeholders 51 Annex 6: Site maps 52 53 54 55 56 Annex 7: NGO sub-awardees for the Dialogue Project Table 3: Dialogue Project NGO sub-awardees Kazakhstan # NGO Sub-Awardees Key population 1 As. Of Legal Entities "CAAPL with HIV" PLHIV 2 PF "Adali" MSM 3 PA "SPLWHIV "Kuat" PWID/ PLHIV 4 PF "Mental Health" PLHIV 5 PA "Umit" PWID 6 PF "KU PLWH" in RK PLHIV 7 PA "Sau Urpak" FSW 8 PCF "Shapagat" PLHIV 9 RPATAT "Ar Namys" PWID 10 PF "Omir LAD" PWID Kyrgyz Republic # NGO Sub-Awardees Key population 1 PA "Kyrgyz Family Planning Alliance" Migrants 2 PA "Kyrgyz Indigo" MSM 3 PA "Antistigma" PLHIV 4 PA "Pravo na jizn" PWID 5 PA "Sotsium" PWID 6 PF "Plus Centre" PWID; PLHIV 7 PF "Zdorovoe Pokolenie" PWID; PLHIV 8 PA Association "AntiAIDS" MSM 9 PF "Podruga" FSW 10 PA "Tais Plus two" FSW 11 PF "Roditeli protiv narkotikov" PWID 12 PF Rans Plus Prisoners 13 Harm Reduction network Prisoners 14 PA"Zadmir" Migrants Tajikistan # NGO Sub-Awardees Key population 1 NGO SPIN plus PLHIV, PWID 2 NGO Marvorid FSW 3 NGO Legal Support MSM 4 NGO Fidokor PWID, FSW 5 NGO Anis PLHIV, PWID, FSW, Migrants 6 NGO Dina PWID 7 NGO AntiSPID FSW 8 NGO Nakukor PWID 9 NGO Volunteer PLHIV, PWID 10 NGO RAN PWID 57 11 NGO Akhtari Bakht Migrants 12 NGO Sino Prisoners 13 NGO Vita Prisoners 14 NGO Hayoti Nav Prisoners 58 Annex 8: Dialogue Project performance data Key populations reached by prevention interventions Figure 5: Number of individuals reached with Dialogue small group or individual HIV prevention interventions by key populations, regional coverage (CAR): Cumulative Years 1 – 4. Source: MIS Figure 6: Number of individuals reached with Dialogue small group or individual HIV prevention interventions by year and by key populations, PLWH and prisoners, Regional coverage (CAR): Years 1-4. Source: Dialogue Annual Report, Year 4. 0 2000 4000 6000 8000 10000 12000 14000 16000 Year1 Year 2 Year 3 Year 4 PWID SW MSM Prisoners PLWH # KP Reached in CAR in Years 1‐4. 59 Referrals and redemptions for HIV testing by key population Figure 7: Number of PWID referred by Dialogue for HIV counselling and testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS Figure 8: Number of FSW referred by Dialogue for HIV counselling and testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS 0 1000 2000 3000 4000 5000 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemed Number of referrals made PWID HIV Testing 0 1000 2000 3000 4000 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemeD Number of referrals made FSW HIV testing 60 Figure 9: Number of MSM referred by Dialogue for HIV counselling and testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS Figure 10: Number of Prisoners referred by Dialogue for HIV counselling and testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS 0 200 400 600 800 1000 1200 1400 1600 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemed Number of referrals made MSM testing and counseling for HIV 0 200 400 600 800 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemed Number of referrals made Prisoners Testing & Counseling for HIV 61 Referrals and redemptions for TB testing by key population Figure 11. Number of PWID referred by Dialogue for TB testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS Figure 12. Number of PLHIV referred by Dialogue for TB testing in CAR: Referrals vs redeemed, Years 1-4. Source: MIS 0 500 1000 1500 2000 2500 3000 3500 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemed Number of referrals made PWID Referral vs. Redeemed for TB Tests 0 500 1000 1500 Year 1 Year 2 Year 3 Year 4 Number of referrals redeemed Number of referrals made PLHIV Referred vs Redeemed for TB Tests 62 Figure 13. Number of prisoners referred by Dialogue for TB testing in CAR: Referrals vs redeemed Years 3 and 4 only (no program or data years 1-2). Source: MIS HIV knowledge by key populations Figure 14. Percent of PWIDs who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission: Turkmenistan and Uzbekistan. Source: TRaC 0 100 200 300 400 Year 3 Year 4 Number of referrals redeemed Number of referrals made Prisoners Referred vs Redeemed for TB Testing. Years 3 and 4 only. 63 Figure 15. Percent of PLHIV in CAR who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV. Source: TRaC. Figure 16. Percent of prisoners who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV: Kazakhstan, Kyrgyz Republic and Tajikistan. Source: TRaC 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Year 2Year 3 KZ KG TJ UZ Percent of PLWH knowledge of HIV in four CAR countries. 64 Annex 9: Quality Project performance data The source of data in this Annex is the Quality Project’s AMEP Indicators Table, which was submitted to USAID in March, 2014. The performance indicators are for the HIV component of Quality’s work. Given the nature of Quality’s work, there are less performance indicators than is the case for Dialogue. Consequently, this evaluation’s assessment of Quality’s work has primarily been qualitative. Table 4: USAID Quality Health Care Project’s AMEP Indicators Years 1-5 # Performance Indicator Country Baseline Actual Year 1 Actual Year 2 Actual Year 3 Target Year 4 Target Year 5 1 Number of pilot or demonstration sites at which public and NGOs collaborate to improve access to services by key populations KZ 0 0 3 5 7 9 KG 0 0 7 9 11 11 TJ NA NA 3 11 14 16 2 Effectiveness of Community Advisory Boards KZ See note KG See note TJ See note 3 Number of individuals who received testing and counselling services for HIV and received their test results (TA only) KZ NA NA NA 1803 1200 1300 KG NA NA NA 2801 2500 2500 TJ NA NA NA 1080 600 700 4 Number of sites offering HIV rapid tests for KPs KZ NA NA NA NA NA NA KG NA NA NA NA NA NA TJ NA NA 3 6 6 6 5 Progress towards adoption of national Comprehensive Package of Services for PWID, FSW and MSM KZ - - - PWID: Stage 3e MSM: Stage 3e FSW: Stage 1a PWID: Stage 4a MSM: Stage 4a FSW: Stage 2d KG - - - PWID: Stage 3e MSM: Stage 3e FSW: Stage 3e PWID: Stage 4c MSM: Stage 4c FSW: Stage 4c TJ - - - PWID: Stage 4a MSM: Stage 3e FSW: Stage 2b PWID: Stage 5a MSM: Stage 5a FSW: Stage 4a 6 Number and percentage of NGOs that received funding from government after training provided by the Quality Project KZ - - - 16 (38%) 18 18 KG NA NA NA NA NA NA TJ NA NA NA NA NA NA 7 Number of HCWs who successfully completed an in￾service training KZ 0 0 464 319 200 100 KG 0 0 660 499 500 200 TJ NA NA 777 1137 (360) 1400 (263) 1150 (150) TM - - 548 500 - - 8 Number of NGOs showing increased capacity in core KZ NA NA NA - - TBD KG NA NA NA - - TBD 65 functions related to HIV programming TJ NA NA NA - - TBD Notes: Indicator 1: This indicator reports on the number of Community Advisory Boards established with Quality’s assistance. Indicator 2: This is a qualitative indicator. Quality has submitted matrices to USAID for this indicator. Indicator 3: Targets for Years 4 and 5 are lower because this new indicator is replacing a previous PEPFAR indicator: ‘Number of individuals who received HCT at USG pilot sites, which are implementing improved HCT’. Indicator 4: In Kazakhstan, there is no national policy on rapid testing for NGOs because these tests are not registered in the country. In Year 4, there are pilot preparation activities for NGO rapid testing. No roll-out is planned. In Kyrgyzstan, all rapid testing activities at the NGO level in Years 3-5 have been cancelled and no roll out is planned. Indicator 5: Data is collected using OGAC’s Stages of Reform tracking criteria to track the progress of each Comprehensive Package of Services. The stages are described in the Table below. Table 5: Method of measurement for Quality Project Indicator 5 Stag e Description Steps 1. Identify baseline policy issues by conducting situation assessment a. Policy analysis research conducted b. Relevant stakeholders identified c. Stakeholders involved and engaged d. Situation assessment implemented e. National deliberative body (or individual) for policy change identified f. Assessment report available as baseline 2 Engagement of stakeholders in developing common policy agenda a. Ongoing stakeholder participation b. Policy dialogue and advocacy c. Specific policy issues to be addressed defined d. White paper, or equivalent defining the policy issues/problems, and response completed 3 Develop policy a. Policy and strategy developed b. Implications of proposed policy with existing legal, policy, and regulatory environments assessed c. Operational barriers identified d. Operational policy issues integrated into policy draft e. Jointly drafted formal/vetted policy text circulated amongst stakeholders 4 Official government endorsement of policy a. Leadership engagement/mobilization b. Revise draft policy accordingly c. Government act/approval making policy official (e.g. passage, endorsement, publication) 5 Implementation of policy a. Costed action/implementation plan developed b. Dissemination; awareness raising and education activities c. Strategy implementation/capacity strengthening activities carried out d. Accountability measures/monitoring plan for implementation determined e. Resources to support implementation (resource mobilization) provided Indicator 7: For Tajikistan, data are cumulative from the beginning of the Project, with the number of new HCWs trained in a year in brackets. 66 Indicator 8: This indicator can be measured only in Year 5. It was not included in the work plan for Year 4 due to the late receipt of the AIDSTAR NGO assessments. Targets for Year 5 will be set in Year 4. 67 68 Annex 12: Endnotes 1 Hereafter referred to as USAID/CAR, reflecting the regional nature of the USAID program. 2 This evaluation includes only USAID managed activities under PEPFAR and not those of CDC. 3 ICAP, 2011 Epidemiological Profile for HIV/AIDS Kazakhstan. 2011 and CDC Country Epidemiological Profiles for the Kyrgyz Republic, Tajikistan and Uzbekistan. 2011. 4 PEPFAR/CAR Regional Operation Plan, 2014. p.2. Unless otherwise noted, all epidemiological data in the section is drawn from the ROP. 5 Republican AIDS Center, Kazakhstan, Report on AIDS Service Activities for 2012. 2012. 6 Kyrgyz Republic Republican AIDS Centre, PowerPoint presentation to the USAID/CAR HIV/AIDS evaluation team, March, 2014. 7 PEPFAR/CAR Regional Operation Plan, 2014. p.2. Unless otherwise noted, all epidemiological data in the section is drawn from the ROP.. 8 ICAP, 2011 Epidemiological Profile for HIV/AIDS Kazakhstan. 2011. 9 AIDSTAR-One is a global USAID project providing field support and technical assistance to teams working in different countries in the areas of prevention, treatment, HIV counseling and testing, care and support, gender, orphans and vulnerable children and prevention of mother to child transmission. 10 AIDSTAR-Two is a global USAID project that contributes to stronger, more sustainable, country-led HIV/AIDS programs by offering systematic organizational capacity building assistance. The project supports local NGOs, CSOs, public sector institutions and networks that work with the United States President’s Emergency Plan for AIDS Relief (PEPFAR) to maximize the impact of HIV/AIDS programs. 11 Grant Management Solutions is a PEPFAR funded field support mechanism that provides short-term technical support to enhance the efficiency and effectiveness of the Global Fund grants by helping country coordinating mechanisms (CCMs), principal recipients (PRs) and sub recipients (SRs) unblock bottlenecks to implementation, and resolving systemic problems impeding effective governance and grant performance. 12 The Health Policy Project is a global USAID project providing field support to strengthen developing country national and subnational policy, advocacy, governance, and finance for strategic, equitable, and sustainable health programming. HPP works with in-country partners to sustain commitment to and ownership of policy responses that support improvements in family planning and reproductive health, HIV, and maternal health. HPP builds the capacity of governments, civil society, and other local organizations to advocate for and implement policies and plans that shape health services, systems, and budgets - from policy formulation to implementation and monitoring. 13 See map of project sites in Annex 6. 14 In Tajikistan, both prisoners and migrants have been retained in the Dialogue Project in Y4/5 as KPs for TB screening and treatment adherence support through use of TB funds. 15 AIDSTAR-Two, Mid-Term Evaluation of the Dialogue Project Central Asian Republics. 2012 16 Epidemiology was the primary reason for this. Prisoners and migrants at greater risk for HIV who were practicing high-risk behaviors were captured in programs aimed at FSWs, MSM and PWID. 69 17 See map of project sites in Annex 6. 18 USAID/Central Asian Republics, Regional Development Cooperation Strategy 2014 – 2018. 19 PEPFAR Fiscal Year 2014 Country Operational Plan (COP) Guidance (October 2013) 20 Central Asia Operational Plan Report FY2013, http://www.pepfar.gov/documents/organization/222160.pdf 21 USAID Dialogue on HIV and TB Project, Annual Performance Report: Year 4. pp 106-113. 22 Overview of TRaC - Tracking Results Continuously, http://www.psi.org/trac 23 USAID Dialogue on HIV and TB Project: Performance Monitoring Plan (PMP) updated: CE _September 2013 24 Messner, Lyn and Tatiana Kazantseva. 2013, Gender Assessment: Access to HIV Services for Key Populations in Kyrgyzstan. Arlington, VA: USAID’s AIDS Support and Technical Assistance Resources, AIDSTAR-One, Task Order 1. 25 USAID Dialogue on HIV and TB Project, Annual Performance Report: Year 4. 26 Dialouge MIS data. 27 USAID Dialogue on HIV and TB Project, Financial Year 2014 Workplan. 28 USAID Dialogue on HIV and TB Project, Annual Performance Report: Year 4. pp 108-109. 29 USAID Dialogue on HIV and TB Project, Financial Year 2014 Workplan. 30 UNAIDS Report on the Global AIDS Epidemic (2012). 31 USAID Dialogue on HIV and TB Project. LOP Results, (September 30, 2009 – December 31, 2013). PowerPoint Presentation to USAID's HIV Program Evaluation Team, March 11, 2014, Almaty. Note: this data is to December 31, 2014 whereas the data in Figure 4 is to 30 September 2013. 32 Dialogue Project’s “UNISON” Model description 33 Schluger NW1 et al. Tuberculosis, drug use and HIV infection in Central Asia: an urgent need for attention. Drug Alcohol Depend. 2013 Nov;132 Suppl 1:S32-6. doi: 10.1016/j.drugalcdep.2013.07.012. Epub 2013 Aug 6. 34 Boltaev A. et al., Assessment of medication assisted therapy in the Republic of Tajikistan, ICAP/Columbia University, 2013, Almaty 35 Grenfell P et al. Tuberculosis, injecting drug use and integrated HIV-TB care: a review of the literature. Drug Alcohol Depend. 2013 May 1;129(3):180-209. 36 Quality Health Care Project, Annual Report October 2013 37 Dialogue PMP. 38 Quality Quarterly Report, Year 4, Quarter 1, page 65. 39 Quality Quarterly Report, Year 4, Quarter 1, page xiv. 40 Quality Quarterly Report, Year 4, Quarter 1, pages 62-63. 70 41 Quality Quarterly Report, Year 4, Quarter 1, pages 5-6. 42 Quality quarterly report, year 4, quarter 1 page 37-38. 43 AIDSTAR-Two, Central Asian Capacity Building Strategy Project, Kazakhstan. 2013. p. 5. 44 Quality Quarterly Report, Year 4, Quarter 1, pages 7-8