May 2011 by David Lowe, Independent Consultant & Team Leader; Tonya Himelfarb, Independent Consultant; Jenne Roberts, Independent Consultant; Billy Pick, USAID/Washington; Sarah Berk, USAID/Washington EVALUATION HIV/AIDS Portfolio Evaluation USAID/Cambodia Public Version Final Report credit i HIV/AIDS Portfolio Evaluation USAID/Cambodia PUBLIC VERSION Final Report by David Lowe, Independent Consultant & Team Leader Tonya Himelfarb, Independent Consultant Jenne Roberts, Independent Consultant Billy Pick, USAID/Washington Sarah Berk, USAID/Washington July 2011 ii Contents ACRONYMS ........................................................................................................................................................ III EXECUTIVE SUMMARY ................................................................................................................................... V EVALUATION CRITERIA AND RATINGS ..................................................................................................IX 1. INTRODUCTION....................................................................................................................................... 1 1.1 PURPOSE OF THE EVALUATION AND SCOPE OF WORK.............................................................................. 1 1.2 SUMMARY OF METHODOLOGY................................................................................................................ 1 1.3 EVALUATION TEAM ................................................................................................................................ 2 2. THE CONTEXT........................................................................................................................................... 3 2.1 THE HIV/AIDS EPIDEMIC IN CAMBODIA ............................................................................................... 3 2.2 OVERVIEW OF THE CAMBODIAN NATIONAL RESPONSE TO HIV/AIDS.................................................... 3 2.3 OVERVIEW OF THE USAID/CAMBODIA HIV/AIDS PORTFOLIO ............................................................. 5 3. PREVENTION ............................................................................................................................................. 8 3.1 OUTLINE OF THE PREVENTION PROGRAM................................................................................................ 8 3.2 KEY ACHIEVEMENTS: ............................................................................................................................. 9 3.3 STRENGTHS AND WEAKNESSES............................................................................................................. 13 3.4 REPLICATION AND SUSTAINABILITY ..................................................................................................... 17 4. CARE AND TREATMENT ...................................................................................................................... 19 4.1 OUTLINE OF THE CARE AND TREATMENT PROGRAM ............................................................................. 19 4.2 KEY ACHIEVEMENTS ............................................................................................................................ 20 4.3 STRENGTHS AND WEAKNESSES............................................................................................................. 21 4.4 CHALLENGES........................................................................................................................................ 23 4.5 REPLICATION AND SUSTAINABILITY ..................................................................................................... 24 5. STRATEGIC INFORMATION................................................................................................................ 25 5.1 MONITORING AND EVALUATION........................................................................................................... 25 5.2 SURVEILLANCE .................................................................................................................................... 26 5.3 RESEARCH............................................................................................................................................ 27 6. HEALTH SYSTEM STRENGTHENING ............................................................................................... 29 6.1 HEALTH SERVICE DELIVERY................................................................................................................. 29 6.2 HEALTH WORKFORCE........................................................................................................................... 31 6.3 HEALTH INFORMATION SYSTEMS.......................................................................................................... 31 6.4 ACCESS TO ESSENTIAL MEDICINES........................................................................................................ 31 6.5 HEALTH SYSTEMS FINANCING .............................................................................................................. 32 6.6 LEADERSHIP AND GOVERNANCE........................................................................................................... 32 7. USAID/CAMBODIA MANAGEMENT OF THE HIV/AIDS PORTFOLIO....................................... 34 7.1 OVERVIEW ........................................................................................................................................... 34 ANNEX 1: SCOPE OF WORK .................................................................................................................. 36 ANNEX 2: METHODOLOGY ................................................................................................................... 38 ANNEX 3: INTERVIEW GUIDES ............................................................................................................ 42 ANNEX 4: EVALUATION SCHEDULE: KEY DATES, TASKS AND DELIVERABLES................. 47 ANNEX 5: OPHE STRATEGIC FRAMEWORK FOR HEALTH 2009 -2013 ..................................... 49 ANNEX 6: KEY INFORMANTS AND SITE VISITS.............................................................................. 50 ANNEX 7: BIBLIOGRAPHY..................................................................................................................... 54 iii Acronyms AAD Activity Approval Document AIDS Acquired Immune Deficiency Syndrome ANC Antenatal Care ART Antiretroviral Therapy ATS Amphetamine Type Stimulant BCC Behavior Change Communication BSS Behavioral Surveillance Survey CA Cooperative Agency CDC GAP Centers for Disease Control and Prevention Global AIDS Program CHBC Community and Home Based Care CoC Continuum of Care CoPCT Continuum of Prevention to Care and Treatment CPICT Community/Peer Initiated Counseling and Testing CQI Continuous Quality Improvement CSDA Cambodian Socio-Economic Development and Democracy Association DTOP District Team on Outreach & Peer education DU Drug User (non-injecting) EW Entertainment Worker FP Family Planning FY Financial Year GFATM Global Fund to fight AIDS, Tuberculosis and Malaria HAARP HIV and AIDS Asia Regional Program HAART Highly Active Antiretroviral Therapy HCBC Home and Community Based Care HE His Excellency HEF Health Equity Fund HIS Health Information System HIV Human Immunodeficiency Virus HRUM High Risk Urban Male HSP II Health Sector Strategic Plan II HSS Health System Strengthening IBBS Integrated Biological and Behavioral Survey IDU Injecting Drug User IP Implementing Partner IPC Inter-Personal Communicator IR Intermediate Result KHANA Khmer HIV/AIDS NGOs Alliance LNGO Local Non-Government Organization MARP Most at Risk Populations MCH Maternal Child Health MHC Men’s Health Cambodia MoH Ministry of Health MSM Men who have Sex with Men MTCT Mother-to-Child Transmission NAA National AIDS Authority iv NACD National Authority to Combat Drugs NCHADS National Centre for HIV/AIDS, Dermatology and Sexually Transmitted Diseases NGO Non-Governmental Organization NSP III National Strategic Plan for Comprehensive and Multi-sectoral Response to HIV/AIDS III (2011-2015) in Cambodia OD Operational District OI Opportunistic Infection OPHE Office of Public Health and Education (USAID) OVC Orphans and Vulnerable Children PAO Provincial AIDS Office PEPFAR President’s Emergency Plan for AIDS Relief PHD Provincial Health Department PLHIV People Living with HIV PMP Performance Monitoring Plan PMTCT Prevention of Mother-to-Child Transmission PSI Population Services International/Cambodia RACHA Reproductive and Child Health Alliance of Cambodia RGC Royal Government of Cambodia RH Reproductive Health RHAC Reproductive Health Association of Cambodia SOP Standard Operating Procedure STI Sexually Transmitted Infection TA Technical Assistance TB Tuberculosis TLC+ Testing and Linkage to Care and Treatment TWG Technical Working Group UIC Unique Identifier Code UNAIDS Joint United Nations Program on HIV/AIDS URC University Research Corporation USAID United States Agency for International Development USG United States Government VCCT Voluntary Confidential Counseling and Testing WHO World Health Organization v Executive summary The purpose of this evaluation of USAID/Cambodia’s HIV/AIDS portfolio was to assess the performance of the current program and make recommendations to guide USAID investments in HIV/AIDS for the next five years. The scope of work specified two overarching questions: 1. How well have past Mission investments in HIV/AIDS performed? 2. How should the Mission invest resources in HIV/AIDS during the next five years? USAID technical assistance in Cambodia has a long track record of testing potential innovations and model development, piloting and adaptation. There are numerous examples of replication and scale up of USAID-developed models. These include replication of prevention program models developed by USAID, using Global Fund monies, and the national roll-out of a continuum of care (CoC) model, based on a USAID pilot. Nonetheless, Global Fund grants have had an insufficient emphasis on technical oversight and quality assurance. USAID continues to add value beyond its own programming in these areas. USAID/Cambodia has successfully developed the technical and organizational capacity of national and community level NGOs to be major contributors to prevention and care and treatment programming. This has been critical to program success. Some USAID Cooperating Agencies (CAs) have also contributed significantly to strengthening the capacity of the National Centre for HIV/AIDS, Dermatology and Sexually Transmitted Diseases (NCHADS) in the areas of surveillance, analysis of data and the development of guidelines and standard operating procedures (SOPs) for a range of HIV-related health services. Prevention Cambodia’s response to HIV/AIDS has been remarkably successful as evidenced by the significant reduction in HIV prevalence which has taken the country from a low-level generalized epidemic to a concentrated epidemic. It is estimated that adult HIV prevalence declined from 2.4% in 1998 to 0.8% in 2010. Prevention programming by civil society and government agencies, particularly early interventions focusing on female sex workers and widespread condom social marketing, appear to have had a significant effect in reducing the prevalence of HIV in Cambodia. USAID/Cambodia has been in the forefront of developing effective prevention models for most at risk populations (MARPs) and paving the way for this sensitive work to be accepted as core business in Cambodia’s national response to HIV/AIDS. Prevention programming has been responsive to trends in the epidemic, with programs targeting low-risk groups being phased-out, while MARPs programs have been scaled up. There have been significant increases in the number of MARPs reached by USAID funded prevention projects, although crackdowns on sex work and IDUs has subsequently limited the ability of prevention programs to reach key populations. There has been a doubling of the number of pregnant women tested for HIV in ante natal care settings with USAID support between 2008 and 2010. The inclusion of prevention within the CoC by development of the continuum of prevention to care and treatment model (CoPCT), coupled with a renewed emphasis on referral from prevention programs to HIV testing and counseling, has increased the number of MARPs tested. However, the number of MARPs presenting for testing is still too low and a substantial effort in demand creation is needed. In addition, entry points or opportunities for providing prevention interventions in the ‘care and treatment’ component of the CoPCT need to be better defined. vi Strengths of USAID’s prevention work include the appropriate focus on MARPs, with programming recognizing overlapping risks; a core package of evidenced based services is in place for MARPs; and the quality of prevention interventions has improved through development of standard packages of activities, development of innovative franchising models, and development of messaging based on operational research. Prevention programs working with IDUs have been facing an increasingly hostile environment which has restricted operations, including needle and syringe exchange. USAID has provided strong support for evidence based harm reduction programming, both through funding and advocacy to address current difficulties. Both USAID and its CAs have engaged in advocacy to address the policy and legal constraints increasingly facing prevention programs for MARPs. A key finding of the evaluation is that this work needs to be significantly enhanced. While there are good lines of communication between CAs there is room for greater collaboration and efficiencies in prevention programming through joint research, program design, trainings, supervision visits and minimization of overlap and duplication. Care and treatment The remarkable progress made by the Royal Government of Cambodia (RGC) in scaling up both the quality and coverage of care and treatment services for HIV/AIDS, including provision of antiretroviral therapy (ART), is in large part attributable to USAID financial and technical support. More recently, USAID advocacy has been key to the introduction of point-of-care HIV testing and counseling for MARPs which has been successfully trialed by USAID partners. More than 20% of all people (102,989) receiving HIV testing and counseling in FY 2010 did so at services supported by USAID. In the same year, 23,206 PLHIV were provided with at least one care and support service through USAID partners, representing more than half of the total number of people receiving ART. Support for quality clinical care is demonstrated by the change in focus to screening HIV-positive people for TB in a USAID-supported care and treatment setting achieving 51% in the first year of reporting (FY2010). Another key achievement is the initiation of efforts to move away from funding direct service provision by transitioning implementing partners (IPs) providing services to orphans and vulnerable children (OVC) to other funding sources. This shift will help to reduce the verticality of HIV/AIDS OVC services by focusing on integration with more sustainable RGC health and social service systems. USAID service providers have been instrumental in the shift from the pre-highly active antiretroviral therapy (HAART) high dependency model of home-based care to a more sustainable model based on community support and self-help groups. There is now a need to take this further by developing an overarching strategy to guide the development of new cost￾effective models which take account of geographic diversity (rural versus urban models) and promote a shift from dependency to autonomy. There has been an overall decrease in prioritizing innovative, ‘outside the box’ thinking that characterized USAID care and treatment activities in the past. This may be due to the fact that USAID CAs are in the burdensome position of simultaneously being responsible for scaling up direct service delivery with the increase in GFATM resources, whilst still being responsible for testing innovations. Key challenges to be addressed include a significant number of people first testing for HIV at an advanced stage of HIV-related immune system decline; improving referral systems between vii VCCT and care and treatment; enhancing positive prevention; better integration with other clinical services such as reproductive health; increasing the cost-effectiveness of services; and investigating how care and treatment outcomes can be enhanced through mainstream livelihood programs. USAID could add substantial value by focusing less on direct service provision and more on a strategy for developing cost-effective, state-of-the art interventions based on research to replace outmoded interventions and encouraging replication. Strategic information There is a strong culture among USAID/Cambodia, CAs and their IPs of monitoring and reporting on activities and using data to inform program development. The Office of Public Health and Education (OPHE) has put considerable effort into improving its performance monitoring plan, with further enhancements in the pipeline. USAID/Cambodia recognizes that it needs to develop improved ways of measuring its HIV/AIDS work. Currently indicators primarily measure inputs, outputs and processes, with limited measurement of outcomes. As technical assistance becomes a more significant component of the program, measurement of the outcomes of capacity building needs to be given priority. USAID assistance to NCHADS to improve the measurement of treatment outcomes and coverage should continue. There is no methodology for capturing the quality of programs beyond supervision visits using checklists based on SOPs. However, OPHE is developing a standardized quality assessment tool for measurement of both service quality and data quality. Overall, the emphasis of CAs has been on monitoring. There are, however, signs of a growing emphasis on evaluation. USAID and CDC GAP technical assistance to NCHADS for STI and HIV biological and behavioral surveillance surveys has resulted in considerable strengthening of capacity. Following strong advocacy from USAID, NCHADS has now decided that all future STI and HIV surveillance surveys will be integrated biological and behavioral surveys (IBBS), and a clear timetable for annual surveys has been established. These two things will significantly improve the availability of timely surveillance data to inform programming. All of the CAs have undertaken good quality operational research which has been widely disseminated and applied in programming. There is scope for greater collaboration between CAs in developing research plans and conducting joint research. This could result in cost savings. More jointly conducted research with government could help to develop capacity and enhance sustainability. Priority also needs to be given to further developing the capacity of Cambodian CAs and IPs to design, conduct, interpret and utilize operational research. Priority research areas are more reliable population size estimations for MARPs using improved methodologies; a better understanding of cost-effectiveness and cost-benefit which can be applied to program design; and an improved understanding of the determinants and different levels of risk within most at risk populations to enable more sophisticated and cost-effective geographic and sub-population targeting by prevention programs. The transition of USAID/Cambodia programming from a service delivery to a technical assistance model will require an increased emphasis on strategic information to guide the work of CAs and IPs and maintain the relevance of USAID/Cambodia’s HIV/AIDS program to the evolving nature of the epidemic. Health system strengthening USAID’s investments in HIV/AIDS are incrementally helping Cambodia’s health system to become more robust. HIV/AIDS health services have been used as a platform for improving other viii parts of the system (for example, laboratories). Significant improvements in access to and the quality of HIV/AIDS health services has contributed to demand creation for a better overall health system. USAID’s strategic placement of HIV funding into its broader health system strengthening program has facilitated improvements to health services in specific programs and health systems more generally. USAID and CA advocacy at the central level for linked or integrated services, based on proven Cambodian models (frequently developed by USAID) and international good practices, appears to have influenced key decision-makers, who are now increasingly moving towards integration of HIV into the general health system. However, this is at an early stage, with much work remaining. For example, there are missed opportunities for linking or integrating the work of the myriad of community based care providers from different programs in ways that would result in better overall health outcomes and cost-efficiencies. Placing HIV funding into mainstream service providers like RHAC and RACHA has integrated HIV into their capacity development work with Operational Districts and into reproductive and maternal and child health services in referral hospitals and health centers through the linked response for prevention of mother to child transmission. USAID’s involvement in innovative performance-based health financing which focuses on outcomes and quality is timely. Models need to be documented and evaluated. USAID’s support for social marketing has resulted in expanded access to a broad range of health products and services in the private sector, but supply chain issues in the public sector, especially stock outs of antiretroviral drugs, need to be addressed. USAID should explore opportunities for providing technical assistance to address weaknesses in the supply chain. The active participation of USAID and its CAs in RGC technical working groups has provided an opportunity to influence technical policy and protocols, enhance the quality and cost￾effectiveness of prevention, care and treatment programs, and strengthen surveillance and service delivery systems. USAID involvement in Global Fund governance in Cambodia has helped with the effective use of GF resources, although this is an area which is recognized as requiring substantial ongoing effort. USAID/Cambodia management of the HIV/AIDS portfolio USAID/Cambodia has consistently provided a high level of strategic technical leadership to its HIV/AIDS portfolio. Good working relationships have been established with a wide range of other development partners and key RGC agencies. However, the heavy administrative burden on USAID/Cambodia’s HIV/AIDS staff limits the time they can devote to providing technical direction and improving coordination between CAs. An increased level of technical direction and involvement from USAID is required to successfully implement the transition from a direct service delivery to a technical assistance model. ix Evaluation criteria and ratings Six evaluation criteria were used to provide aggregate ratings for the performance of the USAID/Cambodia HIV/AIDS portfolio as a whole.1 The six Cooperative Agencies (CAs)2 receiving HIV/AIDS funding may have scored differently if the criteria were applied to each agency. Table 1: Evaluation criteria and ratings Evaluation criteria Rating (1-6) Explanation Relevance 6 The program is highly relevant to the Cambodian HIV epidemic with an increasing focus on MARPs and termination of work targeting lower risk people. The program has been responsive to emerging needs such as evolving community care needs. The USAID HIV/AIDS portfolio has placed increasing emphasis on alignment with the Cambodian HIV/AIDS National Strategic Plan. Effectiveness 5 Key HIV/AIDS intermediate results in USAID’s Strategic Framework for Health have been met (improved access to and use of HIV/AIDS preventive, care, and treatment services). There have been significant innovations across programs to maximize effectiveness. HIV prevalence in Cambodia has reduced significantly. It is plausible that USAID supported programs have been a major contributing factor. Significant national legal and policy constraints have not been sufficiently addressed. Efficiency 4 The program has been responsive to changing needs. The quality of USAID technical advice is very good but a heavy administrative burden constrains the ability to provide strategic and technical leadership. Cost effectiveness of interventions has not been established, with some appearing to be high cost per beneficiary reached. There is some duplication and overlap in prevention programming. Sustainability 5 Models developed by USAID CAs are reflected in Standard Operating Procedures developed by the Royal Cambodian Government. The technical and implementation capacity of Cambodian partners has increased significantly. The Global Fund has scaled up HIV programs developed and tested by USAID. There is currently insufficient funding from the RGC and other donors to fully replace USAID funding of direct service delivery. Some service delivery models developed by USAID are not replicable due to their high cost, while others (e.g. HBC, transport allowances) have contributed to dependence. Gender 5 There has been a strong focus on gender-related issues including an emphasis on high risk urban males, men who have sex with men, and an increasing focus on the sexual and reproductive health needs of entertainment workers. Overall, gender has been well integrated into programs. Analysis and learning 5 USAID and CA Monitoring and evaluation frameworks have been established and data quality has been improved. Program data is increasingly used to analyse performance but more at the CA level than at the implementing partner level. A strong emphasis on operational research that is used in program refinement. Progress towards developing a client unique identifier code has been too slow. Rating scale: 6 = very high quality; 1 = very low quality. Below 4 is less than satisfactory. 1 The evaluation criteria and ratings have been adapted from evaluation criteria used by AusAID. 2 The six CAs are FHI, KHANA, PSI, RACHA, RHAC and URC. 1 1. Introduction 1.1 Purpose of the evaluation and scope of work This evaluation of USAID/Cambodia’s portfolio of activities receiving funding under the President’s Emergency Plan for AIDS Relief (PEPFAR) was commissioned by the Mission’s Office of Public Health and Education (OPHE). The purpose of the evaluation was to: 1. evaluate the performance of the current program, and 2. make recommendations to guide Mission investments in HIV/AIDS for the next five years. The scope of work specified two overarching questions: 1. How well have past Mission investments in HIV/AIDS performed? 2. How should the Mission invest resources in HIV/AIDS during the next five years? A series of more detailed questions relating to each of these overarching questions were set out in the Scope of Work. (These are listed in Annex 1.) Findings from the evaluation, coupled with a strategic assessment of how USAID can best support the evolving Cambodian response to HIV, have been used to make recommendations on the design of a new USAID/Cambodia HIV/AIDS program covering the five-year period September 2014 – September 2019.3 However, as existing agreements with four of the Mission’s six HIV/AIDS Cooperating Agencies (CAs) end prior to September 2014, the evaluation was also tasked with making recommendations on the scope of follow-on awards commencing prior to this time. The mission’s current HIV/AIDS CAs and the end dates for their agreements or contracts are: FHI: September 30, 2012 Khmer HIV/AIDS NGOs Alliance: (KHANA): September 30, 2014 Population Services International/Cambodia (PSI): February 4, 2013 Reproductive Health Association of Cambodia (RHAC): September 30, 2013 Reproductive and Child Health Alliance of Cambodia (RACHA): September 30, 2013 University Research Corporation (URC): December 30, 2013 1.2 Summary of methodology The performance evaluation, which took place over a five week period in May-June 2011, covered five years of portfolio performance, with an emphasis on the last two years. The five member evaluation team followed a methodology consistent with USAID’s January 2011 Evaluation Policy, focusing on descriptive and normative questions, including what the program had achieved; how it was being implemented; how it was perceived and valued; whether expected results were occurring; and other questions pertinent to program design, management, and operational decision making. 3 USAID/Cambodia’s current Health Program Design Activity Approval Document ends in September 2014. 2 The evaluation team began with a review of key background documents and key performance indicator and surveillance data. It then initiated a series of key informant interviews in both Phnom Penh and selected provinces. Ongoing analysis of data at an individual level fed into regular group analysis, which allowed for emerging issues to be identified and explored as the evaluation progressed. A comprehensive group analysis session, following the conclusion of the key informant interviews, led to the development of a preliminary presentation to USAID/Cambodia for the purpose of receiving feedback, validation and further input. This feedback was then incorporated into the draft evaluation report. USAID/Cambodia provided additional feedback on the draft report prior to its finalization. To supplement the evaluation team’s analysis of USAID support for different program areas, an overall assessment of USAID/Cambodia’s HIV/AIDS program was undertaken using six criteria adapted from standard evaluation criteria used by the Australian Agency for International Development (AusAID).. A full description of the evaluation methodology is in Annex 2. Interview guides for different categories of key informants are in Annex 3. The evaluation schedule, including key dates, tasks and deliverables, is in Annex 4. Key informants and site visits are set out in Annex 6. The bibliography is in Annex 7. 1.2.1 Limitations As described in greater detail in Annex 2, several factors limited the evaluation’s findings, conclusions and/or recommendations. For the most part, these revolved around: The high level, strategic nature of the evaluation meant that a reasonably broad focus was taken rather than a detailed evaluation of all project components implemented by each of the six CAs; limited availability of data to be able to assess the coverage and quality of services; the selective nature of site visits, making generalizations across the program difficult, and language barriers. These types of limitations are commonly encountered for evaluations of this type which are conducted within a limited time frame. 1.3 Evaluation team The five-person evaluation team was made up of three independent consultants and two USAID/Washington staffers. Collectively, the team had considerable expertize across all aspects of HIV technical knowledge and programming, from prevention to care and treatment, and health systems strengthening. Most team members had extensive knowledge of Cambodia’s national response to HIV/AIDS from previous work. 3 2. The context 2.1 The HIV/AIDS epidemic in Cambodia Cambodia’s response to HIV/AIDS has been remarkably successful as evidenced by the significant reduction in HIV prevalence which has taken the country from a low-level generalized epidemic to a concentrated epidemic. It is estimated that adult HIV prevalence declined from 2.4% in 1998 to 0.8% in 2010.4 Broadly paralleling this trend, HIV prevalence among women attending ante-natal clinics declined from 2.1% in 1999 to 0.4% in 2010.5 Estimates of HIV prevalence in the adult general population for 2010 indicate that there was no difference in prevalence between males and females (i.e. prevalence in males and females was the same at 0.8%).6 HIV prevalence is concentrated in most-at-risk populations (MARPs), which in Cambodia are entertainment workers (EWs) (previously known as female sex workers), men who have sex with men (MSM), (including male sex workers and transgender people), and injecting drug users (IDU). HIV prevalence among female sex workers (FSWs) declined dramatically from 44.7% in 1996 to 14.7% in 2006.7 The 2010 HIV surveillance survey (HSS) found an overall HIV prevalence among EWs of 4.6%, although prevalence among EWs with more than seven clients per week was 14%. In 2005, HIV prevalence among long-hair MSM in Phnom Penh was 17% and among short-hair MSM was 5%.8 In 2010, HIV prevalence among long-hair MSM in Phnom Penh was 3.6% and among short-hair MSM was 3.1%.9 In 2007 HIV prevalence among IDU was 24.4%.10 HIV prevalence in non-injecting drug users (DU) in 2007 was 1.1%, which was not significantly above the level of infection in the general population. No trend data is available for IDU/DU as only one HIV surveillance study has been conducted for these groups. 2.2 Overview of the Cambodian national response to HIV/AIDS A key factor in Cambodia’s successful national response to HIV/AIDS has been a high level of commitment and leadership within the Royal Government of Cambodia (RGC). The two major public sector organizations with specific HIV/AIDS mandates are the National AIDS Authority (NAA) and the National Centre for HIV/AIDS, Dermatology and Sexually Transmitted Diseases (NCHADS). The NAA is responsible for providing overall strategic direction, leadership and coordination for the national response. Capacity limitations have affected the efficiency and effectiveness with which these functions have been performed. NCHADS provides strategic and technical direction to the health sector’s HIV response and coordinates the work of partners. The efficiency and effectiveness of NCHADS is widely recognized. The other major RGC sector with 4 NCHADS, Estimation of the HIV Prevalence among General Population in Cambodia, 2010. July, 2011. 5 NCHADS, ibid. 6 NCHADS, ibid. 7 NCHADS, HIV Sentinel Surveillance, 2006. 8 NHCADS, Cambodia STI Survey, 2005. 9 NCHADS, Estimation of the HIV Prevalence among General Population in Cambodia, 2010. July, 2011. The MSM data for 2010 is from FHI’s Bross Khmer study which was officially incorporated into the 2010 HSS. Comparisons between the 2005 and 2010 survey results for MSM are limited as different recruitment strategies were used in 2010. 10 NCHADS, HIV Prevalence among Drug Users in Cambodia, 2007. 4 responsibility for HIV/AIDS is the Ministry of Interior, which covers police and public security functions, including the National Authority for Combating Drugs (NACD). Effective prevention programming by civil society and government agencies, particularly early interventions focusing on FSWs, appear to have had a significant effect in reducing the prevalence of HIV in Cambodia.11 USAID/Cambodia has been in the forefront of developing prevention models for marginalized and stigmatized people engaged in high-risk behaviors. This initially sensitive work paved the way for acceptance by the RGC of the need to prioritize work with MARPs. USAID/Cambodia continues to contribute substantially, both technically and financially, to prevention programming for MARPs. Even with the significant growth in funding by the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), in 2010 the United States Government (USG) was providing approximately 72% of total funding for HIV prevention.12 The RGC has rapidly scaled up access to care and treatment for PLHIV. Since the first HIV continuum of care (CoC) site was established in 2003 with USG support, Cambodia has opened 52 CoC sites, extending HIV-related care and treatment to the vast majority of PLHIV. Access to antiretroviral therapy (ART) has been scaled up significantly. NCHADS estimates that 86% of PLHIV in need of treatment, (defined as a CD4 count of below 350), are receiving ART. Others have suggested that this these data may over-estimate the number of PLHIV on ART (see Section 4.3). The delivery of public sector health services in Cambodia is organized around vertical programs such as HIV, tuberculosis (TB), malaria, and maternal and child health (MCH), with generally insufficient attention to quality, efficiency, and overall population health priorities. The verticality of service delivery and significantly different levels of funding and capacity between national programs has limited linkages and integration and distorted relative priorities. There has, however, been an effort in recent years to develop better linkages between some national programs such as HIV/AIDS and TB and HIV/AIDS and MCH. There has also been a growing emphasis on the need to strengthen the health sector’s response to a range of public health problems through a more integrated approach. In recent years the legal and policy environment for HIV has weakened, with the introduction of laws and policies which have had the indirect effect of making it more difficult for prevention programs to reach some MARPs. For example, the law on Suppression of Human Trafficking and Sexual Exploitation and the Village Commune and Sangkat Safety Policy have resulted in brothel closures and the dispersal of previously easy to reach FSWs into different types of entertainment establishments and street based work. Similarly, the environment for harm reduction programs with IDU has become hostile, which has limited the operation of the needle and syringe exchange program and made outreach work more difficult. Other challenges faced by Cambodia in its response to HIV/AIDS include maintaining low HIV incidence rates in the face of a likely reduction in available resources and the deterioration in the 11 UNAIDS/Cambodia, Turning the Tide: Cambodia’s Response to HIV and AIDS 1991 – 2005. 12 National AIDS Authority, National AIDS Spending Assessment, 2010 (NASA III). The high level of funding support for prevention partly reflects the significant level of GFATM funding for treatment and the relatively low level of prevention funding from other donors and the RGC. 5 enabling environment; the need for more timely and comprehensive strategic information to guide programming; the need for more effective mechanisms to set national priorities; the need to improve the cost-efficiency and effectiveness of HIV interventions; how to transition from a vertical program model of service delivery to an integrated model; the effects of a weak health system on the quality of care and treatment; limited performance-based incentives; and heavy dependence on donor support. 2.3 Overview of the USAID/Cambodia HIV/AIDS portfolio USAID/Cambodia’s HIV/AIDS portfolio comes under the OPHE’s Cambodia Health Program Design FY 2009 – 2013 Activity Approval Document (AAD). The program has been designed on the basis of OPHE’s Strategic Framework for Health, 2009 – 2013 which sets out program goals and intermediate results (IRs) for HIV/AIDS and other USAID health program components (see Annex 5). The OPHE has contributed to the development of the draft USG PEPFAR Cambodia HIV Strategy, 2011-2015. The goals of the PEPFAR Strategy in Cambodia are: 1. To reduce the number of new HIV infections through scaled targeted prevention. 2. To provide care and support to people living with and affected by HIV/AIDS. 3. To alleviate the socioeconomic and human impact of AIDS on the individual, family, community and society. 4. To support and build capacity of the Cambodia government to respond to HIV/AIDS programs and strengthen the Cambodia health care system The first three goals are fully aligned with (i.e. identical to) Cambodia’s National Strategic Plan for Comprehensive and Multi-Sectoral Response to HIV/AIDS, 2011-2015 (NSP III). The fourth goal is additional. USAID support for the national response is primarily focused on four specific strategies in NSP III: Increase coverage, quality and effectiveness of prevention interventions Increase the coverage and quality of comprehensive and integrated treatment care and support while addressing the needs of a concentrated epidemic Improved capacity and involvement of civil society institutions, especially organizations and community networks representing PLHIV and MARPs in the national response Ensure the availability and use of strategic information for decision-making through HIV monitoring and evaluation, including impact evaluation and research. Key features of the PEPFAR strategy are improved collection and use of strategic information to guide the national response; a refined focus for prevention interventions to address changes in the epidemic; enhanced quality and cost-effectiveness of care and treatment services; and strengthened HIV-related health service delivery systems. The directions set out in the PEPFAR Strategy are broadly in line with the findings and recommendations of this evaluation. The primary focus of USAID/Cambodia’s programming has been on developing the capacity of civil society to effectively respond to HIV/AIDS. This has taken the form of developing the technical and organizational capacity of national and community level non-government organizations (NGOs). A clear finding by those evaluation team members who have been long term observers and participants in Cambodia’s response to HIV/AIDS is the significant advance in sophistication of civil society’s response. Some USAID CAs have also contributed significantly to strengthening the capacity of NCHADS in the areas of surveillance, analysis of data and the development of guidelines and standard operating procedures (SOPs) in a range of areas. 6 While USAID/Cambodia’s HIV/AIDS work has always been aligned with the national response, historically the focus was primarily on civil society. In recent years there is clear evidence of a heightened level of partnership between USAID/Cambodia and key RGC coordination mechanisms and closer working relationships with other international development partners working in the health sector. For example, USAID staffs participate in the development of national and provincial annual action plans for HIV and USAID is now a regular participant in health development partner forums in an effort to promote health system strengthening (HSS). In Cambodia, USAID technical assistance has a long track record of focusing on developing, piloting and adapting models and potential innovations for broader replication. There are numerous examples of replication. These include RGC SOPs such as the CoC, which was based on a USAID pilot, and replication of prevention program models developed by USAID, using Global Fund monies. However, Global Fund monies have had an insufficient emphasis on technical oversight and quality assurance. The strong technical assistance focus of USAID’s programming has been accompanied by a significant service delivery component. Consistent with current global PEPFAR guidance, USAID/Cambodia is now placing greater emphasis on technical assistance and lesser emphasis on service delivery. This is referred to as a ‘transition from service delivery to technical assistance’. The rationale is to further develop Cambodian capacity to independently manage the response to HIV/AIDS as part of a long-term exit strategy. However, for the time being, service delivery remains an important component of USAID’s portfolio as it is needed to demonstrate further model development, testing and refinement and to promote replication. Phasing out of service delivery by USAID/Cambodia also needs to take account of the availability of replacement funding from other sources such as the RGC and the GFATM. The transition from a service delivery model to a technical assistance model will require active management by OPHE. There will be a need for all CAs and their implementing partners (IPs) to understand what the transition means in practical terms to avoid a ‘business as usual’ approach. Key to this is understanding the difference in approach to technical assistance and where service delivery fits within the transition. For many years, USAID was the single largest HIV donor in dollar terms in Cambodia. In recent years, the level of HIV funding by the GFATM has surpassed that of USAID, which is now the second largest donor. In 2010, total national HIV/AIDS funding came from the following sources: Global Fund 40%; PEPFAR 20%; NGOs 11%; Other donors (World Bank, DFID, AusAID, EC, and minor level donors) 11%; World Food Program 7%; United Nations agencies 5% and the RGC 3%.13 The very low level of government funding means that Cambodia is heavily reliant on donor’s resources. Since 2006, USAID/Cambodia’s core HIV/AIDS budget has been straight lined at around $14 million dollars per year. The total PEPFAR budget for Cambodia in FY 2011 was $18.5 million. In financial year (FY) 2011 USAID/Cambodia’s investments in HIV/AIDS consist of $7.9 million for prevention; $2.3 million for care; $0.9 million for treatment; $1.3 million for HSS; $0.8 million for strategic information, and $0.9 million for management and staffing. The division of HIV/AIDS 13 National AIDS Authority, National AIDS Spending Assessment, 2010 (NASA III). 7 funding by CA in FY 2010 was FHI $4.5 million; KHANA $2.5 million; PSI $2.3 million; RHAC $1.5 million; URC $1 million; and RACHA $0.5 million. The geographic scope of USAID’s HIV/AIDS programming covers Phnom Penh and the following provinces which have been identified as those with the highest HIV prevalence: Banteay Meanchey, Battambang, Kampong Cham, Kampong Speu, Koh Kong, Pailin, Pursat, Siem Reap and Svay Rieng. 8 3. Prevention 3.1 Outline of the prevention program In OPHE’s Cambodia Health Program Design FY 2009 – 2013 Activity Approval Document, USAID supported prevention programming falls under Intermediate Result 1.1 – ‚Improve access to and use of HIV/AIDS preventive services among target populations‛. USAID funding supports prevention programs that are tailored to the needs of Cambodia’s low-level, concentrated HIV epidemic that disproportionately affects MARPs. Currently, MARPs include sex and entertainment workers, men who have sex with men, (including male sex workers and transgender people), and injecting drug users. Programming also appropriately targets men who frequent entertainment venues, known as high risk urban males (HRUM) and the female sexual partners of IDU. The core components of prevention programs for MARPs are behaviour change communication (BCC), (primarily through peer educators and BCC materials), condom social marketing and distribution, referral to STI and VCCT clinics, harm reduction services for IDU, and the integration of HIV prevention with care, support and treatment services. There is a strong emphasis on outreach and drop-in-centres. Prevention activities take place in the context of the continuum of prevention to care and treatment (CoPCT) model, including positive prevention targeting PLHIV and prevention of mother to child transmission (PMTCT). There is a growing recognition of the need to link HIV prevention programming with existing reproductive health programs, although the extent to which this is occurring is variable. USAID supports the only access to HIV post exposure prophylaxis for rape victims, through RHAC clinics. In FY 2011, USAID allocated $7.9 million to HIV prevention programming. This represents 56% of USAID’s total HIV/AIDS budget for Cambodia.14 Over three-quarters of these prevention funds (i.e. three-quarters of $7.9 million) are directed at preventing HIV among sexual partners, which is appropriate for the Cambodian epidemic. (See Figure 1 for a breakdown of how HIV prevention funds are spent). The 2010 National AIDS Spending Assessment estimated that $11 million or 19% of Cambodia’s total annual HIV/AIDS expenditure was in the area of prevention.15 This includes expenditure from all funding sources. In 2010-2011, approximately 72% of Cambodia’s total HIV prevention expenditure was funded by USAID.16 It is therefore reasonable to assume that a significant proportion of overall prevention programming achievements can be attributed to USAID. 14 Sok, B. PEPFAR Cambodia Overview and USAID HIV Program Overview. May 25, 2011. This figure includes $0.57 million for HIV testing and counseling. In NSP III, HCT comes under treatment. 15 Nitsoy, A. National AIDS Spending Assessment Draft Results. NASA III Technical Validation Meeting, Naga World (Phnom Penh), 1 April 2010. 16 These figures are approximate because the National AIDS Spending Assessment years are calendar years, while USAID uses a fiscal year starting on 1 October, making exact analysis not possible. 9 3.2 Key achievements: The key achievements of USAID/Cambodia’s prevention programming include: 3.2.1 Increased coverage for prevention programs which reflect epidemiological priorities Coverage of HIV prevention programs reaching EWs, MSM and IDU/DU increased significantly between FY 2008 and FY 2009, but was not sustained in FY 2010, especially for EWs; (see Figure 2). In addition, significant numbers of HRUM are being reached, with 274,235 men reached in FY 2010. A number of factors may have contributed to the decline in coverage in FY 2010 compared to FY 2009. These include improved cleaning of data to eliminate double counting of the same person, some changes in indicator definitions and a more hostile legal and policy environment which, according to key informants, made it harder for IPs to reach MARPs; (see 3.3.6 for further details). It is not possible to reliably estimate the percentage of MARPs being reached by USAID funded HIV prevention programs. This is because the population size estimates that have been developed by different partners for different MARPs are regarded by key informants as unreliable, or at least contestable, and in some cases, such as for MSM, the estimates only cover limited geographic areas and hence are not national. As epidemiological data confirmed that Cambodia’s epidemic is concentrated among MARPs, USAID has appropriately withdrawn support for prevention programming targeting populations at low risk (e.g. the military and garment factory workers). Figure 1: USAID/Cambodia HIV prevention program financial allocations, FY 2011 Source: OPHE/Cambodia PEPFAR Cambodia Overview and USAID HIV Program Overview. May, 2011 10 3.2.2 A well-established condom and lubricant social marketing program Condoms and lubricant are widely available through both condom social marketing through more than 5,400 (FY 2010) traditional and non-traditional outlets and free distribution to MARPs. Condom availability (PSI and non-PSI condoms) in outlets within 50 metres of hot spots has increased significantly between 2008 and 2010.17 For example, in 2008 only 41% of karaoke venues in Phnom Penh had condoms available within 50 meters. In 2010, this had increased to 72%. The number of condoms sold or distributed through PSI social marketing was above 23 million in both FY 2009 and FY 2010. 18 Figure 2: Prevention reach with MARPs, FY 2008 - 2010 (Cambodia/USAID Program) Source: USAID/Cambodia, OPHE Performance Monitoring Plan 3.2.3 Behavioural data indicates sustained high levels of consistent condom use by MARPs In 2010, self-reported consistent condom use by female EWs with clients for those with more than two sexual partners per day was 84% for the past 3 months and 89% for the last week.19 Behavioural surveillance surveys between 1997 and 2007 show significant increases in consistent 17 Condom availability within 50 meters of other types of hot spots improved between 2008 and 2010 as follows: massage parlor: from 52% to 63%; beer garden: from 60% to 71%; and guest house from 90% to 92%; (source: PSI). With the closure of many brothels it has become important for condoms to be readily available in or near places where male clients meet entertainment workers who sell sex, such as beer gardens, karaoke bars and massage parlors. 18 This represented a decline from approximately 31 million condoms sold or distributed in FY 2008. PSI increased the price of their Number One condom in FY 2009 as part of a long-term sustainability strategy segmenting consumers by their ability to pay and this had an immediate impact on sales volume. High stock levels in 2008 also contributed to the reduction in condoms sold in 2009. Tracking of behavioural trends through PSI research indicates there was no decrease in consistent condom use due to the price increase. 19 NCHADS, Behavioral Sentinel Surveillance, 2010. Direct comparisons with the 2007 BSS are not possible due to the reduction in the number of direct sex workers (i.e. brothel based) which meant that the 2010 data was collected primarily from EWs rather than direct sex workers and was presented using number of sexual partners per day rather than distinguishing between direct and indirect sex workers. 11 condom use by direct sex workers with clients (from 42% to 94%) and for beer promoters (from 15% to 83%). From 2003 to 2007 consistent condom use in sex workers plateaued at a high level. Although there are difficulties in making comparisons between the 2007 BSS and 2010 BSS for sex workers/entertainment workers, consistent condom use in 2010 was at very high levels. There were also significant increases between 1997 and 2007 in consistent condom use by sex workers with their sweethearts; (for direct FSW from 20% to 52% and for beer promoters from 19% to 46%). In the 2010 BSS, consistent condom use with sweethearts by EWs with more than two sexual partners a day was 48% and for EWs with two or less partners a day was 39%. PSI’s 2010 TraC study of MSM in Phnom Penh and four provincial cities found that consistent condom use with all male partners in the last 3 months was 81%.20 This was significantly higher than levels of consistent condom use among MSM in the 2005 STI Sentinel Surveillance study where consistent condom use in the past month varied from a low of 18% to a high of 66% depending on partner type and geographic location.21 3.2.4 A significant decline in HIV prevalence Multiple factors can contribute to a reduction in HIV prevalence; (e.g. deaths and those at highest risk may already have been infected). However, given the significance of the decline in prevalence among the general population and MARPs and the large proportion of HIV prevention programming supported by USAID, it is plausible that USAID prevention programs have made a significant contribution to the reduction of HIV prevalence in Cambodia. Caution is, however, needed in claiming that HIV prevention programming has contributed to declining HIV prevalence among MARPs. This claim can be made in relation to sex workers/entertainment workers and MSM as trend data for HIV prevalence exists (see Section 2.1). The sex worker surveillance data indicates a significant decrease in prevalence from 1996 to 2006. Although a further decline was observed between 2006 and 2010 the results are not strictly comparable with previous surveys due to changes in the structure of the sex/entertainment industry which necessitated the 2010 surveillance survey recruiting and categorizing sex workers in a different way.22 The MSM surveillance data also indicates significant declines in HIV prevalence among MSM in Phnom Penh between 2005 and 2010. It should, however, be borne in mind that comparisons between cross-sectional surveys can be flawed, especially when different recruitment strategies are used as is the case for the MSM studies. There is no recent HIV prevalence data for IDU/DU and no trend data as HIV surveillance has only been conducted on one occasion, in 2007. 20 PSI, Cambodia (2010): TRaC Study evaluating consistent condom use with male partners among MSM from Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey. Round One. Condom use data from the 2010 HSS for MSM was not available at the time this report was being written. The locations where the 2005 SSS and the 2010 TRaC studies were conducted were not fully identical. 21 NCHADS, Cambodian STI Survey 2005: Key Risk Behaviors and STI prevalence. Due to different methodologies and questions the results are not fully comparable. 22 Previously HIV sentinel surveillance surveys categorized sex workers as brothel based (direct) and entertainment based (indirect). For 2010 the survey categorized sex workers as those with more than 7 clients per week and those with 7 or less clients per week. While those with more than seven clients per week may be a proxy for direct sex workers, changes in how the 2010 survey recruited EWs compared to 2006 make comparisons difficult. 12 3.2.5 Significant increases in the number of MARPs referred to and regularly presenting for STI check-ups and HIV testing and counseling. The CAs have placed a strong emphasis on referrals to increase the number of MARPs who regularly have STI check-ups and HIV tests. This has been facilitated by development of a standard referral slip by NCHADS, with input from USAID CAs. The increase in the number of referrals made can best be illustrated by example. Men’s Health Cambodia (MHC) increased the number of referral slips given to MSM from 2,931 in 2008 to 9,391 in 2010, an increase of 220%.23 Over this same period, the number of MHC’s MSM clients presenting for STI check-ups increased from 1,242 to 1,990, an increase of 60%. The number MSM presenting for HIV testing and counselling increased from 1,113 to 1,433, an increase of 29%. (Strengths and weaknesses in relation to STI and HIV testing for MARPs are discussed in 3.3 below.) 3.2.6 USAID’s cooperative agencies have been actively involved in model development The prevention program models developed, refined and documented by USAID CAs for EWs and positive prevention have been adopted as national SOPs, replicated and scaled up. CAs have also been key partners in developing SOPs for HIV prevention for MSM and IDU. Every major prevention- related policy document, plan and SOP developed over the last five years lists USAID CAs and their IPs as key informants. 3.2.7 Significant capacity building of local non-government organizations Technical and organisational capacity building by USAID CAs with local non-government organizations (LNGOs) to enable them to plan, deliver, monitor and report on robust prevention programming has contributed to an LNGO sector that demonstrates it is engaged, informed and confident. The capacity of implementing agency partners and other key stakeholders to implement and manage HIV prevention interventions has increased significantly. This is evidenced by their delivery of more sophisticated HIV prevention programs of a higher quality; (see 3.3.1). 3.2.8 Programming to support positive prevention has improved The CAs have led the development of programs to support positive prevention. After five years of almost exclusive focus on PMTCT, positive prevention to reduce intimate partner transmission in now receiving attention. CAs report that in FY 2010, 25,077 PLHIV over the age of 18 were reached with care services, of whom 8,895 people received positive prevention interventions from FHI, KHANA or RACHA.24 3.2.9 A significant increase in the number of pregnant women tested for HIV Support for the national PMTCT program and the linked response by strengthening ties and referrals between HIV/AIDS services and ante natal care (ANC) services has resulted in a significant increase in HIV testing among pregnant women. The number of pregnant women with known HIV status as a result of USAID supported programming for HIV testing of ANC women almost doubled from 30,661 in FY 2008 to 57,599 in FY 2010.25 23 Data for referrals by all CA for all MARPs is not centrally collected so this needs to be illustrated by example. The data from other CAs and implementing partners seen by the evaluation team shows similar trends. 24 USAID/Cambodia Performance Management Plan. 25 USAID/Cambodia Performance Management Plan. 13 3.3 Strengths and weaknesses 3.3.1 A core package of evidenced based prevention services is in place for MARPs The core components of prevention programs for MARPs developed by FHI, KHANA and PSI, as outlined in 3.1 above, are similar and consistent with evidence-based regional recommendations issued by UN agencies. A strong feature of USAID programming is an emphasis on the provision of technical assistance by CAs to support quality programming by their IPs. Monitoring visits by all three CAs utilise standardised quality control checklists and on the spot feedback is given. This is an important quality assurance mechanism. FHI has demonstrated considerable innovation in the development of SMARTgirl for EWs and M-Style for MSM. Although the core components of these interventions are the same as interventions developed by other CAs for the same target groups, SMARTgirl and M-Style use a sophisticated branded franchise approach to develop a sense of identification and loyalty by the target groups. This has been accompanied by the packaging of standardized prevention messaging to improve the quality, consistency and variety of behavior change communication. Technical leadership in design and quality assessment of HIV prevention remains within FHI. PSI has also shown innovation in its approach to prevention programming. For example, venue mapping and coverage is carried out in a methodical way using GIS technology; standardized messaging is determined by research findings on behavioural determinants; topics are changed every three months, accompanied by quarterly training, and a range of sub-topics linked to the main themes are covered each quarter. While PSI uses volunteer peer inter-personal communicators (IPCs) to work with EWs and MSM, employed IPCs are used to reach HRUMs given the challenges of working with this group in entertainment establishments. The quality of peer education/IPC sessions observed by the evaluation team varied from satisfactory to excellent. The standardization of prevention messaging by CAs means that peer educators are no longer expected to come up with their own topics and messages. It has improved the quality of messaging and ensured that it is evidence based. However, as the content of messages is determined by CAs it may be too top-down, with insufficient responsiveness to the needs of MARPs and little room for creativity by the better IPs. BCC is usually delivered in a didactic manner, although sessions conducted by PSI’s IPCs with HRUM were highly interactive and well above average quality when compared to other education sessions observed by the evaluation team. The CAs and IPs should encourage more interactive approaches in BCC sessions with peers. All three CAs have been broadening out their prevention messaging to include a range of topics such as reproductive health, family planning and drug and alcohol use. KHANA has developed standard packages of activities for prevention programs for MARPs and has recently opened demonstration projects for EWs, MSM and IDU/DU, using a direct implementation model. The stated purpose of these sites is to generate local evidence to support best practice and to share learning with IPs and other LNGOs. (KHANA’s demonstration projects are also discussed in Sections 5.3 and 6.6.) 3.3.2 Prevention programming recognizes multiple risk factors Prevention programming had recognized and responded to multiple risk behaviors for some people. For example, surveys have found high levels of drug use among sex and entertainment workers and MSM. For example, the 2010 BSS found that 27% of EWs with more than two 14 partners a day had ever used drugs and that in the past year 2% had used heroin, 21% had used yama, and 7% had used amphetamine type stimulants (ATS).26 Drug education is now provided for EWs and MSM and safe sex education is promoted with IDU. 3.3.3 Although the number of referrals for STI check-ups and VCCT has increased, systemic improvements are needed The introduction of a standardized NCHADS referral slip for use by the CAs IPs has significantly increased the number of MARPs presenting for STI check-ups and HIV testing and counselling (see 3.2.4 above). This result is also attributable to the CAs placing a far greater emphasis on referrals to these services. There is, however, room for significant improvement. For example, FHI reports that M-Style provided in excess of 12,000 referral slips to MSM in quarter four of 2010, which resulted in approximately 950 MSM accessing VCCT and 2,000 STI tests. Similarly, in quarter two of 2011, SMARTgirl reached approximately 12,000 EWs of whom 4,224 had an STI test and 2,304 presented for VCCT. Other CAs report broadly similar results. Factors limiting the number of MARPs presenting for STI screening and HIV testing appear to be the judgemental attitude of some staff, internalized stigma by MARPs, limited opening hours, testing only being done at fixed clinical sites, insufficient knowledge of the benefits of early HIV treatment, and insufficient effort at demand creation. The NCHADS referral slips appear to be given out routinely in the absence of messaging to reinforce the importance of regular testing, thus reducing the opportunity for impact. Following successful advocacy by USAID, NCHADS agreed to pilot Community/Peer Initiated Counseling and Testing (CPICT), using rapid tests in community settings such as drop-in-centres. USAID CAs provided technical assistance (TA) in development of SOPs and their IPs working with EWs and MSM have assisted with the pilot. Early results indicate that it is feasible to conduct rapid HIV testing in community settings, with targets for the number of persons to be tested being met or exceeded. Adoption of this model on a widespread basis will provide CAs and IPs with the opportunity to undertake demand creation for easily accessible HIV counselling and testing and STI screening. 3.3.4 Condoms are widely available, but there is room for improvement in programming Behaviour change communication is supported by broad coverage of condom and lubricant social marketing and free distribution. However a number of problems are apparent. While condom social marketing is the primary strategy being used, there is no clear guidance to IPs on when free condoms and lubricant should be made available to MARPs. In the absence of this guidance it is possible that some IPs may be undercutting condom and lubricant social marketing by excessive free distribution or alternatively not distributing free condoms and lubricant in situations where it is warranted. During site inspections the evaluation team found four IPs in receipt of USAID funding from different CAs distributing free condoms but not lubricant to MSM. Why this is occurring is not fully clear but it would appear to be a systemic problem. Its occurrence is an example of 26 NCHADS, Behavioral Sentinel Surveillance, 2010. 15 inadequate supervision of quality by CAs and problems with addressing operational problems in the field. Whether this example is generalizable to other areas of programming is not known. The issue of overlap and duplication by CAs has been partially addressed by FHI agreeing to give up condom social marketing where FHI and PSI are working with the same IP in the same area. In effect this means that FHI supported peer educators will not be selling OK condoms and lubricant, although they may distribute free condoms and lubricant which could possibly undercut condom social marketing. Effective condom social marketing from the perspective of clients (rather than of CAs) should see provision of the means of prevention (e.g., condoms and lubricant) as a core element of prevention programming. Accordingly, all peer educators/IPCs should be selling subsidized condoms and lubricant to MARPs regardless of which CA is funding their work. 3.3.5 Improved collaboration between CAs is needed While there is good communication between the CAs, overall there appears to be little substantive collaboration to promote efficiencies, cost-effectiveness and learning. Each agency has invested in separate operational research, program design, training and other capacity building, and monitoring and supervision. There is, however, some limited sharing of supervisory roles between two of the CAs. A degree of healthy competition between CAs can have benefits in terms of innovation but there is a need for greater collaboration. FHI, KHANA and PSI are conducting HIV prevention programs for EWs, MSM and IDU with funding from USAID and other sources such as the GFATM. The issue of overlap and duplication was recently identified by USAID and the CAs were asked to jointly address this. In response, the CAs have documented 60 instances where one IP is receiving USAID and/or other donor funding via multiple CAs to work with the same target group in the one location. Of the 60 overlaps identified by the CAs, some involve dual USAID funding, while others involve USAID and GFATM or European Union funding, and a small number do not involve USAID funding but rather GFATAM and GFATM funding or GFATM and European Union funding. Geographic overlap is small with EWs, significant for MSM and limited for IDU/DU. One of many instances of overlap seen by the evaluation team was USAID funding of FHI which supports outreach by MHC peer educators to MSM in a Phnom Penh hotspot on certain days of the week, while GFATM funding of PSI supports outreach to MSM by a separate set of MHC peer educators (called inter-personal communicators) in the same hotspot but on different days of the week. In other instances, CAs may be working with the same target group in the one city but in different locations. For example, in Siem Reap, there is competition between IPs to gain approval from the Provincial AIDS Office (PAO) for permission to work in particular entertainment establishments. While the PAO only permits one IP to work in each entertainment establishment, having multiple CAs targeting the same MARP in the one city increases overhead costs. The three CAs have identified some areas where they can rationalize their work and have committed to improved coordination to address overlap and duplication. However, in general, the response of the CAs has been to downplay the extent of overlap/duplication and to justify it. For example, they argue that repeat exposure is desirable and needed to achieve behavior change. While there is evidence to support this, repeat exposure can be achieved by one single funding source which avoids expensive overheads and the possibility of uncoordinated duplication. 16 3.3.6 Policy and legal constraints As mentioned in 3.2.1 and illustrated in Figure 2, the coverage of prevention programs has been negatively impacted by constraints in the enabling environment, including harassment and arrest of MARPs and outreach workers and raids on hotspots. This has dispersed sex workers across a range of entertainment venues and driven MSM and IDU underground. The NAA has been challenged to deliver on governance, coordination and leadership for the national response, and this is particularly evident in relation to the enabling environment for HIV prevention programming among MARPs. Technical leadership to address this shortfall has come from USAID and CAs, working closely with the Joint United Nations Program on HIV and AIDS (UNAIDS) and the World Health Organization (WHO) as well as AusAID’s HIV and AIDS Asia Regional Program (HAARP). USAID’s leadership was reported by other development partners and acknowledged by the NAA. Difficulties in achieving an enabling environment have been addressed through high level advocacy and participation in technical, planning and decision making forums by USAID and its CAs. Advocacy has been informed by evidence generated from programs. It is widely acknowledged that this leadership and input, especially from the USAID, has contributed significantly to the development of effective interventions that are still reaching MARPs. However the situation remains difficult and some CAs report being unable to meet their targets. For example KHANA reported in April 2011 that ‚due to the Government’s recent Village and Commune Safety Policy, during this period MSM, EW, and IDU have become more hidden and are reluctant to access services due to raids and harassment from the police.’’ At that time only 59% of their coverage target was achieved. There is a greater than ever need for policy advocacy. 3.3.7 USAID has demonstrated leadership in HIV prevention for IDU and DU, although the priority given to interventions with ATS users needs reviewing USAID has continued to support evidence based harm reduction interventions targeting IDU, despite constraints in the social and policy environment. WHO and Korsang acknowledged considerable support from USAID for a rights based public health approach which has allowed key organisations targeting injecting drug users to continue to operate, albeit with restrictions related to needle and syringe programing. Most of the IDU being reached with prevention services are the clients of one IP, Korsang, who report that the operating environment has become complex and hostile. The relationship between the Government and Korsang is strained. It has been reported that the RGC has approached HAARP with a request to shift funds currently supporting Korsang for IDU projects to the Ministry of Health (MoH). Since IDU are not always best served by Government programs this may have a negative impact on HIV outcomes among IDU. In 2010, KHANA applied to NACD for a needle and syringe exchange license for itself and all its key sub-partners, including Korsang. NACD granted a license only to KHANA and all sub￾partners were invited to submit their requests individually. Currently, Korsang does not have a license to distribute/exchange needles and syringes. This significantly impacts on Korsang’s ability to provide a comprehensive, evidence based program. Only two other organizations currently have such a license. 17 In mid-2010 KHANA set up its own HIV prevention program for drug users a few kilometres away from Korsang’s centre. This service, the Meanchey Demonstration Site, receives 80% of its funding from AusAID’s HAARP and 20% from USAID through KHANA. After one year its client base is overwhelming ATS users, with only a relatively small number of IDU reached.27 (KHANA’s demonstration sites are also discussed more in Sections 5.3 and 6.6.) The evaluation team found that HIV services provided by IPs for ATS users in one or two places are assisting police in identifying drug users, leading to their arrest or involuntary detention. This practice should cease immediately. USAID supported HIV prevention programs target both IDU and non-injecting drug users. The latter are primarily ATS users. While IDU are at significant risk of HIV, ATS users face only a low risk unless they also engage in established high risk behaviors. This is confirmed by the low HIV prevalence data among ATS users which is similar to HIV prevalence in the general population; (see Section 2.1). 3.3.8 The scope of positive prevention has been broadened to address intimate partner transmission The integrated CoPCT demands active case finding in the community through positive prevention and a system that facilitates partner testing. The new FHI Bomnong Initiative is a good example of innovation in this area. The full package of positive prevention includes support for disclosure and condom use among intimate partners and promotes adherence, regular STI check-ups, family planning and access to PMTCT. It does not address gender based violence or the impact of high consumption of alcohol on PLHIV and their families. However, it does indirectly address depression and hopelessness through the theme of future hopes and dreams. The partnership with the Cambodian Community of Women living with HIV should increase the sustainability and credibility of this branded approach. Some IPs, for example, Save Incapacity Teenagers, were able to describe how their peer educators conduct regular outreach to HIV-positive female EWs that includes individualized risk assessment and risk reduction planning. 3.3.9 Some HIV-positive pregnant women are lost to follow up and do not deliver safely Despite the significant increase in the numbers of pregnant women being tested for HIV, a review of the data indicated that some of these women are lost to follow up and do not deliver safely. In response to this, KHANA has introduced a more intensive monitoring system that links all HIV-positive pregnant women to the local home and community based care team to ensure they receive the full course of PMTCT. This initiative further integrates HIV prevention into the CoPCT. This approach needs to be adopted more broadly to cover areas where KHANA is not working. 3.4 Replication and sustainability Extensive technical and organisational capacity building among the IPs has occurred. This has contributed to establishing an engaged, informed and confident LNGO sector and provides a good foundation for the sustainability of HIV prevention programs. There is, however, an 27 Data presented by KHANA at a meeting with the evaluation team. 18 ongoing need for high level technical assistance to guide overall program development and implementation. This is currently provided by the CAs. As part of a longer term exit strategy, planning for who will undertake this role is essential. The models developed by CAs for prevention with entertainment workers and positive prevention have been adopted as the national SOPs and been replicated and scaled up. CAs have also been key partners in developing new SOPs for HIV prevention and care for MSM and IDU. The linked response for PMTCT has been institutionalized into the Cambodian public health system at the service delivery level. This should ensure its sustainability. Given the significant decline in HIV prevalence among ANC women, a more cost-effective way of screening pregnant women for HIV needs to be found. Recognizing that HIV prevalence among ANC women varies by place (e.g., 0.6% in Phnom Penh and 0.3% in remaining districts28), NCHADS is considering development of a two-tier or dual strategy for HIV screening of pregnant women that will deliver more cost-effective programming.29 While issues related to PMTCT and ANC remain important, given the institutionalization of the linked response and the concentrated nature of the HIV epidemic, USAID should consider withdrawal of HIV/AIDS funding support for PMTCT. If all socially marketed condoms are sold at highly subsidized prices or there is excessive free distribution, it will be impossible for private sector marketers of condoms to compete. To ensure sustainability, condom social marketing should be done in such a way that it does not prevent or discourage other entrants to the market, whilst at the same time ensuring that condoms are available at a price where those at risk can afford them. The approach PSI has taken recognizes that there are a range of market segments with different price sensitivities. Accordingly, the price of the OK brand which is marketed to MARPs has been kept low while the price of the Number 1 condom which is marketed to the middle class has been increased. As condom use and buying habits become more established and incomes grow there may be less overall price sensitivity in the market. The approach PSI has been taking is consistent with international best practice to encourage sustainable condom markets. PSI estimates that the private sector’s share of the total condom market in Cambodia has increased from 5% in 2006 to 13% in 2010. Expansion of condom social marketing has been accompanied by significant capacity building of the private sector and it can be expected that a significant number of non-traditional outlets will continue selling condoms with or without PSI support. PSI is well placed to provide technical assistance to potential private sector marketers of condoms on a range of marketing and logistical issues. The major risk to sustainability of HIV prevention programming is that approximately 72% of Cambodia’s total expenditure on HIV prevention is funded by USAID. Priority needs to be given to advocacy to significantly increase the RGC’s funding of HIV prevention. 28 NCHADS, Estimation of HIV prevalence among general population in Cambodia, 2010. July, 2011. 29 HE Dr Mean Chhi Vun, personal communication, 2011. 19 4. Care and treatment 4.1 Outline of the care and treatment program In the USAID/Cambodia Health Program Design FY 2009 -2013 Activity Approval Document care and treatment activities fall under IR1.2 – improve access to and quality of HIV/AIDS care services; IR1.3 - improve access to and quality of HIV/AIDS treatment services; and IR1.4 – increase TB case detection and successful treatment. The activities designed under these IRs support the treatment and care objectives in the RGC’s NSP III. These objectives focus on expanding coverage and improving the quality of HIV treatment services; improvement in the quality and coverage of community and home-based care (CHBC) services for PLHIV and their families; and ensuring access to quality HIV testing and counseling services in the public and private sectors. Current USAID activities include support for voluntary confidential counseling and testing (VCCT), adult and pediatric care, support and treatment, including home and community -based care (HCBC); programs for orphans and vulnerable children (OVC); and TB/HIV-related activities. Successful grants applications to the GFATM have allowed the RGC to undertake a rapid scale up of services, with an emphasis on Continuous Quality Improvement (CQI). In FY 2011, USAID allocated $3.1 million to HIV care and treatment programming. This represents 22% of USAID’s total HIV/AIDS budget for Cambodia.30 Current USAID financial support to HIV care and treatment are described in the Figure 3 below. Figure 3: USAID/Cambodia HIV Care and Treatment Program financial allocations, FY 2011 (in $m) Source: OPHE/Cambodia PEPFAR Cambodia Overview and USAID HIV Program Overview. May, 2011 30 USAID/Cambodia, PEPFAR Cambodia Overview and USAID HIV Program Overview. May 25, 2011. This figure does not include $0.57 million for HIV testing and counseling. In NSP III, HCT comes under treatment. 20 In 2010, the MoH, with technical support from USAID, expanded the CoC model to include prevention to create the CoPCT model. The CoC was originally developed in 2003 in Battambang and Moung Ruessei with TA and funding from USAID. The CoPCT model encompasses a network of services linking facility-based opportunistic infection/antiretroviral therapy (OI/ART) services with community-based providers, and now includes TB, reproductive health (RH) and STI services. In order to assist the RGC provide services based on the CoPCT model, USAID supports a combination of direct service grants and TA through its CAs to local IPs working at the national, provincial and operational district (OD) levels. 4.2 Key achievements The remarkable progress made by the RGC in scaling up both the quality and coverage of care and treatment services for HIV/AIDS is, in large part, attributable to USAID support. The introduction and scale up of ART in Cambodia has been the biggest success in the area of treatment programming. Over 80% of all adults estimated to be infected with HIV were enrolled in care at 51 OI/ART sites by quarter 4, 2010, with 42,799 patients receiving ART and comprehensive care and support services from 337 home-based care teams throughout the country.31 NCHADS estimates that 86% of PLHIV with CD4 counts less than 350 are receiving ART, although some key informants regard this figure as an overestimate (see Section 4.3). The total number of people provided with at least one care and support service through USAID partners in FY 2010 was 23,206.32 Another key achievement has been the 102,989 people who in 2010 received testing and counseling services and obtained their results from service providers supported by USAID. This figure constituted almost 20% of all persons accessing VCCT services in Cambodia (541,080) that year.33 Given USAID’s focus on MARPs, it is likely that many of the 438,091 people who received HIV testing and counseling in 2010 without direct USAID partner support were from low risk populations. A portion of the results for USAID supported testing were achieved through the introduction of point-of-care HIV testing and counseling, using finger-prick approaches and prioritizing access for populations with high HIV prevalence, by USAID partners. Another key achievement has been the support for quality clinical care as demonstrated by the change in focus to screening HIV-positive people for TB in a USAID-supported care or treatment setting with FHI-supported facilities achieving 51% in the first year of reporting.34 These achievements demonstrate the impact that USAID-supported programs have had in assisting the RGC to meet its Millennium Development Goal 6 target related to reversing and halting the spread of HIV and providing universal access to treatment (>80% coverage of those eligible),while focusing on technical innovation and quality improvement. A key achievement is also the initiation of efforts to move away from funding direct service provision by transitioning IPs providing services to OVC to other funding sources. By the end of 31 NCHADS, Estimation of the HIV Prevalence among General Population in Cambodia, 2010. July, 2011. 32 USAID/Cambodia, FY 2010 USG Cambodia (PEPFAR and OPS) APR. This figure includes HIV-positive people and their families/partners. 33 NCHADS Annual Report, 2010. 34 USAID/Cambodia, FY 2010 USG Cambodia (PEPFAR and OPS) APR. 21 the first phase of the Project for HIV/AIDS Strategic Technical Assistance (PRASIT), FHI plans to completely phase out its support of OVC focused IPs after assisting them to link up with relevant local partners and other funding resources. This shift will help to reduce the verticality of HIV/AIDS OVC services by focusing on the integration of these smaller HIV–related activities into existing, more sustainable RGC health and social service systems serving OVC. 4.3 Strengths and weaknesses A major strength of the USAID program is the technical competence of its staff and partners. This competence has been universally recognized by the RGC as well as other stakeholders and is reflected in the inclusion of both USAID and CA/IP staff in various care and treatment technical working groups (TWG), in the development of SOPs related to the CoC, CHBC, VCCT and OVC and in the use of USAID-funded models for scale up of services. For instance, USAID CAs, such as KHANA, have been instrumental in leading the shift from the pre-HAART model of home-based care to a more sustainable model based on community support and self-help groups. KHANA’s HCBC and self-help group model was recently adopted by NCHADS in one their SOPs. While KHANA’s work in this area is an important step forward, the model being developed does not go far enough in looking at key issues related to reductions in morbidity and mortality. There is no overarching strategy to address the verticality of the HIV HCBC activities and to develop new models in terms of cost-effectiveness (linkages to existing poverty reduction schemes versus conditional/unconditional cash transfers), geographic diversity (rural versus urban models) or address the possibility some PLHIV may choose to manage without HCBC after a certain period or on reaching certain milestones (e.g., successful adherence to ART for years and increased psychological and socio-economic stability). This lack of a strategy presents a missed opportunity since some HCBC IPs, such as the Cambodian Socio￾Economic Development and Democracy Association (CSDA) in Banteay Meanchey, are already integrating successful non-HIV livelihood and poverty alleviation programs into their HCBC programs. This local expertise has not been fully utilized in developing new integrated models, despite the fact that livelihood development for PLHIV is recognized as essential for the long￾term success of care and treatment services. Similarly, clinical innovations such as the CPICT (point-of-care rapid HIV testing and counseling in venues such as drop-in-centers, with same day results) have helped to improve the numbers of persons, particularly MARPs, who receive an HIV test and obtain their results. IPs have demonstrated the ability to follow up on those who need ART immediately. On the other hand, late HIV testing as reflected by a low CD4 count at the time of first presentation to an OI/ART site or testing upon presentation with AIDS symptoms remains a persistent issue, especially in respect to MARPs, often compromising treatment outcomes.35 The NCHADS estimate of ART coverage (86% of those with a CD4 count of less than 350) appears to be too high. USAID supported research by FHI indicates that in 2011 the mean CD4 level of PLHIV at first presentation to an OI/ART site was 167. While this is an improvement on 2008 when the CD4 count on first presentation was 127, clearly people are still presenting late. It is not clear if this is because HIV incidence has declined substantially and these late presentations are outstanding cases of people who were infected some time ago or whether there is a significant 35 Personal communication, Laurent Ferradini, FHI, 2011. 22 number of undiagnosed cases of HIV in Cambodia (which would mean that the estimates of HIV prevalence may be too low). There is some data to support both of these hypotheses. For example, preliminary NCHADS data from piloting of CPICT among EWs and MSM by FHI and KHANA supported IPs in April and May 2011 found only ten HIV-positive cases from a total of 1,558 people tested. This was an HIV prevalence rate of 0.6%, which is less than the estimated HIV prevalence of 0.8% in the general population in 2010.36 On the other hand, HIV testing of Chhouk Sar’s VCCT clients from quarter 3, 2010 to quarter 2, 2011 found an overall HIV prevalence rate of 9%. Prevalence for different population groups was MSM: 20%; IDU/DU: 22%; EWs: 7%; low risk women: 9%; and other males: 11%.37 The KHANA data could be interpreted to mean that the HIV prevalence rate among EWs and MSM has dropped significantly (and much below the prevalence for these groups in 2010 NCHADS surveys) or that the MARPs tested as part of the CPICT were not representative of these populations more generally. The FHI data could be interpreted to mean that Chhouk Sar is attracting sub-populations with higher prevalence than is the case for these populations as a whole or that there is a significant number of undiagnosed HIV-positive people. It is not possible to come to any conclusion on these competing hypotheses as there is insufficient data. Improved case finding would be an important strategy if the hypothesis of a significant number of undiagnosed cases of HIV is correct. To take a broader view, in 2010, the total number of VCCT clients tested for HIV was 350,763 of whom 8,639 or 2.5% returned HIV-positive results.38 Not surprisingly the HIV prevalence rate is above estimates for the general population as MARPs are more likely to be tested for HIV compared to other people. The number of HIV diagnoses in VCCT clients has dropped from 8.1% in 2006 to 2.5% in 2010. The decline in the prevalence rate among VCCT clients could be seen as broadly paralleling declines in HIV prevalence among the general population and some MARPs. However, the number of persons tested for HIV increased by approximately 138,000 over this period which may mean there is a higher proportion of low risk people being tested. USAID partners are doing little to improve the overall CoPCT system related to linking and follow-up those who are not in need of immediate ART, but who can benefit from early, regular clinical care, positive prevention, RH/birth spacing, nutritional assessment, counseling and other services. Novel ideas for improving linkages from testing to care and treatment, such as Testing and Linkage to Care and Treatment (TLC+), that will become important in the light of clinical advances in HIV, are not being pursued. TLC+, which is being trialed in various countries, aims to: encourage local programs to integrate HIV testing, care and treatment with prevention efforts by expanding regular HIV testing as a routine part of medical care; target testing to high-risk individual; link HIV-positive people to care and social services by working intensively with newly diagnosed patients and those who already know their status; 36 NCHADS, Progress Report on CPITC for Q2 2011. 37 FHI, Chhouk Sar Clinic as a model of HIV service integration for MARPs. PowerPoint presentation to the Evaluation Team, May, 2011. Only very small numbers of MSM and IDU/DU were tested so HIV prevalence data for these groups should be treated cautiously. The numbers tested for other populations groups were EWs: around 800; other males: more than 590; and low risk women: approximately 540. 38 NCHADS, Annual Report, 2010. pp 13-14. 23 promote regular doctors’ visits after diagnosis with quick evaluation of the need for HIV therapy and advice about options for when to start; counseling HIV-positive people about how to prevent passing HIV whether or not the person decides to start treatment; helping patients understand the importance of taking their medicine consistently and helping them to identify issues that need to be addressed to support adherence; and assuring linkage to supportive services such as substance abuse treatment, livelihood development and childcare, which are essential for many patients to enter and remain in care. (See www.projectinform.org/tlc+) The examples cited are indicative of an overall decrease in prioritizing innovative, ‘outside the box’ thinking that characterized USAID activities in the past. This may be due to the fact that USAID CAs are in the burdensome position of simultaneously being responsible for scaling up direct service provision and quality assurance. That is, making sure that IPs follow national SOPs and monitoring achievement of targets while also being held up as technical leaders by USAID and other stakeholders. In some cases, the CAs (and others) appear reticent to raise difficult issues that affect care and treatment, such as the OI/ART supply chain logistics and management. The inability of the system to ensure a timely and adequate supply of ART leaves OI/ART site staff fearful of stock outs. They therefore provide patients with a short-term supply of drugs, forcing them to return to their OI/ART sites on a monthly or even weekly basis. ART insecurity results in the need for PLHIV to remain in close proximity to OI/ART sites and prevents healthy PLHIV from traveling for work or even accessing a regular employment schedule, thus contributing to dependence on HIV service providers and continuing the cycle of poverty. The need for more frequent visits due to rationing also increases the patient burden on OI/ART site health care providers and increases the need and demand for more transportation subsides from HCBC programs. The ramifications of an ineffective supply chain management system also contribute to an overall increase in care and treatment costs by negatively impacting on adherence, with resultant drug resistance and an increase in the need for more expensive second line ART, particularly if a PLHIV has to choose between being forced to remain close to health services or pursuing an income. 4.4 Challenges The major challenge to the current care and treatment system is also a product of its success. As more people test, those who receive a positive result will at some point access OI/ART and care services, increasing the overall number of potential program activity beneficiaries. Unless funding increases, there may be an ‘overload’ on the current system and quality may suffer. The continued reliance on SOPs as something to be religiously followed as opposed to serving as a minimum standard for quality also compromises the ability to respond quickly to clinical advances in care and treatment. Efficiently programming limited human and financial resources for innovation is another challenge. The RGC realizes the need to develop ‘high impact, low cost’ interventions, while continuing to maintain current levels of quality and coverage. While USAID contributes to the coverage and quality of care and treatment programs through direct service provision, it also represents one of the few funding sources which can assist in testing new, innovative and cost￾effective models. USAID, together with other stakeholders, faces the challenge of initiating a dialogue with the RGC to develop a strategy whereby cost-effective, state-of-the art interventions are identified to replace outmoded interventions and brought to scale. 24 Some USAID-supported CAs also receive funding for scaling up services from other funding sources such as the GFATM. Unless there is agreement by other funders to allow USAID funding support to these CAs to be used for cutting-edge operations and implementation research to develop cost-effective care and treatment models, it will be difficult for the co-funded programs to find the space needed to actively pursue innovations. 4.5 Replication and sustainability One of the lessons learned during USAID’s involvement in HIV care and treatment in Cambodia is that a technically competent, quality system for delivering HIV care and treatment can be scaled up. The primary funding source for HIV care and treatment in Cambodia, the GFATM, is constrained in its use of funds for innovation. However, the GFATM has provided funding to scale up models developed with USAID funding, providing proof that replication of quality, cost￾effective models is not an issue. Sustainability, on the other hand, is a major concern. Cambodia’s inability to secure GFATM funding in Round 10 and its ineligibility for Round 11, has brought home the need to find more innovative and cost-effective ways to provide care and treatment. The challenge will be for the RGC and its development partners and stakeholders to analyze the changing environment and devise an agreed upon strategy to address the current state of affairs. 25 5. Strategic information This section deals with monitoring and evaluation, surveillance and research. Health information systems is discussed in Section 6: Health Systems Strengthening. 5.1 Monitoring and evaluation There is a strong culture of monitoring and reporting on activities and using this data to inform program development by USAID, CAs and their IPs. The Strategic Information Advisor at USAID/Cambodia has developed a consolidated performance monitoring plan (PMP) which is made up of PEPFAR indicators and a select number of other indicators drawn from the monitoring frameworks developed by each CA. The list of indicators in the PMP integrates indicators from all health programs supported by USAID/Cambodia. Development of the PMP is an achievement, although there is room for improvement. OPHE rightly sees this as an iterative process. Currently the PMP has an impact indicator (i.e. reduced HIV prevalence), and IRs which are mostly at the output level (e.g. improve access to and use of HIV/AIDS preventive services), although some IRs are at the outcome level (e.g. improved quality of HIV care and treatment services). Most of the indicator data collected from CAs is at the input, process or output levels. The most significant gap is the limited number of outcome indicators. This is recognized by USAID staff. There is a need for OPHE to consider the best ways of measuring outcomes. This may include a combination of approaches involving development of outcome indicators and a limited number of focused outcome evaluation studies. A particular area of concern is the measurement of treatment outcomes, including treatment failure as the CQI methods used by NCHADS are not considered statistically robust. Both CDC GAP and FHI have offered technical assistance to NCHADS to develop indicators for analyzing treatment effectiveness, although the issue remains sensitive. At present, Results Framework indicators relate primarily to service delivery but not to capacity building. As the program transitions from service delivery to technical assistance, outcome measurement should shift to measuring the outcomes of capacity building. The OPHE is currently working with each CA to improve their existing results frameworks. A life of project cycle approach is being taken to measure progress in achieving targets set out in cooperative agreements or contracts. The revised frameworks will contain baselines, annual targets, actual performance by year, and the extent of the CA’s contribution to USAID and national targets for all indicators they are reporting on to USAID/Cambodia. OPHE plans to complete this work by August, 2011. The consolidated results framework for each CA will then constitute an enhanced USAID Results Framework which will improve CA accountability. Core PEPFAR indicators are aligned with global level WHO, UNAIDS and GFATM indicators which are also used in the RGCs HIV/AIDS Monitoring and Evaluation Framework. A challenge for USAID/Cambodia as it develops a more sophisticated monitoring system is to promote alignment with the national HIV Monitoring and Evaluation Framework. This alignment already exists in the area of HIV treatment as most of the indicators used by USAID are reporting on MoH services. There is considerably less alignment with prevention services as the RGC is not providing this type of service. At USAID and CA levels there is demonstrated use of data and operational research as a tool to inform programming. This is less evident in the CA’s IPs. This may relate to limited analytical 26 skills at this level. Examples of where data has been used to inform programming are the scaling down or terminating HIV prevention programs for groups at low risk (by RHAC, RACHA and FHI) and the use of monitoring data and population size estimates to calculate coverage. However, problems with population size estimates for MARPs limit the ability to come to reliable conclusions on coverage. Monitoring of contacts with MARPs and reducing double counting has improved, with all CAs shifting from counting episodes of service delivery to recording the number of individuals reached and frequency of contact. The lack of a common unique identifier code (UIC) across CAs has resulted in double counting where the same person is reached by multiple CAs or their IPs. From FY 2010, the methodology for de-duplication has been improved by OPHE. Work currently underway to develop a common UIC is discussed in the health information systems part of Section 7. Overall, the emphasis of CAs has been on monitoring. There are signs of a growing emphasis on program evaluation. For example KHANA has been developing baselines for each program area and doing external evaluations for several years. A greater emphasis on evaluation could be achieved using existing data from multiple sources (e.g. monitoring data, surveillance data and small scale operational research), to answer key questions about the epidemic and program effectiveness. This should be supplemented by small scale evaluation studies, focused on priority questions. Currently, there is no methodology or tool that adequately captures the quality of programs. At present, quality is primarily assessed by regular monitoring visits by CAs to their IPs, using checklists based on standard operating procedures. The strength of this approach is that standard operating procedures have generally been developed using evidence based best practice. The monitoring checklists provide a standardized criteria upon which to assess quality, allowing for comparisons over time. The weakness is that monitoring visits are reportedly focused on checklists, with insufficient attention given to encouraging a dialogue between CAs and IPs on problems encountered and how to improve the quality of services. 5.2 Surveillance There has been a high level of collaboration between CDC GAP and FHI in the provision of technical assistance to NCHADS for STI and HIV and biological and behavioral surveillance surveys. This has resulted in considerable strengthening of the capacity of NCHADS. Over the last decade, NCHADS has usually collected biological and behavioral data for sentinel most at risk populations separately. These surveys have been conducted infrequently, resulting in a lack of data to guide programming. For example, until the 2010 HIV Sentinel Surveillance data was released in mid-2011, the most recently available HIV surveillance data for MSM was from 2005 and for FSWs/EWs was from 2006. Following strong advocacy from USAID, NCHADS has now decided that all future STI and HIV surveillance surveys will be integrated biological and behavioral surveys (IBBS). The NCHADS draft Strategic Plan 2011-2015 provides a clear timetable for annual IBBS surveys, with different MARP groups being surveyed each year. This transparent approach, also advocated for by USAID, will allow CAs to plan for evidence-based reviews of programming. 27 5.3 Research All of the CAs have undertaken good quality operational research funded by USAID which has been widely disseminated and applied in programming. For PSI this includes TRaC surveys of EW, MSM and HRUM to identify determinants of consistent condom use; to monitor changes in behaviors and behavioral determinants over time, and to evaluate the impact of communication campaigns; and MAP studies to measure condom availability in the vicinity of high risk venues (e.g. entertainment establishments). FHI studies include Bros Khmer I and II which has evolved from a cross sectional to a cohort study to provide longitudinal biological and behavioural data on at-risk urban men. Use of technology has been applied by FHI research projects including GPS mapping of hot spots and use of ACASI for collection of sensitive behavioural data. FHI has also been a leader in applied clinical research. For KHANA this includes the 2010 KHANA Network Household Economic Livelihoods Survey. KHANA and RHAC have conducted surveys of HIV knowledge, attitudes and risk practices among MARPs and other targets and RACHA has undertaken some small scale operational research examining the family planning needs of PLHIV in Koh Kong. While there has been some joint research between CAs, (e.g. Let’s Go for a Walk, the study of sexual decision making by clients of entertainment workers was conducted by FHI and PSI), most research is conducted by one CA working alone or as a collaboration between the CA and an RGC counterpart. There is scope for greater collaboration between CAs in discussing research plans and where appropriate in conducting joint research. This could result in cost savings. More jointly conducted research with government could help to develop capacity and enhance sustainability. In general, operational research capacity is strongest in international CAs. Priority needs to be given to further developing the capacity of Cambodian CAs to design, conduct, interpret and utilize operational research. Currently, there is no national HIV research agenda. The NCHADS draft Strategic Plan 2011-2015 recognizes the importance of research and sets targets for the number of research studies per year (3), but does specify priority areas. Given the mandate of NCHADS, their involvement has mostly been in the area of clinical and health services-related research in partnership with international institutions and driven by opportunities presented to rather than initiated by them. This is likely to continue to be the case. Therefore CAs will need to identify research needs and funding opportunities and approach NCHADS, rather than wait for opportunities to arise. In addition to this, social and operational research of relevance to prevention programming is needed and, when possible, should be guided by a prioritized national research agenda. USAID and its CAs are well placed to assist in development of such a plan. Although population size estimates for some MARPs have been conducted by NCHADS, FHI and KHANA, the consensus of expert opinion is that none of these studies are reliable. This makes it difficult to determine the number of MARPs that need to be reached and estimate the extent of coverage achieved. The self-identity of people externally identified as MARPs are not well understood by those conducting the estimates, many of these people are ‘hidden’ and discrimination inhibits participation in studies. The methodology for estimating the number of entertainment workers varies between provinces. The extent to which non-entertainment establishment sex workers (e.g. street based and freelance) are included in estimates is limited. Estimating the number of men who have sex with men is particularly challenging due to many MSM being hidden 28 Surveillance studies have indicated that not all members of most at risk populations are at the same level of risk for HIV. For example, the 2006 HIV Sentinel Surveillance found that HIV prevalence among FSWs varied significantly by province, age and duration of sex work. More recently, changes to the structure of the sex industry following the new anti-trafficking laws and the Community Safety Policy may have introduced another differential with an increase in the number of indirect sex workers (i.e. entertainment workers) and segmentation within this sector. An improved understanding of the determinants and different levels of risk within MARPs would enable more sophisticated geographic and sub-population targeting by prevention programs. That is, concentrating programs on those MARPs at highest risk and scaling down or terminating interventions for those where the risk is lower or low. This would result in greater cost efficiency in prevention programming. The integration of biological and behavioural surveillance surveys will improve knowledge of relative levels of risk. Understanding would be further enhanced by a modes of transmission study which typically undertakes an analysis of the most recent HIV infections in relation to behavioural, biological, socio-economic and demographic data to identify the key factors driving the epidemic. The results allow for an assessment of the degree of alignment of national prevention resources with the populations at greatest risk. There is also a need for qualitative studies such as ethnographic studies (e.g. a study of sexual networking among MSM and bridging to female partners). KHANA has opened four services known as demonstration sites, three of which target different MARPs (EWs, MSM, IDU/DUs) and one focussing on livelihood support for PLHIV. The stated purpose of these sites is to develop local evidence of best practice models to inform model development and replication. The evaluation team has concerns regarding KHANA’s shift into direct service delivery and the implications this may have for relations with its IPs, possible duplication, and hence cost inefficiencies. There is no evidence of a clearly articulated operational research plan or monitoring framework in relation to demonstration of best practice. 29 6. Health system strengthening Cambodia’s health system remains fragmented, with several disease-specific programs, such as those focusing on HIV and TB, having achieved their Millennium Development Goals without significantly improving broader health outcomes in areas such as maternal mortality, nutrition, and child survival. Nevertheless, the evaluation team concluded that USAID’s investments in HIV/AIDS are incrementally helping Cambodia’s health system to become more robust, and believes that the current overall health system would be in a much poorer state had the HIV program not been implemented and used as a platform for building other parts of the system. Since its early days, USAID’s approach to HIV/AIDS programming has been consistent with the principles of health system strengthening. The focus on the CoC model and its evolution into the CoPCT has paved the way to better linkages and integration of services, with a focus on quality, coverage, using data for decision-making, and developing cost-effective models. 6.1 Health service delivery The response of Cambodia’s health sector to HIV/AIDS, under the mandate of MoH/NCHADS, and with significant assistance from USAID, has demonstrated that the public health system is capable of delivering a quality health service. The impact of this accomplishment on both the supply of and demand for other health services should not be underestimated. On the supply side, the HIV program provides an aspirational goal for other parts of the public health system. Health care workers in other areas are able to see what quality services should look like at all levels of the system and learn by example. Meanwhile, clients can see the quality of HIV services and begin to demand the same quality in other services. USAID’s strategic placement of HIV funding into URC’s broader health system strengthening program facilitated the improvement of health elements across platforms. For example, the HIV and TB programs benefited from URC-sponsored infection control activities as well as from on￾going efforts to reduce barriers to transferring patients across wards. HIV funding of URC also resulted in an expansion of pre-identification criteria for the Health Equity Fund (HEF) so that MARPS and PLHIV would more likely be eligible for its benefits. Placing HIV funding into mainstream service providers like RHAC and RACHA has also benefited the overall public health system--in at least two important ways. First, these partners worked intensively at the OD level in areas such as training and integrated supervision systems, which helped the system to function across vertical national programs. Secondly, these two partners were instrumental in modeling integration of HIV education into reproductive health and family planning services in health facilities and through use of community mechanisms such as village health volunteers, including providing support for the linked response and linking communities with the health system. These efforts, complemented by integrated models such as FHI’s Chhouk Sar clinics and by social marketing programs that build the capacity of the private medical sector to deliver integrated services, were early steps in the breakdown of parallel structures and are helping move Cambodia toward a more integrated model of health service delivery. To a great extent, they have now accomplished what they set out to do. For example, the linked response has been institutionalized within the health system. USAID and CA advocacy at the central level for linked and integrated services, based on proven Cambodian models (frequently developed by USAID) and international good practices, appears to have influenced key decision-makers, who are now increasingly moving towards integration 30 of HIV into the general health system. Most notably, the MoH is now moving the 19 Family Health Centers (i.e. STI services) so that they are co-located at referral hospitals, where VCCT is available. Additionally, in late mid-2011, family planning (FP) and RH will begin to be integrated into the OI/ART clinic at Battambang, as a first step in broader integration. As described in several sections of this report, USAID-supported service delivery models have consistently been used by the RGC as the basis for SOPs. These SOPs have played an important role in enhancing and ensuring the quality of HIV services. The evaluation team, however, observed that many health care providers now use the SOPs as rote checklists, viewing them as maximum service standards, rather than minimum service standards. Unless deliberate effort is put into continuous improvement of the SOPs, the SOPs may stifle future innovations and improvements in quality. Cambodian health services remain provider-centered, rather than client-centered. The burden is on the clients to follow up on his/her referrals and move from one service to the next. While the recently instituted tracked referral system is a positive step in facilitating referrals, the system lacks a mechanism for ensuring that people actually use their referral (particularly the one from VCCT to OI/ART), and follow-up is often weak. Although progress has been made, many HIV clinical services are still not MARP-friendly, with the attitudes of health care workers towards MARPs, hours of operation, and lack of confidentiality creating barriers to access. Several USAID partners are working on these barriers. For example, clients in RHAC clinics and Chhouk Sar I stated that they preferred these clinics due to the good attitudes of the staff. The evaluation team also noted that the RHAC clinics it visited had successfully recruited good MSM service providers, had designed their clinics to be more attractive to EWs (i.e. the brightly painted examination room for EWs in Siem Reap), and included youth-friendly corners – dedicated areas for youth where they can read health information, watch videos and play games while waiting to see the health care worker. These innovations and good practices, however, have not expanded into the public sector. Similarly, while opportunities for outreach to MARPs are beginning to be explored with USAID support, much remains to be done. The evaluation team also observed that there are many missed opportunities for cross-platform work using community-based care providers. A myriad of community-based systems exist for linking community members to specific health care services, including TB directly observed treatment short course, integrated management of childhood illnesses, community-based distribution of family planning products, and HIV-related HCBC. In addition, Commune Councils and Community-based Health Cooperatives are becoming more active. While each system and group has value, the evaluation team believes that there are opportunities for making linkages in ways that would result in better overall health outcomes and cost-efficiencies. There may also be opportunities to further use community-based groups for active case findings, thus helping to more cost-effectively identify hard-to-find HIV positive individuals who are not yet accessing care and treatment services. Given Cambodia’s concentrated epidemic, the evaluation team recognizes the challenge inherent in using HIV funding for strengthening health service delivery in a way that improves overall health outcomes, while also measurably improving HIV-related outcomes. 31 6.2 Health workforce USAID CAs have trained health care workers at various levels of the health system. Several partners have also worked to address the attitudes of health care workers towards MARPS and PLHIV and reduce levels of stigma and discrimination, which are still high in health care settings and create a barrier to accessing services. Nevertheless, health workforce challenges include low salaries, lack of motivation and limited training. Until recently, many health care providers received salary supplements from GFATM and other donors, which, while boosting morale, often resulted in distorted priorities. The RGC has determined that salary supplements are no longer allowed. The evaluation team was unable to assess the impact of this decision on the HIV/AIDS response. While USAID is not using its HIV funding to specifically address overall health workforce issues, several of the health financing options discussed below are expected to have an impact on the health workforce (as well as on the quality of health services) by helping to link pay to performance. In addition, URC’s work with licensing physicians, developing clinical practice guidelines, and developing and implementing continuing medical education, while not targeted at HIV, will help to strengthen the health workforce. 6.3 Health information systems USAID assistance, through URC, has achieved significant success in helping Cambodia to unify its Health Information System (HIS). The web-based HIS developed by URC, in collaboration with the MoH, is now fully functional within the public sector. It currently collects aggregated data from over 1,000 health facilities. Although HIV data is not yet included in the system, URC’s continued advocacy with NCHADS seems to be having a positive effect. Work on electronic medical records and other systems that will make the HIS more robust is proceeding. To institutionalize the data base, URC provides training in HIS and data entry to the MoH. Data quality audit tools are also under development to ensure the quality of data generated by the HIS. Several USAID CAs have been working on the development of a UIC that will allow monitoring systems to track individuals and the services they receive from one IP to another, thus enabling better continuity of care for clients reached by prevention programs and accessing the public health system. A UIC will eliminate double counting of individuals, resulting in improved estimates of coverage. While there is general agreement that the development of a UIC is a priority, the evaluation team concluded that progress has been slow, in part due to different visions among the USAID CAs, lack of agreement around issues such as confidentiality and data privacy, an inability to work together more collaboratively, and some mixed signals from OPHE. 6.4 Access to essential medicines Access to essential medicines continues to be a barrier to improved health outcomes in Cambodia. USAID’s support for social marketing has resulted in expanded access to a broad range of health products and services in the private sector, but supply chain issues within the public sector, particularly concerns with stock outs of antiretrovirals, limit the ability of Cambodia to move from an emergency response to a chronic care model of treatment for HIV (see Section 4 for additional detail). 32 6.5 Health systems financing The RGC’S complacency with its very low level of funding of the national HIV/AIDS response was recently shaken by the failure of Cambodia’s Round 10 Global Fund proposal and the GFATM decision that Cambodia was not eligible for Round 11 funding. In its discussions with senior RGC officials, the evaluation team observed increasing recognition that levels of outside investment in HIV could decline in the coming years. As a result, the RGC is interested in ‘low cost, high impact’ models that do not compromise quality or coverage. This need for cost efficiency is partially driving the RGC’s interest in integrating HIV into the broader health system. Given the above, USAID’s involvement in innovative health financing is particularly timely, although much of it is still in the planning or early implementation stages. The work around performance-based contracting being carried out by URC, FHI and PSI is beginning to break traditional mind-sets that view funding as a ‘right’ rather than as something that should be tied to service quality. Given the positive impact of incentives on other aspects of the HIV portfolio, the potential for performance-based contracting seems quite high. Nevertheless, as USAID has control over only some 20% of Cambodia’s annual HIV funding, its ability to influence systemic changes is somewhat limited. Given that each of the USAID partners is adopting a slightly different approach to performance-based financing, it will be necessary to carefully document, monitor and evaluate each model to be able to compare impacts and outcomes. URC’s work with the HEF and Community-based Health Insurance is also innovative and shows great promise. Expanding HEF eligibility criteria to better reach MARPS and PLHIV is a clear accomplishment. However, the evaluation team believes that many poor PLHIV are not being adequately reached by the HEF. FHI’s plans to franchise their branded programs (SMARTgirl and M-Style) offer another cost￾effective means of funding prevention program. Again, this will need to be carefully monitored for effectiveness. 6.6 Leadership and governance Coordination of Cambodia’s HIV response is challenged by the existence of four parallel spheres of coordination, each with its own structures that link the center to provinces, districts and villages: (1) the NAA, (2) MoH/NCHADS, (3) Global Fund and the various donors, and (4) LNGO and MARP networks.39 Difficulties with streamlining leadership and coordination, setting priorities and addressing other health sector inefficiencies jeopardizes the RGC’s standing with donors. To date, USAID has focused its efforts on collaborating with and building leadership and technical capacity within the NCHADS, while also supporting capacity building technical and organizational capacity within the LNGO and MARP networks. The sustainability of these efforts is threatened, however, by the dependence that the various organizations have on key personnel. As of yet, the organizational capacity of many institutions is not robust enough to continue to function effectively should their leadership change. 39 Personal communication, Dr Tia Phalla, June 2, 2011. 33 USAID was actively involved in the development of Cambodia’s NSP III and has aligned its priorities to that plan. The active participation of USAID and its CAs in NCHADS TWGs have provided an opportunity to influence technical policy and protocols, enhance the quality and cost effectiveness of prevention and care and treatment programs, and strengthen surveillance and service delivery systems. As NCHADS and the MoH increasingly move toward integrated, more cost effective health services, it will be critical for USAID to flexibly tap into its strengths in policy development, health system strengthening and strategic information for decision making to respond to Cambodia’s changing needs and priorities. USG involvement in the Global Fund, including the CCC, the oversight committee, and the proposal development committee, have helped ensure effective use of GFATM resources—many of which are used to replicate and scale up models developed by the USG. Support for effective use of Global Fund resources will be enhanced by the new Global Fund Liaison position within USAID/Cambodia, offering increased scope for USG assistance in harmonizing programs and reducing duplication of efforts. USAID support for technical and organizational capacity building within LNGO and MARP networks has been critical to the success of prevention, care and treatment efforts. KHANA has been USAID’s primary partner in this area and currently works with a network of 41 partner organizations, using its Key Performance Monitoring System to track organizational performance. The evaluation team is concerned, however, with KHANA’s recent decision to develop four demonstration sites, managed directly by KHANA. The evaluation team questions whether direct implementation is the an appropriate approach, versus an approach of strengthening current partners—which is KHANA’s recognized strength. 34 7. USAID/Cambodia management of the HIV/AIDS portfolio 7.1 Overview USAID/Cambodia’s HIV team has consistently provided strategic leadership to the portfolio of activities, appropriately targeting MARPs, responding flexibly to the changing needs of the epidemic in Cambodia, supporting the RGC’s national HIV/AIDS plan, and focusing on building the capacity of local partners. At the national level, the USAID team has developed strong working relationships with the RGC, particularly NCHADS, and the other donors. Both the Director of NCHADS, His Excellency (HE) Dr Mean Chhi Vun, and the Deputy Vice Chairman of the NAA, HE Dr Tia Phalla, explicitly stated their appreciation of the role USAID has played in the national HIV response, above all in developing models for replication and scale up. They clearly expect USAID to continue to play a critical future role in developing low cost, high impact models that will help the national response become more sustainable. Dr Vun specifically praised the addition of the Global Fund Liaison position at USAID as a means of strengthening quality and partnerships. At the donor level, the active involvement of Dr Michael Cassell, USAID/Cambodia’s Senior Technical Advisor for HIV/AIDS, as chair of the donor coordination working group has contributed to increased consensus around donor approaches, as evidenced by the recent ‘principles document’. The six CAs receiving HIV funding uniformly appreciate the clear and strategic guidance received from the USAID HIV/AIDS team. Dr Cassell is particularly valued for his combination of both technical and administrative knowledge, as well as for his excellent follow up on outstanding issues. In addition, CAs feel that information on new directions and areas of emphasis are quickly and effectively communicated to them. CAs also appreciate the solid technical strength of Dr Sok Bunna, OPHE’s Team Leader for HIV/AIDS, for bringing continuity and vision to the program and for ensuring that the Cambodian context is understood. The important contribution of Prateek Gupta, the Strategic Information Technical Advisor, was also noted by the CAs, particularly his efforts at improving the performance monitoring plan, eliminating double counting and ensuring data quality. One CA also mentioned that USAID/Washington has consistently provided good support to the program, with experts from Washington visiting frequently and providing quality input. The strong coordination and collaboration between USAID and CDC GAP was evident. CDC GAP was included in key evaluation-related meetings with USAID staff and participated in the evaluation team’s field visits. Several USAID CAs described their close working relationship with CDC GAP on various activities. The US Peace Corps also expressed strong interest in collaborating closely with USAID and CDC GAP under the PEPFAR program, but has not yet identified an appropriate role through which they could add substantive value. Given the urban nature of Cambodia’s concentrated epidemic, the Peace Corps’ potential to make a significant contribution to the USG program may be constrained by its focus in rural areas. This highly positive feedback is particularly notable in light of the small number of people working in the HIV/AIDS team in USAID/Cambodia. Despite its small size, the team is required to complete virtually the same amount of planning, monitoring, reporting and responding to ad hoc requests as larger PEPFAR countries with significantly more staff. The evaluation team observed that this heavy administrative burden has negative repercussions on performance of the portfolio and limits the ability of USAID’s staff to use their strong technical skills to maximum 35 effect. For example, the heavy administrative demands appear to keep staff tied to their desks, rather than in the field where their technical input would improve the quality and effectiveness of programming. Even site visits in Phnom Penh, where a large proportion of USAID-funded activities take place, are rare. Yet, many of the ‘issues’ identified by the evaluation team would have been quickly picked up by the USAID HIV/AIDS team and resolved had they been able to get out of the office (for site visits, not meetings) more frequently. The heavy administrative burden has also affected the HIV team’s ability to adequately coordinate and direct its partners. USAID has relied too much on the CAs to coordinate amongst themselves. This is most obvious in the area of duplication of prevention efforts, as described in more detail in Section 3. Given that the incentives for CAs to resolve this issue are limited, strong leadership on the part of the HIV team is needed. The CAs noted that regular partner meetings that provide a forum for coordination are not taking place. The evaluation team also noted that greater direction and involvement from USAID’s technical staff is required to successfully implement the on-going transition from a direct service delivery model to a technical assistance model. Again, CAs incentives to build local capacity and thus ‘work themselves out of a job’ are limited; unless outcomes are clearly defined and USAID closely monitors the process, progress could be slow. In considering how to reduce the administrative burden placed on technical staff, OPHE should consider options such as task shifting and hiring additional administrative, non-technical staff. One approach to this would be for the HIV team and OPHE management to keep a detailed inventory of the time they spend on all tasks. This could be used as a baseline and to determine how much time should be spent on different tasks. This would provide information on what tasks need to be shifted. Consideration could be given to who might be able to take on particular tasks and whether additional staff are needed. In thinking of how much time should be spent on different tasks, OPHE should consider the technical direction that will be needed to support follow-on awards and not just current workloads. 36 Annex 1: Scope of work USAID/Cambodia seeks an expert team to evaluate the performance of the mission’s current HIV/AIDS investments, and to make recommendations for investments over the next five years based on epidemiological priorities, good and promising practices, and service delivery gaps. Key evaluation questions In particular, the mission anticipates that the evaluation team will focus its attentions on addressing the following priority questions, relevant to 1) past program performance, and 2) strategic priorities for the future: Question 1: How well have past mission investments in HIV/AIDS performed? In particular: To what extent has the USAID-funded HIV/AIDS program achieved its objectives as specified in the mission’s Activity Approval Document and in the existing partner agreements? What have been the program’s major achievements and constraints with respect to these objectives? What, if any, additional impacts have the HIV/AIDS program had on the quality, coverage, and/or efficiency of HIV services in Cambodia? To what extent are HIV resources placed in ways that strengthen the overall health system and improve health outcomes more broadly? What evidence do we have of impacts on health system? To what extent do the activities make a significant contribution to the MoH’s HSP II (2008-2015)? To what extent have existing investments been strategic in terms of addressing key local epidemiological priorities and service gaps and avoiding duplication of effort? To what extent are partners addressing felt or actual beneficiary needs? To what extent are existing partners serving as technical leaders and advancing good and promising practices in HIV prevention, care and treatment? How have existing projects contributed towards priority policy reforms? How well have partners collaborated to improve the quality, coverage, and impacts of HIV/AIDS programming? How well are existing service linkage and referral systems working? Are there any noteworthy areas of synergy or duplication across USAID-funded and other donor-funded HIV/AIDS activities? To what extent has the HIV/AIDS program produced the data needed to inform strategic planning? What recommendations does the evaluation team have for near-term modifications, of existing strategies or agreements to enhance and improve overall program performance and impact? What key lessons have we learned from current mission HIV/AIDS investments and activities? 37 Question 2: How should the mission invest resources in HIV/AIDS during the next five years? In particular: What should be the scope and priority focus areas for future USAID investments in HIV/AIDS in Cambodia? What should the mission consider doing: 1) more of? 2) less of? 3) differently? What kinds of surveillance or special studies need to be done to better align programming to beneficiary needs in the future? What strategies should USAID adopt to promote sustainable HIV prevention for MARPS? How should USAID condom social marketing investments evolve? What should be USAID’s level of focus and involvement in implementation of care and treatment efforts in future? How could HIV resources be better placed to strengthen the overall health system and improve broader health outcomes? What strategies should USAID adopt to improve the quality and coverage of integrated services to address the comprehensive health needs of clients, particularly the need for access to voluntary family planning services? How can USAID investments help Cambodia to develop more cost-effective intervention models and reduce its dependency on donor support? How can or should USAID compliment the support and activities of other donors and RGC? Given the staggered project end-dates among the current HIV/AIDS activities, what is the most suitable course of action for follow-on design/s to prevent potential gaps in service delivery and program implementation? To what extent should USAID consider restructuring its investments and procurements to focus in the following four key areas: 1) prevention programming for most-at-risk-populations and strategic information, 2) social marketing, 3) improving HIV case management, and 4) FP/HIV integration. 38 Annex 2: Methodology Timeframe The framework for the evaluation was the last 5 years. However, given the USAID Cambodian Health Program Design Activity Approval Document covers the years FY 2009-2013 and the dynamic nature of the HIV response in Cambodia, the evaluation primarily focused on the FY 2009-2010 and the first eight months of FY 2011 as these are most relevant in assessing current performance and providing the basis for a strategic assessment of future directions. Evaluation methodology The evaluation methodology was designed to be consistent with USAID’s Evaluation Policy (January, 2011). The type of evaluation specified in the Scope of Work fell within the category of evaluation defined as a ‘performance evaluation’ by the Evaluation Policy. This type of evaluation focuses on ‚descriptive and normative questions: what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management, and operational decision making.‛40 The key components of the methodology for this evaluation were as follows: 1. Document review The following categories of key background documents were reviewed: USAID/Cambodia documents: including the Cambodia Health Program Design 2009-2013 Activity Approval Document; the USG PEPFAR Cambodia HIV/AIDS Strategies 2006-2010 and 2011-2015 (draft); USG PEPFAR Cambodia Country Operational Plan FY 2010 and 2011 (Executive Summary); the cooperative agreements or task orders for the six Cooperative Agencies; the current year work plans for the six CAs in receipt of USAID HIV/AIDS funding; and available data on mapping of USAID funded HIV services. Government of Cambodia documents: including the National Strategic Plan for Comprehensive and Multi-sectoral Response to HIV/AIDS III (2011-2015); the NSP III costing document; key sub￾strategies (NCHADS, Ministry of Interior, MSM, Illicit Drug Use & HIV/AIDS; PMTCT); and key standard operating procedures and operational plans (including the Linked Response, Female Entertainment Workers, MSM, Positive Prevention). Global Fund documents: including the Round 7 and Round 9 Grant Agreements and Performance Reports. Reviews and evaluations of USAID and non-USAID HIV/AIDS programs. These will include key reviews of the national response such as Turning the Tide and the Situation and Response Assessment conducted prior to development of NSP III. Relevant reviews of key projects and programs, particularly of USAID funded projects, will also be studied. 40 USAID, Evaluation Policy. Bureau for Policy, Planning and Learning, January 19, 2011. p. 4. 39 2. Review of performance indicator data The performance indicator data collected by USAID/Cambodia was reviewed to identify key outputs and where possible outcomes. It should, however, be noted that this data primarily measures outputs rather than outcomes. This was supplemented by a review of key performance indicators for the six CAs and review of key indicator data for IPs who were included on site visits. Key focus areas were trends in output data for key activities (e.g. outreach, drop-in-centres, referrals, clinical services); coverage of services – both geographic and total number of clients in relation to population size estimates; and frequency and intensity of contact with beneficiaries. As this was a review of the HIV/AIDS portfolio as a whole the indicator review was primarily focussed on aggregate data from the six CAs. However, individual CA and implementing partner data was examined to assess performance in relation to some key indicators where data was not collated centrally. 3. Review of surveillance data The review of CA performance related data was supplemented by a review of HIV and behavioural sentinel surveillance data for key population groups. Where HIV prevalence data is available this will provide partial evidence of the impact of the national response, to which USAID has been a significant contributor. Of particular interest will be outcome data on trends in risk behaviour (e.g. consistent condom use) and health seeking behaviour (e.g. STI and HIV testing) for key MARPs. (See Section 1: Introduction for an outline of limitations, including a brief discussion on the extent to which impacts and outcomes identified in surveys can be attributed to programs.) 4. Key informant interviews An extensive range of key informant interviews were conducted to address the focus questions in the SOW. As most of these questions are of a high level strategic nature and many are seeking information on processes that have led to outcomes, the evidence needed to address the questions was best sought by interviews with a qualitative focus. Where relevant, available quantitative data was used to supplement this qualitative evidence to answer evaluation questions (see points 2 and 3 above). The following categories of key informants were interviewed: USAID CAs; the IPs of CAs; the beneficiaries of USAID funded projects (i.e. MARPs and PLHIV); community networks; Cambodian government agencies; select multilateral and bilateral donors; USG partners such as CDC GAP; and USAID/Cambodia Mission staff. Interviews were conducted in a way that promoted the opportunity for all key informants to meaningfully participate in the evaluation. Collaboration and dialogue with CAs was promoted by taking the opportunity, as needed, to conduct follow-up interviews on any issues that emerged from interviews and site visits with their IPs and USAID’s other partners. Collaboration and dialogue was further promoted by CA participation in a debriefing and feedback meeting conducted by the evaluation team on the last day of in-country work. Interview guides were developed for each category of key informant to ensure a consistency in approach in interviews. These interview guides are at Annex 3. Sub-teams: Initial interviews with the CAs were conducted by the full evaluation team. To maximize the number of CA implementing agencies able to be interviewed the evaluation team divided into two sub-teams: Team 1: primary orientation – Prevention 40 Team 2: primary orientation – Care and treatment This delineation parallels how many USAID funded HIV services are provided. However, as HIV services in Cambodia are meant to adopt a CoPCT, each team took a reasonably broad cross￾program approach to their work. This involved assessing whether linkages and referrals between different types of services were working; assessing the success of an integrated approach to service delivery across the continuum; and assessing the extent to which prevention has been successfully incorporated within care and treatment services and vice versa. System strengthening (health system and community systems) and gender, particularly as it relates to sexual and reproductive health, were cross cutting themes that both teams examined. Phnom Penh interviews: For interviews with implementers in Phnom Penh, the prevention team visited prevention services and the care and treatment team visited care and treatment services. Provincial interviews: The provincial visits were conducted by mixed sub-teams. That is, one member of the prevention team and one member of the care and treatment team visited one lot of provinces and the other member of the prevention team and the other member of the care and treatment team visited the other provinces. During the provincial visits a mix of prevention and care and treatment services were inspected and interviewed. 5. Analysis The evaluation team conducted an ongoing analysis of data through individual analysis of data collected, regular, scheduled team meetings and informal discussions among team members. This iterative process allowed for emerging issues to be identified and explored and potential findings to be tested as the evaluation progressed. Immediately following the completion of key stakeholder interviews the evaluation team met for 1.5 days to conduct an analysis of all data and to develop preliminary findings in relation to the evaluation questions in the scope of work. A summary of this analysis was presented to OPHE, USAID/Cambodia by way of a PowerPoint presentation. The purpose of this was to receive feedback, validation and further input from OPHE. The analysis, incorporating feedback from USAID, formed the basis for writing the draft report. USAID’s Cambodia Health Program Design FY 2009-2013 Activity Approval Document contains USAID’s Strategic Framework for Health, including IRs. The Strategic Framework for Health is set out in Annex 5. IRs relevant to the HIV/AIDS have been used as the main criteria for evaluating the performance of the program components that make up USAID’s HIV/AIDS portfolio. For example, for USAID’s HIV prevention program, IR 1.1 ‚Improve access to and use of HIV/AIDS preventive services among target populations,‛ is the principal criteria for evaluation. The evaluation analysis has also been guided by the evaluation questions set out in the scope of work (see Annex 1). In addition to using relevant IRs and the SOW’s evaluation questions as the criteria for evaluating program components, the evaluation team decided to make an overall assessment of USAID/Cambodia’s HIV/AIDS program using evaluation criteria suitable for this purpose. Six evaluation criteria were used and ratings of 1-6 were applied to each criterion. The criterion are relevance, effectiveness, efficiency, sustainability, gender, and analysis and learning. A rating of 6 equals very high quality and a rating of 1 equals very low quality. Ratings below 4 are less than satisfactory. The evaluation criteria and ratings were adapted from standard evaluation criteria 41 used by AusAID. These criteria and ratings were applied to the USAID/Cambodia HIV/AIDS portfolio as a whole. The ratings for each criteria and brief explanations are provided in Table 1 on page ix. The HIV/AIDS work of particular CAs may have scored differently if the criteria had been applied to each agency. 6. Report writing and program concepts for follow-on activities The analysis, incorporating feedback from USAID, as described in point 5 above, formed the basis for writing the draft report. Limitations Key limitations were: This was a high level strategic evaluation of the USAID/Cambodia HIV/AIDS portfolio rather than a detailed evaluation of each CA and each program area. This somewhat limited the depth of inquiry, although the time available for interviews and site visits was adequate. Strategic decisions were made as to which IPs and which provinces were visited. The evaluation team was guided by USAID/Cambodia’s knowledge of its IPs in making these choices. While there are national impact indicators such as declining HIV prevalence and national outcome indicators such as improved consistent condom use, it is likely that these have been achieved by the collective work of a number of partners in the Cambodian national response to HIV/AIDS and possibly other factors. It is, however, reasonable to assume that the large size of the USAID program has meant that USAID funded activities have contributed to these impacts and outcomes. Nonetheless, it is not possible to determine the extent to which impacts and outcomes can be attributed to USAID programs. The evaluation team did not have access to the 2010 HIV Surveillance Survey data until after the draft report had been written as the results had not been released. Within the time available it was not possible to collect quantitative data. This limitation was minimized by use of existing performance related data collected by the CAs and their IPs and the use of HIV and behavioural surveillance data. There are limitations on the extent to which an evaluation using the methodology described above can assess the quality of services. Collection of data to measure quality was not feasible in the time frame for the evaluation and nor was this required by the SOW. The USAID/Cambodia results framework for HIV/AIDS does not include indicators that measure the quality of services. Site visits allowed for a limited assessment of quality through observation (e.g. of peer education sessions) and through questioning. However, care needs to be taken in generalizing observations from one site into findings that apply to an entire program. Where observations were consistent across a number of sites and supported by other evidence, findings can be made on a more reliable basis. Quality was primarily assessed on the basis of whether systems were in place to measure and improve upon quality. For example, monitoring visits using checklists based on standard operating procedures or some other criteria. Given the broad scope of the evaluation it was not, however, possible to make a detailed assessment of the adequacy of these systems. Two members of the evaluation team were not able to be present for the full duration of the in-country work. This limitation was minimized by the familiarity of these members with the USAID/Cambodia HIV/AIDS portfolio and USAID/Cambodia agreeing to an additional independent consultant for the team. Language differences presented barriers to in depth conversations during some key informant interviews and site visits. The evaluation team members had to rely on translations by CA and/or IP staff, some of whom had limited English skills. 42 Annex 3: Interview guides Interview guides were developed for each category of key informant to ensure a consistency of approach in interviews. The questions were based on the questions specified in the Scope of Work which are listed in Annex 1 and tailored to the categories of key informants. Cooperating Agencies questions 1. What have been the major achievements over the last 5 years with your USAID funded programming? (For CAs with multiple program areas this question could be answered for each program area.) 2. What have been the major challenges, barriers and constraints encountered over the last 5 years, how have these affected your projects and how have you responded? Supplementary: Have there been legal and policy barriers and how have you addressed these? (For CAs with multiple program areas this question could be answered for each program area.) 3. How do you go about determining programming priorities and beneficiary needs? To what extent do your projects reflect the priorities of men, women, transgenders, most at risk populations and their sexual partners and meet beneficiary needs? How could you go about achieving a better alignment between priorities, programs and meeting beneficiary needs? 4. How do you and your implementing partners collaborate with other CAs, other implementing partners and the programs of others? What evidence is there of effective collaboration? How could collaboration be improved? How has collaboration improved your outcomes? 5. Are there any noteworthy areas of synergy or duplication across USAID funded and other donor funded HIV activities? 6. How well are existing service linkage and referral systems working? Do they work equally well for men and women, or are there some differences? To what extent have TB/HIV and PMTCT outcomes been improved? To what extent is there a true continuum of prevention to care and treatment? Can you give some examples of how USAID support has facilitated the development of a true CoPCT? What improvements need making? 7. How do you go about promoting and measuring quality in your work, including technical assistance and the quality of service delivery by your implementing partners? How do you respond when there are concerns regarding quality? How much quality do you think Cambodia can afford in an environment of shrinking resources and limited country funding for the national response? 8. What strategies have you adopted to reduce the unit cost of interventions or to increase their cost effectiveness? What are the negative consequences and how can these be minimized? 9. What evidence do you have to demonstrate how your work has contributed to systems strengthening (HSS, strengthening of other sector’s systems, community SS)? What should be the key priorities areas for how HIV resources can contribute to system strengthening over the next 5 years? 43 10. Under PEPFAR II, USG programs are meant to shift from direct program implementation to an increased technical assistance focus. Can you outline what you think this means and how you are going about implementing this shift? How are you doing things differently? 11. What are the key factors that will determine whether HIV incidence can be kept low in Cambodia? What are the key risks and what is the likelihood of these risks becoming real? 12. What should be USAID’s priorities in HIV programming over the next 5 years using the following criteria: epidemiology; good and promising practices; interventions with the greatest likelihood of success; addressing structural drivers and key service delivery gaps? Supplementary: What should USAID be doing more of, less of and differently? 13. What strategies should USAID’s HIV program adopt to address the comprehensive health needs of clients? In practical terms, what would you and partners be doing differently? 14. What strategies should USAID adopt to promote sustainable HIV prevention for MARPs and their regular sexual partners? How should USAID condom social marketing programming evolve? 15. What are the key areas where strategic information needs to be improved and how will these improvements be applied? 16. How would you describe USAID’s management of its HIV portfolio? How effective has this management been? What improvements could be made to how USAID manages the program? CAs implementing partners questions 1. Thinking about the HIV work you have been doing with USAID funding, what have been your major achievements over the last 5 years? 2. What have been the major challenges, barriers and constraints encountered over the last 5 years? How have these affected your work and how have you responded? 3. To what extent do your activities respond to the priority needs and most important risk factors of men, women, transgenders, most at risk populations and their regular sexual partners and meet beneficiary needs? How do you know these are their most pressing needs? 4. What other HIV projects do you work with? How do you work with these projects and is there effective collaboration? How could you improve your work with these projects? (Seek information on the existence and effectiveness of service linkages and referral systems and the extent to which CoPCT exists. Where relevant, seek information on the current status/improvements in linkages and referrals for TB and HIV services and between MCH and HIV services in relation to PMTCT.) 5. Are there any areas where the work of your project is duplicated by other USAID funded agencies or by other donor funded HIV activities? 44 6. What types of support do you receive from [name of CA]? How effectively do they support your organization? Are there areas where their support could be improved? Are there types of support you would like to get but which are not available? 7. How do you know what standards your activities and interventions should meet to be considered good quality and effective? How do you go about improving the quality of your work? What assistance do you get from [name of CA] in showing you how to measure quality and how to improve quality? 8. Some people say that to reach universal access targets within the available funding for HIV it will be necessary to increase the efficiency of projects. For example, by stretching funding to do more with the same amount of funding. Has your project been able to do this already or do you think your project would be able to do this in future? How have you done this or how would you plan to go about doing this? 9. How has your work has contributed to systems strengthening (HSS, strengthening of other sector’s systems, community SS)? What additional ways can you think of where HIV work can contribute to broader systems strengthening? [Give an example of systems strengthening relevant to the agency as a preamble to the question.] 10. Thinking of the needs of your target groups, are there any significant unmet needs or gaps in services? These gaps or unmet needs might be HIV-related or for broader health needs. How could you or other agencies go about meeting these non-HIV needs? 11. In addition to evaluating USAID’s HIV program we need to make recommendations about USAID’s future programming. What should be USAID’s priorities in HIV programming over the next 5 years? Supplementary: What should USAID be doing more of, less of and differently? MARPs questions 1. Which services of this organization do you use? 2. How often do you use these services? 3. Do your sexual partner/s have contact with this service? In what way? What is their opinion of the service? 4. How do you find the staff and volunteers? 5. Would you say you are satisfied or dissatisfied with the services here? Why? Has access to this service made any differences to your life? In what ways? 6. Do you contribute anything to this service to benefit others or help the service? 7. How could the services be improved? 8. Are there any other HIV and STI services you use? Are you satisfied with those services? How do they compare to this service? 45 9. Are there any services you need that are not provided here and which are not available anywhere? What are the type of services you need that are not available? 10. Name all of the types of places where condoms and lubricants and needles and syringes are available either for free or for sale? Can you easily get these products when you need them? 11. For you and your friends, where do you mostly get condoms and lubricants and/or needles or syringes? Why? Government agencies, multilaterals and other donors questions 1. What have been the major achievements over the last 5 years of the USAID funded HIV projects in Cambodia? 2. What have been the major challenges, barriers and constraints encountered over the last 5 years by USAID funded HIV programs? How effectively has USAID and its implementing partners responded to these challenges, barriers and constraints? 3. In what ways does USAID and the agencies USAID funds to implement HIV projects collaborate with your agency and others? How effective is this collaboration? Are there ways it could be improved? 4. Are there any noteworthy areas of synergy and/or duplication across USAID funded and other donor funded HIV activities? 5. How well are existing service linkage and referral systems working? To what extent have TB/HIV and PMTCT outcomes been improved? To what extent is there a true continuum of prevention to care and treatment? What improvements need making? 6. In what ways have USAID funded HIV projects contributed to systems strengthening (HSS, strengthening of other sector’s systems, community SS)? What should be the key priorities areas for how USAID HIV resources can contribute to system strengthening over the next 5 years? 7. What are the key factors that will determine whether HIV incidence can be kept low in Cambodia? What are the key risks and what is the likelihood of these risks becoming real? 8. What should be USAID’s priorities in HIV programming over the next 5 years using the following criteria: epidemiology; good and promising practices; interventions with the greatest likelihood of success; and key service delivery gaps? Supplementary: What should USAID be doing more of, less of and differently? 9. In your view, should the services USAID funds address a broader range of client’s health needs and not just their HIV needs? If so, what would be the other health needs you would like to see USAID supporting? 46 10. How sustainable do you think USAID funded HIV prevention programs for MARPs are? What could be done to make these programs more sustainable? 11. What are the key areas where strategic information needs to be improved? How would this improved strategic information be used? 47 Annex 4: Evaluation schedule: key dates, tasks and deliverables Dates Task Deliverable 1. Initial preparation and planning May 19-21 Review of background documents and offshore preparation work May 24 Team arrives in Phnom Penh May 25 Evaluation Team briefed by PEPFAR Team and USAID OPHE. Evaluation Team planning meeting: development of Evaluation Framework 2. Interviews with CAs, their implementing partners & USAID partners in Phnom Penh and provinces May 26 Initial meetings with CAs: FHI and PSI May 27 Evaluation Framework presented to USAID/Cambodia Initial meetings with CAs: RACHA and KHANA Evaluation Framework May 29 Evaluation Team meeting to discuss meetings with Mission, CAs and to plan for the following week May 30 Initial meetings with CAs: RHAC and URC May 31 – June 3 Evaluation Team forms two sub teams (Prevention Team and Care and Treatment Team) and conducts interviews and site visits with the CAs implementing partners in Phnom Penh June 4 Evaluation Team meeting to share findings from interviews with implementing partners and plan work for the following week June 6-10 Evaluation Team visits provinces. One sub-team to visit Kampong Cham and Siem Reap and the other to visit Battambang, Pursat and Banteay Meanchey. Each sub-team to consist of members from the Prevention Team and the Care and Treatment Team. June 12 Evaluation Team meeting to share findings from provincial field work and plan for the following week June 13-15 Interviews with government agencies, multilateral and bilateral donors, and community networks and follow-up interviews with CAs 3. Analysis of data, development of key findings and mid-point debriefing June 16 Team meeting to analyse all data and develop key findings June 17 Team meeting to analyse all data and develop key findings (continued – morning only) Presentation of key preliminary findings to USAID/Cambodia Debriefing presentation of preliminary findings 4. Writing of draft Evaluation Report and draft technical scope of AAD June 18 Team meeting to discuss feedback on preliminary findings from USAID/Cambodia and finalize analysis prior to writing June 19-22 Writing of draft evaluation report and draft program description concepts for a revised Activity Approval Document and relevant follow-on activities June 23 Consolidation of draft report Development of presentations for debrief to PEPFAR, USAID/OPHE and CAs Draft evaluation report and program concept note for AAD and follow-on activities 5. End of field work debriefings June 24 Debrief with USAID Acting Mission Director, PEPFAR Team and OPHE, and CAs (separate meetings) Evaluation debriefing presentations 6. Review and finalization of Evaluation Report and technical scope of AAD June 27 – July 8 USAID/Cambodia reviews draft report and program concept note for ongoing activities and provides comments USAID/Cambodia feedback July 11-15 Evaluation Team Leader reviews USAID/Cambodia’s feedback and finalizes the report and concept note Final Evaluation Report and final concept note for ongoing activities 48 Dates Task Deliverable July 18 – 29 USAID/Cambodia reviews and approves final report and concept note 49 Reduce maternal and under-5 mortality by 25% Increase modern contraceptive prevalence to 33% Reduce prevalence of TB by 20% Reduce prevalence of HIV in the 20-24 age group by 10% Annex 5: OPHE Strategic Framework for Health 2009 -2013 PROGRAM GOALS BY END OF FY2013 IR 1 – Reduce impact of HIV/AIDS, TB and other infectious diseases IR 1.1 – Improve access to and use of HIV/AIDS preventive services among target populations IR 1.2 – Improve access to and quality of HIV/AIDS care services IR 1.3 – Improve access to and quality of HIV/AIDS treatment services IR 1.4 – Increase TB case detection and successful treatment IR 1.5 – Build shared platforms for addressing dengue and influenza-like diseases IR 2 – Increase delivery of maternal, child and other reproductive health services IR 2.1 – Increase availability and use of life￾saving interventions that address major killers of mothers, children and newborns IR 2.2 – Expand access to high quality voluntary family planning services and reproductive health care IR 2.3 – Improve availability of micro￾nutrients for mothers and children IR 3 – Build health systems capacity IR 3.1 – Reduce financing constraints on access to/provision of quality care IR 3.2 – Strengthen clinical and non-clinical quality assessment tools IR 3.3 – Improve other organization and management systems to support service delivery IR 3.4 – Institutionalize a standard approach to improving community-level health outcomes IR 3.5 – Build local policy formulation and advocacy skills IR 4 – Change key client health behaviors IR 4.1 – Improve clients’ care-seeking behaviors with respect to timeliness and sourcing decisions IR 4.2 – Strengthen self-care and care-giving practices in the home IR 4.3 – Empower clients to demand quality through greater awareness of their rights 50 Annex 6: Key informants and site visits The evaluation team conducted interviews with a wide range of key informants and site visits to the range of service types funded by USAID. The key informants and site visits are listed below by category. USAID/Cambodia and the USAID PEPFAR Team An initial briefing meeting was held with the USAID PEPFAR Team. Agencies present were USAID/Cambodia OPHE, CDC GAP/Cambodia, and the Peace Corps/Cambodia. In-depth discussions were held with the HIV/AIDS Team in OPHE during in-country field work. Staff members who participated in these discussions were Michael Cassell, Sok Bunna and Prateek Gupta. Interviews were also conducted with Carol Allbaugh, Peace Corps/Cambodia; and Dr Ly Vanthy and Dr Perry Killam, CDC GAP/Cambodia. Following all field work (site visits and interviews) and prior to drafting its report the evaluation team presented its preliminary findings to OPHE for feedback and validation. Following completion of the draft report the evaluation team presented its key findings and recommendations to the Acting Mission Director, USAID/Cambodia and the USAID PEPFAR Team. USAID Cooperating Agencies Separate meetings were held early in the evaluation with each of the six Cooperating Agencies that receive PEPFAR funding through USAID/Cambodia. These are FHI, KHANA, PSI, RACHA, RHAC and URC. At these meetings the CAs presented their HIV/AIDS programs, followed by questions and discussion. After interviews and site visits to the CAs IPs, follow-up interviews were held with FHI, KHANA, PSI and URC to address questions that had arisen from the field work. Following completion of the draft report, the evaluation team presented its key findings and recommendations to a meeting of CAs for the purpose of feedback and validation. Cooperating Agencies Implementing Partners The evaluation team split into two sub-teams to meet with the CA’s IPs. Site visits were conducted in Phnom Penh, Kampong Cham, Siem Reap, Pursat, Battambang and Banteay Meanchey. Typically, these meetings were with the management, staff and volunteers working for a particular implementing agency. During site visits most time was devoted to in-depth interviews to generate information to answer the evaluation questions. In addition to this, services were inspected (e.g. drop-in-centers, hospitals, VCCT clinics) and the evaluation team accompanied the staff of IPs during outreach work. This enabled the evaluation team to observe HIV prevention activities and talk with peer facilitators, peer educators and beneficiaries. Decisions on which IPs to visit were made by USAID in consultation with the CAs. A list of the IPs visited is in Table 1 below. Table 1: Interviews and site visits with Cooperating Agencies Implementing Partners Implementing partner Location ACTED-PSF: implementing HIV prevention work with EW with TA from FHI (SMARTgirl). Included site visit to observe outreach with EWs in a karaoke bar Phnom Penh Angkor Hospital for Children: implementing pediatric AIDS care and treatment with TA from RHAC. Included site visit to observe home based pediatric care Siem Reap 51 Implementing partner Location AUA: implementing a range of care and support services for PLHIV with TA from FHI Kampong Cham Battambang Referral Hospital OI/ART service: providing care and treatment services with TA from FHI Battambang Cambodian Women for Peace and Development: implementing HIV prevention work with EW with TA from FHI (SMARTgirl). Included site visits to observe outreach with freelance sex workers in Toul Kok and outreach with EWs at NagaWorld Phnom Penh Cheung Chhnok Health Centre: implementing the linked response with TA from RHAC Kampong Cham Chroy Commune: Health Centre Management Committee and Village Health Support Group volunteers with TA from RACHA Siem Reap CSDA: implementing community care and support in Monkul Borei Operational District with TA from KHANA Banteay Meanchey CSI: implementing a range of prevention and care and treatment services for PLHIV with TA from FHI Phnom Penh Health Centre, SGK Operational District: implementing VCCT and the linked response with TA from RHAC Battambang IDA: implementing linked response between PMTCT and home and community based care for PLHIV with TA from KHANA Khan Chamkarmorn, Phnom Penh Kampong Cham Referral Hospital OI/ART service: providing care and treatment services with TA from FHI Kampong Cham Khmer Women’s Cooperation for Development: site visit to observe outreach with EWs in a karaoke bar Phnom Penh Korsang: implementing HIV prevention work with IDU with TA KHANA. Included a visit to Korsang’s drop-in-center Phnom Penh KOSHER: implementing CoPCT between OI/ART sites and home and community based care for PLHIV with TA from KHANA Kilometer No 6, Phnom Penh Kravanh Health Centre: implementing the linked response with TA from RACHA Pursat Leach Village: Commune Council members with TA from RACHA Pursat Men’s Health Cambodia: implementing HIV prevention work with MSM with TA from FHI (M-Style) and KHANA. Included site visits to observe outreach to MSM in Phnom Penh and outreach to MSM and EWs in Siem Reap Phnom Penh and Siem Reap Men’s Health Social Services: implementing HIV prevention work with MSM and EW work with TA from KHANA Battambang New Hope for Cambodian Children: care and support for children affected by HIV with TA from URC Phnom Penh OEC: implementing HIV prevention with drug users with TA from KHANA Battambang Prohos Village, Samrong Commune: observe health promotion contest on HIV/AIDS. Receiving TA from RACHA Pursat PSOD: implementing HIV prevention work with EW with TA from FHI (SMARTgirl). Included site visits to observe outreach with EWs in karaoke bars Kampong Cham PSOD: implementing HIV prevention work with drug users with TA from FHI Banteay Meanchey Salvation Centre Cambodia: implementing home and community based care for PLHIV and OVC with TA from KHANA Siem Reap SEADO: implementing community care and support in Monkul Borei Operational District with TA from KHANA Banteay Meanchey Save Incapacity Teenagers: implementing HIV prevention work with EW with TA from KHANA Prek Lap, Phnom Penh Srey Snam Health Centre: implementing the linked response and VCCT with TA from RACHA Siem Reap 52 Implementing partner Location Vithey Chivit: implementing home and community based care for PLHIV with TA from KHANA Khan Sen Sok, Phnom Penh WOMEN: implementing home and community based care for PLHIV with TA from KHANA Stoeng Mean Chey, Phnom Penh Women’s Media Centre of Cambodia: HIV radio program with TA from KHANA Phnom Penh In addition to visiting the CAs IPs listed in the table above, the evaluation team visited the following services/sites which are implemented directly by the CAs and met with provincial based CA staff providing TA to IPs, as listed in Table 2 below: Table 2: Interviews and site visits to services implemented directly by Cooperating Agencies Service/Site Cooperating Agency Location Chouk Sar I and II Clinics FHI Phnom Penh Dragon Fly Centre: MSM demonstration site KHANA Siem Reap FHI field staff supporting care and treatment work with IPs FHI Kampong Cham and Battambang Man Up’s work with high risk urban men: included visits to observe outreach activities PSI Entertainment establishments in Phnom Penh and Battambang Mean Chey Centre: IDU/DU demonstration site KHANA Phnom Penh RACHA field staff supporting the linked response in Kralanh Operational District RACHA Siem Reap RACHA field staff supporting the linked response in Pursat RACHA Pursat RHAC Clinics RHAC Tul Sangker District Clinic, Phnom Penh; Battambang Clinic; Siem Reap Clinic RHAC field staff supporting the linked response in Prey Chhor￾Kang Meas Operational District RHAC Kampong Cham URC’s activities in Battambang Referral Hospital: HEF, infection control and quality improvement URC Battambang Other Provincial level key informants Interviews were held with a range of key informants working at the provincial level. These were in addition to interviews/site visits to CA IPs which are listed above. Other provincial level key informants are listed in Table 3 below Table 3: Other Provincial level key informants Name Agency Dr Kim Sourphirun Provincial Health Director, Kampong Cham Mr Siv Meng Se Provincial AIDS Manager, Kampong Cham Family Health Clinic and VCCT Clinic Kampong Cham Family Health Clinic and VCCT Clinic Battambang 53 Name Agency Dr Kross Sarath Provincial AIDS Manager, Siem Reap Dr Khlem Sokun Provincial Health Director, Pursat Dr Sor Hong Provincial AIDS Manager, Pursat Provincial Health Directors Provincial Health Director, Battambang and Banteay Meanchey Provincial AIDS Managers Provincial AIDS Manager, Battambang and Banteay Meanchey National level key informants Interviews were held with a range of key informants working at the national level. These included RGC agencies, development partners and national civil society networks working in HIV. The agencies met are listed in Table 4 below: Table 4: National level key informants Name Agency Dr Masami Fujita and Dr Michel Thieren WHO Dr Premprey Suos HIV/AIDS Asia Regional Program, AusAID Dr Ross Seilavath National AIDS Authority HE Dr Mean Chhi Vun National Centre for HIV/AIDS, Dermatology and Sexually Transmitted Diseases HE Tia Phalla Chair, CCC, GFATM Leng Monyneath & Mok Sokha National MSM Network (Bandanh Chaktomuk) Keo Chen Cambodian People Living with HIV/AIDS Network (CPN+) Dalise Cambodian Community of Women living with HIV Sophon Positive Women of Hope Organization Tony Lisle and Tia Phauley UNAIDS Sonia Bezziccheri UNODC 54 Annex 7: Bibliography Buhler, M. et al. (2006). Turning the tide: Cambodia's response to HIV & AIDS 1991-2005. Cambodia, Ministry of Education, Youth and Sports (2011) Most at risk young people survey Cambodia 2010. Cambodia, Village Commune and Sangkat Safety Policy. Cambodia, Ministry of Health. (2008). National strategic plan for prevention mother-to-child transmission of HIV 2008-2015. Cambodia, National AIDS Authority. (2008). A situation and response analysis of HIV and AIDS in Cambodia 2007 update. Cambodia, National AIDS Authority, (2010) The National Strategic Plan for Comprehensive Multisectoral Response to HIV/AIDS (2011-2015) in Cambodia. Cambodia, National AIDS Authority. (2010). The financial resource for the comprehensive and multi￾sectoral response to HIV/AIDS III (2011-2015) in Cambodia. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2008). Strategic plan for HIV/AIDS and STI prevention and care in the health sector in Cambodia 2008-2010. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2009). Standard operating procedures (SOP) for continuum of prevention to care and treatment for women entertainment workers in Cambodia. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2010). Annual operational comprehensive plan 2010. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2010). Annual Report 2009. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2010). Fourth quarterly comprehensive report, 2010 HIV/AIDS and STI prevention and care programme. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2010). Guide for implementation of Positive Prevention among PLHIV in Cambodia. Cambodia, National Center for HIV/AIDS, Dermatology, and STD. (2011). Annual Report 2010. Cambodia, (2008) National Center for HIV/AIDS, Dermatology, and STDs. Cambodia 2007 Behavioral Surveillance Survey: HIV/AIDS related sexual behaviors among sentinel groups Cambodia, National Center for HIV/AIDS, Dermatology and STDS. (n.d.). Concept of continuous quality improvement for HIV/AIDS care. Lecture presented at NCHADS CQI Launch. Cambodia, National Center for HIV/AIDS, Dermatology and STDS. (2008, August). HIV Sentinel Surveillance 2006. Lecture. Cambodia, National Center for HIV/AIDS, Dermatology and STDS. (2010). Draft of strategic plan for HIV/AIDS and STI prevention and care in health sector 2011-2015. Cambodia, NCHADS, & NMCHC. (2007). Standard operating procedures (SOP) to initiate a linked response for prevention, care and treatment of HIV/AIDS and sexual and reproductive health issues (Cambodia, Ministry of Health). Cambodia, NCHADS, FHI, US CDC-GAP, & NIPH. (2006, September). Cambodia STI survey 2005: key risk behaviors and STI prevalence. Lecture. Cambodia, (2010) National Center for HIV/AIDS, Dermatology, and STDs. Behavioral Sentinel Surveillance 2010. Cambodia, (2008) National Center for HIV/AIDS, Dermatology, and STDs. HIV Prevalence among Drug Users 2007 in Cambodia. FHI, KHANA and PSI, (2011, March 18) Coordination and Collaboration on USAID-funded Programs. (Paper). FHI, KHANA, PSI, RACHA, RHAC, URC, (2010) Work plans for each CA for FY 2011. FHI, KHANA, PSI, RACHA, RHAC, URC, (2010) Annual Reports from each CA (dates vary). 55 Gender and Development for Cambodia. (2010). Deoum troung pram hath in modern Cambodia: a qualitative exploration of gender norms, masculinity, and domestic violence. Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM). (2011). Continued achievement of universal access of HIV/sexually transmitted infections prevention, treatment, and care services in Cambodia (Rep. No. CAM-S11-G16-H). Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM). (2011). Program grant agreement for single stream of funding. (Cambodia, National Center for HIV/AIDS, Dermatology, and STD). Hecht, R. et al. (2010, October 9). Financing of HIV/AIDS programme scale-up in low-income and middle-income countries, 2009–31 : The Lancet. Retrieved June 29, 2011, from http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(10)61255-X Joint United Nations Programme on HIV/AIDS, & Hok, G. (n.d.). Leadership (Issue brief). Joint United Nations Programme on HIV/AIDS, & Lisle, T. (n.d.). Global fund to fight AIDS, Tuberculosis, and Malaria (Issue brief). Joint United Nations Programme on HIV/AIDS, & Lisle, T. (n.d.). Governance and human rights issues (Issue brief). Joint United Nations Programme on HIV/AIDS, & Phauly, T. (n.d.). Drug users and injecting drug users (Issue brief). Joint United Nations Programme on HIV/AIDS, & Phauly, T. (n.d.). Entertainment workers (Issue brief). Joint United Nations Programme on HIV/AIDS, & Phauly, T. (n.d.). Men who have sex with men and high risk men (Issue brief). Joint United Nations Programme on HIV/AIDS, Phauly, T., & Hok, G. (n.d.). Civil society (Issue brief). Kanal, K. (2006). PMTCT Cambodia framework. Lecture presented at Bi-regional Consultation on Integration Prevention and Management of STI/HIV/AIDS into Reproductive, Maternal and Newborn Health Services, Kuala Lumpur. KHANA, (2008) Strategic Plan 2008-2010. KHANA, (2010 Second Edition). Standard package of activities: Drug Users (DU/IDU). KHANA, (2010 Second Edition). Standard package of activities: Entertainment Workers. KHANA, (2010 Second Edition). Standard package of activities: Men who have Sex with Men. KHANA, (2010 Second Edition). Standard package of activities: Orphans and Vulnerable Children. KHANA, (2010 Second Edition). Standard package of activities: People living with HIV. KHANA, (2010). Analysis in Action: A Household Asset Based Livelihood Approach. KHANA, (2010). The Baseline Survey on HIV/AIDS Knowledge, Attitudes, Practices and Related Risk Behaviors among Most-at-Risk Populations. KHANA, (2010) Mid-Term Review of the Integrated Care and Prevention Project Regarding PLHIV and OVC. KHANA, (2011) Strategic Plan 2011-2015. Levine, R., & DeRuiter, D. (2011, January 19). United States Agency for International Development evaluation policy. USAID. Retrieved May, 2011, from http://www.usaid.gov/evaluation/USAID_EVALUATION_POLICY.pdf?020911 Ministry of Interior, National Authority to Combat Drugs, National AIDS Authority. (n.d.). National strategic plan for illicit drug use related HIV/AIDS 2008 - 2010 (Cambodia, Royal Government of the Kingdom of Cambodia). Phalla, T., & Fowler, D. (2009). Strategic plan for the response to HIV and AIDS 2009-2013 (Cambodia, Ministry of Interior). 56 Population Services International, & Nauman, E. (n.d.). Cambodia (2008): HIV TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia. Population Services International, (2011) TraC study evaluating condom use among high risk urban men from Phnom Penh, Siem Reap, Battambang and Sihanoukville. Round Three. Population Services International, McDougall, J., & Nauman, E. (n.d.). Cambodia (2006): HIV/AIDS TRaC study evaluating condom use among sexually active men with sweethearts in Phnom Penh Population Services International, (2011) TraC study evaluating consistent condom use with male partners among men who sex with men from Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey. Round One. Population Services International, (2011) TraC study evaluating condom use among female entertainment service workers in Phnom Penh Population Services International and Family Health International, )2007, September) Let’s Go for a Walk. Sexual Decision-making among Clients of Female Entertainment Service Workers in Phnom Penh, Cambodia. Population Services International, (2008, December) MAP Study Evaluating the Availability of HIV Prevention Products. 4th Round. Population Services International, (2008) Informed Decisions. Recent evidence provides programmatic guidance for HIV prevention interventions in Cambodia. Population Services International, (2009) Informed Decisions 2009 Update. Evidence-driven programmatic guidance for HIV prevention interventions in Cambodia. Population Services International, (2009) Informed Decisions. Evidence-driven programmatic guidance for HIV prevention interventions targeting female entertainment workers in Cambodia. Population Services International, (2010) Informed Decisions. Evidence-driven programmatic guidance for HIV prevention interventions targeting men who have sex with men in Cambodia. Reproductive Health Association of Cambodia (RHAC). (2008). Evaluation: report of the HIV/AIDS for vulnerable group program. Riggs-Perla, J et al. (2011, March) Cambodia, USAID Health Program Mid-Term Evaluation. Schwartländer, B. et al. (2011, June 3). Towards an improved investment approach for an effective response to HIV/AIDS. Retrieved June 29, 2011. DOI:10.1016/S0140-6736(11)60702-2 Schwartländer, B. et al. (2011, June 3). Supplementary web appendix; Towards an improved investment approach for an effective response to HIV/AIDS. The Lancet. DOI:10.1016/S0140- 6736(11)60702-2. Smart, T. (2010). Bridging the divide: HIV and health systems. HATiP, (168), 1-8. The United Nations Children's Fund, & Campbell, P. (n.d.). Orphans and Vulnerable Children (Issue brief). United States, U.S. Agency for International Development, Bureau for Policy, Planning and Learning. (2011). Evaluation policy (pp. 1-15). USAID/Cambodia, & Gupta, P. (2010, September 30). 2010 MARPS Geo Coverage in Battambang. Unpublished data. USAID/Cambodia, Cooperative Agreements between USAID and KHANA, PSI, RACHA, RHAC and the Task Order between USAID and FHI. (various dates) USAID/Cambodia Office of Public Health and Education, (2009) Cambodia Health Program Design FY2009-2013 Activity Approval Document. United States Government, (2010) USG Cambodia HIV/AIDS Strategy 2011-2015. Draft. World Health Organization, & Seguy, N. (n.d.). Care and treatment, PMTCT, and health system strengthening (Issue brief). 57 World Health Organization Maximizing Positive Synergies Collaborative Group. (2009). An assessment of interactions between global health initiatives and country health systems. The Lancet, 373(9681), 2137-2169. doi: 10.1016/S0140-6736(09)60919-3