August 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by David Lowe (Independent Consultant and Team Leader), Srey Mony (Independent Consultant), Jenne Roberts (Independent Consultant), Marta Levitt, (USAID/Washington), and Billy Pick (USAID/Washington). End-of-Project Evaluation of the Sustainable Action Against HIV and AIDS in the Community Project (SAHACOM) Cover Photos by KHANA Contents ACRONYMS............................................................................................................................................................III EXECUTIVE SUMMARY.................................................................................................................................... V 1. EVALUATION PURPOSE AND METHODOLOGY........................................................................................ 9 1.1 PURPOSE OF THE EVALUATION AND SCOPE OF WORK ............................................................................................ 9 1.2 METHODOLOGY ............................................................................................................................................ 9 1.2.1 Limitations...................................................................................................................................... 9 2. PROGRAM OUTLINE..............................................................................................................................10 2.1 THE CAMBODIAN CONTEXT ...........................................................................................................................10 2.2 OVERVIEW OF THE SUSTAINABLE ACTION AGAINST HIV AND AIDS IN THE COMMUNITY PROJECT ..............................11 3. COMMUNITY BASED SUPPORT FOR PLHIV AND OVC ............................................................................13 3.1 OUTLINE OF COMMUNITY BASED SUPPORT PROGRAMMING FOR PLHIV AND OVC...................................................13 3.2 FINDINGS AND KEY RESULTS...........................................................................................................................14 3.2.1 Coverage .......................................................................................................................................14 3.2.2 Linking PLHIV with services...........................................................................................................15 3.2.3 ART uptake rates and retention and adherence ...........................................................................16 3.2.4 Self-help group sustainability........................................................................................................17 3.3 CONCLUSIONS ............................................................................................................................................18 3.4 FUTURE DIRECTIONS.....................................................................................................................................19 4. LIVELIHOODS AND ECONOMIC STRENGTHENING..................................................................................21 4.1 OUTLINE OF LIVELIHOODS AND ECONOMIC STRENGTHENING PROGRAMING .............................................................21 4.2 FINDINGS AND KEY RESULTS...........................................................................................................................22 4.2.1 Village savings and loans scheme ................................................................................................. 22 4.2.2 Livelihoods skills training and application ....................................................................................23 4.2.3 Small cash grants..........................................................................................................................24 4.2.4 Linkages with other livelihoods and economic strengthening initiatives......................................24 4.3 CONCLUSIONS ............................................................................................................................................25 4.4 FUTURE DIRECTIONS FOR LIVELIHOODS AND ECONOMIC STRENGTHENING PROGRAMMING..........................................26 4.5 RECOMMENDATION.....................................................................................................................................26 5. FOCUSED PREVENTION.........................................................................................................................27 5.2 FINDINGS AND KEY RESULTS...........................................................................................................................27 5.2.1 Uptake of targeted HIV prevention interventions.........................................................................27 5.2.2 Innovative models.........................................................................................................................29 5.2.3 Effectiveness of interventions.......................................................................................................31 5.3 CONCLUSIONS ............................................................................................................................................32 5.4 FUTURE DIRECTIONS FOR FOCUSED PREVENTION................................................................................................33 5.5 RECOMMENDATIONS ...................................................................................................................................33 6. CAPACITY DEVELOPMENT.....................................................................................................................34 6.1 OUTLINE OF SAHACOM’S CAPACITY DEVELOPMENT WORK ................................................................................34 6.2 FINDINGS AND KEY RESULTS...........................................................................................................................34 6.2.1 Capacity building of NGOs/CBOs and communities......................................................................34 6.2.2 Capacity of KHANA as a technical assistance provider................................................................. 36 6.3 CONCLUSIONS ............................................................................................................................................37 6.4 FUTURE DIRECTIONS FOR CAPACITY BUILDING WORK...........................................................................................37 6.5 RECOMMENDATIONS ...................................................................................................................................37 7. CROSS CUTTING AREAS.........................................................................................................................37 7.1 GENDER ....................................................................................................................................................37 i 7.1.1 Overview of KHANA’s approach to gender ...................................................................................37 7.1.2 Findings and key results................................................................................................................37 7.1.3 Conclusions ...................................................................................................................................38 7.1.4 Future directions and recommendations......................................................................................39 7.2 MULTISECTORAL PARTNERSHIPS .....................................................................................................................39 7.2.1 Overview of KHANA’s approach to multisectoral partnerships ....................................................39 7.2.2 Findings and key results................................................................................................................39 7.2.2 Conclusions ...................................................................................................................................40 7.2.3 Future directions and recommendations......................................................................................40 7.3 SUSTAINABILITY...........................................................................................................................................40 7.3.1 Overview .......................................................................................................................................40 7.3.2 Findings.........................................................................................................................................41 7.3.3 Conclusions ...................................................................................................................................42 7.3.4 Future directions and recommendations......................................................................................42 8. USAID/CAMBODIA MANAGEMENT OF SAHACOM................................................................................43 8.1 RECOMMENDATIONS ...................................................................................................................................43 ANNEX 1: SCOPE OF WORK .....................................................................................................................44 ANNEX 2: EVALUATION METHODOLOGY................................................................................................. 47 ANNEX 3: BIBLIOGRAPHY........................................................................................................................51 ANNEX 4: INTERVIEW GUIDES.................................................................................................................54 ANNEX 5: EVALUATION SCHEDULE..........................................................................................................60 ANNEX 6: ORGANIZATIONS CONSULTED................................................................................................. 62 ANNEX 7: SAHACOM OPERATIONAL FRAMEWORK AND STRUCTURE OF TECHNICAL APPROACHES........65 ANNEX 8: DATA - GRAPHS AND TABLES ..................................................................................................67 ANNEX 9: FOCUSED PREVENTION SERVICE PACKAGE..............................................................................74 ANNEX 10: CONFLICT OF INTEREST DISCLOSURE.......................................................................................75 ii Acronyms AAD Activity Approval Document AIDS Acquired Immune Deficiency Syndrome ANC Ante natal care ART Antiretroviral Therapy BCC behavior change communication CA Cooperative Agency CHBC Community and home based care CBPCS Community Based Prevention Care and Support CC Commune Council CDC GAP Centers for Disease Control and Prevention Global AIDS Program CDP Commune Development Plan CoC Continuum of Care CoPCT Continuum of Prevention to Care and Treatment CPN+ Cambodian People Living with HIV Network CSO Community Support Officer CSV Community Support Volunteer DAI Development Alternatives Incorporated DIC Drop in Center ES Economic strengthening EW Entertainment worker FP Family planning FY Financial Year GBV Gender based violence GF/GFATM Global Fund/Global Fund to fight AIDS, Tuberculosis and Malaria HBC Home based care HCT HIV counselling and testing HEF Health Equity Fund HIV Human Immunodeficiency Virus HSS Health System Strengthening IBBS Integrated Biological and Behavioral Survey ICP Integrated care and prevention IEC information, education, and communication IGA Income generating activity IP Implementing Partner IR Intermediate Result KPs Key populations LoE Level of effort M&E Monitoring and evaluation MMM Mondul Mith Chuoy Mith MMT Methadone maintenance treatment MoH Ministry of Health MoSVY Ministry of Social Affairs, Veterans and Youth and Rehabilitation MSIC Marie Stopes International/Cambodia iii MSM Men who have Sex with Men NAA National AIDS Authority NCHADS National Center for HIV/AIDS, Dermatology and STD NGO Non-Governmental Organization OD Operating District OI Opportunistic infection/illness OW Outreach worker OVC Orphans and vulnerable children OVCSG Orphans and vulnerable children support group PE Peer Educator PEPFAR President’s Emergency Plan for AIDS Relief PF Peer Facilitator PHD Provincial Health Department PLHIV People Living with HIV PMP Performance Monitoring Plan PMTCT Prevention of mother to child transmission PP Positive Prevention PSI Population Services International/Cambodia PWID People who inject drugs PWUD People who use drugs RGC Royal Government of Cambodia SAHACOM Sustainable Action Against HIV and AIDS in the Community SGL Support Group Leader SHG Self-help group SOP Standard operating procedure SOW Scope of work SRH Sexual and reproductive health SROI Social return on investment STI Sexually Transmitted Infection TA Technical Assistance TasP Treatment as prevention TB Tuberculosis TG Transgender TWG Technical Working Group UIC Unique Identifier Code UNAIDS Joint United Nations Program on HIV/AIDS USAID United States Agency for International Development USG United States Government VCCT Voluntary, Confidential Counselling and Testing VFC Vision Fund Cambodia WFP World Food Program VSL Village Savings and Loans WHO World Health Organization iv Executive summary This end of project performance evaluation of the Sustainable Action Against HIV and AIDS in the Community (SAHACOM) project was commissioned by USAID/Cambodia’s Office of Public Health and Education. The purpose of the evaluation was 1) to assess the project’s performance and the extent to which it was able to meet its intended objectives at all result levels; and 2) to document lessons learned and best practices as well as make recommendations to inform and improve future program directions and effectiveness. The evaluation used a performance based methodology. SAHACOM is a five year (October 2009 – September 2014) $13.4 million project, implemented by the Khmer HIV/AIDS NGO Alliance (KHANA). SAHACOM consists of three main components (with the major focus being on component 1): 1. Community and home based care (CHBC) for people living with HIV (PLHIV) and orphans and vulnerable children (OVC) through Community Support Volunteers (CSVs), self-help groups (SHGs) and livelihood and economic strengthening for PLHIV, with support for access to health facilities and treatment retention and adherence. 2. Focused prevention programming for key populations (KPs), with a particular emphasis on under-served and marginalized groups, including the promotion of HIV testing and linking HIV-positive people to treatment and care services. 3. Capacity building of national networks, SAHACOM’s NGO implementing partners (IPs) and SHGs. The SAHACOM model represented a shift away from direct NGO service delivery to NGO support for PLHIV CSVs and SHGs. The objective was to develop a more sustainable community leadership and participation approach with reduced reliance on external support by increasing individual social capital and community responsibility. The shift from NGO direct service delivery to a CSV and SHG centered model, with NGO support, has been successful. CHBC services have been provided in Phnom Penh and eight provinces with the highest HIV prevalence in Cambodia. The project has supported 540 SHGs (currently 361), involving a total of 9,250 PLHIV and 500 (currently 435) OVC support groups, involving 10,139 OVC. This accounts for 52 per cent of KHANA’s total CHBC activities which in turn contribute to 60 per cent of the national CHBC coverage. Therefore the USAID-funded project makes up around 31 per cent of the national response. Through the work of CSVs, SAHACOM has successfully linked the vast majority of PLHIV who attend SHGs with health and other services. On average, each PLHIV received six referrals and OVC received seven referrals to health services each year. Eighty per cent of PLHIV at end-line rated their overall health as good or fair compared to 52 per cent at baseline. Similarly, 72 per cent of PLHIV at the end-line rated their overall quality of life as good or fair compared 35 per cent at baseline. In FY 2013, 96 per cent of PLHIV reported that they were satisfied with the CHBC services of SAHACOM. According to the end-line survey, 95 per cent of PLHIV respondents received support from SAHACOM IPs in the past 12 months. The number of SAHACOM’s PLHIV beneficiaries in need of antiretroviral therapy (ART) and currently on treatment increased from 90 per cent at baseline to 93 per cent at mid-term and to 96 per cent at end-line. SAHACOM data indicates that 89 per cent of PLHIV on ART were retained in treatment at 12 months after initiation, compared to a national retention in treatment rate of 85 per cent. These data are impressive by regional and international standards. v Prevention of HIV infection among sero-discordant couples has been a focus of SAHACOM. Of the 872 sero-discordant couples identified by SAHACOM, 98 per cent of positive partners are on ART which largely eliminates the risk of transmission, with only a handful of new HIV infections occurring amongst sero-discordant couples. The CHBC and livelihoods models are primarily oriented towards the needs of PLHIV living in rural areas and has not been tailored for urban PLHIV and HIV-positive KPs. It is important to note this limitation of the project, as most of Cambodia’s PLHIV and KPs reside in peri-urban areas and towns, including Phnom Penh. Financial resources for community-based care for PLHIV and OVC are quite vulnerable in the context of reduced donor funding. In regard to future CHBC programming, key features should be development of a more streamlined, cost effective model of community support for PLHIV; a community based case management approach which prioritizes and tailors flexible support for those most in need; a greater focus on meeting the needs of urban PLHIV; and exploring the possibility of using the Health Equity Fund (HEF) for HIV services. While all newly diagnosed people require an immediate health assessment in relation to treatment eligibility and community based psycho-social support, those stabilized on ART generally will require much less support, but may need referral for skills or business training to address livelihoods needs. A streamlined model based on PLHIV needs would enable CSVs to tailor levels of support for PLHIV to changing needs, including phasing support to PLHIV in and out, as needed, based on health status and such vulnerabilities as risk of loss-to￾follow up or treatment adherence challenges. CSVs are the backbone of the SAHACOM model. It is clear that CSVs provide a link and functional referral system between the community and the health facilities, in particular the ART/OI centers. It is essential that the community based comparative advantage of CSVs continues to be recognized and supported. Sole reliance on facility based case management would be much less effective. Key questions are how much technical support and supervision is required for CSVs and whether the model can still be effective with fewer IPs and fewer CSVs who cover more SHGs and a larger catchment area. One option to consider is a professionalized, community based case management approach by PLHIV (possibly utilizing the best of the CSVs). The HEF provides an alternative way to pay beneficiaries transport and healthcare fees without the IP management costs. A 2010 KHANA survey found that 40 per cent of SAHACOM’s PLHIV beneficiaries were particularly vulnerable in terms of food, water and economic security and lacked skills and capacity to significantly increase household incomes and nutritional consumption. SAHACOM’s livelihoods and economic strengthening programming aims to strengthen the socio-economic status, resilience and health outcomes of poor PLHIV, OVC and KP households, through mitigating vulnerability and reducing dependency. SAHACOM trained 572 PLHIV in 28 residential training sessions in agricultural skills, coupled with small cash loans to be used as start-up capital, and ongoing technical support, post training. Of those trained, 62 per cent had applied the skills learned at training and 65 per cent of those who had applied the skills had done so successfully. That equates to an overall success rate of 40 per cent of those who had been trained, although some IPs achieved better uptake application and success rates. A real benefit of livelihoods programming is that PLHIV with significantly improved health as a result of ART have been encouraged to plan for the future by becoming more self-reliant. For this reason, livelihoods support should continue, but on a reduced scale, targeted to PLHIV most in need. Future livelihood programming should have strengthened linkages with vi mainstream livelihoods programs to take advantage of their technical expertise and ensure the engagement of local authorities. The Village Savings and Loans (VSL) scheme successfully provided financially vulnerable PLHIV in rural communities with a supportive and safe space to learn through doing about the benefits of saving and investing and has encouraged participants to become financially self-reliant. A total of 174 VSL groups were established out of 540 SHGs and most groups have been sustained, albeit with KHANA support. VSL groups have built social capital through peer support and the trust required by pooling savings and loaning money. SAHACOM’s prevention activities are appropriately focused on the KPs of entertainment workers (EWs), men who have sex with men (MSM), transgender persons (TGs) and people who inject drugs (PWID), in Phnom Penh and three provinces. The number of KPs reached with individual and/or small group level interventions that are based on evidence and/or meet minimum standards increased from 6,016 in 2010 to 11,575 in 2013. The program exceeded in reaching targets for MSM, TGs and EWs but was under target for the number of PWID reached, possibly reflecting the challenges in reaching PWID. SAHACOM has not targeted freelance and street based sex workers. This is a significant limitation. IPs working with KPs have broadened their prevention programming by taking on the role of HIV case detection through promotion of demand for HIV testing as an entry point to care and treatment, and facilitating access to care and treatment for newly diagnosed PLHIV. The mid-term and end-line surveys of SAHACOM clients indicate high HIV testing rates among KPs in the last 6 months (65% - 85%), although SAHACOM’s monitoring data shows significantly lower rates of HIV testing for KPs reached (44% in last 12 months). Possible reasons for the difference in HIV testing rates might include KPs in the end-line survey over￾reporting testing by giving a socially acceptable answer; the limited geographical coverage of the end-line survey may not have been representative of the actual number of KPs who had an HIV test; possible weaknesses in monitoring data in relation to tracking the number of clients who had an HIV test; and/or significant numbers of KPs initiating HIV testing independently of SAHACOM. There is evidence to demonstrate that EWs receiving HIV education are more likely to participate in HIV counseling and testing than EWs who had not received HIV education. In the end-line survey, 71% of EW receiving HIV education had an HIV test in the last six months, whereas only 44% of those who had not received HIV education had an HIV test. SAHACOM data indicates a very high successful referral rate to care and treatment services following facility based testing and confirmation of an HIV positive test result. Of the 86 confirmed HIV-positive cases among the 5,135 SAHACOM KP clients having an HIV test (not rapid testing) in 2012-13, 85 individuals were either enrolled in pre-ART or commenced ART, equating to a 99 per cent uptake rate. Over the life of the SAHACOM program, data from the baseline and mid-term and end-line surveys show an overall improvement in adoption of safe behaviors for MSM and PWID, although there has been a modest reduction in consistent condom use by EWs with commercial partners. Consistent condom use in the last three months by MSM with their regular partners increased from 27 per cent at baseline to 63 per cent at end-line, whereas for EWs condom use fell from 89 per cent at baseline to 81 per cent at end-line. Needle sharing among PWID fell from 63 per cent at mid-term to 25 per cent at end-line, a marked improvement in risk reduction. HIV prevalence among KPs tested recently through rapid testing was 0.5 per cent. The overall HIV prevalence for SAHACOM KP members tested in 2012-13 was only 1.7 per cent, vii although prevalence varied significantly among KPs: 0.5 per cent for EWs; 0.8 per cent for MSM; and 72 per cent for PWID. With the exception of PWID, this is well below prevalence for KPs in the most recent bio-behavioral surveys. Possible explanations for the low prevalence may be: 1) HIV prevalence among KPs being HIV tested through rapid and health facility based testing would be lower than HIV prevalence in IBBS surveys as those who already know they HIV positive would not be seeking repeat testing; 2) prevention programming and treatment as prevention (TasP) may have been effective and there are fewer new and undetected HIV cases than was thought to be the case; and/or 3) the KPs being tested are at lower risk; and harder- to-reach members of KPs who are at higher risk are not being reached and/or tested. There is insufficient evidence to come to any conclusion on these points. The outreach prevention model needs to be refocused on reaching hard to reach KPs who may be at higher risk, including underserved groups such as freelance and non-venue￾based sex workers, PWID, including female PWID, and those with multiple risk factors. SAHACOM’s continuous capacity building for IP staff and CSVs in a wide range of areas contributed significantly to achievement of program targets (e.g., high rates of treatment retention), strong linkages between the community and the health care system and with other organizations, functioning SHGs, improved communication, strong engagement of PLHIV, and improved data quality and reporting. KHANA has emerged as a recognized, effective and respected provider of HIV technical assistance in Cambodia. KHANA’s strong technical capacity has given them a seat at the policy table at national and provincial levels, and they have been able to support the Cambodian government in developing key policy documents that guide programs for PLHIV and OVC. SAHACOM was designed to develop and scale up a self-help approach to community support for PLHIV and OVC. The key principles of SAHACOM emphasize community leadership and participation, adaptability and innovation, and sustainable and cost effective models to reduce reliance on external support by increasing individual social capital and community responsibility. The model also places emphasis on development of partnerships and collaboration to promote synergy and maximize use of resources and linkages with integrated health and non-health services. The SAHACOM model set out to demonstrate how to empower and create community ownership by persons living with HIV, with these individuals serving as Community Support Volunteers, leading self-help groups and support groups for orphans and vulnerable children to implement Community and Home Based Care and focused HIV prevention for Key Populations by peer facilitators and educators. SAHACOM has been a successful program against all these measures. viii 1. Evaluation purpose and methodology 1.1 Purpose of the evaluation and scope of work This end of project performance evaluation of the Sustainable Action Against HIV and AIDS in the Community (SAHACOM) project was commissioned by USAID/Cambodia’s Office of Public Health and Education. The purpose of the evaluation was twofold: 1. to assess the project’s performance and the extent to which it was able to meet its intended objectives at all result levels; and 2. to document lessons learned and best practices as well as make recommendations to inform and improve future program directions and effectiveness. The scope of work (SOW) for this evaluation specified a number of questions to address these two overarching aspects of this study. These questions are in Annex 1. 1.2 Methodology The evaluation team followed a performance based methodology consistent with USAID’s January 2011 Evaluation Policy, focusing on descriptive and normative questions, including: what the program had achieved; how it was being implemented; how it was perceived and valued; whether expected results were occurring; and other questions pertinent to program design, management, and operational decision making. A more detailed description of the methodology is in Annex 2. The evaluation team began with a review of key background documents (see Annex 3) and SAHACOM’s performance indicator data. Interview guides, based on the evaluation questions, were developed by the evaluation team for different categories of stakeholders to ensure a consistent approach (see Annex 4). This was particularly important as two sub teams were formed and conducted separate interviews and site visits. Key informant interviews included KHANA, the main implementing partner for SAHACOM, NGOs contracted by KHANA to implement SAHACOM, beneficiaries and volunteers, and other national and provincial level stakeholders. These interviews were conducted in Phnom Penh and each of the eight high HIV burden provinces in which SAHCOM was implemented. (See Annexes 5 and 6 for the evaluation schedule and a list of organizations consulted and site visits.) Ongoing analysis of data by individual team members fed into regular group analysis, which allowed for emerging issues to be identified and explored as the evaluation progressed. A series of comprehensive analysis sessions, utilizing quantitative and qualitative data, were conducted by the evaluation team, following the conclusion of the key informant interviews, for the purpose of developing preliminary key findings and conclusions. These were presented to USAID/Cambodia at a mid-point debriefing for the purpose of receiving feedback, validation and further input. This feedback was incorporated into the drafting of the evaluation report. At the end of the in-country work, the evaluation team conducted a further debriefing for USAID/Cambodia and US CDC. The evaluation’s preliminary findings and conclusions were also presented to KHANA and other stakeholders. Feedback from all these meetings was taken into account in further drafting of this evaluation report. 1.2.1 Limitations In 2010 SAHACOM collected ‘baseline’ data through a desk based review of existing data, along with field visits and consultations with program staff. This work did not involve a quantitative baseline survey. While the ‘baseline’ documentation gives a reasonable overview of the situation at project commencement, some of the data is not directly comparable with key indicators for SAHACOM and therefore cannot be regarded as comprehensive baseline data. This means that for some indicators, performance data at the end of the first year of 9 implementation is the closest proxy to a baseline. In 2012 and 2014, mid-term and end line surveys of SAHACOM beneficiaries were conducted by KHANA that provide further data for comparison purposes. A limitation of the mid-term and end line survey is not all population groups were surveyed in each of the survey sites. For example, people living with HIV (PLHIV) were not surveyed in Phnom Penh. The evaluation team has drawn extensively on KHANA’s SAHACOM monitoring data in developing findings and conclusions. Any weaknesses in the monitoring data may have resulted in the development of invalid findings and conclusions. The evaluation team did not have time to make an assessment of the strengths and weaknesses of SAHACOM’s monitoring data. The evaluation team conducted interviews in Phnom Penh and all the provinces where SAHACOM is working. However, due to the large number of SAHCOM stakeholders, it was not possible to interview all. The number of stakeholders interviewed was maximized by the evaluation team splitting into two sub-teams in Phnom Penh and for provincial visits, and further splitting into additional sub-teams in the provinces. Proportionally, more time was spent interviewing NGOs implementing SAHACOM in the provinces than in Phnom Penh. While there was limited data that directly measured the quality of services, quality was assessed by the extent to which expected results and outcomes were achieved and the level of sophistication by which stakeholders were able to answer questions. Another constraint was language barriers and the need for translation of interviews, particularly for KHANA’s NGO implementing partners and project beneficiaries. While the translation was of a high standard, the use of translators creates a communication barrier and is time consuming, thereby limiting the scope of interviews. 2. Program outline 2.1 The Cambodian context Cambodia had demonstrated considerable success in slowing its HIV epidemic by reducing adult HIV prevalence in the general population by more than half from 1.7 per cent in 1998 to 0.7 per cent in 2012. The epidemic is concentrated among the key populations (KPs) of entertainment workers (EWs),1 men who have sex with men (MSM), transgender people (TG), and people who inject drugs (PWID). HIV prevalence for these populations is 2.1 per cent for MSM (2010), 13.9 per cent for EWs who have more than seven partners per week, and 24.8 per cent for PWID (2013).2 In 2013, 68 per cent of the estimated 74,500 Cambodian PLHIV (adults and children) were receiving lifesaving antiretroviral therapy (ART). This represented 83 per cent of those who met treatment eligibility requirements, which is high by international standards for comparable countries.3 1 Entertainment workers is the Cambodian term used to refer to women working in karaoke bars, beer gardens, massage parlors, casinos, bars and night clubs who may sell sex. The definition has evolved recently to women involved with free-lance sex work as well. 2 National AIDS Authority, Cambodia Progress Report, January 2012 – December 2013. 2014. No estimate for HIV infection among transgender people is available. 3 Ibid. Treatment eligibility is a CD4 count of 350 or below. 10 The Royal Government of Cambodia (RGC) has demonstrated strong strategic leadership in support for scale up of HIV prevention and treatment programming, within a strong policy framework that encompasses the enabling environment. RGC expenditure on health has increased but from a low base and at a slow rate. Cambodia’s HIV response remains heavily donor dependent, with the RGC financial contribution only amounting to 11 per cent of total funding in 2012. The United States Government remains the largest bilateral donor, with the Global Fund (GF) being the largest of all donors. In mid-2014 Cambodia was notified that its allocation under the GF’s new funding model would need to last for 4 years, rather than two years. This fifty per cent reduction in anticipated funding has caused a crisis for Cambodia’s national HIV response, with all areas of the budget being examined for possible savings. As a result, stakeholders are particularly interested in the future directions recommended by this evaluation. Despite progress in reducing HIV prevalence and increasing access to care and treatment, significant challenges remain. These include further reduction of new infections among KPs, particularly those who are hidden and hard to reach, and building upon Cambodia’s considerable achievements in treatment scale up by further strengthening community support mechanisms to ensure retention in treatment and ART adherence. These challenges need to be addressed within the context of reduced financial resources and strategies to ensure the sustainability of programs. 2.2 Overview of the Sustainable Action Against HIV and AIDS in the Community Project SAHACOM is a five year (October 2009 – September 2014) $13.4 million project, implemented by KHANA. SAHACOM’s goal, expected results and intermediate results are set out in Figure 1, below. SAHACOM consists of three main components: 1. Community support to PLHIV and OVC through Community Support Volunteers (CSVs), self-help groups (SHGs) and livelihood and economic strengthening for PLHIV, with support for treatment retention and adherence. 2. Focused prevention programming for KPs, with a particular emphasis on under-served and marginalized groups, including the promotion of HIV testing and linking HIV￾positive people to treatment and care services. 3. Capacity building of national networks, SAHACOM’s NGO implementing partners (IPs) and SHGs. SAHACOM’s major focus is on component 1. Cross cutting focus areas for each of the three main program components are gender, poverty reduction, multi-sectoral partnerships, and sustainability. The operational framework for SAHACOM and the structure of technical approaches to implementation of Components 1 and 2 are illustrated in Figures in Annex 7. The SAHACOM design was based on four principles: • Community leadership and participation: to promote relevance and local ownership. • Adaptability and innovation: to meet changing needs and draw on emerging best practice. • Sustainable and cost effective models: to reduce PLHIV reliance on external support by increasing individual social capital and community responsibility. • Partnership and collaboration: to promote synergies and maximize use of resources and linkages with health and non-health services. The program is tailored to increasing service delivery coverage; improving quality; ensuring a continuum of comprehensive prevention, care and treatment services, with HIV testing as an 11 entry point to treatment; meeting the needs of KPs; addressing the complexity of overlapping risk behaviors; and reducing stigma faced at health service and in communities. SAHACOM works in Phnom Penh and eight high burden HIV provinces: Banteay Meanchey, Battambang, Kampong Cham, Kampong Chhanang, Pailin, Pursat, Siem Reap, and Takeo. Implementation of Component 1 is supported by 15 NGO IPs and Component 2 by five different NGO IPs. Figure 1: SAHACOM logical framework OUTPUTS Program Components/ Expected Results ER1. Improved coverage, quality & sustainability of comprehensive and integrated services for PLHIV (including KPs) and OVC, which have successfully linked communities with public health and non-health services ER2. Improved uptake of innovative and targeted HIV prevention interventions and services to KPs, with a particular focus on under-served and neglected groups ER3. Strengthened capacity and leadership of NGOs/CBOs and communities (especially those representing KPs & PLHIV) leads to their meaningful participation in delivering quality and sustainable community-based HIV prevention and care services within the national response USAID, KHANA, Implementing Partners, Strategic Partners, Collaborative Partners, Private Sector Partners IMPACT Goal Improved health & quality of life of people in Cambodia by reducing the impact of HIV and AIDS, especially amongst the most vulnerable population groups OUTCOME ESW MSM PLHIV IDU OVC Strategic Objective In partnership with the Royal Government of Cambodia and other stakeholders, enable people living with/and or affected by HIV and AIDS to meet their health and related socio-economic needs through sustainable, community-driven programming approaches INPUTS Intermediate Results IR1.1 Full coverage achieved and maintained in project sites of high quality comprehensive care, treatment and support services for PLHIV (including KPs) and OVC IR1.2 Improved integration of HIV-related services for health, social welfare and impact mitigation with existing home & community-based care services IR1.3 Increased capacity and sustainability of self-help groups IR2.1 Increased access to services, HIV knowledge & related behavior change among KPs IR2.2 Supportive environment established for HIV prevention programming with KPs at national and sub-national levels IR3.1 Increased technical and organizational capacity of community-based service providers and NGOs, incorporating best practice approaches IR3.2 Increased involvement of PLHIV and KPs in program design and advocacy for an enabling environment 12 3. Community based support for PLHIV and OVC 3.1 Outline of community based support programming for PLHIV and OVC Over the last 15 years, community and home based care models in Cambodia have evolved and been refined to meet changing needs, with practice standardized through the adoption of standard operating procedures (SOPs) developed under the leadership of the National Centre of HIV/AIDS, Dermatology and STD (NCHADS). At the early stage of the epidemic, prior to the scale up of HIV treatment, demand for home based care (HBC) services was high due to large numbers of people experiencing HIV-related opportunistic illnesses (OIs) and as more people entered the continuum of care. Initially, HBC teams were composed of NGO and health center staff. Where there was limited staff at health centers, HBC services were mainly implemented by NGOs and PLHIV. Over time, as coverage spread, the HBC teams evolved to the point where NGOs began to coordinate activities, while volunteers from PLHIV SHGs played a stronger role in actual service delivery. Later, given the improved health status of PLHIV in the era of ART scale up, the HBC model was modified to community and home based care (CHBC) which included impact mitigation, economic strengthening, livelihood support and nutrition. This reflected the need for a more nuanced approach to offer case-specific support, focusing effort and resources on linking persons newly diagnosed with HIV infection to HIV treatment services, positive prevention, under-served areas, and on HIV-positive people from KPs. A key feature of the new model was encouragement of self-reliance by increasing individual social capital and community responsibility. The shift to CHBC also incorporated the need for improvement in referral systems and linkages with tuberculosis (TB), sexual reproductive health (SRH), family planning, and other health services. The SAHACOM CHBC model represents a shift from reliance on the NGO staff for implementation, which had been necessary earlier in the epidemic when HIV-related stigma and discrimination discouraged PLHIV from coming forward to participate in self-help groups. The SAHACOM model is spearheaded by Community Support Volunteers (CSVs) who are all PLHIV from the local community and local Self Help Groups, led by the CSVs. NGO IPs support the CSVs and SHGs by developing resource material to be used during meetings, training CSVs as group leaders, and providing transportation fees for members to go to OI/ART clinics for monitoring and medication. During monthly group sessions, members review key topics, get information on the latest changes in policies affecting services and medication, and share information about good practices among themselves. The group members form a network of people that they can turn to when ill or in need of support and information. The SHGs integrate livelihood activities such as village saving and loans (VSL), vocational training, and coaching and mentoring. The CSVs work closely with the Mondul Mith Chuoy Mith (MMM) Coordinator from the Cambodian People Living with HIV Network (CPN+) at ART clinics and the newly created Active Case Manager in ensuring PLHIV retention in care. In line with the SOP, one part of the CHBC model is tailored to meet the needs of orphans and vulnerable children (OVC) from affected families, including those living with HIV. CSVs ensure children have access to a basic social safety net, including food and welfare support, support to attend school, and in-kind support from community members and faith-based groups. OVC Support Groups (OVCSGs) are affiliated with a local PLHIV SHGs. Adolescent OVC are provided with the skills to become Support Group Leaders (SGL) so that OVCSGs build autonomy. 13 Although SAHACOM is formally implementing the CHBC model, the reality is that the improved health status of PLHIV means that the need for home based care has largely evaporated. The model now focuses on community support, as described above. While this report uses the term CHBC, it should be understood to include the model described above. 3.2 Findings and key results SAHACOM’s achievement of its objectives and results has been measured against Expected Result 1: “Improved coverage, quality and sustainability of comprehensive and integrated services for PLHIV (including KPs) and OVC, which have successfully linked communities with public health and non-health services” and Intermediate Result 1: “Full coverage achieved and maintained in project sites of high quality comprehensive care, treatment and support services for PLHIV (including KPs and OVC)”. In addition, KHANA is in the process of finalizing an internal SAHACOM End-of-Project Evaluation, based on a survey of the project’s beneficiaries in relation to key indicators (referred to as the end-line survey in this report). Results from this survey can be compared with results from the SAHACOM mid-term evaluation’s survey of beneficiaries, conducted 2012. The sample for the end-line survey was 71 per cent rural and 67 per cent female. The mean age was 43 years. The end-line results are still in draft form and have not been stratified by KPs or analyzed for differences in sub-groups yet. Specific indicators and results will be discussed in the sub-sections below. Note: A summary of performance by SAHACOM against key indicators for CHBC is in Table 7, Annex 8. 3.2.1 Coverage Over the five years of SAHACOM implementation, CHBC services have been provided in eight provinces and Phnom Penh, including 29 Operational Districts (ODs) (currently 24) and 191 Health Centers (currently 156). These areas have the highest HIV prevalence in Cambodia. The project has supported 540 SHGs (currently 361), involving a total of 9,250 PLHIV and 500 OVCSGs (currently 435), involving 10,139 OVC. This accounts for 52 per cent of KHANA’s total CHBC activities4 which in turn contribute to 60 per cent of the national CHBC coverage. Therefore the USAID-funded project makes up around 31 per cent of the national response. The SAHACOM model is the model being implemented, with minor variations, throughout the country, supported by all funding sources. The model was adapted to the 2010 draft NCHADS SOP. The shift from greater NGO involvement in CHBC to a CSV and SHG centered model, with NGO support, has been successful. The CHBC model is primarily oriented towards the needs of PLHIV living in rural areas and has not been tailored for urban HIV-positive KPs. This is partly the result of how Cambodian national health policies are structured. Activities directed at EWs, MSM, TGs and PWID come under the NCHADS Boosted Continuum of Prevention to Care and Treatment (COPCT) SOP, whereas PLHIV SHG and CSV activities are implemented in line with the NCHADS Boosted Continuum of Care and Linked Response targeting the general population PLHIV in 4 KHANA is implementing CHBC activities with other sources of funding in addition to USAID funding for SAHACOM. 14 rural areas and pregnant women. Due to a variety of issues related to the illegal nature of behaviors and stigmatization in society, many KP PLHIV do not want to be identified or do not want to access care and treatment elements of the COPCT, making service provision for this population more challenging. Peer Facilitators (PFs) working with KPs who are HIV positive must address the added dimension of targeted prevention. While there are some SHGs established for HIV-positive KPs, these groups do not seem to be as institutionalized or as organized as the village-based SHGs. 3.2.2 Linking PLHIV with services The number of HIV positive adults and children receiving a minimum of one clinical service increased from 4,374 at baseline in FY09 to 8,974 in FY 2013 and 7,407 in the first six months of FY14. The number of number of eligible adults and children provided with a minimum of one care service increased from 23,948 at baseline in FY09 to 38,903 in FY 2013. During the first six months of FY2014, SAHACOM reached 6,854 adult PLHIV and 11,998 OVC (of which 1,499 were HIV positive) with prevention, care and support services. The activity package included assistance with HIV case detection among priority groups, including pregnant women, sero-discordant couples, and TB patients; positive prevention for sero-discordant couples through treatment as prevention (TasP); partner tracing; referral of HIV positive cases to Pre-ART/ART services, including immediate enrollment on ART for newly diagnosed eligible patients; and livelihoods support. SAHACOM has successfully linked most PLHIV who attend SHGs and have a CSV with needed services. On average, each PLHIV received six referrals and OVC received seven referrals to health services each year. This included financial assistance with the cost of transport for the most-in-need PLHIV on ART, with priority afforded to HIV positive pregnant women and children and newly identified PLHIV. According to the end-line survey, 95 per cent of PLHIV respondents received support from SAHACOM IPs in the past 12 months. The most common support was financial assistance for transport to a health facility (96%) which also was perceived as the most important need. Other support received included psychological support, home visits and counseling (27%), food support (24%), and child education support (24%). Material support provided through SHGs appeared to be a significant motivator for PLHIV participation in the group. For example, during a site visit to a SHG, participants were asked if there were any PLHIV in the local area who did not participate in the SHG, and if so why. The answer was that there were five PLHIV who did not participate because they had sufficient money to support themselves. For OVC, the majority of SAHACOM’s beneficiaries were currently in school (93%). OVC at end-line were more likely to report that they attended school regularly in the past 12 months compared to OVC interviewed at mid-term (89.6% vs. 85.3%; OR= 1.8 , 95% CI= 1.4 – 2.3). Sixty eight per cent of the participants reported that they had received educational support from any sources in the past six months. Support for schooling almost doubled from the mid-term survey (46%) to the end-line survey (81%). The results of SAHACOM in relation to beneficiaries’ quality of life are impressive. Eighty per cent of PLHIV at end-line rated their overall health as good or fair compared to 52 per cent at baseline. Similarly, 72 per cent of PLHIV at the end-line rated their overall quality of life as good or fair compared 35 per cent at baseline. In FY 2013, 96 per cent of PLHIV reported that they were satisfied with the CHBC services of SAHACOM. 15 Accelerated case detection: Prevention of HIV infection among sero-discordant couples has been a focus of SAHACOM’s CHBC activities, as part of its support to scale up TasP. In the SAHACOM baseline survey, 73 per cent of PLHIV reported their spouse/partner was also HIV positive. Over the life of SAHACOM, CSVs have identified 872 sero-discordant couples, provided positive prevention education, and monitored HIV infection trends among them. Overall, 98 per cent of all positive partners in sero-discordant relationships are currently on ART. The remaining two per cent are receiving pre-ART services. In FY14, only two new HIV infections were identified among sero-discordant SAHACOM beneficiaries. CSVs helped by supporting open discussion in SHGs or couples groups to help decrease the stigma and fear among sero-discordant couples. In the end-line survey, only 5.2 per cent reported not knowing their spouses or partner’s HIV status. In line with the national Boosted Linked Response, CSVs identify and refer pregnant women for HIV testing and, if positive, early initiation of ART to reduce mother to child transmission. This resulted in 1,221 pregnant women being tested, with only two being newly identified as HIV positive. HIV positive pregnant women receive ART Option B+. There have been no cases of mother to child transmission among SAHACOM beneficiaries. CSV and IP efforts on prevention of mother to child transmission (PMTCT) have positively engaged community leadership and broadened acceptance of the HIV response at the village level. 3.2.3 ART uptake rates and retention and adherence Maximizing Retention in Care: The number of SAHACOM’s PLHIV beneficiaries in need of ART and currently on treatment increased from 90 per cent at baseline to 93 per cent at mid-term and to 96 per cent at end-line, with a mean length of 6.9 years (SD= 3.9) on ART. This represents a high treatment uptake rate and is above the national treatment coverage rate of 83 per cent of those eligible. SAHACOM data indicate that 89 per cent of PLHIV on ART were retained in treatment at 12 months after initiation, compared to a national retention in care rate of 85 per cent. Of the 11 per cent not retained, one per cent were lost to follow up and 10 per cent had moved to new locations, primarily seeking employment. CSVs work closely with CPN+’s MMM coordinators and health clinic staff to follow PLHIV who do not keep appointments. Access by PWID to ART is constrained by arrest, detention in drug treatment and incarceration. The end-line survey found that 46 per cent of PWID respondents had been arrested for drug use or trafficking in the last year. More than half (56%) had gone to a drug rehabilitation center at least once in their life, and 20 per cent had been to a rehabilitation center in the past year. About a third of the group (31%) had been incarcerated at least once. These factors limit access to ART for the large number of PWID who are HIV positive (around 25%). Methadone maintenance treatment (MMT) has been shown to increase ART adherence and retention. The number of HIV positive PWID receiving MMT at the Khmer-Soviet Hospital Clinic in Phnom Penh has declined from 24 per cent in 2012 to 11 per cent in 2014, although the total number of PWID receiving MMT has remained largely unchanged. There appears to be a lack of initiative by MMT clinic staff in relation to HIV-related care. A contributory factor is the limited linkages between the National Center for Mental Health, which is responsible for MMT, and NCHADS. This structural and institutional barrier has been recognized by the Ministry of Health and, in August 2014, a restructuring aimed at strengthening MMT service delivery and retention rates was initiated. 16 The mobility of EWs raises a similar concern. Both the mid-term and end-line surveys found that most EWs stay at their current place for less than one year. This presents challenges for follow up and retention in the care and treatment continuum for HIV positive EWs. Integration of HIV and Sexual and Reproductive Health, Family Planning, PMTCT and Tuberculosis: CSVs provided PLHIV and adolescent OVC with comprehensive, non￾judgmental sexual and reproductive health rights (SRH) information and education. Informed consent and voluntary uptake of FP services has been a principle observed throughout the life of the SAHACOM Project, with CSVs trained to counsel regarding the full array of options available in Cambodia for birth spacing. Non-discriminatory counseling is provided to PLHIV wishing to conceive a child, with referrals to PMTCT services at local antenatal care facilities. SRH, FP, PMTCT and TB education and counseling are offered through one-to-one and quarterly group sessions. The end-line survey found that 66 per cent of SAHACOM’s PLHIV cohort had been referred to SRH services and 63 per cent to FP services by CSVs. The number of PLHIV and adolescent OVC who received counseling and referrals to access modern contraceptive methods in FY 2013 was 2,144. This fell short of planned estimates to reach 4,000 PLHIV and adolescent OVC with FP educational information and counseling services. In year two of SAHACOM, KHANA contracted Marie Stopes International/Cambodia (MSIC) to conduct an assessment of KHANA’s SRH/FP work, including that of IPs, and develop a roadmap for HIV-SRH integration. Key findings were that while HIV and SRH/FP had been integrated, KHANA’s SRH/FP curriculum needed strengthening to support capacity building needs in this area. While both KHANA and MSIC were encouraged by USAID to collaborate, there is insufficient evidence to conclude that this occurred effectively. Constraining factors appear to be insufficient leadership by KHANA to IPs on SRH/FP and integration being seen by IPs as a burden on top of existing work. SAHACOM contributed to efforts for HIV-TB integration by encouraging TB screening among PLHIV, and HIV counseling and testing (HCT) among suspected TB patients. CSVs administered a screening questionnaire for suspected TB cases among PLHIV and OVC in the community, followed by referral to testing at health centers. In FY 2013, 444 PLHIV were screened for TB in HIV care and treatment settings and 97 started on TB treatment. In the first six months of FY 2014, 171 PLHIV were screened for TB, and 53 of them commenced treatment. While these numbers appear on the low side, the use of the screening questionnaire to determine who needs to be referred for testing at health centers probably explains this. CSV worked collaboratively with health center staff to follow up on TB treatment adherence. In FY 2013, the number of registered TB patients who were tested for HIV was 495, against a target of 1,500. The evaluation team was unable to determine whether treatment outcome success rates for SAHACOM beneficiaries infected with TB were better than success rates for TB patients who were not HIV infected, in the same geographical areas. This warrants further investigation, as PLHIV in clinical care may receive more support than non-PLHIV TB patients and be more adherent. 3.2.4 Self-help group sustainability There is now a well-trained cohort of CSVs and SGLs who are confident in their work with PLHIV, health care facility staff and local community leaders. On the other hand, prospects for them remaining in their positions are tenuous. Despite increases in the remuneration for CSVs and SGLs (US$50 per month), many feel this is insufficient. As PLHIV get healthier and employment opportunities increase, many trained CSVs will be employed in other 17 primary jobs, with CSV activities being seen as secondary. IPs and CSVs complained about their heavy reporting load. Weekly and quarterly reports are required by KHANA and NCHADS along with a host of other forms to keep track of inputs and outputs and patient health status. For example, CSVs are collecting individual clinical patient data (e.g., CD4 counts) that would normally be kept at health facility level. There appears to be a need for KHANA to streamline the data being collected by IPs and CSVs by only collecting information that is directly related to SAHACOM project activities. In addition, OVC SGLs leave their position at age 18 which means there is limited continuity in leadership for the OVCSGs. Identifying and training CSV and SGL replacements will impact the on-going sustainability of SHG. In many cases, the SHGs act as the venue where PLHIV receive their transport allowances and it is difficult to determine whether, over the long term, SHG continue to be necessary to meet the psychological, social and economic support needs of PLHIV. There may be an element of duplication between SHG meetings and MMM meetings. It was unclear as to why there is a need for both meetings, although MMM meetings are now held less frequently. If there is a need for MMM meetings, these should be held when PLHIV are receiving clinical services at the hospital, rather than clients coming at another time. This would reduce costs. There may also be some duplication of effort between CSVs and MMM coordinators. Due to time constraints, it was not possible to adequately examine this during the evaluation. A rapid assessment and analysis of the duties of each would be worthwhile. Drop-in centers (DIC) provide psychological support, including peer counseling, and act as safe places for KPs to meet each other, but appear to be hosting a limited number of PLHIV SHGs. Consideration needs to be given to the question of whether the current SHG model, which demands a long-term commitment from members, is the best modality for HIV positive KPs, especially in urban areas. Sustainability of SHGs has been fostered by creating linkages to local authorities. The IPs and KHANA have worked with village chiefs and Commune Councils (CCs) to sensitize them to the needs of PLHIV and the benefit of SHGs. IPs built on their strong relationships with local authorities to ensure that HIV has been integrated into Commune Development Plans (CDPs). Nonetheless, this is a nascent process since most CDP funding is targeted toward infrastructure and communes have relied on NGOs to support the service delivery component of the plan. For the most part, IPs have not discussed the possibility of SHGs being directly supported by local government entities such as the CC. Improved links between OVCSGs and their SGLs and CCs and Commune Committees for Women and Children have supported collaboration on provision of care and protection from child trafficking, exploitation and abuse. Regular coordination meetings were held to share information and updates on issues and problems faced by children in the community. Issues relating to sustainability are further discussed in Section 7.3. 3.3 Conclusions The SAHACOM project has been successful in linking PLHIV with health services, contributing towards Cambodia’s high HIV treatment coverage rate. SAHACOM’s effectiveness in providing community support has resulted in high rates of treatment retention, with 89 per cent of PLHIV on ART being retained in treatment at 12 months after initiation. PLHIV also report significant improvement in their quality of life, including improved health status. 18 SAHACOM has been successful in transitioning to a community based support for PLHIV model, centred around CSVs and SHGs in rural and peri-urban areas. The evaluation did not identify any negative consequences from the shift to a more community-based model. KHANA’s Social Return on Investment (SROI) study of a European Union funded HIV integrated care and prevention project (ICP), similar to SAHACOM, found that the social return on investment was 96 per cent. That is, for every $1 invested, $1.96 was generated in social, health and economic value.5 With the improved health status of PLHIV resulting from ART, it could be argued that poor PLHIV in rural villages are not that different to other poor villagers and therefore all support should be mainstreamed. One key difference is that PLHIV villagers need to regularly go to clinics for checkups and to receive medication, and the cost of transport over a year can be considerable. Another possible difference may be that the economic capital of PLHIV may have been substantially reduced over time, especially through health problems for long term PLHIV, prior to more effective treatments becoming available. For example, PLHIV may previously have sold land and other possessions to deal with health shocks prior to ART initiation and may consequently be in a worse position than other poor villagers. There are, unfortunately, no data to indicate whether this is the case. The shift to greater community involvement in and ownership of support services and promoting reciprocal responsibility and self-reliance among PLHIV has laid a foundation for sustainability. Devolution of responsibility for support and increasing autonomy of CSVs and SHGs enabled IPs to reduce direct intervention. This was an important step in the move to reduce AIDS exceptionalism at the village level. There has been some success in mainstreaming SHGs at the village level. In some cases, the support that PLHIV have received has been from village or commune leaders from non-HIV funds. 3.4 Future directions Development of a streamlined, cost effective model for community support for PLHIV: In the context of reduced funding for the Cambodian response to HIV, there is a need to develop a streamlined, more cost effective model for PLHIV community support. This model needs to identify what are the core elements of community support that need to be maintained and the most cost effective way of providing that support. The Social Return on Investment (SROI) study uses an innovative form of cost-benefit analysis which can be used to monetize project outcomes that may otherwise be difficult to quantify. It has been applied to the KHANA ICP project to assess the impact of its community based responses to HIV prevention, care and treatment. In addition, URC is conducting a costing study is to provide recommendations on how community based support for PLHIV can be modified or improved to reduce costs with the goal of providing information for the next Global Fund proposal. Move to a community based case management approach which supports those most in need. The current SAHACOM model assumes all PLHIV have similar support needs. A streamlined model based on PLHIV needs would enable CSVs and PFs to tailor levels of support for PLHIV to changing needs, including phasing support to PLHIV in and out, as needed, based on health status and such vulnerabilities as risk of loss-to-followup or treatment 5 KHANA, Social Return on Investment. ‘Doing more with less’. Evidence based operational research on the KHANA Integrated Care and Prevention Project in Cambodia. 2012. 19 adherence challenges. This would enable CSVs and PFs to focus on providing support to those who are currently most in need. For example, support would be phased in for a PLHIV who was missing clinic appointments or who had poor adherence, with phasing out of this support once the person had become stabilized. While all newly diagnosed people require an immediate health assessment in relation to treatment eligibility and community based psycho￾social support, those stabilized on ART generally will require much less support, but may need referral for skills or business training to address livelihoods needs. This approach recognizes that the needs of PLHIV have changed significantly as a result of their better health status resulting from high ART coverage rates. A package of community support needs to be tailored to the specific needs of different beneficiaries at particular points in their lives along the continuum of HIV care and treatment (e.g., KPs, newly diagnosed general population spouses, pregnant women). Someone who has been stable on ART for many years and who has a job and a support group needs a different package of services than an active PWID who may be engaged in sex work or homeless. Parallel to a streamlined community support model, KHANA needs to streamline the data being collected by IPs and CSVs by only collecting information that is directly related to SAHACOM project activities. Explore options for increasing ART accessibility: Currently, PLHIV need to attend ART clinics more regularly than is clinically needed in order to get ART supplies. This is because stocks of ART held by clinics are generally insufficient to allow prescribing more than two months supply of ART. This increases transport subsidy costs for SAHACOM. The recognized need to improved ART supply chain management needs to be addressed. Other options that could be explored include piloting mobile ART clinics and community distribution of ART to reduce patient transport subsidies. Focus more on urban areas and key populations and design models accordingly. There has been insufficient emphasis under the SAHACOM Project in aligning the CHBC model to address the specific community support needs of urban PLHIV, particularly HIV-positive KPs. Dynamics among urban KP PLHIV, and the mobility and “hidden” situation of many KPs may preclude SHG formation and participation. There is a need for an urban and KP-relevant model. And, PLHIV who are urban-dwelling general population may not find affinity with SHGs focused on meeting the distinct needs of PLHIV KP. Address possibilities for mainstreaming current CHBC activities, particularly at the village level, by integrating PLHIV in other schemes. CSV participation in mechanisms promoting community access to health services and participation in health service management, such as Village Health Support Groups and Health Center Management Committees, can help mainstream HIV services through the participation of PLHIV in local decision-making, accountability of public services, and effectiveness of public services. In the villages, many SHG are now functioning around VSL schemes. If these can be supported after the USAID (or other donor) funding ends or integrated with other VSL schemes, the likelihood that SHGs will continue will be higher. Explore the possibility of using the Health Equity Fund for HIV services: The Health Equity Fund (HEF) currently covers 61 ODs and is expected to cover all ODs and Health Centers by 2015. The government contributes 40 per cent of the funding for HEF and donors fund 60 per cent. The geographic overlap of HEF and SAHACOM areas was not available to the evaluation team, but during visits to IPs and hospitals, it was obvious where HEF was available. In areas with HEF, the SAHACOM IPs were often involved in the ID Poor identification process in the communities where they worked. All HIV and AIDS services at ART/OI Centers are currently free to the patient and thus HEF does not provide payments for these services. It is understood that transport subsidies for PLHIV are currently being paid by 20 SAHACOM rather than HEF. A HEF Plus is being considered in the future which would have the capacity to cover HIV services. USAID/Cambodia has recently awarded a new, multi-year project activity which has as one of its objectives exploring and advocating for options related to HEF Plus for addressing the issue of access and non-free HIV services for PLHIV and KPs. This has potential to provide an efficient system for payment of transport subsidies to PLHIV. As current funding of HEF is limited, it would be necessary for NCHADS and/or donors to contribute funds to cover the cost of extending HEF to cover HIV services. Continue emphasis on developing an understanding about the reduction in HIV resources at village and commune level. KHANA and the IPs should explain to village and commune leadership that donor supported HIV resources are shrinking and that villages and communes will need to contribute to sustaining existing support mechanisms for PLHIV and OVC through their Village and Commune Development Plans. Address the impact of laws, policies and practices on PLHIV access and retention in care. An area of particular concern are the high rates of incarceration of PWID and the non￾availability of ART and HIV care and treatment in custodial settings. Also, laws which have affected the structure of commercial sex industry and the resulting focus of HIV programs on EWs, while freelance and street based sex workers are not reached, means that a continuum of prevention to care and treatment for freelance and entertainment based sex workers does not exist. 4. Livelihoods and economic strengthening 4.1 Outline of livelihoods and economic strengthening programing A 2010 KHANA survey found that 40 per cent of SAHACOM’s PLHIV beneficiaries were particularly vulnerable in terms of food, water and economic security and lacked skills and capacity to significantly increase household incomes and nutritional consumption.6 Of the 42 per cent involved in agricultural work, 25 per cent had no skills and 40 per cent lacked financial resources to support this work. Of the 17 per cent involved in small scale trading, 37 per cent lacked skills and 40 per cent lacked financial resources to support their business. SAHACOM’s livelihoods work has been focused on those without skills and financial resources. KHANA’s livelihoods and economic strengthening programming aims to strengthen the socio-economic status, resilience and health outcomes of poor PLHIV, OVC and KP households, through mitigating vulnerability and reducing dependency. It aims to maximize household assets using village based saving and loans scheme, skills building in horticulture and animal husbandry, small cash loans, and on-going technical support for application of skills learned. The rationale for this work is that households with more secure livelihoods will have a better access to health services, send children to school and be more resilient to health and financial shocks.7 6 Household economic livelihood survey conducted by KHANA in 2010. A further 35 per cent of respondents were vulnerable, with 25 per cent of beneficiaries being ranked as less vulnerable. 7 SAHACOM Year 5 Work Plan Narrative, KHANA, September 2013. 21 For the first two years of livelihood programming, Development Alternatives Inc. (DAI) provided technical assistance to KHANA on the development of strategic directions, intervention approaches, tools and materials in livelihoods and economic strengthening. KHANA livelihoods staff reported that following the integration of the livelihoods unit within KHANA’s program management unit, the interest of program staff in livelihoods work improved. 4.2 Findings and key results 4.2.1 Village savings and loans scheme The VSL scheme has three core products: savings, loans to group members, and emergency loans. As savings accumulate, VSL members are motivated to invest money from their savings and loans into income generating undertakings. While KHANA provides training and model rules to new VSL groups, each group sets its own rules, such as who is eligible to participate, the maximum amount for loans and the frequency of borrowing. The VSL scheme is supported by livelihoods skills training and small start-up cash loans. A pilot Village Savings and Loans (VSL) scheme from 2010-2012, funded by the European Commission found that 100 per cent of respondents reported benefitting from the scheme and the social benefits of group support. Respondents expressed a high degree of commitment to continuing their pilot VSL groups.8 Since the commencement of SAHACOM supported VSL groups in late 2010 through to June 2014, a total of 174 VSL groups were established out of 540 SHGs. Total PLHIV membership of VSL groups was 1,800, which is approximately 20 per cent of all SHG members. Women make up 65 per cent of VSL group membership. Total saving across the 174 VSL groups was KHR 246,634,000 (US$ 60,897). On average, each VSL member saves around US$35 per 12 month cycle. Loans have totaled KHR 241,700,000 (US$ 59,679), which is 98 per cent of total savings. The interest rate on loans is two per cent per month, which is considerably lower than interest rates charged by money lenders. Loans have primarily been provided for strengthening existing livelihoods activities including horticulture, animal husbandry, and small scale entrepreneurship. Members of VSL groups are encouraged to make voluntary contributions to the Emergency Fund at their monthly savings and loans meetings. As with ordinary loans, decisions on the allocation of emergency loans are made by the VSL group. Contributions to emergency funds to May 2014 total KHR 21,266,700 (US$5,251) of which 38 per cent has been loaned. The VSL groups operate on 12 month cycles. At the end of each cycle, the VSL group decides whether they wish to continue the operations of the VSL group for a further 12 month cycle. Ten VSL groups were established in 2011, 37 groups in 2012 and a further 127 groups in 2013. Table 1 provides data on the number of VSL groups that have continued into their second and third cycles. For the ten groups established in 2011, 80 per cent continued to cycle two and 70 per cent to cycle three. For the 37 groups established in 2012, 81 per cent continued to cycle two and 73 per cent to cycle three. This demonstrates a strong commitment to the VSL groups by their participants. 8 KHANA’s VSL Survey Report 22 Table 1: Continuation of VSL groups after first year of operation Year 2011 2012 2013 2014 First generation of VSL groups Cycle 1 = 10 VSL groups Cycle 2 = 8 VSL groups (80% continuation rate) Cycle 3 = 7 VSL groups (70% continuation rate) Second generation of VSL groups Cycle 1 = 37 Cycle 2 = 30 (81% continuation rate) Cycle 3 = 27 (73% continuation rate) Third generation of VSL groups Cycle 1 = 127 Cycle 1 not yet complete Note: Generation refers to the year in which the VSL group was established The work of VSL groups is supported by a manual which KHANA adapted from a mainstream VSL scheme guide, with technical support from DAI. The manual includes methods for oral account verification which has been shown to be more effective with illiterate, semi-literate and innumerate groups in rural Cambodia.9 Members of the general community are allowed to join the VSL groups, which contributes to integrating PLHIV and OVC into the community and breaking down stigma and discrimination. 4.2.2 Livelihoods skills training and application In partnership with CEDAC, a local agricultural development organization, KHANA developed a livelihoods training program on a residential basis at the KHANA Livelihoods Centre in Kampong Chhanang. Selected IPs in priority areas identified PLHIV and OVC families in need of training. After training, technical support was provided to promote skill application and increase productivity. To May 2014, SAHACOM conducted 28 training sessions in various aspects of horticulture and animal husbandry (chicken, pig and fish raising, home gardening and micro businesses) and VSL establishment procedures for 572 PLHIV, including 327 CSVs. Female PLHIV made up 52% of PLHIV trainees. Training was also provided to 11 OVC household participants and 34 members of KPs. Eighty six IP staff were also trained to enable them to provide ongoing support to PLHIV, post training. A survey of 589 people who had undertaken this training found that 367 (62%) had applied the skills learned at training and of those, 237 (or 65% of those who had applied the skills) had done so successfully. That equates to an overall success rate of 40 per cent of those who had been trained. Reasons for not applying the skills were lack of motivation, lack of resources (e.g., water and land), lack of start-up capital, migration to Thailand, had another business with better prospects, concern over animal disease outbreaks, and an unstable health condition. Reasons for failure in those who had applied the skills were no real interest or lack of consistent commitment, lack of support from other family members, animal disease outbreaks, drought, insufficient start-up capital and migration. There was a difference in the rate of skill application and success for those trained, based on the sponsoring IP. For example, of 56 trainees supported by Cambodian Poverty Reduction (CPR) IP in Pursat, 50 (89%) applied skills learned, of whom 34 (68% of those who had 9 2010 KHANA VSL Program Guide 23 applied the skills) had done so successfully. This equated to an overall success rate of 61 per cent of those trained. CPR has demonstrated considerable commitment to livelihoods work. Possible reasons for its greater success rate may have included better selection of candidates for training and/or more IP support, post training. It can be concluded that the sponsoring IP makes a difference to the success rate of trainees. Skills training may have had a higher success rate if selection of trainees was based on a better assessment of their existing resources and suitable farming for those resources. For example, a couple in Pursat who completed training chose to grow papaya but found that their land did not receive sufficient rain to sustain the crop. They did not have enough money to build a well (US$250). It may have been preferable for the couple to be advised to grow a less water dependent product. Related to this example, the survey of trainees found that reasons for failure to apply training skills and lack of success in application of the skills included insufficient startup capital and lack of resources (e.g., water). This is an important lesson learned. The aim of livelihoods training is to equip PLHIV who have significantly improved health as a result of ART to plan for the future by becoming more self-reliant and less in need of SAHACOM support. Although there was no quantitative data available to the evaluation team on increased income for those who had undertaken livelihoods training, there was evidence from site visits that this was the case. For example, a PLHIV widow in Pursat with two children reported that as a result of income she generated through livelihoods work, she now paid for her transport costs to the clinic rather than seeking support from the SAHACOM IP. 4.2.3 Small cash grants Small cash grants, to be used as capital for setting-up small scale enterprises and to support animal husbandry and home gardening, were available to households following completion of livelihoods training and approval of a business plan. The maximum grant amount was US$120. A total of 505 PLHIV and OVC households were provided with small grants. The SHGs visited in areas where training was available said that, together with their VSL loans, these additional small cash grants provided them with capital for business/livelihoods start-up or expansion. No data was available to the evaluation team on outcomes related to these grants. KHANA plans to conduct an impact assessment in the second half of 2014. 4.2.4 Linkages with other livelihoods and economic strengthening initiatives HARVEST Project: Ten PLHIV households, supported by SAHACOM IPs in three provinces, have been linked with USAID’s HARVEST project. HARVEST provides technical support and skills training in modern agriculture technologies and management practices. The reason there were only 10 PLHIV households linked to HARVEST is because of HARVEST’s selection criteria that focuses on a higher level of capacity, resources and potential business viability than would be commonly found amongst SAHACOM’s beneficiaries. KHANA has entered into an agreement with HARVEST which will create more opportunities for PLHIV SHGs to access agricultural technologies and skills. Given the imminent close-out of SAHACOM, this partnership will need to be continued under the Flagship project and GF￾funded program. Vision Fund Cambodia: The small cash loans (see 4.2.3 above) were implemented through a partnership agreement with Vision Fund Cambodia (VFC). This enables PLHIV and OVC households to access VFC financial services and microloans beyond the life span of SAHACOM. VFC also had some technical inputs to KHANA’s livelihoods programming which KHANA found to be beneficial. For example, advice on the monetary level of livelihood grants. 24 4.3 Conclusions SAHACOM’s livelihoods and economic strengthening work evolved: Initially, programming simply involved one-off cash transfers of US$31-41 to selected beneficiaries, based a non￾systematic client assessment of their need for income generating activities (IGA). These grants were provided without any technical support and systematic follow-up. The grant amount was also insufficient. An evaluation of a pre-SAHACOM pilot project with VFC found that a grant of US$150-180 was needed to provide sufficient support for IGA. 10 In response, KHANA increased grants to US$120 and tied them to completion of a training course and approval of a business plan. Over the last three years, programming has evolved to a more strategic approach which focuses on 1) practical skills training;2) post training assistance on implementation of skills learned at training; and 3) provision of business development skills. This has been supplemented by establishing VSL groups. VSL as an empowerment model: VSL provides financially vulnerable PLHIV in rural communities with a supportive and safe space to learn through doing about the benefits of saving and investing. VSL differs from microfinance institutions as all money is generated from within the group, and the group, rather than an outside lender, administers all loans and decides on all policies. VSL also encourages participants to become financially self- reliant. Other benefits of livelihoods and economic strengthening programming have been: • Helped reduce stigma and discrimination. For example, in some villages VSL groups have been opened up to non-PLHIV members and villagers have been happily buying food and other products from PLHIV. • PLHIV with significantly improved health as a result of ART have been encouraged to plan for the future by becoming more self-reliant and less dependent on SAHACOM. • VSL groups have built social capital through peer support and the trust required by pooling savings and loaning money. • VSL groups have built beneficiaries confidence in and understanding of the use of credit. Rural oriented vs urban oriented: SAHACOM’s livelihoods and economic strengthening activities have a strong rural focus. VSL groups are confined to rural areas and the primary orientation of the KHANA Livelihoods Training Centre is on agricultural skills training (although some other activities that could be undertaken in urban areas such as cake production, tailoring and small grocery shops, were observed by the evaluation team). While continuing support for vulnerable PLHIV in rural areas is important, future livelihoods and economic strengthening programming should consider development of models suitable for urban areas. IPs have a limited multisectoral focus: Although IPs have developed strong links with PHDs, there was no evidence of IPs establishing partnerships with other provincial government departments covering areas such as social welfare, agriculture and rural development that could link with livelihoods programming. With the skills gained by PLHIV in training, coupled with savings they have made and complementary grants, beneficiaries may have the potential to produce more food for their 10 Evaluation of microfinance pilot project for HIV affected families by Vision Fund Cambodia and supported by KHANA-August 2008 25 own consumption and generate additional income to cover transportation costs to ART clinics, children’s education and other health and non-health related expenses. IGA may help reduce their need to leave home to seek employment which in turn may increase ART retention and adherence, and gradually reduce donor-reliance among beneficiaries and enhance their capacity to become self-reliant in dealing with financial and health shocks. As noted above, while there is some evidence from field visits by the evaluation team to indicate a greater level of self-reliance by some PLHIV involved in livelihoods activity, there is no quantitative data available. Further evaluation activities planned by KHANA may be able to provide such data. 4.4 Future directions for livelihoods and economic strengthening programming Continue livelihoods and economic strengthening support but on a reduced scale, targeted to PLHIV most in need.: With the improved health status of PLHIV, who can now expect to have close to a normal life expectancy, livelihoods and economic strengthening programming has become an important feature of community support to PLHIV, with the aim of supporting the move away from a culture of dependency to financial independence and self-reliance. Future programming should incorporate both rural and urban models of livelihoods development. Improve technical sophistication: In future programming, a higher success rate for IGA could be expected if the reasons for not applying skills learned at training or lack of success in applying those skills are systematically addressed (see 4.2.2 above). Strengthen linkages with mainstream livelihoods and economic strengthening programs: While SAHACOM has benefited from the support of KHANA staff with livelihoods expertise, the level of support for livelihoods in an HIV oriented organization such as KHANA is likely to be limited. Any ongoing livelihoods and economic strengthening work would benefit from stronger linkages with mainstream development partners specialising in this area. Engage local authorities in support for livelihoods programming with local development NGOs: Local authorities can play an important role as a catalyst in connection of PLHIV households to mainstream development agencies and specialized livelihoods and economic strengthening programs. This mainstreaming approach will be essential given the likely decline in HIV funding over the next decade. Where there is commitment from local authorities, this approach can succeed. For example, in Pursat province the engagement of a village chief in livelihoods and economic strengthening activities provided the opportunity for a PLHIV widow to access livelihood support from a mainstream agency. Local authorities, such as Village Chiefs, can also play a role in advocating for the inclusion of HIV and social protection into the commune investment plan which is one of the major local development programs of the RGC. 4.5 Recommendation 1. Due to the lack of quantitative evidence for outcomes related to SAHACOM’s livelihoods work, a study should be conducted to determine whether there is a relationship between increased household incomes and changes in health care outcomes of PLHIV (includes CD4 counts, and retention and ART adherence rates). This should include a comparison between the strong livelihood and economic strengthening programming for PLHIV and no/weak programming in these areas. 26 5. Focused prevention The expected result for SAHACOM’s component 2 is improved uptake of innovative and targeted HIV prevention interventions and services for KPs, especially by those from under￾served and neglected groups. The expected intermediate results are increased access to services, HIV knowledge and related behavior change among KPs; and a supportive environment established for HIV prevention programming with KPs at national and sub￾national levels. The national Continuum of Prevention to Care and Treatment (CoPCT) policy11 requires all IPs to link a package of prevention interventions and support, provided by peer facilitators (PFs), supported by NGO staff, with HIV care and treatment services. See Annex 9 for details of the package. Particular emphasis has been placed on promoting HIV counseling and testing (HCT) for KPs as a prevention strategy and as an entry point to life saving HIV treatment. 5.2 Findings and key results SAHACOM’s prevention activities are appropriately focused on the KPs of EWs, MSM, TGs and PWID. Interventions have been primarily implemented by 96 outreach workers (OWs) engaged by five IPs12, working in 21 administrative districts of the three priority provinces of Siem Reap, Banteay Meanchey and Battambang, and Phnom Penh. SAHACOM activities have appropriately been delivered in the highest burden districts. 5.2.1 Uptake of targeted HIV prevention interventions The key PEPFAR indicator adopted to measure SAHACOM’s focused prevention work with KPs was number of KPs reached with individual and/or small group level interventions that are based on evidence and/or meet minimum standards. 13 Figure 1 below shows the numbers of KPs reached each year by SAHACOM with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet minimum standards. The program reached 11,465 KP in 2014, exceeding the target of 8,185. The program exceeded targets for MSM, TGs and EWs. In 2013, 4,692 MSM/TG were reached, substantially exceeding the target of 3,633; and the program reached 5,268 EW, exceeding the target of 4,759. These results were attributed to the identification of new hotspots. However the program struggled to meet the target of reaching 1,130 PWID. The target was revised down to 350, after the program reached 244 PWID in FY 2013, and KHANA subsequently reached 347 PWID in 2014. The difficulty meeting the initial target of 1,130 reflects the challenges of identifying and reaching a hidden population whose behavior is criminalized. Focused prevention for MSM, TGs and PWID has primarily been undertaken by ‘specialist’ IPs; that is, NGOs primarily run by and for KPs (KORSANG, MHSS and MHC). They provide peer outreach, networking, support services, information and advocacy. Through observations at site visits, the evaluation found that providing peer outreach in familiar 11 National Center for HIV/AIDS, Dermatology and STD (2009) Continuum of Prevention to Care and Treatment for HIV/AIDS. Ministry of Health 12 These IPs are CARAM and SIT for EWs, KORSANG for PWID, and MHC and MHSS for MSM and TGs. There were originally eight IPs for focused prevention. Due to the limited ability of three IPs to reach KPs, three IPs were not re-contracted to perform these activities. 13 Note: positive prevention is discussed in Section 3. 27 locations where KPs gather and feel safe contributes to reaching the target number of KPs, and that the relaxed environment supported the quality and effectiveness peer education. Figure 1: KPs reached by SAHACOM with focused prevention services, 2010-2014 6016 6314 10278 11575 11465 4279 4763 6590 6311 6331 880 1047 3550 5020 4787 857 504 138 244 347 0 2000 4000 6000 8000 10000 12000 14000 2010 2011 2012 2013 2014* MARP (#) # of EW # of MSM # of PWID Source: SAHACOM Performance Management Plan. Note: MARPs refers to key populations (KPs) HIV counseling and testing: The peer OW model has been able to integrate HIV prevention alongside communication covering a range of topics of interest to KPs and this has created a context for promoting the value of knowing your HIV status. The IPs report that KPs are not experiencing discrimination at ART sites, so treatment is accessible, and this, combined with access to free treatment, also encourages testing. The mid-term and end-line surveys of SAHACOM clients indicate high HIV testing rates among KPs (see Table 2, below), although there was a decrease in testing among EWs and MSM in the end-line survey. Some caution should be exercised in interpreting this data as KPs were only surveyed in some sites. (MSM were surveyed in Siem Reap and Battambang; EWs in Phnom Penh and Siem Reap; and PWID in Phnom Penh). Table 2: HIV testing by KPs at SAHACOM mid-term and end-line Key population Tested in the last 6 months Mid-term 2012 (%) Tested in the last 6 months End-line 2014 (%) Ever had an HIV test (%) – from end-line survey 2014 Entertainment workers 68 65 82 Men who have sex with men 94 77 83 People who inject drugs No data 83 93 Source: SAHACOM mid-term and end-line surveys. Data on HIV testing from the mid-term and end-line surveys is, however, not consistent with SAHACOM performance monitoring data, with the latter showing lower rates of HIV testing. Of the 11,575 KPs reached by SAHACOM in the 12 months to September 2013, only 5,135 28 or 44 per cent had an HIV test in that period.14 Similarly, in the six months to March 2014, of the 8,916 KPs reached by SAHACOM, only 2,680 or 30 per cent had a rapid (finger prick) HIV test.15 Possible explanations for the lesser number of HIV tests in 2014 are a shortage of test kits for two months, turnover of OWs, and delays in training. The reason for the difference in HIV testing rates between the mid-point and end-line surveys and SAHACOM’s monitoring data are not readily apparent but may include KPs over-reporting the rate of HIV testing by giving socially acceptable answers; and/or possible weaknesses in SAHACOM’s performance data in relation to tracking the number of clients reached who have an HIV test. It should, however, be noted that the mid-term and end-line surveys only sampled KPs in a limited number of sites, whereas the KHANA monitoring data is meant to cover all members of KPs in contact with the project. Hence, the monitoring data may give a more reliable picture. Alternatively, the higher rates of self-reported HIV testing in the surveys may be because some members of KPs initiate HIV testing independently of SAHACOM. It is, however, not known whether any these explanations are valid. There is evidence to demonstrate that EWs receiving HIV education are more likely to participate in HCT. The end-line survey found that only 44 per cent of EW who had not received any HIV education in the past year reported getting tested for HIV in the last six months. In comparison 71 per cent of EW who received HIV education had been tested in the past six months. Enabling environment: From the inception of SAHACOM, the environment for reaching KPs has been compromised by the 2010 Village/Sangkat and Commune Safety Policy and the 2008 Law on the Suppression of Human Trafficking and Sexual Exploitation. These policies have created a hostile environment for implementing programs that work with sex workers and PWID. As a result of the policy environment there has been a shift from brothel-based sex work to venue based entertainment work, where commercial sex transactions are initiated in karaoke bars and beer gardens, and other entertainment establishments as well as in guesthouses. Despite police attention, sex work in urban areas continues to take place in the street and parks. However, SAHACOM OWs report being afraid to seek out sex workers in these places for fear of being harassed by police. Also some IPs do not ask their OWs to work evening hours when EWs are most likely to be at work, for their own safety. It should be noted that other HIV implementers, including PSK/PSI and Flagship IPs and other non￾USAID-funded implementers specializing in EW/SW have been successful in accessing street-based/park-based and entertainment venue – based EW during the same time period as SAHACOM’s project period. 5.2.2 Innovative models Adoption of Flagship innovations: For example, SmartGirl was adopted by KHANA as the technical model for EW interventions across all programs, including SAHACOM. Innovative models that increased HIV/FP integration for EW: SAHACOM has taken a number of steps to increase access to FP and SRH services for EWs. IPs were supported to provide EWs with financial support (transport fees) and promote referrals to SRH services. The number of EW and PLHIV are shown in Annex 2. 14 KHANA, SAHACOM Annual Progress Report to USAID/Cambodia, October 2012 - September 2013. 15 KHANA, SAHACOM Semi Annual Progress Report to USAID/Cambodia, October 2013 - March 2014. 29 EWs, MSM and former drug users have been mobilized to reach out to their peers to spread knowledge and awareness of HIV, and mobilize individuals and groups to access health and support services. A key challenge is to build relationships of trust, so KPs feel safe contacting and using services without fear of stigma or discrimination. The communication observed by the evaluators appeared to be open and unstilted, and good relationships among peer OWs and beneficiaries were apparent. Peer provided rapid HIV testing: SAHACOM has participated in the roll-out of peer provided rapid HIV testing for KPs (by finger prick). Key findings from an assessment of this rapid testing program were 1) for large numbers of people in KPs in Cambodia, it is acceptable to be counseled and tested by a peer; 2) testing appeared to be voluntary; 3) significant problems were found in relation to confidentiality (although some of these have been addressed); and 4) there was some evidence of counseling and testing leading to an increased capacity to stay HIV negative.16 The lack of privacy and confidentiality related to community based finger prick testing remains an issue. For example, even when results are given in a small room at an EW venue, OWs note that “their face shows the result”. In addition, the evaluation team heard reports from OWs that people from KPs who already know they are HIV-positive feel the need to have a rapid test if their peers are there because not to test could be interpreted by their peers that they already know they are HIV-positive. Some OWs reported that now that they know who their HIV positive clients are, in this type of situation, they take the client into the room as though they are going to conduct a test, but do not conduct the test. The lack of privacy led to fingerpick testing being not well accepted by PWID when initially introduced. Korsang, however, have introduced an innovative approach where they invite PWID to sit and talk in the curtained outreach tuk tuk.17 Pre-test counselling is provided after the vehicle has moved away from peers and on lookers. If PWID give informed consent for testing they are finger prick tested, and counselled about how to stay negative. If the PWID screens positive, they are already moving towards a facility where a confirmatory test can be done. At this initial visit another appointment is made to get a CD4 count test and treatment for opportunistic infections is provided. The rate of confirmatory testing for PWID is much higher than for other KPs. 16 Jan W de Lind van Wijngaarden, Peer-provided HIV testing and counseling in Cambodia. Preliminary results of a qualitative documentation study. 30 May, 2014. 17 Small cart with seating drawn by a motorbike. 30 A major concern is the reasonably high number of cases identified as newly-HIV positive by finger prick testing who were lost to follow-up. Twenty out of 74 newly-identified HIV cases in 2013 and the first quarter of 2014 did not go to a VCT clinic for confirmatory testing, which means these cases were lost and not linked to clinical services for ART assessment. This can be attributed to the delay in implementing a case management system that was meant to operate in collaboration with peer provided rapid HIV testing. HIV reactive cases were meant to be immediately referred to a care manager who would be responsible for ensuring an HIV confirmation test and linking confirmed cases into care and treatment. Innovative models that increased effective referrals to OI/ART as measured by increased initiation and retention in pre-ART and ART As discussed in Component 1, there is widespread access to treatment in Cambodia, and treatment uptake rates are the highest in the region. IPs working with KPs have broadened their prevention programming by taking on the role of HIV case detection through promotion of demand for HIV testing as an entry point to care and treatment, as part of the national Boosted Continuum of Prevention to Care and Treatment SOP. One Provincial Health Department chief said: “Before, the NGO only focused on prevention education, just talking, talking, but now they are responsible for finding the new cases. When they find a new case they refer them to us, and they work closely with us to get the person into treatment. They must follow up, and they even look for people with opportunistic infections, and they bring them to us, as well as support them to get on treatment and come for their appointments.” Referral to treatment following an HIV positive test result appears to be highly effective and efficient, even for HIV-positive PWID. Of the 5,135 SAHACOM KPs having an HIV test in 2012-13, there were 86 confirmed positive cases. Of these, 42 individuals subsequently commenced ART and 43 were enrolled in pre-ART care (one positive case moved away), equating to 99 per cent successful referral rate. OWs and CSVs have promoted understanding of the benefits of early treatment initiation, the efficacy of ART in reducing HIV transmission, and the importance of adherence. Although this was not tested extensively, during site visits KPs were able to articulate the benefits of early initiation of treatment and also discuss the implications of treatment interruptions, indicating that these SAHACOM efforts have been successful. 5.2.3 Effectiveness of interventions Over the life of the SAHACOM program, data from the baseline and the mid-term and end￾line surveys show an overall improvement in adoption of safe behaviors for MSM and PWID, although this is not the case for EWs. • For MSM, consistent condom use in the past three months with their regular partners increased from 27 per cent at baseline to 64 per cent at mid-term, and had plateaued at 63 per cent by end-line. • For PWID, consistent condom use with their regular partners in the past three months increased from 30 per cent at baseline to 32 per cent at mid-term and to 50 per cent at end￾line • For PWID, needle sharing in the previous three months fell from 63 per cent at mid-term to 25 per cent at end-line. 31 Despite SAHACOM efforts, there has been a declining use of condoms among EWs. KHANA data indicates that consistent condom use in the past three months among EWs with commercial partners decreased steadily from 89 per cent at baseline to 85 per cent at mid-term, to 81 per cent at end-line. Consistent condom use with regular partners remained steady at around 34 per cent. Despite the reduction in condom use, the percentage of EWs that reported experiencing STI symptoms in the past three months dropped from 39 to 22 per cent from baseline to end-line. The reasons for the decline in safe behaviors by EWs is not apparent and requires further investigation. 5.3 Conclusions The SAHACOM project has been successful in helping Cambodia attain good HIV prevention coverage for MSM, TGs and EWs in three high HIV burden provinces and Phnom Penh. However, efforts have been less successful in reaching PWID, although risk activities among PWID have declined. ‘Specialist’ IPs have been more successful in reaching key populations with HIV counselling and testing than facility based services and non-specialist IPs. Peer OWs are reaching and responding to the specific needs of groups that are often marginalized. Implementing through the right partners facilitates the discussion of issues and practices that may be considered too sensitive or stigmatized to discuss with people not regarded as peers. The HIV prevalence among KPs participating in the peer provided rapid HIV testing initiative was only 0.5 per cent, which is well below HIV prevalence in the most recent KP surveys. However, the HIV prevalence from rapid testing may be lower than is actually the case as ‘many’ HIV negative people were tested “over and over again”.18 The number of tests performed was counted, rather than the number of individuals tested. Nonetheless, there is other evidence of low HIV prevalence among KPs recently tested. The overall HIV prevalence for SAHACOM KP members tested in 2012-13 was only 1.7 per cent, although prevalence varied significantly among KPs: 0.5 per cent for EWs; 0.8 per cent for MSM; and 72 per cent for PWID. Possible explanations for the low prevalence among certain of the SAHACOM KP focus groups may be: 1) HIV prevalence among KPs being HIV tested through rapid and health facility based testing would be lower than HIV prevalence in IBBS surveys as those who already know they HIV positive would not be seeking repeat testing; 2) prevention programming and TasP has been effective and there are now fewer new and undetected HIV cases than was thought to be the case; and/or 3) the KPs being tested are at lower risk, and harder- to-reach members of KP populations who are at higher risk are not being reached and/or tested. There is insufficient evidence to come to any conclusion on these points. However, it can be concluded that the current outreach model appears to have exhausted its usefulness for finding new KPs and significant numbers of new/undetected HIV cases. The model needs to be refocused on reaching currently un-served and hard to reach KPs who may be at higher risk and underserved groups such as freelance and non-venue-based sex workers, PWID, including female PWID, and those with multiple or overlapping risk factors. This type of outreach is more resource intensive. Although fewer OW may be needed, the right peers 18 Jan W de Lind van Wijngaarden, Peer-provided HIV testing and counseling in Cambodia. Preliminary results of a qualitative documentation study. 30 May, 2014. 32 will be critical to increasing effective reach for most at risk KPs, as would be an adjustment in their working hours and approaches used to reach target populations. Consideration should be given to upgrading these roles to better supported and remunerated staff positions. A more enabling environment is needed to facilitate improved reach to non-venue based sex workers and PWID. End-line survey results indicate that BCC efforts to promote condom usage with EWs are not sufficiently effective. This may because the interventions themselves are problematic, or because interventions are not reaching the EWs most frequently engaged in sex work. During site visits, the evaluation team did not observe any innovative, dynamic, interactive community-based activities or approaches to outreach, interpersonal communications, and BCC. Risk assessment is primarily focused on the number of commercial partners EWs have, rather than on assessment of risk behaviour, including inconsistent or low levels of condom use. Better targeting may be achieved by screening for risk factors and for other variables that lead to increased risk. Field-proven, evidence based interventions exist which are not being implemented under SAHACOM. These include: • Access to alcohol and other drug treatment:- International evidence shows that using drugs and alcohol is associated with increased sexual risk-taking.19 • Advocacy on human rights issues and legal and policy barriers faced by EWs, MSM, TGs and PWID, particularly those that affect access to HIV prevention, testing and treatment and social protection. 5.4 Future directions for focused prevention Future programming therefore needs to target those KPs most at risk in priority locations, address overlapping high risk behaviors and continue to successfully reduce stigma faced at health service sites and in communities. Access to prevention products such as condoms and lubricant and clean injecting equipment needs to be improved. Ongoing analysis of the community based HIV screening data, triangulated with other data sources and coupled with and follow up of all newly diagnosed cases could help determine risk profiles and if there are few new cases to find or if IPs are not reaching and testing those at highest risk. Securing an enabling environment that allows health and social services to reach populations targeted by anti-trafficking, anti-drug, anti-prostitution and community safety laws is critical to ongoing success of community based prevention, care and support. To reduce HIV among drug users, Cambodia will have to fully address drug related issues such as a functioning opioid substitution therapy service, other elements of harm reduction services, employment opportunities for ex-drug users, and social support for stabilized clients on methadone. 5.5 Recommendations Greater effort and innovation will be needed to continue to prevent new HIV infections in KPs, and this has a number of implications for programming: 19 Stall, R., Purcell, D. (2000). Intertwining epidemics: a review of research on substance use among men who have sex with men and its connection to the AIDS epidemic. AIDS and Behavior, 4(2), 181–92. 33 1. Maintain ‘specialist ‘community based partners but re-think the traditional ‘peer outreach’ approach: it may be time for new peers, new BCC materials, and more engaging approaches, fewer OWs providing services in times and places where risk behavior is occurring but with enhanced training and professionalization of their roles, greater remuneration and realistic caseloads. 2. Influence the hostile policy and legal environment and address human and civil rights abuses of KPs to enable more success in reaching hidden populations and enabling access to HIV and other health services and improved health seeking behavior 3. Continue to utilize mechanisms such as contact tracing, analyzing GIS data for ‘outbreak surveillance’, analysis of behavioral and service uptake data to increase KPs coverage, reduce new HIV infections and risk behavior and follow up all new cases. 6. Capacity development 6.1 Outline of SAHACOM’s capacity development work The expected result for Component 3 is strengthened capacity and leadership of NGOs/CBOs and communities (especially those representing KPs and PLHIV) leads to their meaningful participation in delivering quality and sustainable community-based HIV prevention and care services within the national response. The intermediate results are increased technical and organizational capacity of community-based service providers and NGOs, incorporating best practice approaches; and increased involvement of PLHIV and KPs in program design and advocacy for an enabling environment. These results reflect the vision that KHANA would develop as a technical leader and national resource on CHBC and focused prevention. SAHACOM has placed a strong emphasis on capacity development. This has included supporting increased participation of PLHIV and KP networks in policy and programming; increasing institutional capacity and good governance among local NGOs to support high quality, innovative, targeted and effective programming; and empowering SHGs and strengthening their sustainability. 6.2 Findings and key results 6.2.1 Capacity building of NGOs/CBOs and communities Over the five years, the following capacity building results were achieved against life of project targets: • Number of local organizations provided with technical assistance for strategic information was 37 against a target of 27 • Number of individuals trained in strategic information was 159 against a target of 54 • Number of local organizations provided with technical assistance for HIV-related institutional capacity building was 32 against a target of 27 • Number of Community Support Officers (CSOs), CSVs and PEs/PFs/OWs who successfully completed an in-service training program was 6,138 in 2010, 5,436 in 2011, 6,636 in 2012, and 1,685 in 2013 against a life of project target of 3,114. (These figures represent persons being counted multiple times because each was trained more than one time over the five year life of SAHACOM. Training of new recruits was also needed to fill vacancies through turnover.) KHANA provided numerous capacity building opportunities for IP staff and community workers that included training in leadership, management, monitoring and evaluation (M&E), the CHBC package (basic health treatment, psychosocial support, general hygiene, counseling, referral to health services, OIs, HIV treatments, and adherence and retention), and the focused 34 prevention package (counseling, life skills, negotiation skills, SRH, STIs and HIV transmission, prevention and support). KHANA worked to strengthen the capacity of IPs, CSVs and peer facilitators (PF) and peer educators (PE) through trainings, workshops, exchange visits and on-site mentoring in programming and M&E. IP and CPN+ staff received training in financial management. The majority of training was provided in the first two years and in subsequent years KHANA added training on new topics, such as integration of SRH/FP, TB and livelihoods. (see Table 9 in Annex 8 for a list of training and capacity building activities completed.) Capacity to implement the NCHADS SOPs and SAHACOM’s CHBC model was evident throughout site visits by the evaluation team. IP staff from directors to project managers to CSVs were all able to articulate the model and structure, including the roles and responsibilities of CSVs, SHG membership and meetings, and use of the flipcharts to review health promotion information for PLHIV. What was less evident was the ability of IP staff to innovate to achieve results and for CSVs to have more in-depth discussion on topics beyond the flipchart. Though IP staff understood some factors that put people at risk for HIV, for example, people with TB, people who use drugs, clients of EWs, they did not have an action plan for finding new cases/infections. Nevertheless, IP staff learned and followed the SOPs. For example, in line with the SOP for the Linked Response, IPs in rural areas tried to identify pregnant women in their community and refer them for VCT, though some IPs were only referring HIV positive pregnant women and others were trying to reach out to all pregnant women in their community and refer them. The latter approach yielded very few if any new HIV cases in rural areas. KHANA provided technical support and regular field support visits in areas of financial management and program implementation. The M&E team provided intensive coaching on database management, monitoring, and reporting to improve capacity and refresh knowledge and skills in quality data collection and data analysis. In 2009 KHANA conducted a baseline capacity assessment of SAHACOM IPs. In December 2013, KHANA conducted a further, more intensive NGO capacity assessment of 32 local organizations, including 17 SAHACOM IPs. A secondary technical assessment was done on 13 IPs using a technical checklist. The results of these assessments are set out in Tables 10 and 11 and Figure 4 in Annex 8. In the 2013 assessment, SAHACOM employed a standardized tool called the Purple-O-Meter to help organizations systematically measure their organizational capacity against 13 indicators and 91 criteria in the area of: 1) Partnerships, referral systems, coordination, communication and advocacy; 2) HIV/AIDS, TB, SRH, FP, impact mitigation and technical capacity; 3) Organizational systems and policies; and 4) Promotion of participation of PLHIV, KPs and other affected communities. This was a three-step process which includes self-assessment, field review and certification. The Purple-O-Meter is more stringent and comprehensive than the tool used to assess IP capacity in 2009. The results are therefore not comparable. Also, it is not uncommon for organizations to see themselves as strong at a baseline and as they learn about organizational development, come to the realization that they are not as strong as they initially thought. The 2013 assessment, when compared to the baseline, indicated that SAHACOM built the capacity of IPs in the areas of M&E and in access to technical resources and knowledge. There was little change in organizational governance, strategy and structure, which is likely to be due to the fact that many of the SAHACOM IPs were fairly strong in these areas prior to the project. In the 2013 survey, out of a possible score of six, only two SAHACOM IPs 35 received a score of five (exemplary) and were certified by KHANA. Twelve SAHACOM implementing partners received a score of three (satisfactory) and three SAHACOM IPs received a low score of two (poor). For those that received a score of two, KHANA performed a field review to further investigate the organization’s capacity. Despite the low scores of the three IPs, the findings of these field reviews were used as the basis for signing new partnership agreements and grant agreements for each IP. The (not very detailed) explanation of why KHANA decided to renew funding for these IPs was not, in the view of the evaluation team, sufficient to override the poor scores of the IPs. To renew funding for low scoring IPs, KHANA should have an objective explanation, based on field observations, that indicates that the score was not representative of the IPs actual capacity. Given the considerable efforts and organizational strengthening by KHANA, it is a concern that some IPs had low scores in the 2013 assessment. Nonetheless, through the use of the Purple-O-Meter and other tools, over the five years, KHANA reduced the number of IPs from 30 to 20, eliminating some who were poor performers and transitioning other good performers to the Global Fund. Challenges that SAHACOM faced in capacity building included high turnover of CSVs and CSOs due to the low remuneration provided. In one IP, an entire new group of CSOs (young, well educated, but inexperienced) were recently hired when the organization’s experienced CSOs left for better paying opportunities. This results in a significant loss of institutional memory and expertise. SAHACOM provided intensive institutional strengthening support to revitalize CPN+ to become an active and sustainable network of PLHIV. Support included strengthening governance and institutional bylaws, policies and administrative SOPs, short-term secondment of a National Coordinator and Finance Manager, support for CPN+ offices in ART/OI centers, along with financial support for program implementation and operations. As a result of SAHACOM’s capacity building efforts with CPN+, there is now stronger leadership in CPN+. The co-location of CPN+ provincial offices in ART clinics has strengthened the involvement of PLHIV in case management and quality assurance. This clinical focus may, however, detract from CPN+’s advocacy work. 6.2.2 Capacity of KHANA as a technical assistance provider There is significant evidence of KHANA being an effective technical assistance provider at national and local levels. KHANA is recognized and respected by partners and stakeholders for its extensive program implementation experience and technical expertise. This is particularly evident by the long list of technical work done by KHANA in support of government ministries and departments responsible for HIV, particularly NCHADS, and for OVC, particularly the Ministry of Social Affairs, Veterans and Youth and Rehabilitation (MoSVY). KHANA has been involved in developing and revising strategies, key HIV prevention, care and treatment SOPs, guidelines, training material and tools. The SAHACOM model of community-based support became the basis of the working/draft version of the SOP for CBPCS for PLHIV, Affected Families, and Pregnant Women. Recently, KHANA has taken on consultancies on a fee for service basis and has been invited to participate in assessments such as the HIV/FP integration assessment conducted by Marie Stopes International. KHANA has also developed capacity in research and over the last few years conducted numerous research studies and demonstration projects on livelihoods, sero￾discordant couples and harm reduction. (See Table 9 in Annex 8 for a list of some of the main technical support provided by KHANA). 36 6.3 Conclusions SAHACOM’s continuous capacity building for IP staff and community-based workers in a wide range of areas contributed significantly to achievement of program targets (e.g., high rates of treatment adherence and retention), strong linkages between the community and the health care system and with other organizations, functioning SHGs, improved communication, strong engagement of PLHIV, and improved data quality and reporting. KHANA has emerged as a recognized, effective and respected provider of HIV technical assistance in Cambodia. KHANA’s strong technical capacity has given them a seat at the policy table and they have been able to support the government in developing key policy documents that guide programs for PLHIV and OVC. 6.4 Future directions for capacity building work KHANA can further consolidate the number of IPs and transition only the strongest organizations to reduce overheads and the management and capacity building burden. 6.5 Recommendations 1. Develop more creative, interactive and dynamic approaches to adult learning and organizational capacity building, including materials appropriate to specialist NGOs and Peer Facilitators that are tailored to the distinct needs of the different KPs (i.e., EWs/SWs, MSM, TGs, and PWID). In the area of focused prevention, stronger adaptive skills are called for to effectively: a) identify individuals with undiagnosed HIV infection and link them to care and treatment; b) reach the most at risk and hidden individuals; and c) in facilitate dynamic interpersonal and group behaviour change activities that result in positive behaviour change. 2. Discontinue funding organizations that have poor institutional capacity and performance to mitigate risk and make the most of limited resources. 3. In the last few months of SAHACOM, to increase the sustainability of good performing IPs, conduct proposal writing workshops for the remaining IPs so they can seek additional financial resources. 4. Seek other opportunities whereby KHANA or the IPs could utilize their capacity to implement programs at the community level. 7. Cross cutting areas 7.1 Gender 7.1.1 Overview of KHANA’s approach to gender KHANA has developed and implemented a gender strategy that covers KHANA and all IPs. The strategy recognizes that the gender norms contribute to HIV vulnerability and that the epidemic has a unique impact on men, women and transgender persons. 7.1.2 Findings and key results Equity in access and utilization of services Representatives of IPs were able to articulate how their programming responds to the unique needs of male and female sex workers, EWs, MSM, TGs, and male and female PWID. Additionally, the needs of TGs have been recognized and they are no longer integrated into MSM activities. Specific resources have been developed for TGs under the new “Srey Sros” branded package of HIV prevention interventions and materials tailored to the unique needs of Cambodian TG, following along the lines of the Smart Girl and M Style materials and 37 approaches developed for EWs and MSM. TG OWs have been recruited and are tasked with providing information, support and service linkages for this often marginalized group. Gender equitable strategies are evident in activities. For example, Korsang, the IP working with PWID, have added female OWs to their previously all-male team. Korsang reports that while male OWs were able to reach female PWID, they found it difficult to discuss issues effectively, and were not confidant that the package of prevention and care was being delivered appropriately. Although reach has remained relatively stable, and PWID continue to be difficult to link to health services, Korsang reports that female OWs seem better able to link female PWID to HIV, STI and TB testing and treatment services. The number of clients reached by SAHACOM is disaggregated by male, female and TG. However the sex breakdown is not usually included in either reports from IPs or in KHANA’s reports to USAID. Addressing gender norms The SAHACOM program has made sound efforts to address gender norms. For example, workshops and training sessions were provided to raise awareness of gender based violence and to equip staff and volunteers to raise this issue and respond accordingly. This was in response to the mid-term survey findings that 21 EWs reported that a recent client had expressed the wish not to use a condom, 59 per cent of whom said the client did this by offering more money, while almost 30 per cent reported the client threatening them either verbally or with a weapon. 7.1.3 Conclusions The evaluation team did not see any evidence of women being excluded from HIV services and community based support. Program data indicates high levels of female involvement in care and support services for PLHIV and OVC, as well as participation in livelihoods and economic strengthening activities. Female PLHIV have equitable access to HIV treatment. With the exception of female non-venue based sex workers and PWID, there is equitable access to services and gender responsive programming. Although KHANA has substantially increased KP, PLHIV and community involvement in the program, it is not possible to determine whether men, women and transgender persons have been equitably supported to take up paid positions within the program. Although the evaluation timeline did not allow for an in-depth investigation of BCC, it seems not enough emphasis is placed on the social and transactional contexts of sexuality and the way in which gender relations and power norms within sexual relations might serve as an obstacle to condom use. This is particularly relevant to EWs and their ‘sweethearts’. Thirty seven per cent of EWs reported that they had sexual intercourse with a sweetheart in the past three months. Only 31 per cent self-reported always using a condom. BCC addressing cultural norms and habits related to condom use by EW with ‘sweethearts’, appears to be ineffective as condom use with ‘sweethearts’ has remained at around this low level for a number of years. This deserves further investigation to determine correlations among those with new infections, inconsistent condom use and multiple partners and what might constitute an effective intervention. Key conclusions in relation to gender are that: • The KHANA Gender Strategy is being implemented and SAHACOM has demonstrated good gender responsiveness • Specific strategies exist for KPs including EWs, MSM, TGs, and male and female PWID 38 • Female and male PLHIV have equitable access to HIV services, however it is not clear whether these same opportunities are available for TG • HIV prevention interventions include information about and referral to FP and SRH services for PLHIV, EWs and female PWID • Sex workers and female PWID have been pushed underground as a result of government policies and laws. HIV programming has not adequately focused on reaching and responding to the needs of these underserved women. 7.1.4 Future directions and recommendations Continued capacity building, data analysis and technical assistance on gender for IPs is needed. This should include developing understandings of how to best identify and reach hidden or underserved KPs and those most at risk; how to reduce their risk behavior and effectively link them to the COPCT. 7.2 Multisectoral partnerships 7.2.1 Overview of KHANA’s approach to multisectoral partnerships KHANA has built relationships and closely collaborated with relevant stakeholders including CBOs, local and international NGOs, and government agencies including the National AIDS Authority (NAA), NCHADS, MoSVY, the Ministry of the Interior’s Department of AIDS, and the National Authority to Combat Drugs (NACD). KHANA is an active member of several national technical working groups (TWGs), including the TWGs for MSM, Standard Operating Procedures for the Continuum of Prevention to Care and Treatment, Drugs and HIV/AIDS, Home Based Care, Reproductive, Maternal and Child Health, and OVC. Through meetings and collaboration with government and donor agencies and international organizations, KHANA has built strategic alliances and provided technical inputs for programming and planning, while also influencing long-term change in policy and practice. 7.2.2 Findings and key results KHANA and implementing partners conduct regular coordination meetings with government, local authorities, law enforcement agencies and entertainment establishment owners to build an enabling environment for community based HIV prevention, care and treatment programming. KHANA and its IPs have closely linked their activities with relevant stakeholders at national and provincial levels, including HIV/AIDS Coordinating Committee, NAA, NCHADS, the National Center for TB and Leprosy Control, NACD, and the National Maternal and Child Health Center. Through USAID education funding, school attendance of OVC was secured and the quality of learning and collaboration with public schools was improved. A total of 2,646 OVC were supported for schooling from project commencement. KHANA provided technical support to MoSVY’s National OVC Task Force in the development of SOPs and guidelines for OVC and was awarded the Mohasena and Samrith Medals by the Minister of Social Affairs, Veterans and Youth Rehabilitation. KHANA received funding from Australian Aid to support IPs to roll out needle and syringe distribution in two provinces: Kampong Speu and Steung Treng. KHANA, under the Asia Action for Harm Reduction initiative, signed a MoU with the AIDS Department of the Ministry of the Interior to implement a policy and advocacy project on harm reduction utilizing a Police Community Partnership Initiative, with support from the European Commission and the International HIV/AIDS Alliance. 39 Through collaboration with the World Food Program, KHANA provided 6,162 PLHIV households and 3,232 OVC households with 2,848 metric tons of food support from October 2010 to September 2011. Partnerships in the area of livelihoods and economic strengthening are outlined in Section 4. No evidence of partnerships with the private sector was found, although this was part of SAHACOM’s design. 7.2.2 Conclusions KHANA has collaborated with a wide range of multisector partners and leveraged additional resources that enhanced community-based support for PLHIV and OVC and focused prevention among KPs. Associations and alliances with both government agencies and donors have enabled SAHACOM to influence national policy and programs focused on HIV prevention and expand non-HIV community based support for PLHIV, OVC and KPs. However, the absence of partnerships with private sector means the opportunity to leverage additional market-based economic strengthening and livelihood opportunities for PLHIV, OVC, and KPs, was not realized. Although USAID has democracy, rights and governance activities that focus on human rights, including legal aid, LGBT rights, and engaging civil society, there were no apparent links established with SAHACOM partners such as CPN+ or the specialist IPs working with MSM, TGs, EWs and PWID. This was another missed opportunity. 7.2.3 Future directions and recommendations 1. In urban areas such as Phnom Penh and Siem Reap, establish links with handicraft and other industries that can provide market driven economic opportunities, such as the newly established Ministry of Industries and Handicraft and groups such as Angkor Artisans. 2. Create opportunities for the stronger SAHACOM IPs to connect with USAID Democracy, Rights and Governance, Education20 and Economic Growth programs. Develop and disseminate organizational profiles for the SAHACOM IPs describing the multisector capabilities of the IPs to funding agencies in non-health sectors. 7.3 Sustainability 7.3.1 Overview SAHACOM was designed to develop and scale up a self-help approach to community support for PLHIV and OVC. The key principles of SAHACOM emphasize community leadership and participation, adaptability and innovation, and sustainable and cost effective models to reduce reliance on external support by increasing individual social capital and community responsibility. The model also places emphasis on development of partnerships and collaboration to promote synergy and maximize use of resources and linkages with integrated health and non-health services. The SAHACOM model set out to demonstrate how to empower and create community ownership through PLHIV CSVs, SHGs for PLHIV and OVCSGs to implement CHBC and focused prevention implemented for KPs by PFs and PEs. 20 Potential linkages with USAID’s education program could be in the following areas: 1) focus on keeping girls and boys who are vulnerable to HIV in school; 2) SAHACOM review of life skills curricula related to STI/HIV prevention, sexuality, gender identity, and use of drugs; and 3) SAHACOM identification of NGOs that could be strong education partners. 40 7.3.2 Findings SAHACOM was built on predecessor home based care models with deep roots in communities and long experience in providing support for PLHIV. This was evident in the high number of PLHIV reached by IPs in year one of SAHACOM. During SAHACOM, the focus was on scaling up the SHG model, ensuring more effective and self-sustaining involvement of PLHIV, building capacity of CSVs in planning, budgeting and reporting, ensuring poor PLHIV had access ART/OI clinics, and economic strengthening of households through livelihoods training, loans, and VSL groups. SAHACOM has done a good job of building capacity of IP staff, CSOs and CSVs in implementing the community-based model. NCHADS considers CHBC essential for the successful operation of the continuum of care in recognition of its important role in facilitating access to care and promotion of treatment retention and adherence. The commitment of PLHIV to maintaining SHGs appears to be greater for those groups that have established VSL groups as the savings and loans activities appear to be highly valued. These groups are, therefore, likely to be more sustainable than SHGs without VSL groups. However, even with a strong commitment by PLHIV to maintaining these groups, the question remains what is the minimum level of support needed from CSVs and IPs to keep groups functional. The IP component of the SAHACOM model, which is undertaken through NGOs, is quite expensive, especially the costs going to the IPs for project management, oversight and capacity building. CSVs are the backbone of the SAHACOM model. It is clear that CSVs provide a link and functional referral system between the community and the health facilities, in particular the ART/OI centers. If a person missed their appointment at the OI/ART center, the clinic staff would contact either the IP or a designated CSV of that area and ask for assistance in locating that person. Apart from CSVs, all other case management systems are facility based. These systems rely on the capacity of CSVs to engage with PLHIV in the community in facilitating access to care and retention and adherence. It is essential that the community based comparative advantage of CSVs continues to be recognized and supported. Sole reliance on facility based case management would not be as effective. There are continued challenges for sustaining community-based support programs for KPs. Most KP work is urban based and the SAHACOM CHBC model is more of a rural based model. Hostile policies such as the Village and Commune Safety Policy and the Anti￾Trafficking and Prostitution Policy that drive sex workers and PWID underground. Continued stigma, discrimination at health centers create barriers to care and treatment services. KHANA submitted a transition plan for SAHACOM to USAID in October 2013 that heavily relied on a transition to Global Fund for its entire community work for PLHIV, for its prevention with key populations in priority areas, and for a reduced package of care and support for OVC. Because of PEPFAR budget reductions, in 2013 KHANA transitioned some IP from SAHACOM to Global Fund Phase II earlier than originally planned. More recently, Cambodia has been informed that it will have fifty per cent less Global Fund resources for HIV than was anticipated. This funding crisis is currently being addressed. Decisions are being made to determine what components and subcomponents of the HIV response, including the community support and focused prevention models, will need to be discontinued or streamlined. A new Global Fund Concept Note for HIV and AIDS will be developed and submitted in October 2014. Key stakeholders are hoping that the findings of this SAHACOM evaluation would inform such decisions. 41 Recommended future directions for programming are set out in Sections 3, 4, 5, and 6. Although costing of the CHBC model has not been undertaken as it was not within the scope of this evaluation, a costing study of the CHBC model will be conducted in 2014 by URC’s HIV Innovate and Evaluate project with funding from USAID. This will provide much needed costing data which can be used to examine how the CHBC model can be modified or improved to reduce costs. 7.3.3 Conclusions “The celebrated MDG gains in the health sector are fragile, and greater country responsibility for funding and leadership is needed to strengthen and sustain health systems.”21 Financial resources for community-based care for PLHIV and OVC are quite vulnerable in the context of reduced donor funding. NCHADS, KHANA, USAID and other stakeholders need to develop a more affordable, streamlined model that is targeted to priority needs. The most expensive components of the SAHACOM model are the IP operational costs, stipends for CSVs, CSOs, and PFs, livelihood programming and referral/transport fees for beneficiaries. The CSVs and PFs are the backbone of the community-based model and perform critical tasks such as identifying potential new cases, ensuring PLHIV access OI/ART services, supporting ARV adherence, and tracing those who miss appointments or are lost to follow-up, and should be continued. Key questions are how much technical support and supervision is required for these positions and whether the model can still be effective with fewer IPs per OD/Province and fewer CSVs who cover more SHGs and a larger catchment area. One option to consider is a professionalized, community based case management approach by PLHIV (possibly utilizing the best of the CSVs) to improve follow-up from referrals and to facilitate access to care and treatment and retention and adherence. This position, while based in the community would work closely the MMM Coordinators and Active Case Manager. HEF provides an alternative way to pay beneficiaries transport and healthcare fees without the IP management costs. 7.3.4 Future directions and recommendations 1. USAID should advocate for the expansion of HEF coverage to all HIV/AIDS services as a cost-effective way to ensure poor PLHIV have access to ART/OI services, including transport costs. 2. KHANA should pilot a modified version of the current SAHACOM model for community-based support of PLHIV and OVC with fewer IPs and CSVs covering larger catchment areas and holding SHG meetings less frequently. CSV roles and responsibilities should be streamlined and focused on key tasks such as supporting PLHIV newly initiated on ART, supporting ARV adherence, and referring for ART/OI and linked services, and tracing those who miss appointments or are lost to follow-up, and support for VSL groups. 3. KHANA should work closely with NCHADS in the development of a new SOP for community based support for PLHIV. 21 USAID Country Development Cooperation Strategy 2014-2018, November 2013. 42 4. As the URC costing study will provide important information for the Global Fund Concept Note for HIV, USAID should consider this activity as a priority. It would also be useful for the protocol of the study to be reviewed by USAID headquarters health economists working on PEPFAR. 5. Focused prevention activities will need to be continued through the funding of specialized IPs with unique capacity to work with EWs, MSM, TGs and PWID, although the focus of work may need to shift to targeting of hard to reach/currently unreached members of KPs who are at highest risk. 6. The PEPFAR interagency team needs to focus on leveraging government commitment to increasing domestic resources for the HIV response in light of declining donor funding. USAID may consider providing technical assistance to the government in generating resources. 8. USAID/Cambodia management of SAHACOM SAHACOM took some time to transition from the HBC model to the community support model based on SHGs led by and for PLHIV. In the first two years, KHANA had to focus on building the capacity of IPs in this new approach. This required that the USAID Agreement Officer’s Representative (AOR) spend time with KHANA to help accelerate this shift. USAID/Cambodia’s Office of Public Health and Education has limited staff. As a result, staff are over committed with multiple roles and responsibilities. The AOR for SAHACOM held monthly meetings with KHANA to review progress against work plans and to discuss technical issues and challenges, burn rate and pipeline. The AOR is also the Agreement Officer’s Representative for the Flagship Project awarded to a consortium led by KHANA as the prime. After several quarters of separate meetings with KHANA on each project, it was decided to combine the meetings on a quarterly basis. This has helped to reduce the management burden for the AOR and KHANA management and provides an opportunity for discussing collaboration and synergies between the two projects. In addition to these meetings, the AOR conducts field visits to meet with beneficiaries and IPs on a quarterly basis. This leaves little time for engaging with NCHADS and other stakeholders and attending TWG meetings. 8.1 Recommendations 1. The Global Fund liaison officer should be included on Technical Evaluation Committees for health procurements on HIV, TB and malaria to facilitate synergistic technical support and co-funding with Global Fund. 2. USAID/Cambodia should ensure that it has inputs to decisions on SAHACOM transition planning and the future shape of technical models for community based support for PLHIV and OVC and focused prevention for KPs, which will form the basis of the Cambodia’s GF HIV Concept Note. 3. USAID should encourage URC to give high priority to the timely completion of the CHBC costing study. 4. USAID should encourage exploration of the expansion of the Health Equity Fund to all SAHACOM/Global Fund community-based support locations. 5. USAID should consider requesting the Flagship Project to undertake a study to compare different community-based support models, such as an urban model for key populations and a streamlined rural model for general PLHIV populations. 43 Annex 1: Scope of Work This annex contains extracts of the key elements of the scope of work for this evaluation. Background and program description information is in Sections 1 and 2 of this report. Purpose of the evaluation The purpose of the evaluation was to: 1. Assess SAHACOM’s performance and the extent to which it was able to meet its intended objectives at all result levels. 2. Document lessons learned and best practices as well as make recommendations to inform and improve future program directions and effectiveness. Evaluation questions 1) To what extent did the project achieve its objectives and expected results? 1.1. To what extent did the CHBC services improve coverage, quality, and sustainability? (50% of LOE) • Did the CHBC achieve the coverage of a package of care services for PLHIV and OVC? • Was the project successful in linking communities with public health services (ART and pre ART, Family Planning, PMTCT and other clinical services)? Link community with non-health services? • To what extent did the CHBC services improve retention and adherence in the Continuum of Care (CoC)? • What is the likelihood/feasibility of Self-help group model being self￾sustaining? • To what extent were the various livelihood models successful in generating sustainable income and improving access to HIV services and commodities for PLHIV? For example: cash grants, vocational training, village savings/loan schemes. 1.2. To what extent did the different project activities improve uptake of innovative and targeted HIV prevention interventions and services by KPSs, especially by those from currently under-served and neglected groups? Which approaches were most successful? (20% of LOE) • Effectiveness of interventions such as drop-in centers (DICs) for IDUs, MSM, and TG; • Innovative models that increased HIV testing and use of condoms among KPs, as well as lubricant use for MSM, HIV/FP integration for EW, and effective referrals to OI/ART as measured by increased initiation and retention in pre￾ART and ART. 1.3. To what extent did the project strengthen the capacity and leadership of NGOs/CBOs and communities (especially those representing KPs and PLHIV) leading to their meaningful participation in delivering quality and sustainable HIV prevention and care services within the national response? (20% of LOE) • How effective was KHANA’s approach to capacity building of local NGOs and the networks of PLHIV at the National, Provincial and community/self￾help group levels on financial management, program management and technical implementation? 44 • What is the capacity of KHANA to provide TA to the government and capacity building to local organizations? 1.4 How effectively has the project addressed cross-cutting areas? (10% of LOE) • Linkages between community and clinical services as well as with non-HIV support and social protection (e.g. HTC, OI and ART services, TB services, family planning services and contraceptives, health equity funds, livelihood, legal and human rights services) • Gender (equity in access and utilization of services, addressing gender norms) • Poverty (livelihood, skills training) • Partnerships (multisectoral partners such as agricultural NGOs, NCHADS and Provincial Health Department) 2) Were there any unintended consequences or results of the project interventions, e.g. SAHACOM model being taken to scale by NCHADS with funding Global Fund? Or were there any negative consequences of the project interventions? 3) What are key lessons learned? 3.1 Based on recommendations of the mid-term review and the portfolio review, what changes to the SAHACOM program were made? 3.2 Which interventions, based on evidence, should be continued or expanded to improve access to and the quality of CHBC services? • In light of increased coverage of OI/ART services and changing needs of beneficiaries, is there a continued need for a comprehensive package of CHBC? (If so, how should the CHBC be adapted to meet the changing needs of beneficiaries?) • What are the key CHBC services that are recommended to be retained for optimum PLHIV client outcomes? (What services and support should be sustained to improve or maintain health outcomes?) • What cost savings and efficiency gains might be made with a more streamlined CHBC package? (How can we make CHBC further efficient?) 3.3 How might future investments be refocused or reduced? The level of effort (LoE) specified after key evaluation questions refers to the level of effort the evaluation team was required to put into different aspects of the evaluation. It broadly reflects the level of effort for SAHACOM across different program components. As such the evaluation team was asked to primarily focus on the CHBC/community support for PLHIV component of SAHACOM. Not within scope The evaluation was not required to undertake a costing analysis, as this will be done by USAID’s HIV Innovate and Evaluate Project. In addition, the second project objective on prevention with KPs will be explored more fully by the HIV Innovate and Evaluate Project through a study on barriers experienced by Cambodian KPs in accessing prevention, care, and treatment services and other evaluations of new interventions being rolled out. Audiences and intended use for the evaluation report The audiences for this evaluation report will be the USAID/Cambodia Mission, the PEPFAR/Cambodia team, the Asia Bureau, the Global Health Bureau Office of HIV/AIDS, Office of the Global AIDS Coordinator, USAID implementing partners, the MoH/NCHADS, 45 and other HIV/AIDS key stakeholders in Cambodia (especially the GF). USAID will consider the findings, particularly the evidence-based findings, in its strategic approach to HIV/AIDS. It is expected that Cambodian partners, the Global Fund, and other donors will also be able to use the report to assist them in defining their future goals. The findings of this end of project evaluation will also be used to strengthen the interventions of the USAID/Cambodia HIV Flagship Project. 46 Annex 2: Evaluation methodology This evaluation took place over a six-week period in June-July 2014, of which four weeks were spent in-country. The evaluation was designed to be consistent with USAID’s Evaluation Policy (January, 2011). The Scope of Work for this evaluation is consistent with what the Evaluation Policy defines as a ‘performance evaluation’. This type of evaluation focuses on: “descriptive and normative questions: what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management, and operational decision making.”22 Evaluation team The five-person evaluation team was made up of three independent consultants and two USAID/Washington experts from the Office of Global Health, Office of HIV/AIDS.: External consultants: David Lowe (Team Leader) Srey Mony Jenne Roberts USAID/Washington: Marta Levitt Billy Pick This evaluation report was written by the core team members listed above. Collectively, the team had considerable expertise across all relevant aspects of HIV technical knowledge and programming, from prevention to care and treatment, and health systems strengthening. All team members had extensive knowledge of Cambodia’s response to HIV and USAID/Cambodia HIV programming, from previous work. The evaluation team was joined by Mr Panus Na Nakorn from the USAID/Regional Development Mission Asia for its initial planning meeting and consultations with stakeholders in Phnom Penh and the provinces. The evaluation team was also joined by Shivani Murthy and Dr Ly Vannthy from the US Centers for Disease Control/Cambodia for the planning meeting and provincial consultations. Dr Ly Vannthy also participated in some consultations in Phnom Penh and in team meetings to analyse data and develop findings and conclusions. Translation for stakeholder interviews was conducted by Heng Thona and Sreng Sopheap from USAID/Cambodia. Given the relatively large size of the evaluation team, two sub-teams were formed for consultations in Phnom Penh and the provinces. This allowed the evaluation team to maximise the number of stakeholders who could be interviewed. An initial in-depth interview with KHANA senior management and SAHACOM technical staff was conducted by the full evaluation team. 22 USAID, Evaluation Policy. Bureau for Policy, Planning and Learning, January 19, 2011. p. 4. 47 Methodology: key components The key components of the methodology for the evaluation were as follows: 1. Document review Evaluation team members reviewed the following categories of key background documents: Government of Cambodia documents: including the National Strategic Plan for Comprehensive and Multi-sectoral Response to HIV/AIDS II and III (2006-2010 and 2011- 2015); the NSP III costing document; the National Strategic Framework and Operational Plan for MSM; and NCHADS documents, including relevant guidelines and Standard Operating Procedures, documents relating to the Cambodia 3.0 initiative, and Annual Reports. USAID documents: including USAID’s Evaluation Policy, the Cooperative Agreement for the SAHACOM Project; and USAID/Cambodia’s comments on SAHACOM work plans and performance monitoring plans. SAHACOM documents: including annual work plans, progress reports, the M&E plan and performance monitoring data, baseline documentation, the mid-term review, data from the end line assessment, relevant KHANA research reports; and the KHANA strategic plan. A full list of documents reviewed is in Annex 3. 2. Review of performance related data The performance monitoring data for SAHACOM was reviewed to identify key outputs and where possible outcomes. Key focus areas for data review were trends in output data for key activities (e.g. PLHIV coverage, PLHIV receiving care and treatment, prevention outreach, referrals to clinical services, HIV testing rates among key populations, livelihoods coverage, etc.). Key outcome data reviewed included retention in ART, PLHIV deaths, OVC’s attending school, and HIV infections among sero-discordant couples. The performance indicator data was compared to targets. 3. Key informant interviews An extensive range of key informant interviews were conducted to address the focus questions in the SOW. The following categories of key informants were interviewed: • USAID/Cambodia Health Office staff and US CDC • KHANA senior management and SAHACOM technical staff • SAHACOM implementing partners, strategic partners and collaborative partners • Key Cambodian Government agencies at national and provincial levels • Program beneficiaries, volunteers and community networks • Selected multilateral organisations and bilateral development partners All interviews with beneficiaries, volunteers and other stakeholders were conducted without the presence of KHANA staff and their implementing agencies staff to minimize the risk of bias in responses. Interview guides were developed for each category of key informant, based on the evaluation questions in the SOW, to ensure a consistency in approach by the sub-teams conducting interviews. These interview guides are in Annex 4. Given that information relating to many of the evaluation questions was contained in SAHACOM’s progress reports and performance monitoring data, the interview guides focused on seeking information not contained in those sources. The interview guides were not used a rigid list of questions to be asked in sequential order, but more as a checklist to ensure that all key areas were covered. Interviews were 48 conducted in a way that promoted the feeling that stakeholders were being given an opportunity to meaningfully engage in a dialogue with the evaluation team. Following the completion of stakeholder interviews, the evaluation team had a meeting with KHANA management and technical staff to ask follow up questions and discuss issues that had arisen during the consultations with other stakeholders. Collaboration and dialogue with stakeholders was further promoted by their participation in a debriefing meeting conducted by the evaluation team at the end of the field work where feedback on preliminary findings, conclusions and proposed future directions was sought. 4. Analysis The evaluation team undertook ongoing analysis of all data through analysis by individual team members of performance monitoring data and data collected during interviews, team meetings and informal discussions among team members as we traveled throughout Cambodia. This iterative process allowed for emerging issues to be explored and potential findings to be tested as the evaluation progressed. Following the completion of key stakeholder interviews and site visits, the evaluation team conducted a thorough analysis of all data, both qualitative and quantitative, and developed preliminary findings and conclusions in relation to the evaluation questions in the SOW. This analysis formed the basis upon which the evaluation report written. 5. Mid-point de-brief with USAID/Cambodia The evaluation team’s preliminary findings and conclusions were presented to USAID/Cambodia for the purposes of feedback, validation and further input. This debriefing took place in week three of the in-country work so that the Mission’s inputs could be considered in drafting the evaluation report. 6. Report writing The evaluation report was written consistent with criteria to ensure the quality of evaluation reports, as set out in Appendix 1 of USAID’s Evaluation Policy. In particular, emphasis was placed on demonstrating the quantitative and qualitative evidence on which findings are based. A draft table of contents for the evaluation report and writing allocations were agreed upon at the evaluation team’s initial planning meeting at the commencement of in-country work so that team members had a clear understanding of the key deliverable and their inputs. 7. End of evaluation mission debriefings On the last day of the in-country work, the evaluation team presented a summary of key findings, conclusions and proposed future directions and recommendations to three separate meetings. One presentation was made to USAID/Cambodia and US CDC/Cambodia, a discussion was held with KHANA and a further presentation was made to KHANA and SAHACOM’s partners and stakeholders. Feedback was sought at all debriefing meetings to inform revisions to the draft report. 8. Review of draft evaluation report and finalisation Revisions were made to the draft evaluation report in response to feedback received from USAID/Cambodia. Limitations In 2010 SAHACOM collected ‘baseline’ data through a desk based review of existing data, along with field visits and consultations with program staff. This work did not involve a 49 quantitative baseline survey. While the ‘baseline’ documentation gives a reasonable overview of the situation at project commencement, some of the data are not directly comparable with key indicators for SAHACOM and therefore cannot be regarded as comprehensive baseline data. This means that for some indicators, performance data at the end of the first year of implementation is the closest proxy to a baseline. In 2012 and 2014, mid-term and end line surveys of SAHACOM beneficiaries were conducted by KHANA that provide further data for comparison purposes. A limitation of the mid-term and end line survey is that not all population groups were surveyed in each of the survey sites. For example, PLHIV were not surveyed in Phnom Penh. The evaluation team has drawn extensively on KHANA’s SAHACOM monitoring data in developing findings and conclusions. Any weaknesses in the monitoring data may have resulted in the development of invalid findings and conclusions. For example, SAHACOM reports on the number of KP clients who had an HIV test following a referral. The effectiveness of the monitoring system in accurately tracking clients who had an HIV test is not known. It is possible that the numbers tested may be under-reported. The evaluation team did not have time to make an assessment of the strengths and weaknesses of SAHACOM’s monitoring data. Within the available time it was not possible for the evaluation team to collect qualitative data. This, however, is not regarded as a limitation due to the availability of SAHACOM’s performance monitoring data and the mid-point and end-point surveys conducted by KHANA. The evaluation team conducted interviews in Phnom Penh and all the provinces where SAHACOM is working. However, due to the large number of SAHCOM stakeholders, it was not possible to interview all. The number of stakeholders interviewed was maximized by the evaluation team splitting into two sub-teams in Phnom Penh and for provincial visits, and further splitting into additional sub-teams in the provinces. Proportionally, more time was spent interviewing NGOs implementing SAHACOM in the provinces than in Phnom Penh. This could be regarded as over-representation of provincial stakeholders, given estimates that 60 per cent of PLHIV live in Phnom Penh. While there were limited data directly measuring the quality of services, quality was assessed by the extent to which expected results and outcomes were achieved and the level of sophistication by which stakeholders were able to answer questions. Another constraint was language barriers and the need to use translators for interviews, particularly for KHANA’s NGO implementing partners and project beneficiaries. The SOW specified that 50 per cent of the team’s level of effort (LoE) should be expended on evaluating community based support for PLHIV; 20 per cent LoE on prevention programming; 20 per cent LoE on capacity building; and 10 per cent LoE on cross cutting areas. As this broadly reflected the LoE of KHANA and its partners in implementing SAHACOM, this was both appropriate and did not limit the work of the evaluation team in that the LoE for each of these areas was regarded as sufficient. 50 Annex 3: Bibliography de Lind van Wijngaarden JW, Peer-provided HIV testing and counselling for most-at-risk populations in Cambodia: Lessons learned and ways forward. 2014. de Lind van Wijngaarden JW, Peer-provided HIV testing and counselling in Cambodia. Preliminary results of a qualitative documentation study. May 2014. Powerpoint presentation. HIV Innovate and Evaluate Project, Research Protocol. Barriers for Most at Risk Populations to Access Clinical Services at Health Facilities. 2014. KHANA, Analysis in action – A household asset based livelihoods approach. 2010. KHANA, Annual Report 2013 – Advancing technical expertise for better service delivery. 2014. KHANA, Baseline Documentation: Sustainable Action Against HIV and AIDS in Communities (SAHACOM). 2011. KHANA, Briefing: KHANA harm reduction. 2014. Powerpoint presentation. KHANA, Characteristics of Serodiscordant couples in Cambodia. 2014. KHANA, Economic livelihoods program. 2014. Powerpoint presentation. KHANA, Flagship Annual Workplan, October 1 2013 - September 30 2014. KHANA Grant for USAID SAHACOM, October 1 2009 – May 31 2014. KHANA, Mid-Term Review of the Sustainable Action against HIV and AIDS in Communities (SAHACOM) Program. 2013. KHANA, Network household economic livelihoods survey analysis. 2010. KHANA, Onward Granting Manual. 2013. KHANA, Revised KHANA Strategic Plan (Boosted KSP 15) Highlights 2013-2015. 2013. KHANA, SAHACOM Annual Workplan, October 1 2009 - September 30 2010. KHANA, SAHACOM Annual Workplan, October 1 2010 - September 30 2011. KHANA, SAHACOM Annual Workplan, October 1 2011 - September 30 2012. KHANA, SAHACOM Annual Workplan, October 1 2012 - September 30 2013. KHANA, SAHACOM Annual Workplan, October 1 2013 - September 30 2014. KHANA, SAHACOM Annual Progress Report to USAID/Cambodia, October 1 2009 - September 30 2010. KHANA, SAHACOM, Semi-Annual Progress Report to USAID/Cambodia October 1 2010 - March 31 2011. KHANA, SAHACOM, Semi-Annual Progress Report to USAID/Cambodia October 1 2011 - March 31 2012. KHANA, SAHACOM Annual Progress Report to USAID/Cambodia, October 1 2011 - September 30 2012. KHANA, SAHACOM, Semi-Annual Progress Report to USAID/Cambodia October 1 2012 - March 31 2013. 51 KHANA, SAHACOM Annual Progress Report to USAID/Cambodia, October 1 2012 - September 30 2013. KHANA, SAHACOM, Semi-Annual Progress Report to USAID/Cambodia October 1 2013 - March 31 2014. KHANA, SAHACOM – Sustainable Action against HIV and AIDS in Communities. 2014. Powerpoint presentation. KHANA, SAHACOM End-of-Project Review: Key findings. 2014. Powerpoint presentation. KHANA, SAHACOM Monitoring and Evaluation Plan 2009-2014. KHANA, The Sustainable Action against HIV and AIDS in Communities (SAHACOM) End￾of-Project Evaluation. Draft 3. 2014. KHANA, SAHACOM Transition Plan. Undated. KHANA, SAHACOM Closeout Plan 2014. KHANA, Social Return on Investment. 2012. KHANA, Village Savings and Loan (VSL) Field Operations Manual. 2010. MMT Partner Meeting, MMT: Meeting with MMT Partners at MMT Clinic, 28 April 2014. Powerpoint presentation. NACD, NCHADS and KHANA, National Population Size Estimation, HIV Related Risk Behaviors and HIV Prevalence among People who Use Drugs in Cambodia in 2012. 2014. National AIDS Authority, National Strategic Framework and Operational Plan on HIV, AIDS and STI for Men who have Sex with Men in Cambodia. 2008. National AIDS Authority, National Strategic Plan for a Comprehensive and Multisectoral Response to HIV/AIDS 2006-2010. National AIDS Authority, NSP II Costing. 2006. National AIDS Authority, Revised National Strategic Plan II for a Comprehensive and Multisectoral Response to HIV/AIDS 2008-2010. 2007. National AIDS Authority, National Strategic Plan for a Comprehensive and Multisectoral Response to HIV/AIDS III 2011-2015. 2010. National AIDS Authority, The Financial Resource for the Comprehensive and Multisectoral Response to HIV/AIDS III (2011-2015) in Cambodia. 2010. National Programme for Mental Health, MMT Statistic 2013. Powerpoint presentation. NCHADS, Annual Report. 2013. NCHADS, Conceptual Framework on Virtual Elimination of New HIV Infections in Cambodia by 2020. Cambodia 3.0. Undated. Powerpoint presentation. NCHADS, Conceptual Framework for Elimination of New HIV Infections in Cambodia by 2020. 2012. NCHADS, Standard Protocol for Baseline Assessment for Cambodia 3.0 Initiative. 2014. NCHADS, Standard Operating Procedures for Boosted Continuum of Prevention to Care and Treatment for Most at Risk Populations in Cambodia. 2013. 52 NCHADS, Standard Operating Procedures for Community Based Prevention Care and Support (CBPCS) for People Living with HIV, their Affected Families and Pregnant Women. Working Version. 2010. Panith P, Summary report of NGO capacity assessment for non-Flagship IPs. 2013. USAID, USAID Evaluation Policy. 2011. USAID/Cambodia comments on each of the SAHACOM Annual Workplans 2009 – 2014. USAID/Cambodia, Cooperative Agreement with KHANA for the SAHACOM Project. 2009. USAID/Cambodia, Cooperative Agreement with KHANA for the HIV/AIDS Flagship Program. 2012. Wolfe J, Economic strengthening in the PEPFAR Program. 2014. Powerpoint presentation. Zelnick J, Village Saving and Loan Pilot Program Summary and Analysis. Undated. 53 Annex 4: Interview guides KHANA questions Management questions 1. How has the award of the Flagship Program to KHANA impacted on SAHACOM from both a management and programming perspective? CHBC 2. If the presentations by KHANA do not deal with key achievements (unlikely), we will ask a question on this. 3. What are the core principles of the CHBC model? 4. What have been the major challenges, barriers and constraints encountered in implementing the CHBC component, how have these affected SAHACOM and how have you responded? (How did KHANA manage the transition in roles for NGO/CBO under the SAHACOM model?) 5. How have you built the capacity of SHGs? What is their current level of capacity and in what ways does vary between SHGs? 6. What do you see as the key future roles for SHGs? What are their key roles or functions? 7. What is the feasibility and likelihood of the self-help group model being self-sustaining? Explore current strengths and weaknesses of the model in terms of sustainability. What more needs to be done by whom to develop the sustainability of the model? 8. The Community Support Volunteers clearly have a central role in making the CHBC model work. What has been SAHACOM’s experience with CSVs – how effective have they been and what are the key strengths and weaknesses you have encountered with CSVs in implementation? How have you responded? 9. How do SHGs cater to different types of beneficiaries and their different needs? 10. How has the CHBC package changed over time since commencement of the project? 11. To what extent has support for PLHIV been mainstreamed into larger national programs (e.g., microfinance)? At the local level has SAHACOM supported buy-in from commune leadership – for example, are PLHIV needs/CHBC been included in commune development plans? CHBC future directions 12. Scenario 1: retain existing package: In light of increased coverage of OI/ART and changing needs of beneficiaries, is there a continued need to retain the existing comprehensive package of CHBC? If yes, is there any need for any adaptation to meet the changing needs of beneficiaries? If the existing package of CHBC was retained, are there any ways it could be made more efficient and money saved? (Issue of number of CBOs being supported.) 13. Scenario 2: streamline existing package: How could the CHBC model be changed or streamlined? With a streamlined model, what are the key CHBC services that need to be maintained or enhanced for optimum PLHIV outcomes? 54 With a streamlined model, what could you stop doing, do less of, or do more efficiently? Prevention 14. For prevention which types of interventions or programming approaches did you find were most successful in terms of increasing coverage or service uptake by KPs, especially those from under-served groups? Supplementary: what evidence do you have to support that these approaches were most effective? 15. What are the key barriers to HIV testing and how can these best addressed? What approaches have you found worked best in increasing HIV testing among KPs? 16. What approaches have you found worked best for referrals to OI/ART services and increased initiation and retention in pre-ART and ART? 17. What have been the major challenges, barriers and constraints encountered in implementing the prevention component, how have these affected SAHACOM and how have you responded? 18. Based on SAHACOM’s experience with prevention programming, what recommendations would you make for how prevention programming in Cambodia can be improved? (not just KHANA’s programming, but more broadly) Capacity building 19. What have been the major challenges, barriers and constraints encountered in capacity building of local NGOs, community self-help groups and community networks? How have these affected SAHACOM and how have you responded? 20. How do you measure capacity development in your implementing partners? What changes in capacity can you demonstrate? 21. What are the key areas of need for ongoing capacity building of local NGOs, self-help groups and community networks? Compared to the level of effort in capacity building in SAHACOM, what level of effort is needed in future for capacity building of these groups? 22. How has KHANA’s own capacity to provide TA to the government and local organizations changed over the last 5 years? (Baseline and current) What are the key areas where KHANA sees the need to develop its own capacity? Cross cutting: Linkages, gender, poverty and partnerships 23. For each of the cross cutting areas, what have been the major success stories and challenges 24. How has SAHACOM addressed gender and what have been the results? (Explore issues related to lower utilisation of services by men.) Overall questions 25. Have you found any key differences in implementing SAHACOM in rural areas compared to urban areas? Is the model equally applicable to both areas or have you had to adapt to cater for rural and urban differences? 26. What were the key changes made to SAHACOM follow the mid-term review and the USAID portfolio review? What resulted from those changes? Were there areas where changes were not made in response to recommendations? Why? 55 27. What have been the key lessons you have learned as you have implemented SAHACOM? and how have you responded and adapted to those lessons (both positive and negative lessons)? 28. Have there been any unintended or unplanned consequences as a result of SAHACOM – both negative and positive? 29. In your work with collaborating partners have there been any noteworthy areas of synergy or duplication? 30. How do you go about promoting and measuring quality in your work (in all components), including technical assistance and the quality of service delivery by your implementing partners? How do you respond when there are concerns regarding quality? 31. How effective has USAID’s management of SAHACOM been? What improvements could be made to how USAID manages the program? 32. What are the key issues upon which SAHACOM has been advocating? What have been the results of advocacy? Collaborating partners This includes FHI, PSI, Marie Stopes, UN agencies and government bodies Context: What have been the key areas in which your organisation has worked with SAHACOM? 1. What have been SAHACOM’s major achievements? 2. What have been the major challenges, barriers and constraints encountered by SAHACOM and how has the project responded? 3. How effectively do you think SAHACOM has collaborated with your organization and other partners? Are there any noteworthy areas of synergy or duplication? 4. To what extent does SAHACOM meet beneficiary needs in CHBC and prevention programming? 5. To what extent has SAHACOM’s capacity building of government, NGOs/CBOs and national networks been effective? What types of capacity building have worked best? What are the key priority areas for capacity building in the future? 6. How has KHANA’s own capacity to provide TA to the government and local organizations changed over the last 5 years? (Baseline and current) What are the key areas where KHANA sees the need to develop its own capacity? 7. What have been the key lessons learned from SAHACOM – both positive and negative? 8. How effective do you think SAHACOM’s prevention programming has been in terms of increasing KPs coverage, increasing HIV testing and linking people to treatment services? What approaches have been effective and ineffective? 9. Based on SAHACOM’s experience with prevention programming, what recommendations would you make for how prevention programming in Cambodia can be improved? (not just SAHACOM’s programming, but more broadly) 10. What support is your organisation planning for CHBC in the future? (especially TA) 56 11. Government agencies question: Has KHANA/SAHACOM had technical support inputs to the work of your agency? If so, have you been satisfied with this support? Have you seen an increase in KHANA’s capacity? CHBC future directions (may not be relevant for PSI) 12. Scenario 1: retain existing package: In light of increased coverage of OI/ART and changing needs of beneficiaries, is there a continued need to retain the existing comprehensive package of CHBC? If yes, is there any need for any adaptation to meet the changing needs of beneficiaries? If the existing package of CHBC was retained, are there any ways it could be made more efficient and money saved? 13. Scenario 2: streamline existing package: How could the CHBC model be changed or streamlined? With a streamlined model, what are the key CHBC services that need to be maintained or enhanced for optimum PLHIV outcomes? With a streamlined model, what could you stop doing, do less of, or do more efficiently? Implementing partners For NGOs/CBOs and national networks funded by SAHACOM 1. Thinking about the work you have been doing as part of SAHACOM, what have been your major achievements over the last 5 years? 2. What have been the major challenges and difficulties you have encountered in implementing SAHACOM? How have these affected your work? What have you done to address these challenges and difficulties? 3. How well has SAHACOM met the needs of your target group? Any key gaps or areas for improvement? 4. What support have you received to improve your organisations capacity to implement the SAHCOM Project? Has this support been helpful and if so, how? Are there areas where capacity building and support could be improved? Are there types of support you would like to get but which are not available? 5. How do you go about improving the quality of your work? What assistance do you get from SAHACOM in showing you how to measure quality and how to improve quality? 6. Who are the main partners you work with in implementing SAHACOM and in what ways do you work with them? (e.g. health and non-health services, other NGO/CBOs, local government, etc.). Check for effectiveness of linkages, referrals and collaboration. Are there any areas of duplication? CHBC specific questions (may not apply to all partners) 7. How strong are the self help support groups? Do you think they can continue their work with less or no donor support? Do they need any ongoing support and if so, what types of support? 57 8. The SAHACOM CHBC model places strong emphasis on self-help support groups. Are all PLHIV happy to be a part of support groups? How do you support PLHIV who don’t want to be part of a SHG? 9. Now that more people are on ART and in better health, are all parts of the comprehensive package of CHBC services still needed? Why? Are some services/support not needed now? 10. What are the most important parts of the comprehensive package of CHBC services – the things people most need and value? 11. Are there any needs of clients not being met? Does the CHBC package need changing in any ways to better meet client needs? 12. Would it be possible to make changes to the CHBC package to save money and make it more efficient? Prevention specific questions (may not apply to all partners) 13. What have you done to improve coverage of prevention programs for KPs, especially those from under-served groups? What were the results? 14. What are the key barriers and difficulties faced in getting people to have an HIV test? What have you done to increase HIV testing? What were the results? What approaches worked best? 15. What approaches have you found worked best for referrals to OI/ART services and increased initiation and retention in pre-ART and ART? 16. How can prevention programs in Cambodia can be improved? Self-help group questions 1. Tell us about the self-help group – how often does it meet and what happens at meetings? 2. How does the SHG help you? What are the most helpful/needed areas of support? Has the work of the SHG changed over the last 5 years? In what ways? 3. How do you contribute to the self-help group? (exploring the ‘self’ aspect of help) 4. Are there any ways in which the SHG could be improved to better meet your needs? 5. Do you think that SHGs need to be continued? If yes, what are the most important things they do? 6. Do you know PLHIV who don’t want to be a part of the SHG? Why don’t they want to join the group? If they are not part of the group, how are their needs met? What do they miss out on? 7. Does the SHG help PLHIV with treatment adherence and staying on treatment? How? Is this helpful? Why? 8. What HIV and other health and social services you use? Do you get referred to those services? Who refers you? Does the referral system work? Any difficulties in accessing these services? 9. Are you satisfied or dissatisfied with the HIV and other health and social services you receive? Why? How could they be improved? 10. Thinking about all the HIV and other services you use, which are the most important ones which you most need? 58 11. Are there any services you need that are not available anywhere? What are the type of services do you need that are not available? Focus groups with KPs re prevention programming and links to health services 1. Find out about their contact with the HIV prevention program being provided by NGO/CBO under SAHACOM. Can you tell us about your experiences with the outreach, DICs, etc. 2. How often do you use these services (i.e., see outreach workers, go to DICs or receive referrals)? 3. How do you find the staff and volunteers? 4. Would you say you are satisfied or dissatisfied with the services? Why? 5. How could the services be improved? 6. Explore condom, lubricant and needle availability through NGOs, venues, entertainment establishments and social marketing. 7. What ideas do you have for increasing the number of people from your group [i.e. KPs] who have an HIV test? Explore barriers/difficulties associated with HIV testing. 8. For self-help groups for HIV positive people, are separate SHGs needed for different KPs populations or are HIV positive people from your group happy to join general population SHGs? Explore experiences with KPs specific and general population SHGs. 9. What HIV and other health and social services you use? Do you get referred to those services? Who refers you? Does the referral system work? Any difficulties in accessing these services? 10. Are you satisfied or dissatisfied with the HIV and other health and social services you receive? Why? How could they be improved? 11. Thinking about all the HIV and other services you use, which are the most important ones which you most need? 12. Are there any services you need that are not available anywhere? What are the type of services do you need that are not available? 59 Annex 5: Evaluation schedule Table 3: Evaluation schedule Dates Tasks Deliverables 1. Initial preparation and planning June 2-6 Review of background documents and preparation work (home base) June 8 Team arrives in Phnom Penh June 9 Evaluation Team planning meeting: review of draft Evaluation Framework, interview guides, team roles, logistics, etc. June 10 Evaluation Team in-brief with USAID Public Health and Education Office. Evaluation Framework presented to USAID/Cambodia Interview with KHANA senior management and SAHACOM technical staff Evaluation Framework 2. Interviews with SAHACOM, implementing partners and other stakeholders June 10-13 Phnom Penh interviews and site visits Day trip to Takeo Province on June 13 June 15-21 Provincial trips: two sub-teams Team 1: Battambang Pailin Pursat Kampong Channang Team 2: Kampong Cham Siem Reap Banteay Meanchey June 23 &25 Interviews with Phnom Penh stakeholders Follow up interview with KHANA 3. Analysis of data and development of key findings June 23-25 Team meeting to analyse all data and develop key findings. (Done concurrently with additional interviews with Phnom Penh stakeholders and follow up interview with KHANA) 4. Mid-point debriefing with USAID/Cambodia June 25 Preparation of presentation on key preliminary findings for presentation to USAID/Cambodia (afternoon) June 26 Presentation of key findings to USAID/Cambodia (morning) Mid-point debriefing presentation 5. Writing of draft Evaluation Report June 26- July 2 Writing of draft evaluation report 6. End of field work debriefings July 2 Development of debrief presentations for debrief to USAID/Cambodia and implementing partners (two presentations) - afternoon 60 Dates Tasks Deliverables July 3 Debrief presentations: 1. USAID/Cambodia and US CDC 2. KHANA 3. SAHACOM implementing partners and stakeholders End of in-country work Evaluation debriefing presentations 7. Editing of evaluation report July 4-9 Revisions and editing of the evaluation report Draft evaluation report submitted to USAID Cambodia (COB July 9) 8. USAID/Cambodia review of draft evaluation report July 10-30 USAID review of draft evaluation report Consolidated feedback from USAID/Cambodia due COB July 30 9. Finalization of evaluation report July 31 – August 6 Evaluation report revised and finalized taking account of USAID feedback Final Evaluation Report submitted by COB August 6 61 Annex 6: Organizations consulted 1. SAHACOM implementing partners In depth interviews were conducted with 18 of SAHACOM’s implementing partners in Phnom Penh and seven provinces. The implementing partners consulted are listed in Table 4, below. Consultations with implementing partners typically involved an interview with the management and program staff and separate focus group discussions with Self Help Groups, Village Savings and Loans Groups (often combined with the SHG interview), and Orphans and Vulnerable Children Support Groups. Where implementing partners were conducting livelihoods work, the evaluation team also conducted site visits to inspect the livelihoods work underway (e.g., fish farming) and interview the beneficiaries. For implementing partners working in focused prevention with key populations, the evaluation team held focus group discussion with members of the key populations who had been reached. All interviews with implementing partners were conducted without KHANA staff being present. Similarly, all interviews with project beneficiaries were conducted in the absence of KHANA and implementing partner staff to minimize respondent bias in answers to questions. Table 4: Interviews and site visits with SAHACOM Implementing Partners Implementing partner Location CARAM: Coordination of Action Research on AIDS and Mobility Phnom Penh Korsang Phnom Penh KOSHER: Key of Social Health Educational Road Phnom Penh WOMEN: Women Organization for Modern Economy and Nursing Phnom Penh PC: Partners in Compassion Takeo Province MHSS: Men’s Health Social Services Battambang Province BFD: Buddhism for Development Battambang Province BWAP: Battambang Women Against AIDS Battambang Province CWPD: Cambodian Women for Peace and Development Battambang Province BWAP: Battambang Women Against AIDS (Pailin Province project) Pailin Province CPR: Cambodia Poverty Reduction Pursat and Banteay Meanchey Provinces BSDA: Buddhism for Social Development Action Kampong Cham Province SPEAN Kampong Cham Province NAS: Nak Akphiwath Sahakum Kampong Cham Province MHC: Men’s Health Cambodia Siem Reap Province SCC: Salvation Centre Cambodia Siem Reap Province SEADO: Social Environmental Agricultural Development Organization Banteay Meanchey Province KBA: Khmer Buddhist Association Banteay Meanchey Province 2. Other Provincial level key informants Interviews were held with a range of key informants working at the provincial level. These were in addition to interviews/site visits to IPs which are listed above. Other provincial level key informants are listed in Table 5 below. 62 Table 5: Other Provincial level key informants Agency Location Provincial Health Department and Provincial AIDS & STI Program Team Battambang Province Clinical staff, Pre-ART/ART Clinic, Battambang Referral Hospital Battambang Province MMM staff, Battambang Referral Hospital Battambang Province United States Centers for Disease Control staff Battambang Province Commune Council for Women and Children, Anglong Vil Commune, Sangke District Battambang Province AHEAD: Action for Health Development Battambang Province Provincial Health Department and Provincial AIDS & STI Program Team Pailin Province Clinical staff, Pre-ART/ART Clinic, Pailin Referral Hospital Pailin Province MMM staff, Pailin Referral Hospital Pailin Province Provincial Health Department and Provincial AIDS & STI Program Team Pursat Province Clinical staff, Pre-ART/ART Clinic, Pursat Referral Hospital Pursat Province MMM staff, Pursat Referral Hospital Pursat Province KHANA Livelihood Centre staff Kampong Chhnang Province Provincial Health Department and Provincial AIDS & STI Program Team Kampong Cham Province Clinical staff, Pre-ART/ART Clinic, Kampong Cham Referral Hospital Kampong Cham Province MMM staff, Kampong Cham Referral Hospital Kampong Cham Province Kampong Cham Provincial PLHIV Network (PPN+) Kampong Cham Province Provincial Health Department and Provincial AIDS & STI Program Team Siem Reap Province Clinical staff, Pre-ART/ART Clinic, Siem Reap Referral Hospital Siem Reap Province MMM staff, Siem Reap Referral Hospital Siem Reap Province Commune Council for Women and Children, Kok Chork Commune Siem Reap Province Siem Reap Province OVC Taskforce Siem Reap Province Caritas Siem Reap Province Siem Reap Provincial PLHIV Network (PPN+) Siem Reap Province Provincial Health Department and Provincial AIDS & STI Program Team Banteay Meanchey Province Clinical staff, Pre-ART/ART Clinic, Banteay Meanchey Referral Hospital Banteay Meanchey Province MMM staff, Banteay Meanchey Referral Hospital Banteay Meanchey Province 3. National level key informants Interviews were held with a range of key informants working at the national level. These included RGC agencies, development partners and national civil society networks working in HIV. The agencies met are listed in Table 6 below. Table 6: National level key informants Name Agency HE Mean Chhi Vin Director, NCHADS Khum Kim Eam Vice Chief, Technical Bureau, CENAT Thong Sokunthea National Authority for Combatting Drugs Khlang Pichet Ministry of Social Affairs, Veterans and Youth and Rehabilitation Marie-Odile Emond Country Director, UNAIDS/Cambodia 63 Fujita Massami World Health Organization Michelle Lang-Alli HIV Team Lead, Office of Public Health and Education, USAID/Cambodia Sok Bunna Technical Team Leader for HIV/AIDS, Office of Public Health and Education, USAID/Cambodia Pamela Teichman Senior Technical HIV/AIDS Prevention Advisor, Office of Public Health and Education, USAID/Cambodia Heng Thona Project Management Assistant, Office of Public Health and Education, USAID/Cambodia Inga Olesky Global Fund Liaison Officer, Office of Public Health and Education, USAID/Cambodia Dora Warren Director, US Centers for Disease Control and Prevention/Cambodia Perry Killam Care and Treatment Officer, US Centers for Disease Control and Prevention/Cambodia Ly Vanthy US Centers for Disease Control and Prevention/Cambodia Suos Premprey Australian Department of Foreign Affairs (Australian aid program) Christophe Grundmann Project Director, University Research Corporation Emerson Mar Deputy Director, Marie Stopes International, Cambodia Monte Achenbach Population Services International, Cambodia Amy Weissman FHI360 Cambodia Laurent Ferradini FHI360 Cambodia Chhit Sophal MMT Clinic, Russian Hospital Sao Sopheap Bandanh Chaktomuk (Cambodia MSM Network) Sorn Sothearidh Cambodian People Living with HIV Network Oum Sopheap Executive Director, KHANA Choub Sok Chamreun Deputy Director, KHANA Tith Khimuy Deputy Director: Programs, KHANA Pen Monorom Deputy Director: Finance, KHANA Sron Samrithea Livelihoods Program Manager, KHANA Prom Channty Programs Team Leader, KHANA Yi Siyan Research Director, KHANA Tout Sovannary Research Manager, KHANA So Kimhai Harm Reduction Manager, KHANA Penh Phanith Technical Advisor: Organizational Strengthening, KHANA Pheak Chhoun Research Fellow: HIV and Health, KHANA Samantha Brunt Research Intern, KHANA Cady Shadwick Research Intern, KHANA Kalet Kea Research Intern, KHANA 64 Annex 7: SAHACOM operational framework and structure of technical approaches Figure 2: SAHACOM operational framework 65 Figure 3: Structure of SAHACOM technical approaches for community support for PLHIV/CHBC and Focused Prevention 66 Annex 8: Data - graphs and tables Table 7: SAHACOM’s achievements against targets for key community support indicators Indicators Baseline FY 2009 Target FY 2010 Actual FY 2010 Target FY 2011 Actual FY 2011 Target FY 2012 Actual FY 2012 Target FY 2013 Actual FY 2013 Revised target FY 2014 Life of project target Quality of life for PLHIV: rate their life as good 35% 35% NA 37% NA 40% NA 45% 73% 50% 50% Quality of life for PLHIV: Health satisfaction among adult PLHIV in the last four weeks 52% 55% NA 57% NA 60% 72% 60% 72% 60% 60% Quality of life for PLHIV: satisfaction with CHBC service 83% 85% NA 87% NA 90% 96% 90% 96% 90% 90% Percentage of adults and children with advanced HIV infection receiving ART >90% >95% 97% >95% NA >95% 92% >95% 92% 100% 100% Number of eligible adults and children provided with a minimum of one care service 23,948 21,910 31,769 27,649 35,811 34,000 42,832 33,300 38,903 38,903 38,903 Number of HIV positive adults and children receiving a minimum of one clinical service 4,374 6,809 5,755 5,950 7,720 7,694 8,560 6,750 8,974 6,989 8,974 Number of HIV positive patients who were screened for TB in HIV care or treatment settings 444 3,748 3,748 Number of HIV positive patients in HIV care or treatment (pre-ART or ART) who started TB treatment 600 500 219 500 163 555 176 150 97 125 555 Number of registered TB patients who received counseling and testing for HIV and received their test results 2,500 918 1,500 495 500 4,500 Number of suspected OVC referred for TB screening 600 684 600 851 600 1,800 Number of pregnant women with known HIV status 2,344 2,450 2,344 438 1,758 7,032 Number of PLHIV and adolescent OVC received support to access contraception methods 5,000 2,248 4,000 2,144 2,200 4,000 Source: SAHACOM Workplan and PMP Matrix, October 2010 – September 2014. 67 Table 8: illustrative SAHACOM capacity development activities for implementing partners 1. Consultative workshop to review and revise Standard Packages of Activities for PLHIV, OVC, MSM, IDU and EW to align with the National Strategic Plan III. 2. Revised 5 training curricula to help guide IPs, PF, PE, CSVs: 3 on prevention for KPs- EWs, IDU and MSM and 2 on Community and Home Based Care (CHBC) for PLHIV and OVC that include HIV basics, facilitation skills, communication skills, nursing care, nutrition, positive living, family planning, sexual and reproductive health, HIV prevention, ART, and community mobilization and collaboration with key stakeholders. 3. Training for Peer Educators and Peer Facilitators on HIV/AIDS health issues including counseling, VCCT, life skills, negotiation skills, sexual and reproductive health, STIs and HIV transmission, prevention and support in order to strengthen their skills on methodologies for education and build confidence in transferring key messages to their peers. 4. Exchange visits for 15 PFs and IP staff to FHI to learn about the SMART girl program. 5. 5-day training for IPs on project operational procedures and TOT for CSOs on CHBC to orient them on: PLHIV, SHG, and OVC activities; organizing, planning and reporting; financial management; and methods of working of CHBC within the new model of SAHACOM. 6. CSOs train, coach and support CSVs and CSVAs to provide health and support services, organize and control the self-help group (SHG) meetings, refer to health services and follow up their group members, for undertaking home visits. Training focused predominantly on the basic of HIV, PMTCT, STI, ART, health care and nutrition, and referral of PLHIV to health services, facilitation skills, communication skills, nursing care, positive living, family planning, sexual and reproductive health. 7. Conducted the ToT workshop on the Convention on the Right of Child from 15-17 June 2010 at Siem Reap Province. 8. Training workshops on NGO Capacity 9. Economic Livelihoods Assessment Tool training workshops for IPs resulting in Assessment Response in Battambang, Preah Sihanouk, and Siem Reap. 10. 5-day Workshops on Project Operational Procedures and ToT as per SAHACOM model on Home Community Based Care in Preah Sihanouk and Focused Prevention on KPs in Siem Reap Province. 11. In collaboration with DAI, KHANA organized Village Saving Loan training at Battambang province to equip IP staff and Community Support Volunteers with economic livelihoods skill and approach on Village Saving Loan (VSL). 12. OPERACY and Village Saving Loan (VSL) training delivered by the trainer from Human Earth Development Center (HEDC) for CSVs. 13. Training to IP staff, CSV, and caregivers in areas including basic health treatment, psychosocial support, general hygiene, counseling, referral to health services, adherence of OIs and ARV, and symptomatic identification HIV suspected person and PEs and PFs were trained in prevention activities. 14. Alliance linking organization, MAMTA, conducted training on sexual and reproductive health and family planning. 15. KHANA contributed to the National Harm Reduction Workshop that aimed to increase awareness on harm reduction for government officials, local authorities and law enforcement. 16. Involved in a training co-organized by NAA and MoI for media to increase awareness on issues and concerns in the HIV/AIDS response. 17. Held “A Most Significant Change” workshop for IP representatives to increase documentation of the changes in the lives of PLHIV, KPs and OVC. 68 18. Livelihood training on Village Savings and Loan schemes, micro-business, home gardening, fish culture, crop production, and chicken-raising for CSV and beneficiaries. 19. Training on Integrated SRHR and TB for IP and KHANA staff. 20. Good governance capacity building through a workshop for IPs and networks to improve organizational development by strengthening internal systems and policies, governance practices, accountability and overall performance and Exchange Visits to two IPs that were accredited by KHANA. 21. KHANA and MSIC developed a Practical Field Guide for Strengthening the HIV/AIDS and SRH Integration and conducted trainings on HIV/SRH integration. 22. Advance HIV Testing and Counseling training for KPs. 23. Orientation on Treatment as Prevention and Informed Choice, which covered early treatment, Option B+, and informed consent. 24. In collaboration with WFP, conducted cascade training on revised Good Food Toolkits on HIV and nutrition and nutrition and positive living for pregnant and lactating women living with HIV, children living with HIV. Source: KHANA/SAHACOM Annual Progress Reports to USAID/Cambodia, FY 10, FY11, FY12, FY13, FY14 (October 1, 2013-March 30, 2014) 69 Table 9: illustrative activities of KHANA’s technical assistance provision 1. Organized a first briefing session on health situation in Cambodia at KHANA with participation from UN country team, RACHA, RHAC, MSI-Cambodia and World Bank. 2. Invited by MoSVY and UNICEF to participate in the “Rapid Assessment Workshop on OVC M&E System” and presented its OVC M&E system and guidelines, data collection tools, database system, and key indicators. 3. KHANA was a member of several NCHADS Technical Working groups: • Played an active role in revising the ART and CHBC SOPs • Participated in the Takeo and Battambang Regional CHBC meetings for HIV/AIDS • As a member of the MSM TWG, helped develop the National Guideline of HIV/AIDS response among MSM and Transgender Population, launch the EW/MSM General Stigma toolkit and for completion of the Purple Sky Network Regional Information System on MSM (PRISM) questionnaire • As a member of the Drug and HIV/AIDS (DHA) TWG, reviewed data, ToR and membership of TWG and issues of Korsang in needle and syringe program implementation, Methadone Maintenance Therapy (MMT) • For Migrant and Mobile Population (MMP), reviewed the strategic plan and operational plan 2006-2008 and helped develop the section on HIV prevention among MMP for the National Strategic Framework 2010-2012 • As a member of the EWs Core Group meeting, helped develop the Standard Operating Procedure for Continuum of Prevention Care and Treatment (CoPCT) • Participated in the NCHADS TWG to develop the Concept Note on Treatment as Prevention (TasP) as a Strategy for Elimination of New HIV Infections in Cambodia 4. Provided training on a consultancy basis to share KHANA good practice. Examples include support Tiny Toons for training of trainers for their harm reduction program with street children and teenagers, as well as support to the NGO Forum, Oxfam Novib, and Padex. 5. With NCHADS/MOH, co-organized the third Symposium on HIV and helped revise the Harm Reduction training curriculum for law enforcement officers 6. Provided input into several key documents: position paper on MCH for the RNMCH working group; manual on Gender and HIV/AIDS along with tools, checklists and core indicators for monitoring gender and HIV; guidelines and discussion of Unique Identifier Code for KPs; review of 5 Year National Strategic Plan on Drug and HIV/AIDS 2011 – 2015; SOP for care and support for OVC; national COPCT SOP for MSM 7. In collaboration with NCHADS and NACD, carried out the National PWID size estimation study and the Integrated Biological and Behavioral Study (IBBS). 8. Conducted the SAHACOM Mid-Term Review and a Livelihood Program study in four provinces. 9. Conducted research of Risk Factors for HIV Transmission in Sero-Discordant Couples that measured biological, demographic, and behavior variables. 10. Conducted Geographic Information System mapping of key populations including the development of guidelines, protocol, and training of data collectors. 11. Developed the concept for the Unique Identification Code for KPs and pre-tested it among EWs and MSM. 12. In collaboration with CPN+, conducted the Stigma Index Study. Source: KHANA/SAHACOM Annual Progress Reports to USAID/Cambodia, FY 10, FY11, FY12, FY13, FY14 (October 1, 2013-March 30, 2014) 70 Source: KHANA. NGO Assessment Analysis, 2014. Table 10: SAHACOM’s organization capacity baseline and end line Average Scores by Indicators Capacity No Indicators Baseline 2009 End line 2014 Baseline￾End line Difference Capacity 1: Partnerships, Referral Systems, Coordination, Communication and Advocacy 1.1 Awareness and working with other organizations 3.8 4.2 0.4 1.2 Referrals 4.6 4.2 -0.4 1.3 Effective, target communication and advocacy work 3.1 3.7 0.6 1.4 Research, consultation and analysis as a foundation for advocacy work 3.0 3.3 0.3 Capacity 2: Technical Capacity on HIV/AIDS, TB, SRH, FP, MCH and Impact Mitigation 2.1 Experience, knowledge and skills 3.4 3.7 0.3 2.2 Access to technical resources and knowledge 3.2 4.0 0.8 Capacity 3: Organizational Systems and Policies 3.1 Governance, strategy and structure 3.5 2.3 -1.2 3.2 Human resources and administration 3.5 3.3 -0.2 3.3 Program Management 3.8 3.8 0.0 3.4 M&E 3.7 4.7 1.0 3.5 Financial Management and Sustainability 3.7 3.2 -0.5 Capacity 4: Promotion of Participation of PLHIV, KPs and other Affected Communities 4.1 Level and range of involvement of key populations (PLHIV, MSM, SW, DU) and other affected communities 3.6 3.9 0.3 4.2 Efforts made to promote involvement of people living with HIV/AIDS and other affected communities 3.9 3.9 0.0 Total 13 3.6 3.7 71 Figure 4: Web of organizational capacity average scores for SAHACOM implementing partners by indicator Source: KHANA. NGO Assessment Analysis, 2014 72 Table 11: Organizational capacity baseline and end line by implementing partner N # of NGO (SAHACOM) Endline 2014 Baseline 2009 Remark 1 BSDA 5.0 No baseline 2 PC 5.0 4.1 3 SCC 3.0 4.0 4 MHC 3.0 3.5 5 MHSS 3.0 3.9 6 IDA 3.0 3.1 7 CPR 3.0 3.6 8 BWAP 3.0 4.2 9 NAS 3.0 3.8 10 MODE 3.0 4.0 11 SEADO 3.0 4.1 12 WOMEN 3.0 4.4 13 KOSHER 3.0 2.2 14 KS 3.0 2.9 15 CARAM 3.0 4.2 16 SIT 3.0 3.0 17 VC 2.0 2.7 18 BFD 2.0 4.4 19 KBA 2.0 3.5 20 NHCC 2.0 No baseline Average 3.0 3.6 KHANA Purple-O-Meter is more enhanced and comprehensive assessment tool with stricter scoring system comparing to the assessment tool used for the baseline. In general, the 91 criteria are categorized into ’23 Essential’ (mainly related to governance, internal control and systems) and ’68 Desirable’. In order to be scored 4 or 5, the organization must meet all the ‘Essential Criteria’ plus at least 50% of ‘Desirable Criteria’, otherwise the highest score is ‘3’ only. In the previous tool (used for baseline assessment) all criteria are treated with the same weight and the scoring is based on the total number of criteria met. Source: KHANA. NGO Assessment Analysis, 2014 73 Annex 9: Focused prevention service package The key prevention strategies included in Focused Prevention comprise: • Outreach workers doing health promotion and providing BCC • Promoting the use of and access to condoms and lubricant • Harm reduction interventions for PWID • Drop in centres • Linking KPs to STI screening, testing and treatment • Accelerated uptake of HIV testing among KPs using rapid tests (finger prick) • Referral and active follow-up of HIV positive KPs from community to health facilities • Support for immediate enrolment in pre-ART and early initiation of ART • Integration of SRH/FP into BCC, health promotion and the referral system • Psychosocial support • Advocacy for an enabling environment • Capacity building for outreach workers 74 Annex 10: Conflict of interest disclosure Disclosure of Conflict of Interest for USAID Evaluation Team Members Name David Lowe Title Mr Organization Not applicable Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) PO AID-486-O-14-00076 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Project evaluated: Sustainable Action Against HIV and AIDS in the Community. Implementer: KHANA I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. No conflict of interest I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature David Lowe Date 75 Name Srey Mony Title Consultant Organization Independent Evaluation Position? Team Leader Team member Evaluation Award Number (contract or other instrument) AID-486-O-14-00079 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Sustainable Action Against HIV/AIDS in Community (SAHACOM) I have real or potential conflicts of interest to disclose. Yes No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I have been engaged in the following consultancies offered by KHANA: 1. Technical Local Consultant to Support KHANA/MSIC HIV/SRH integration project, 10 July 2013 to 10 September 2013. 2. Technical Support Provider to provide technical input and develop VCCT training curriculum, 16-25 December 2013. 3. National Consultant for Documenting Peer-Provided Testing in Cambodia, December 26, 2013 – August 31, 2014. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure 76 for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date 12 August 2014 77 U.S. Agency for International Development/Cambodia No 1 Street 96 Wat Phnom Phnom Penh Duan Penh, Phnom Penh Cambodia 78