Midterm Performance Evaluation of CAP-3D in Burma, Laos, & Thailand and Final Performance Evaluation of CAP-3D in China October 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Po-Lin Chan, Billy Pick and Jamie Uhrig. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 0 Executive Summary A midterm evaluation of CAP-3D was undertaken in Burma, Laos, and Thailand and a final evaluation was undertaken in China in June and July 2013. The overarching strategy for this project is to increase the coverage, quality, and sustainability of a comprehensive package of for HIV prevention, care, and treatment services. A major area of emphasis is reducing dependence on external donor resources by strengthening local institutions and securing local government buy-in and diverse funding sources. To achieve this, PSI and its partners focus on the following objectives:  the scale up of the CPP model, shifting increasingly toward provision of technical assistance to local agencies for CPP implementation (IR1);  evaluation and documentation of CPP models and use of strategic information (SI) to advocate for replication and leveraged funding (IR2); and  enabling local organizations to scale-up the CPP through provision of technical assistance and capacity building (IR3). The key populations at higher risk include men-who-have-sex-with-men, transgendered people (mostly transgendered women) female sex workers and their clients, people who inject drugs, and people living with HIV. For tuberculosis control, the target populations include people living with HIV and people with symptoms of tuberculosis who access private practitioners or community based programs. For malaria control, target populations are on rural populations in endemic areas. The evaluation focused on answering the following five key questions:  Question 1- Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date?  Question 2 - Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches?  Question 3 - Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach?  Question 4 - Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project?  Question 5 - Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? 1 Key Findings Burma CAP-3D funds activities for HIV, tuberculosis, and malaria. Much of the work of the evaluators focused on the first two diseases. Malaria funding went almost exclusively to procurement of bed nets in financial year 2012. It is also noted that some of the HIV funding for financial year 2013 was provided by USAID in Yangon. Performance - While key populations reached with prevention activities for the last full year reported (FY 2012) were under target, performance has improved with the decreasing of constraints on the geographic sites covered and new leadership in place. The continued decline of incident infections to below one thousand infections per year among all people identified as ‘men-who-have-sex-with-men’ and sex workers are largely, but not exclusively, due to PSI activities. Nonetheless, it is difficult to gauge how effectively key populations served by CAP￾3D are linked to care and treatment. The approach to meeting some targets may not be either effective or efficient. Identifying and testing individuals who have never or rarely been tested will have a greater impact through getting people early into care, support, and treatment programs. A similar issue arises with targets set for peer educators who must refer members of key populations for sexually transmitted infection care. The project has utilized approaches to meet and address the needs of highly vulnerable members of the target populations. In tuberculosis, the focus on case finding based upon segmentation according to populations and geographic locations has been relatively successful. Both Sun Quality Health and TOP Centers provide potential models in antiretroviral treatment service delivery which can provide care for different segments of the population and increase equity of ART access. Sustainability- Ensuring sustainability of program approaches remains a challenge. While some progress has been made, there remains a question as to sustainability over the long term. Meanwhile, PSI has worked with PACT and the TOP Center Yangon staff to tailor a specific planning process which may move TOP Centers toward more autonomous status. Also, PSI efforts with Sun Quality Health doctors may represent a possible way to contribute to the national effort to decentralize counseling and testing. Operations and relationships - Despite challenges related to funding, partner capacity and USAID administrative changes, the CAP-3D project has been able to balance a wide range of partnerships in a professional manner. There were no apparent advantages or disadvantages to having the three major communicable diseases dealt with in one platform. Ability to achieve critical results - CAP-3D’s ability to achieve critical results will continue to be constrained by the legal environment for key populations, limited antiretroviral treatment access, and limits to clinical screening and sputum microscopy. There are areas such as the development of community and peer led HIV testing where TOP Centers could possibly take a lead in the future, and while there are questions as to achieving critical results at scale, most beneficiaries find TOP Centers provide a safe and respectful place for them to come and appreciate the ease with which they can receive prevention, HIV testing, and STI treatment. 2 Strategic information - The degree to which monitoring and evaluation activities inform planning and implementation varies. There is a need to improve the use of routine data from programs (TOP or SQH) as there is a very rich database on TOP outcomes including detailed individual longitudinal data on HIV testing (at center level, not at PSI MIS level). In addition, there is lack of regional sharing of knowledge based and specificities to research in key populations. China Performance – The CAP-3D China program has balanced their approaches to prevention, care and treatment going further than other country programs in 1) improving models of delivery through local partners, 2) documenting effectiveness, and 3) advocating for these models to become the regional standard of quality HIV prevention programming for most at-risk populations, implemented by strong local institutions and sustained through local governments and diverse funding sources. Determining the extent of success in achievement of program results for the CAP-3D China program is complicated by the fact that the timeline for planned activities was severely curtailed. Project activities were implemented for only two and a half years instead of over the planned four-five year period. In addition, the impact of several activities were either difficult to measure with current methodologies or the indicators used were not adequate to measure the desired results. Sustainability – Despite the constricted time line, most local organizations have received funding to continue some or all of their activities. It is in the field of sustainability that many lessons can be learned from the experience of the last two years. Both PACT and RTI played vital roles in helping local partners and government achieve sustainable results. Operations and relationships – The manner in which the CAP-3D project was organized, operated and communicated with stakeholders was challenged by the unexpected reductions in funding and earlier phase out. This resulted in difficulties in engaging partners and government, but these challenges were successfully resolved by project end. PSI and PACT staff successfully negotiated the difficulties presented by USAID’s decision to cease activities in China with a great deal of professionalism and managed to re-engage local partners to achieve successful outcomes given the constraints. Ability to achieve critical results – While the national level may not have been influenced by CAP-3D program activities, Yunnan and Guangxi HIV prevention and care practice was certainly positively influenced by project activities to the project end. Strategic information - Despite the fact that CAP-3D China has been in a transition and phase￾out period since 2011, the program has conducted the relevant studies to inform programming and built capacity in strategic information with local partners (community based organizations and government partners). Dissemination and use of strategic information have been well utilized in further developing interventions to address weaknesses in programming. 3 Laos Performance – The CAP-3D program in Laos is contributing to national HIV program impact as measured by HIV prevalence. On the other hand, PSI activities will have limited impact in decreasing the high prevalence of sexually transmitted infections among transgendered people, and cannot be determined for men-who-have-sex-with-men because of the unknown prevalence. Inadequate consideration has been given to data collection by program implementers constraining the ability to determine the extent to which prevention, care and treatment results for the target populations has been achieved. Results for a number of innovations related to condom/lube social marketing, STI treatment and peer education outreach incentives have, so far, been mixed. Sustainability – CAP-3D funded activities have influenced programming for the overall national HIV response. Experience with the models developed by PSI and its partners in implementing CAP-3D were valuable in working with other stakeholders in developing renewal documents so that Global Fund resources could continue to be used for HIV prevention and care in Laos. Operations and relationships – Relationships between PSI and a wide range of partners are, in general, handled professionally and have been well managed. The program has worked closely with both government and civil society partners to build capacity to provide an effective and sustainable approach to the HIV response. Ability to achieve critical results – CAP-3D should be able to achieve critical results at scale related to condoms/lubricant for prevention of HIV. On the other hand, achieving results at scale for HIV counseling and testing and linkages to care and treatment could be constrained by a variety of programmatic and structural factors. Strategic information- At present, indicators are not constructed to track the care cascade from prevention to enrollment in HIV care and treatment. While PSI has conducted some surveys, critical gaps in strategic information remain. In addition, it is unclear whether data from routine monitoring reports is used at the level of the subcontractor/implementing partners and how this contributes to improving local programming. Thailand Performance – The approaches used to achieve results have, for the most part, been successful. There have been remarkably high levels of performance for almost all activities except those involving two governmental institutions. High performance was supported by sound data collection and management. This resulted in ‘treatment cascades’ that could be easily developed and used to improve program outcomes. The CAP-3D funded activities contributed to the national strategic plan calling for increased testing of key populations. CAP-3D activities include several on-going innovations which still need to be evaluated to determine their effectiveness. Sustainability – The CAP-3D funded program has not had a major influence on both national public health and community leaders to adequately prioritize and take action on the public health emergency among MSM and transgendered persons. Thailand is the only one of the four 4 countries with a severe and increasing epidemic. While national program leaders recognized the contributions made by the program through PSI staff work on national working groups, particularly in the development of national MSM guidelines, this did not always translate into sufficient efforts at prioritizing interventions. Separating the impact of CAP-3D capacity building from the impact of capacity building by previously funded activities by other USAID-funded projects is difficult to determine. An enabling environment surrounding local registration of community based organizations may help sustain CAP-3D activities, but continued efforts at developing broader funding bases for some CBOs needs to be addressed. Operations and relationships – The organization and operation of the CAP-3D project in Thailand, with its many implementing partners, both old and new have the potential to result in the ideal of a truly effective consortium of partners. Ability to achieve critical results – CAP-3D’s ability to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project is constrained by national policy related to HIV testing. In addition, the lack of interventions aimed at stigma and discrimination is another area which may negatively affect the project’s ability to achieve critical results. Strategic information – Recent changes in monitoring and evaluation should improve the ability of strategic information to inform planning and implementation. It is also important to ensure that the indicators are aligned to the reporting requirements of the country and global needs, since CAP-3D contributes to the national response. Overall Project Conclusions and Recommendations CAP-3D represents an appropriate regional focus on prevention and HIV testing to ensure continuity and linkages throughout the comprehensive model of prevention, care and treatment. The project is appropriately focused among men-who-have-sex-with-men, transgendered women, and female sex workers. The added activities for people who inject drugs in China were also appropriate and in accordance to the local epidemiology. The comprehensive approach was adapted in each country according to local needs. There remains the challenge, given current funding levels, to obtain a balance between direct service delivery, community capacity building and providing technical assistance. Direct implementation of drop-in center-based programs by PSI should transition to customized technical support to community based organizations. Organizational capacity development needs to be initiated as early as possible in the lifetime of the funding grants, since community based organizations vary in their ability to change rapidly. There contributes to be the belief that HIV counseling and testing is a means of HIV prevention instead of an entry point to care, and that an HIV diagnosis is a death sentence. PSI can use its expertise in marketing to support organizations in creating a demand for testing as a gateway to treatment for a longer and better life in all three countries. 5 A method must be developed to track individuals from testing to treatment so that a ‘treatment cascade’ can be developed and analyzed to increase and improve entry to care. As soon as a person living with HIV takes their first tablet of antiretroviral medications, it is going to be challenging to obtain individual clinical data and monitoring of treatment outcomes by personnel from community based organizations, as this information is usually under the purview of health services. Consideration should be given to revising the signature deliverables or to the approach to monitor this, given this difficulty. Sharing of ideas and experience among PSI staff and implementers across the three countries could be improved. This should be done (1) among key PSI staff and/or other expertise working on research and studies including methods of studies, results and common findings across the different target populations; and (2) among local implementers facing similar challenges, to share local solutions which may prove helpful for others. 6 Table of Contents Executive Summary ................................................................................................................................ 1 Table of Contents .................................................................................................................................... 6 Abbreviations and Short Forms .............................................................................................................. 7 Introduction ............................................................................................................................................. 8 Background ............................................................................................................................................. 8 Methodology and scope of work ............................................................................................................. 9 Findings and recommendations ............................................................................................................ 11 BURMA ........................................................................................................................................... 11 CHINA ............................................................................................................................................. 21 LAO PEOPLE’S DEMOCRATIC REPUBLIC ............................................................................... 30 THAILAND ..................................................................................................................................... 39 Overall CAP-3D project conclusions .................................................................................................... 46 Overall recommendations ..................................................................................................................... 49 To USAID/RDMA ........................................................................................................................... 49 To PSI and implementing partners ................................................................................................... 50 Appendix 1 - Evaluation Scope of Work .............................................................................................. 51 Appendix 2 - Final evaluation design with data collection tools .......................................................... 51 Appendix 3 - Evaluation schedules ....................................................................................................... 51 Appendix 4 - List of people contacted .................................................................................................. 51 Appendix 5 - References ....................................................................................................................... 51 7 Abbreviations and Short Forms AHRN Asian Harm Reduction Network APMG AIDS Projects Management Group ART Antiretroviral treatment, antiretroviral therapy CBO Community based organization CD4 T-helper cell CDC Center for Disease Control, Centers of Disease Control CPP HIV/AIDS Comprehensive Prevention Package for MARPs CXR Chest X-ray DQA Data quality assurance FY Financial year M&E Monitoring and evaluation MIS Management information system MMT Methadone maintenance treatment, methadone maintenance therapy MSM Men-who-have-sex-with-men PWID People who inject drugs SI Strategic information STI Sexually transmitted infection SQH Sun Quality Health TB Tuberculosis TG Transgendered, transgender VCT Voluntary counselling and testing 8 Introduction A midterm evaluation of the USAID Regional Development Mission for Asia (RDMA) Control and Prevention of Three Diseases (CAP-3D) Project was undertaken in Burma, Laos, and Thailand; with a final evaluation conducted in China during a six-week period between June and July 2013. The CAP-3D grant runs from January 15, 2011 through January 14, 2016 with a total estimated $36 million for five years, for the Behavior Change for Infectious Disease Prevention Program (BCC-ID) project, implemented by the consortium led by Population Services International (PSI). Background The goal of the CAP-3D project is to reduce morbidity and mortality related to HIV, TB and malaria in the Greater Mekong Sub-region by increasing an effective regional response - characterized by stronger country ownership - to prevent and mitigate these diseases. For HIV prevention and care, target populations include men-who-have-sex-with-men, transgendered people (mostly transgendered women), female sex workers and their clients, people who inject drugs, and people living with HIV. For tuberculosis control, the target populations include people living with HIV (PLHIV), the elderly, and slum dwellers. For malaria control, target populations focus on rural populations in endemic areas. The overarching strategy for this project is to increase the coverage, quality, and sustainability of a comprehensive package of for HIV prevention, care, and treatment services. Major areas of emphasis are reducing dependence on external donor resources by strengthening local institutions and securing local government buy-in and diversify funding sources. To achieve this, PSI and its partners focus on the following objectives:  IR1: the scale up of the CPP model, shifting increasingly toward provision of technical assistance to local agencies for CPP implementation ;  IR2: evaluation and documentation of CPP models and use of strategic information (SI) to advocate for replication and leveraged funding ; and  IR3: enabling local organizations to scale-up the CPP through provision of technical assistance and capacity building. 9 Methodology and scope of work The evaluation was commissioned by the USAID/RDMA’s Office of Public Health (OPH). The evaluation team comprised of two independent consultants: a Team Leader Evaluation specialist (Jamie Uhrig) and HIV Specialist (Po-Lin Chan). They were joined by Billy Pick (USAID Technical Advisor HIV, Washington) in Burma and China. The purpose of the evaluation was: 1. To analyze the CAP-3D’s project achievements to date in areas related to its performance, ability to achieve critical results, steps made towards sustainability, and its ability to build and maintain critical relationships. This comprised of a multi-country mid-term evaluation for activities in Thailand, Laos and Burma; and an end of project review in China. 2. Make recommendations to strengthen project implementation in countries where CAP-3D will continue to work The evaluation team was tasked with answering the specific following evaluation questions that were included in the Scope of Work (see Appendix 1), as follows: Question 1 - Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? Question 2 - Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? Question 3 - Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? Question 4 - Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? Question 5 - Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? The two consultants adapted the five major questions and a list of sub questions that were included in the scope of work into a matrix and set of questions to be asked of informants at regional level, country level, implementation level, and community beneficiaries. The resulting evaluation tool was disseminated to PSI and consortium partners prior to the country visit and is attached as Appendix 2. The evaluation schedules are attached as Appendix 3 and a List of People Contacted as Appendix 4. The evaluation began with a review of key documents provided by USAID/RDMA and PSI for each country. These included background documents, performance indicators and targets, annual and semi-annual reporting narratives including strategic information documents. The evaluators 10 were briefed by the consortium partners before embarking on the multi-country visit. Data was collected at each country which was verified with existing documents, or other published reports. The evaluative questions were administered at each country according to the level of key informants. For example, the evaluation questionnaire for local implementing partners was asked during discussions with community based organisations in the various countries. The sub questions were asked or questions were asked to elicit an answer to one of the sub questions by one of the evaluators to an informant or a group of informants. Additional focus group discussions with beneficiaries of the CAP-3D services were held in order to verify findings. Over eighty interviews with a wide range of stakeholders were held in four countries. There are limitations to the method used in this evaluation. There was a positive bias inherent in the method. Most of the informants were people who implemented activities that were funded by CAP-3D. In order to confirm or refute findings, the evaluators asked the same questions to multiple informants to triangulate on the findings and included informants who were not directly related to the project activities(such as beneficiaries and community members who were not directly incentivized/hired by PSI or implementing partners, either as staff or volunteers). Language and translation issues were barriers in some in-depth interviews and focus group discussions. Lastly, there was limited time in each country and with each implementing partner for which in-depth evaluation could not be conducted in all areas e.g. monitoring and evaluation indicators and results. 11 Findings and recommendations Key findings are presented for each of the five major questions for each country and recommendations made. Overall conclusions and recommendations follow the country sections. Annotations such as (a), (b) and so on, refer to the sub-questions in the tool. BURMA In Burma, CAP-3D funds activities for HIV, tuberculosis and malaria. Much of the work of the evaluators focused on the first two diseases. Malaria funding went almost exclusively to procurement of bed nets during the financial year (FY) 2012. It is also noted that some of the HIV funding for financial year 2013 was provided by USAID in Yangon. Key Findings Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (a) While FY 2012 reports for key populations reached with prevention activities were under target, performance has improved with the decreasing of constraints on the geographic sites covered and new leadership in place. In HIV, there is a clear prevention focus on the three key populations of men-who-have-sex-with-men, transgender women, and female sex workers. For the first half of financial year 2013, there was 92% achievement of the half year target for key populations reached and 76% achievement for HIV counseling and testing targets. The shortfall in meeting the targets was partly due to conflict in some areas and difficulties in signing a new memorandum of understanding for country operations. There was a vacuum in leadership as both the PSI Country Director and the Director of the TOP Centers staff left and were replaced. Ongoing issues of arrests of sex workers continue however there is no evidence that this is increasing or decreasing, or that it has a significant impact on the CAP-3D program. There has also been a budget decrease from the amount originally expected from USAID and a delay in disbursement. The CAP-3D program in Burma supports the national response. TOP Centers provide a majority of the prevention activities for all three key populations and contribute greatly to the national response: no other organization reaches one sixth of the number of sex workers or men-who￾have-sex-with-men reached by TOP Center activities. However, the evaluation team initially had concerns about duplication and inflation of reported numbers on coverage and reach. There have been recent changes in both the number of townships that are reached and the methods used to count and verify the number of individuals reached for prevention activities. The number of individuals reached had been reported differently in different sites in the past and this led to inaccuracies in the national figures for reach and coverage. This has now been resolved. Although there are three different reporting periods used by PSI, USAID, and the national 12 program, it is now possible to confirm the accuracy of most figures for reach of prevention. With a market share of 80% of commercially available condoms, and distribution of female condoms and sexual lubricant, PSI also contributes to prevention of incident HIV infections. The continued decline of incident infections to below one thousand infections per year among all people identified as ‘men-who-have-sex-with-men’ and sex workers are largely, but not exclusively, due to PSI activities. Maintaining a national aggregate HIV prevalence under ten per cent is also largely due to these activities. (b) It is difficult to gauge how effectively key populations served by CAP-3D are linked to care and treatment. There are at present no methods in the country to track how many people who test seropositive are enrolled in HIV care and treatment services, receive CD4 testing and are begun on antiretroviral treatment. There is no information on HIV-positive persons on ‘waiting lists’ for the limited quota of antiretroviral treatment (ART). The level of adherence to antiretroviral treatment is also unknown. Individuals may be known by drop-in center staff but no records are kept of the number of people who enter care or what their outcomes are. In addition, although there are medical personnel in TOP Centers, their current responsibilities do not include essential HIV and ART care. A few HIV-infected individuals are currently being started on antiretroviral treatment at the Yangon TOP Center with plans for one hundred and fifty persons receiving ART before the end of 2013. This new initiative, where selected TOP centers provide ART care and treatment, leverages on the existing reach of TOP Centers with key population community members and is not funded by CAP-3D. There is a possibility of transferring stable patients on antiretroviral treatment that are currently being treated at the joint National AIDS Program/IUATLD (or The Union) program to the care of the TOP Center in Mandalay. (c) The approach to meeting some targets may not be either effective or efficient. With respect to HIV counseling and testing, each individual peer educator is given a target for the number of individuals that she or he must send to the center for counseling and testing. Although targets may be met, there is a possibility that these are met through repeat testing of individuals already accessing drop-in center services, who undergo testing every six months. The objective of encouraging HIV testing is to get HIV-infected individuals into earlier care, both for their health and to reduce transmission of the virus. Repeat HIV testing alone will have little impact on preventing infections. Currently, HIV counseling and testing is marketed as a prevention activity, rather than an entry to care, and the demand generation for this aspect is weak. Identifying and testing individuals who have never or rarely been tested may have a greater impact through getting people early into care, support, and treatment programs. A similar issue arises with targets set for peer educators who must refer members of key populations for sexually transmitted infection care. CareU co-packaged treatment for both chlamydia and gonorrhea is prescribed; this is sound clinical and public health practice. Syndromic management is usually employed. This approach has been validated to be most effective for men who have urethral symptoms and is less effective for women with vaginal symptoms or people with anal symptoms. Syndromic management is backed up in selected TOP Centers with gram stain microscopy which is specific but not sensitive for gonorrhea and does 13 not detect the usually more prevalent chlamydia.1 Targets may drive relatively ineffective and inefficient routine treatment of asymptomatic individuals which will have little impact on sexually transmitted infections or new HIV infections. Syphilis testing is currently done on all blood samples at the same time as HIV testing. (e) The project has utilized approaches to meet and address the needs of highly vulnerable members of the target populations. The evaluators heard from both community members and peer educators that special efforts were made to reach female sex workers who are just beginning sex work and men-who-have-sex-with-men who have just begun to be in the places where new contacts are being made. There is also a new awareness of the legal instruments being used by police to arrest female sex workers and the mechanisms that individual sex workers can use to defend themselves. However this is variable between TOP Centers in the cities (Mandalay, Yangon) and smaller cities. Among the community members who are defined as ‘men-who￾have-sex-with-men’, peer educators are at present reaching many more beneficiaries who practice receptive anal sex than those who practice insertive sex. (f) In tuberculosis, the focus on case finding based upon segmentation according to populations and geographic locations has been relatively successful. CAP-3D activities have contributed greatly to the national tuberculosis program, with 17% of all case detections and an 85% treatment success rate. Case finding has occurred among the three key prevention populations through testing at TOP Centers, among people living with HIV at TOP Centers, among the general population and people living with HIV at Sun Quality Health clinics with their associated community workers, and among the general population in rural and urban areas of Magwe Region through activities implemented by PACT. There are challenges in producing sputum samples that are not simply saliva when taking samples in the community. People newly diagnosed with tuberculosis were also routinely referred for HIV testing or tested according to new national tuberculosis/HIV guidelines. About one-third of TB suspects and TB patients are tested for HIV through the Sun Quality Health network, increasing from 129 individuals tested for HIV (December 2010) to 11,004 persons tested (2012). This provider initiated counseling and testing program has exceeded its targets with about ten per cent of newly-diagnosed tuberculosis patients testing positive, the same rate as that found during sentinel HIV surveillance among TB patients. Tuberculosis targets are not set except for the Magwe activity ‘TB Heroes’ implemented by PACT where targets have not been met in rural areas, but may be met in an urban one. The total number of people detected to be sputum positive has risen by a thousand over the last two years 1 There is little data on sexually transmitted infections in Myanmar. HIV sentinel surveillance includes syphilis testing. A recent study on 601 male highway coach drivers found prevalence rates of syphilis, gonorrhea, chlamydial infections, and trichomoniasis were 4.8, 4.3, 5.7, and 9.8%, respectively. Aung et al.: Sexually transmitted infections among male highway coach drivers in Myanmar. Available at http://www.tm.mahidol.ac.th/seameo/2013-44-3/10-5493-5.pdf 14 reportedly reaching about 8,000 people with a male to female ratio of 1.45:1. Malaria funding was used to procure and distribute bed nets. It was confirmed that this activity took place but the evaluation team did not visit the sites where bed nets were in use. This activity has also recently initiated and USAID did not plan an evaluation of them. (g) Both Sun Quality Health and TOP Centers provide potential models in antiretroviral treatment service delivery which can provide care for different segments of the population and increase equity of ART access. The Sun Quality tuberculosis model provides services to low and very low income groups and has the potential to function as a ‘ART linked center’ with drug refills for stable antiretroviral treatment patients or a ‘ART prescribing center’ with new patients taking first line medications. Importantly, the Sun Quality model provides an approach to potentially improve rational antiretroviral drug prescription in the private sector. The TOP Center model could be valuable as a community based ART model for key populations. Sustainability is a key consideration in implementing these models, as the government needs to subsidize the costs of drugs or provide drugs in the future. Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? (a) Ensuring sustainability of program approaches remains a challenge. While some progress has been made, there remains a question as to sustainability over the long term. Through involvement in the Technical and Strategy Group of the Country Coordinating Mechanism, PSI Myanmar plays a major role in testing, piloting, and promoting initiatives that lead to changes in practice and policy. Rapid HIV testing is currently being used by PSI implementers. There are indications that community based HIV testing may be an initiative that will be supported by the National AIDS Program in future, although continued advocacy is required. As HIV rates among people newly diagnosed with tuberculosis is currently about ten per cent2 , HIV testing of newly diagnosed tuberculosis patients is the ‘low hanging fruit’ for expanding counseling and testing and PSI is at the forefront of expanding this segment of testing. (b) PSI has worked with PACT and the TOP Center Yangon staff to tailor a specific planning process which may move TOP Centers toward more autonomous status. A major sustainability challenge is determining the future of the TOP Center model for prevention and care for the key populations of men-who-have-sex-with-men, transgendered people, and female sex workers. With a change of high level management of the TOP Center program within PSI and the recent resignation of a community member who had become a manager, the management of TOP centers is evolving. In addition, with half of funding for the centers now coming from the Global Fund, and the increasingly open environment for civil society in the country, the present is a good time for the mature TOP Center model to begin to become autonomous. Expressed needs and pressures from a range of stakeholders - funders, communities served, other institutions serving the three key populations, and governmental institutions - must be balanced. Recognizing that organizational capacity building of civil society organizations is not an 2 Review of the National Tuberculosis Programme, Myanmar, 7-15 November 2011. 15 institutional strength of PSI, the project subcontracted to PACT to support civil society organizations in acquiring and using the skills needed for their development. PSI has initiated the process with a successful human resource management review and plan with the support of PACT. A well-managed stakeholder consultation process is ongoing. There is an opportunity to have some but not all of the TOP Center activities included in the national minimum packages for key populations packages that are being developed by the Principal Recipient for international nongovernmental organizations. Recent allegations of fraud by the subcontracted organization for people living with HIV, Cordial Hands, were handled professionally and with tact. (c) PSI efforts with the Sun Quality Health private practitioner network may represent a possible way to contribute to the national effort to decentralize and improve access to HIV counseling and testing. Nationally, there are plans to decentralize counseling and testing to the grassroots level, including consideration of the use of private sector physicians if they receive training and participate in quality assurance management systems in testing target forecasts. Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? (a) Despite challenges related to funding, partner capacity and USAID administrative changes, the CAP-3D project has been able to balance a wide range of partnerships in a professional manner. During the past ten years, PSI has headed up a Myanmar consortium that has included, at various times, international non-governmental organisations (INGO) such as Save the Children, PACT, RTI, AHRN, and Cordial Hands, a local community based organization. For a variety of reasons, primarily due to USAID funding constraints limiting the consortium partners’ ability to conduct activities, several have either chosen to drop out of the consortium or have had their sub-agreement terminated based upon adjustments to the annual work plans. PACT has remained a partner, albeit in a more limited role. The relationship with PACT appears to be amicable and professional. The partners have worked together to avoid overlap between activities supported by PACT’s TB Heroes and PSI SPH volunteers in villages in Magwe township. There has been limited discussion between the partners regarding differential incentives for the SPH versus TB Heroes. In the case of Cordial Hands, where the organization was accused of fraudulent behavior, PSI responded quickly as the fiduciary agent, informed the appropriate USAID and is currently working closely with USAID/Burma, USAID/RDMA and the regional IG to investigate and proceed to a satisfactory conclusion. The relationship with USAID has become more complicated since the CAP-3D grant was awarded. In July 2012, USAID reestablished its Mission in Burma after a 23-year hiatus. With the advent of the US-Burma partnership for Democracy, Peace and Prosperity, the practical day￾to-day locus of control for USAID activities in-country shifted from the USAID/Regional Development Mission in Bangkok to the local Mission. While the administrative and contractual 16 responsibilities for the CAP-3D agreement will remain with the office in Bangkok for the duration of the project, there is an on-going process where the USAID/Burma is taking more ownership of its portfolio. PSI has worked in an open and transparent way with both missions to minimize any impact on service delivery and performed well in navigating the challenges of responding to two USAID Missions whose relationship is evolving. They have continued to maintain a professional demeanor in addressing their responsibilities for financial and administrative oversight, responding to technical guidance, and transparent and inclusive reporting. (b) Working relationships are cordial with partners and stakeholders. At the national level, PSI and TOP Center management have close working relationships with other stakeholders through various technical working groups. PSI is well regarded as a major key organisation working in prevention, and in strategic information in key populations. A stakeholder reported that staff from TOPS and PSI has been notable for their non-self-interested efforts in the national strategic and planning processes, particularly in the recent Global Fund reviews. At the provincial level, TOP Centers work closely and coordinate with local authorities. (c) There were no apparent advantages or disadvantages to having the three major communicable diseases dealt with in one platform. The only advantage that could be seen was that all three diseases were handled by one institution, so that staff working on the three diseases was co-located and able to meet one another to resolve upcoming issues on a regular basis. Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (a) CAP-3D’s ability to achieve critical results will continue to be constrained by the legal environment for key populations, limited antiretroviral treatment access, and limits to clinical screening and tuberculosis sputum microscopy. Sex work is illegal in Myanmar governed under the Suppression of Prostitution Act of 1949 which was amended in 1998, increasing the penalties for sex work.3 Key informants report that police use condoms as evidence to arrest or fine women, although some peer educators state that they are now able to discuss their HIV outreach work and the fact that they distribute condoms with police and avoid arrest. In Mandalay, volunteer lawyers are in contact with the TOP center to provide legal advice. In Magwe however, there is lack of legal support – in 2012 till mid-2013, 15 sex workers in Magwe have been arrested and imprisoned. Sex between men in Myanmar is punishable by a prison sentence of up to 10 years, although rarely enforced. There are no laws protecting MSM and there are no laws against discrimination on the basis of HIV status. The law does not allow transgender people to change their sex or gender on official documents and records. Laws in Myanmar against ‘public nuisance,’ ‘obscene material,’ and ‘conduct that might affect the morality of an individual, society or the public’ can also be used against MSM and other key affected populations.4 3 Sex work and HIV: Myanmar. Evidence to Action paper. AIDS Datahub, August 2010. 4 MSM country snapshots: Myanmar. AIDS Datahub, December 2012. 17 The influence on TOP Center activities of these legal constraints is variable and dependent on the ability to work effectively with regional key players in authority such as the National AIDS Program and the police. PSI work must be within the terms of the Memorandum of Understanding with the government. This limits the ability to expand TOP services to new areas or emerging hotspots if the townships do not fall within the agreement. Similarly, where hotspots move to other locations, there are issues in responding to these needs. In TB, there are issues with low case detection by clinical screening and sputum microscopy. Of the 58,820 new cases tested in 2012 through SQH, 14% were sputum smear positives. The national TB review 2011 noted that the current definition of TB suspects by cough for two or three weeks can detect only one third of smear positive and one fifth of culture-positive TB cases. This is also reflected in the PSI TB follow up surveys that reimbursement fees usually used for transport costs are increasingly diverted to CXR fees; rather than supporting transport fees which continue to be a barrier to seeking services. Limited antiretroviral access and waiting lists within the national treatment program is a chronic issue. While there are currently discussions led by the National AIDS Program with stakeholders concerning decentralization of governmental antiretroviral treatment delivery to township level concurrent to increasing numbers being treated at government sites, implementation of the plan will take time as capacities are built gradually. There is a draft ‘hospital initiative’ concept note outlining a partnership between nongovernmental and public facilities to increase the number of people on antiretroviral treatment and gradually turn over patients on antiretroviral treatment from nongovernmental to public treatment sites. It is currently being considered by the National AIDS Program. Decentralized ART delivery is being influenced in a limited way through ART pilots delivered at TOP centers as well as Sun Quality Health doctors managing ART. (b) Development of community and peer-led HIV testing is an area where PSI and TOP Centers could possibly take a lead in the future. The HIV national testing policy is evolving rapidly. Informants report recent approvals by government for a policy of one HIV rapid screening by paramedical staff (e.g. midwives, health assistants) with referral to voluntary counseling and testing services for confirmation. HIV testing by community members or peer educators has not been included however key informants indicate that certification and training may provide negotiating space in this area. (c) While there are questions as to achieving critical results at scale, most beneficiaries find TOP Centers provide a safe and respectful place for them to come and appreciate the ease with which they can receive prevention, HIV testing, and STI treatment. Additional services for female sex workers such as cervical cancer screening and treatment of intraepithelial squamous cell lesions has added to beneficiaries’ perception that TOP Centers addresses a broader range of their concerns rather than being limited to HIV prevention services. However, this is not consistent for the range of issues affecting the community. Transgender informants report that hormonal treatment was a major concern. Most buy hormones at local pharmacies without physician care, and there were reports of side effects. TOP medical personnel are not involved and have little knowledge in this area. In a (public) health system where transgender persons’ health is not a priority, TOP services could fill the gap in providing better specific medical support for this aspect and reduce the misuse/inappropriate use of hormones through better treatment literacy in the transgender community. 18 Similarly, TOP centers run Saturday Clubs where HIV positive individuals receive peer support, counseling and education, which are conducted with little medical personnel support. The current medical services model within TOP Centers has not included care and follow-up of HIV–positive individuals and is a gap area which should be addressed to ensure that key populations’ access, remain in care and treatment, and are provided with continued peer support. Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (a) The degree to which monitoring and evaluation activities inform planning and implementation varies. PSI has a monitoring and evaluation plan for both TOP Centers and Sun Quality Health which includes data quality assurance (DQA). Supervision visits have been done a few times in Mandalay but are not regular. Dissemination of the monitoring and evaluation plan to key staff working on data entry/collection has not been done. Nonetheless, staff who were responsible for data collection reported that they understood the importance of data collection. Systems to collect data are established in the TOP Mandalay as outlined below:  Transmission of routine monitoring and evaluation (M&E) information: original paper data sheets from peer educators of the TOP network are sent to the PSI office in Yangon weekly or monthly, and entered individually into the central electronic database. Storage of paper documents is increasingly a problem. The Mandalay TOP center has exceptionally capable staff members who perform site-based data entry and compilation of data from other three centers. A recent Global Fund LFA audit on PSI M&E reports showed that data entry errors were within the margin of ten per cent.  The PSI M&E unit does not conduct DQA checks at TOP centers. This is the responsibility of the TOP drop-in center management team. Discussions with the TOP Mandalay center indicate that internal data quality assurance checks are not being regularly implemented. PSI states that they have a 2013 plan to strengthen the monitoring and evaluation reporting capacity at peer educator level, as there has been discrepancy of data noted between the peer education diary and for the aggregate reports. During the field visits, a brief DQA check of source registers corresponded with the monthly reports available at the Yangon national AIDS program regional office.  The overall design of monitoring and evaluation formats showed that personnel keep many different forms at each level and there has been substantial work in trying to understand the cohort of key populations accessing services (e.g. new/fresh/repeat clients; number of contacts vs. number of individuals; specific services e.g. counseling alone or counseling and HIV testing; longitudinal record of individuals and their history of HIV testing). The system of recording in the registers/logbooks show an attempt to reduce mistakes in tally/counts by using a specific system of daily counts and monthly running tallies. 19  CAP-3D does not fund research in PSI Burma – this is done with Gates funding. TRAC and FoQus5 surveys have been done to examine quality of care in female sex worker and the Sun Quality Health network. Gates funds also specific research in other areas e.g. pediatric diarrhea interventions, Sun Quality Health –TB network etc. There are gaps in national strategic information. There is a need to advocate for second generation surveillance such as integrated bio-behavioural surveillance (IBBS) in key populations, and for qualitative studies in sexual behaviors/networking/identity. The latter study is being planned. A review of the national HIV/STI surveillance system has recently been conducted by the World Health Organization (WHO) and the Ministry of Health; however this report is not yet available. It will be important to see where PSI can play its part in contributing to the overall strategic information of the national program. (b) There is a need to improve the use of routine data from programs (TOP or SQH) as there is a very rich database on TOP outcomes including detailed individual longitudinal data on HIV testing (at center level). Currently the electronic M&E information system at PSI Yangon is used mainly to report numbers to USAID. This could be better analyzed to understand specific detailed outcomes of current programming such as syphilis and HIV prevalence ratios in those coming in to the drop-in center for testing; or incident/new infections within those coming for HIV testing at drop-in centers, within the limitations and biases of the available data. ART retention in the TOP and Sun Quality Health centers will need to be analyzed as a critical outcome and compared to the national aggregate retention level (such as ART retention at 12 months of 87% 6 ). (c) At present, there is lack of regional sharing of knowledge based and specificities to research in key populations. The CAP-3D funds PSI for work in key populations and PSI contributes to research and methods towards the national program in prevention. Thus, there is need to share technical research methods across the PSI offices for better/improved/adapted methods for research in key populations in this region, although there may be contextual variations among countries One example would be the PSI Laos transgender women surveys using self-collected anal swabs which could be a useful approach to navigate the difficulties in doing bio-behavioral surveys in men-who-have sex-with-men and transgendered people in Burma, where clients have difficulty in “undressing for medical checkup”. Burma: Recommendations 1. Continue advocacy through evidence generated by implementation, and engagement with government and other stakeholders in technical and programmatic issues. The overall national context for health and HIV programming is rapidly evolving. While there may be current challenges in negotiating space in conduct of specific tasks, it is essential that USAID and PSI maintain close engagement and negotiation to advance better prevention programming and access to HIV care for key populations. 5 TRAC and FoQus are PSI research approaches. See: http://www.psi.org/resources/research-metrics/research￾approaches 6 Global AIDS Response Progress Report, Myanmar: 2012 20 2. Female sex workers, men-who-have-sex-with-men, and transgendered people with the most sexual partners should be prioritized for activities. As TOP Centers begin to achieve autonomy, there are clear opportunities to increase the efficiency and effectiveness of their HIV prevention and care activities. This may require having outreach workers to continue to move outside their traditional coverage areas to reach more individuals who have not been reached before, and making greater efforts to reach female sex workers who are just beginning work or men-who-have-sex-with-men/transgendered people who recently began to be sexually active. It may also require reducing program activities in places with small numbers of people in key populations and reducing the frequency of repeat HIV testing. TOP Center staff should be involved in the national development of the minimum package of prevention activities. 3. PSI should conduct research on sex worker migration to determine if many sex workers begin sex work in a few key locations. These locations could then be the focus of intensified action to reach sex workers who are just beginning sex work. 4. HIV testing and TB screening activities should be targeted to reach people who are at greater risk than the average. The effectiveness and efficiency of both HIV testing or TB testing activities should be measured against the 'background' prevalence in the population tested. For instance, if the prevalence of HIV in a population is five per cent, then voluntary testing and counseling should aim to test people to produce a rate higher than five per cent. Detailed data is available at the drop-in center for these analyses. 5. In order to better advocate for equitable access to treatment among key populations, data on risk group should be collected, collated and reported. It is currently unknown whether there is equitable access to treatment by the key populations of female sex workers, men-who-have￾sex-with-men, transgendered people, and people who inject drugs. ‘Risk group’ is noted on the forms used for antiretroviral treatment and could possibly be analysed for this purpose. 6. Because tuberculosis incidence and prevalence worldwide in men is double the rate in women, more efforts targeting men as recipients of CAP-3D TB services are needed. Sun Quality Health clinic data showed an equal ratio of men versus women in TB care, suggesting that women access health services more frequently than men, or men do not have adequate access or both. This aspect needs evaluating. 7. PSI should partner with local organizations in efforts toward creating an enabling environment through work on legal, policy, and regulatory issues. Since this is not a core PSI strength, USAID should fund this activity under other work on governance and regulatory environment. 8. The role of medical personnel at TOP Centers could be expanded to provide basic HIV care and to address some of the medical concerns of transgendered people including support for correct use of hormonal therapy. Many transgendered people take oral and injected hormonal treatment. Training on appropriate management of hormonal treatment for transgendered people including side effects and interactions with antiretroviral therapy could be offered to service providers working in TOP centers. As well, medical personnel should take an active role in support basic HIV and ART care for HIV-infected key populations. 21 9. As USAID/Burma and USAID/RDMA continue to work out Mission roles and responsibilities, they should ensure a transparent, cooperative process so that implementation of activities by PSI and partners continues throughout the life of the agreement. CHINA Key Findings Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (a) The CAP-3D China program has balanced their approaches to prevention, care and treatment going further than other country programs in 1) improving models of delivery through local partners, 2) documenting effectiveness, and 3) advocating for these models to become the regional standard of quality HIV prevention programming for most at-risk populations, implemented by strong local institutions and sustained through local governments and diverse funding sources. The CAP-3D models of service delivery have been constantly refined through the provision of services through both PSI and contracting sub￾partners, and providing technical assistance in two provinces of Yunnan and Guangxi. Studies documenting behavior change correlating with program activities have been conducted and advocacy has been done directly by PSI and through partner organizations. The project has worked within the existing national and local government structures to improve models for service delivery for key populations. This is apparent in the project’s work with people who inject drugs (PWID). Community based rehabilitation for PWID began with a decision by Beijing in 2008. In the last five years, the detention period for people with drug dependence to be held in mandatory drug detention centers has slowly been decreasing from two or more years to one year. This is to be followed by a one to three-year period of community based treatment or community based rehabilitation. This can perhaps best be understood as a period of parole in which a person who was detained for drug dependence lives in the community under the supervision of the local administrative authority. He or she signs a contract with the authority agreeing to periodic and spot urine testing for drug use and periodic HIV testing. An arrest for drug use can lead to re-detention in the mandatory drug detention center and repeated positive urine tests can also lead to the same outcome. Re-detention, however, occurs rarely. Methadone maintenance treatment is often a component of community based treatment or community based rehabilitation but does not have to be a part of either Community based rehabilitation programs offer support to people with drug dependence in getting social assistance in the form of income supplements and health care benefits. Handicraft work in a community center, outside job placement assistance and driver’s license reinstatement are also provided to some beneficiaries. An unbroken record of negative urine tests at the end of the contract period may lead to the name of the drug dependent person being removed from the list of people with drug dependence that is maintained by the police and local authorities. The health outcome of community based rehabilitation is long term abstinence from the use of 22 illegal drugs. This is usually heroin. There are other non-health outcomes that can be measured such as wealth or income, employment, and the exercise of civil rights. Some of these may also have an impact on health outcomes. PSI supports community based rehabilitation through employing peer educators - most of whom are currently undergoing community based rehabilitation. Peer educators reach out to drug dependent people in methadone programs, as well as in the community to provide them with harm reduction messages and support them in getting the benefits offered by community based rehabilitation. PSI has also begun a social franchised, key population-friendly primary care and laboratory testing service through four out of fifty-six governmental community health centers at Daguan, Lianmeng, Xiyuanliangyuan, and Jinma in Kunming, called the Clinical Health Network in 2012. HIV, sexually transmitted infections, TB, viral hepatitis screening and counseling are provided using a voucher system for payment. At the Daguan clinic, only one hundred and six individuals, the majority of whom were PWID, have used this service since it was opened in May 2011. (b) Determining the extent of success in achievement of program results for the CAP-3D China program is complicated by the fact that the timeline for planned activities was severely curtailed. The planned program activities were designed to be implemented over five years by a number of international NGOs and local government and civil society organizations serving one or all three major key populations (including PLHIV) in dispersed locations in two provinces. Although PSI and other INGOs do not provide direct services to key populations, their contributions were designed to support local organizations that serve different key populations. Project activities were implemented for only two and a half years from January 2011 to September 2013, instead of over the initially planned five-year period. Performance as measured by meeting targets was low during the first year of implementation as new procedures and working relationships were established. Targets were reached or exceeded in the second year of implementation. This applies to the number of people belonging to key populations reached with both prevention activities and reached with HIV counseling and testing. In the first half of the final year of implementation, there were decreases in service delivery as activities were phased out or turned over to local organizations so that they could no longer be reported on as direct results of PSI activities. Despite these challenges, there were scientifically sound methods used to measure behavior change. The Routine Behavior Tracking surveys were used to monitor the behavior of key populations within the coverage of the program. These surveys provided sound evidence of a correlation between safer behavior and more contact with services provided by local organizations among all three key populations of people who inject drugs, female sex workers, and men-who-have-sex-with-men. In the absence of well-defined population size estimates, it is not possible to accurately measure program coverage among these three populations. Local organizations had their own methods of determining reach and coverage and report that they reach over half of the estimated people at risk in their coverage areas. The use of ‘individuals’ and ‘person-times’ for reach in the China program monitoring and evaluation system is a useful method of determining reach. 23 Change in HIV prevalence as an impact of the program was not measured. Even if it had been, it would only give an indication of the trajectory of the epidemic in Yunnan and Guangxi provinces and not for the national population of China. (c) The impact of several activities were either difficult to measure with current methodologies or the indicators used were not adequate to measure the desired results. For example, there are two innovative activities for which impact was difficult to measure. The number of injections of naloxone given or people administered naloxone can be counted, but as there is no scale for measuring life threatening opiate overdose 7 , there is no standard method of determining the number of lives saved. ‘Break the Cycle’ of injecting is another program activity for which the impact is extremely difficult to measure. If the goal is to prevent or delay the initiation of injecting among people who already use heroin, then an expensive and technically challenging cohort study would be necessary to demonstrate effectiveness. Nonetheless, both these programs are planned for evaluation before phase out in September 2013. In addition, the number of people reached with sexually transmitted infection treatment is not a useful measure of program performance for people who inject drugs. HIV testing targets are not a good measure of program success, particularly in light of the decreasing incidence of injecting drug use compared to other drug use. People who inject drugs are often taking methadone maintenance treatment and the seronegatives among them undergo compulsory HIV testing every three or six months. Female sex workers undergo compulsory HIV testing every six months. This compulsory testing reduces both the need and the motivation for voluntary counseling and testing. A related issue that may affect the measurement of both performance and outcome is the fact that most program beneficiaries are currently taking methadone maintenance treatment. Program implementers call people who are not taking methadone ‘hidden’ drug users. Repeatedly contacting the same beneficiaries in methadone programs may lead to different outcomes than reaching new beneficiaries or people who have left methadone programs prematurely. If changes in behavior are measured, they should be measured both in people who are taking methadone and those who are not, as the latter population is much more likely to be at risk of HIV infection. There are similar issues with the two community based organization reaching female sex workers and men-who-have-sex-with-men. Given that there is no methadone or community￾based rehabilitation program holding them in their home places as is the case with PWID, both populations tend to be more mobile. This mobility, with the addition of newly active men-who￾have-sex-with-men and recently initiated female sex workers leads to more ‘new’ beneficiaries to be reached at all times. There was little program activity directed at the need for maintaining service access continuity for sex workers and MSM who moved from one city to another. Similarly, although there is anecdotal evidence that the activities of the PWID peer educators help provide services to beneficiaries, the evaluation mechanism was not designed to collect data 7 Not all untreated opioid overdoses are fatal, thus some of the persons with reported overdose reversals may likely would have survived without naloxone administration. 24 on peer educator activities to show improvement in health outcomes; or whether the interaction between beneficiaries and the peer educators leads to a more effective abstinence-based program (rather than a more efficient parole program). The value added by the peer education program with respect to positive health outcomes such as longer term abstinence or longer term adherence to methadone has not been measured. Finally, in a related issue, the use of data to follow people who test seropositive into the HIV care system, receive CD4 testing and are begun on antiretroviral treatment is inadequate. Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? (a) Despite the constricted time line, most local organizations have received funding to continue some or all of their activities. It is in the field of sustainability that many lessons can be learned from the experience of the last two years. It was shortly after the grant was beginning to be implemented that a decision was made to terminate the grant in 2013. This led to strained relationships between PSI and some implementing partners and between implementing partners such as PACT and their partner organizations. Limited funding for some activities can be and is being provided to a few implementing partners through local governmental institutions, such as the local Centers for Disease Control (CDC). A major opportunity for future funding is for governmental funds to flow through the social service outsourcing (SSO) initiative from the central, provincial or local government to registered NGOs and affiliated CBOs. This program has not yet been completely rolled out and the criteria for funding are not yet clear. What is clear is that the amount of funding per organization will not be large and staff and program cost (apart from activities) may not be included. Organizations that have a sponsoring management organization in order to register fully with the government have a greatly increased chance to receive funding. A synopsis of current activities and future plans by implementing organizations below examines the range of sustainable models used in the China program: The Yunnan Association of STI and AIDS Prevention and Control (SAA) is a government oriented nongovernmental organization (GONGO) that is increasingly providing organizational capacity building, and technical assistance and capacity building to local community based organizations. It is a registered organization. There is a national counterpart organization in Beijing that may be a source of future funding and the organization is eligible for social service outsourcing funding. There is also the possibility of fundraising from other sources. The Association could be a management agency for local community based organizations. Hu Xiang Hao is a community program implemented by PSI. It has provided services for several years for people who inject drugs in Kunming through a drop-in center and outreach to people in the community and people taking methadone maintenance treatment. Two newer initiatives are methadone rescue through staff in the drop-in center notified by mobile phone and a program encouraging people who inject drugs to discourage people from beginning to inject (‘Break the cycle’ of injecting). Although limited funding has been offered by the government, high recurrent costs make it difficult for the staff to continue programs, and the staff have decided to 25 stop operations. Some of them may find work in other organizations. There are three other community organizations in Kunming that have received technical assistance in organizational capacity but were not supported with funding for program implementation: Rainbow Sky is a community based organization reaching men-who-have-sex￾with-men with prevention messages and has been successfully getting more men tested for HIV. Daytop Drug Rehabilitation and Recovery Center is a long-established, fee-for-service inpatient drug recovery center using a therapeutic community model. There is also a methadone maintenance and treatment (MMT) clinic on-site. Almost all patients are HIV tested on entry and at regular intervals throughout the therapeutic program. Firefly runs a community based HIV prevention project for female sex workers with high coverage in Xi Shan district. Combining with three other CBOs serving MSM, PWID and PLHIV, it has recently been registered as a non-governmental organization renamed the ‘Xi Shan District Healthcare Promotion Association’. There is regular mandatory testing of sex workers in this district. All three organizations have been working with the Yunnan Association of STI and AIDS Prevention and Control and PACT to build institutional capacity and have secured future funding for some of their activities from their local centers for disease control. PSI has begun a social franchised, key population-friendly primary clinical care and laboratory testing service through the governmental community health centers at Daguan, Lianmeng, Xiyuanliangyuan, and Jinma in Kunming, called the Clinical Health Network. HIV, STI, TB, and viral hepatitis screening and counseling are provided using a voucher system for payment. How the participating doctor was chosen could not be determined by the evaluation team. The city health bureau will partly fund continuation of this service, but there is a possibility that users will be asked to co-pay. In view of the 4.1 per cent two-year HIV incidence among men-who-have-sex-with-men in Yunnan, a provincial MSM Technical Working Group has been funded for meetings and this funding will now be provided by the Yunnan provincial government. In Honghe prefecture south of Kunming, PSI helped to establish the Sisterhood Health Home in Mengzi as a community based organization for female sex worker prevention and care with high coverage of activities. Most female sex workers are now routinely HIV tested twice a year and about half of the sex workers who tested positive on the last round of testing are now provided with antiretroviral treatment. PSI provided technical and financial support to the organization until it was transitioned to the Mengzi CDC in 2012. The local CDC provides space for the drop-in center and office, management support and funding albeit at reduced levels. Kangxin Home is a peer-led community based organization providing prevention services to people who inject drugs in Mengzi. PSI provided technical assistance and financial support to promote community based activities. Naloxone rescue has been successfully piloted by this organization and doses are left with the families of people who have overdosed twice. Kangxin will be funded by the Mengzi CDC, similar to the Sisterhood Health Home. Poplar Tree is a community based organization providing prevention services for people who inject drugs in Gejiu. There are plans to pursue continued funding from local health authorities, but no firm commitments have yet been made. 26 A representative of a community based group, the Dali Municipal HIV/AIDS Health Promotion Association, travelled from Dali to Kunming to meet the evaluation team. This organization works in a hospital in Dali to support people living with HIV. The Yunnan Association of STI and AIDS Prevention and Control mediated the linking of this organization with the provincial authorities to sponsor this organization. It is now registered as a social association. There is also now a board of directors, including government officials, so independence and autonomy will need to be carefully maintained. In Nanning in Guangxi, the MSM Technical Working Group was supported by PSI to provide technical assistance and capacity building trainings to twenty local community based organizations with the Guangxi and Nanning centers for disease control. The coordination meetings are now supported by the Guangxi Center for Disease Control. Xin Cheng Xin and Shuguangjiayuan are two small community based organizations for people who inject drugs. PSI provided them with small grants, technical assistance and training in capacity building. Xin Cheng Xin has already mobilized small grants from the district Red Cross and police for on-going activities. Both CBOs plan to register and apply for social service outsourcing funds as well as funding from their local Narcotics Control Committees. In Nanning, the Wangzhou Community Rehabilitation Center is one example of a community based rehabilitation mentioned above. Peer educators from this center sometimes visit the catchment areas of some but not all community based rehabilitation centers in Nanning. It is hoped that the center can hire two or three peer educators through social service outsourcing funding. In Luzhai, the Chengxi Community Rehabilitation Center or Yuxin Home follows the same model. Local governmental funding for the peer educators is hoped for. (b) Both PACT and RTI played vital roles in helping local partners and government achieve sustainable results. Working quickly and beyond their terms of reference, PACT in China worked with the Yunnan Association of STI and AIDS Prevention and Control to support them in their plan to become a leading technical and organizational development assistance provider in Yunnan. Better performance by this organization was monitored using adapted PACT tools. In addition, there is evidence to show that SAA and PACT are working to ensure long term sustainability of organization capacities, including the community trainer network initiative and one of the PACT staff will be hired by SAA after phase out of USAID funding. RTI also provided much needed technical assistance by providing background papers on the context and direction of potential future funding through the social services outsourcing model. Question 3 - Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? (a) The manner in which the CAP-3D project was organized, operated and communicated with stakeholders was challenged by the unexpected reductions in funding and earlier phase out. This resulted in initial difficulties in engaging partners and government, but these challenges were successfully resolved by project end. There was a shift in relationships from the beginning of the project to the end. Soon after the Cooperative Agreement was awarded, the reductions in funding and earlier phase out of the project were announced to partners. This led to confusion and consternation among partners, particularly those in 27 government and GONGOs like SAA. The PSI and PACT staff successfully negotiated the difficulties presented by USAID’s decision to cease activities in China with a great deal of professionalism and managed to re-engage local partners to achieve successful outcomes given the constraints. PSI turned over implementation of all activities to local partners with the exception of Hu Xiang Hao. This is the only organization that is stopping operations when funding runs out. The shift from PSI implementation to local implementation was handled professionally by both PSI and local implementers. (b) In both Yunnan and Guangxi, PSI and partners (PACT and RTI) had close coordination and supportive relationships with the broad range of local implementing partners, as well as government stakeholders. Key informants reported consistently of the high quality of technical assistance and knowledge sharing, useful for improving local programming. Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (a) (b) While the national level may not have been influenced by CAP-3D program activities, Yunnan and Guangxi HIV prevention and care practice was certainly positively influenced by project activities to the project end. The program in China is different than the programs in the three other countries as the main mode of transmission is still blood-borne through the sharing of non-sterile injecting equipment. The program adapted well to this, supporting mostly prevention and care and support for people who inject drugs while still providing resources for organizations serving men-who-have-sex-with-men and female sex workers. (c) With respect to achieving critical results at scale for the most vulnerable populations, there are two concerns for the future of PSI-influenced programming. Firstly, community based organizations serving key populations will continue to be influenced by the strong governmental emphasis on methadone and mandatory testing. The program for people who inject drugs will increasingly focus on people taking methadone. Although people who are taking methadone may still inject drugs and so may still be at risk, the MMT program has been maturing – dosage levels have steadily increased and this may lead to greater retention in MMT. Thus, there is a danger of losing focus on the people who are not engaged in MMT and inject drugs much more often in the community. Secondly, there is a risk that local nongovernmental organizations or community based organizations that are supervised directly by governmental institutions may be co-opted and lose focus on the most vulnerable within the people who inject drugs to serve the easiest to work with, or the easiest to meet targets. Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (a) (b) Despite the fact that CAP-3D/ China has been in a transition and phase-out period since 2011, the program has conducted the relevant studies to inform programming and 28 built capacity in strategic information with local partners (CBOs and government partners). Dissemination and use of strategic information have been well utilized in further developing interventions to address weaknesses in programming. PSI China has a strong monitoring and evaluation system using a web-based MIS system used to collect and track process indicators. A comprehensive DQA evaluation was conducted by USAID in March 2013 with a good overall rating for PSI’s data management systems and data processes. The M&E system was not evaluated in detail as this was an end of project review. Since 2010, PSI has conducted regular operations research studies for baseline, exploratory, Routine Behavioral Tracking (RBT) and FoQus studies in men-who-have-sex-with-men, female sex workers and people who inject drugs in Yunnan and Guangxi. The studies inform program design and implementation, and are shared with government through various platforms including the MSM technical working groups at provincial level, dissemination meetings, local and national conferences. Nine abstracts have been written for various local and international conferences to disseminate knowledge by local staff. Research findings were disseminated to local partners who could discuss during the review visits, the detailed key findings and rationale for changing or tailoring practice and interventions. Examples include the pilot ‘break the cycle program of injecting’ where the PWID RBT survey of 2010 showed that most injecting drug users were initiated into injecting by another injector; and modifying the program education interventions based on the results of the 2012 RBT survey which showed that over half of PWID still shared injecting equipment while most (more than 90%) do not share needles/syringe; had low consistent condom use with regular partners and over half of PWID did not know their HIV status. In general, RBT studies examining program impact showed that individuals who had exposure to the USAID-PSI interventions where significantly more likely to have used services. PWID individuals were significantly more likely to use MMT and accept HIV testing in the last one year. Similarly, MSM who had interventions through drop-in centers and outreach were more likely to have had an HIV test and consistently use condoms (RBT 2010). Female sex workers who had visited drop-in centers at least once or more times, and those with contact through outreach were more likely to have had STI and HIV testing and consistent condom use with regular partners in the past one month; compared to sex workers who had not received interventions (RBT 2010). Excellent efforts by PSI and PACT are being made to support an extensive archive of documents in SAA, spanning the whole period of USAID funding in China. The archive will be made available to other organisations through the web. (c) PACT and PSI have provided excellent capacity building and transfer of skills to SAA and SAA-supported CBOs in the last two years which include monitoring and evaluation. PSI has conducted multiple trainings in research methods and implementation of studies at national level (UNFPA funded national level behavioral tracking survey training in Beijing), provincial and local level to CBOs and government partners. The strategic information area has been consistently cited by all key informants as being essential technical assistance to building knowledge and capacities at local level. Dissemination and use of data have been done particularly well in evolving interventions to address weaknesses in programming. 29 China: Recommendations As this is an end-of-project evaluation, there are no recommendations to improve programming. However, there are several considerations for possible lessons learned which the Mission should take into consideration when considering the wind-down of similar activities in other countries, as well as how to continue to engage Chinese stakeholders on HIV issues :  The Mission should consider a phase-out period in which no new innovations or activities are undertaken as this may help to ensure that program activities are continued at a more secure and comprehensive level.  Before interventions such as the Wangzhou model of using peer educators in community based rehabilitation for PWID are replicated, it is sound public health practice to determine if the peer educators added value to the community based rehabilitation model by producing better health outcomes among people who leave the program at the end of their time in it. They can be compared to people who leave the program without peer educators being involved. This kind of outcome study was of interest to the Nanning city health and drug control leaders, but would require technical assistance.  While no USAID funding can be used to support activities in China directly, the Mission should consider how to include Chinese organizations, both civil society and government, to participate in future USAID-funded regional technical assistance activities related to evaluation of implemented activities, organizational capacity building, technical updates as well as other areas. 30 LAO PEOPLE’S DEMOCRATIC REPUBLIC Key Findings Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (a) The CAP-3D program in Lao PDR is contributing to national HIV program impact as measured by HIV prevalence. By implementing a well-established condom social marketing program distributing over 2.5 million condoms and 60,000 sachets of lubricant in six months ending in March 2013 and contributing to a total market approach for condoms, PSI in Laos has contributed and will continue to contribute to the national goal of low HIV prevalence among the adult population. HIV prevalence among the target group of transgender people has most recently been measured at 7% among those who sell sex and 1% among those who do not, with an aggregate prevalence of 3.1%.8 This is not a significant decrease since HIV prevalence was last measured three years ago. With a survey among men-who-have-sex-with-men last conducted in 2007 in Vientiane demonstrating an HIV prevalence of 5.6% 9 , Laos may meet the criteria for a concentrated HIV epidemic. HIV case finding has recently been elevated in importance in the work of drop-in centers run by PSI Laos and its partners. Between October 2012 and March 2013, over three hundred transgender people, their sexual partners, and men-who-have-sex-with-men were given the service of voluntary counseling and testing. Three have tested positive, reflecting the lower limit of prevalence in this population. One hundred and fifty female sex workers also underwent voluntary counseling and testing and one tested positive. Tuberculosis case finding has been undertaken among people living with HIV and their families who are reached by volunteers of the Lao Red Cross in two cities. A few sputum positive cases have been found, but notably no suspects have been identified among the 122 people living with HIV, most of whom are taking antiretroviral treatment and who have a smaller risk of active tuberculosis than those who have not yet begun treatment. (b) PSI activities will have limited impact in decreasing the high prevalence of sexually transmitted infections among transgender people, and cannot be determined for men-who-have￾sex-with-men because of the unknown prevalence. Measuring the achievement of results with respect to sexually transmitted infections is much more challenging than decreasing HIV transmission. Bacterial sexually transmitted infections are more transmissible than HIV and control of them requires many more inputs than condom distribution and case management of symptomatic cases. Among transgender people, chlamydia prevalence stands at nearly 40% and is not decreasing. Gonorrhea infection & coinfection with both of these organisms is 8 Results of Second Round HIV/STI Prevalence and Behavioral Tracking Survey among Male-to-Female Transgenders in Vientiane Capital and Savannakhet Lao PDR, 2012, Slide 19 9 Sheridan et al, HIV prevalence and risk behavior among men-who-have-sex-with-men in Vientiane Capital, Lao People’s Democratic Republic, 2007 - AIDS DOI:10.1097/QAD.0b013e32831ef510 2009 31 increasing.10 There is also little chance of national targets for sexually transmitted infections being met. A behavioral outcome among transgendered people was measured by consistent use of condoms and lubricant in the last month. The only statistically significant result was that consistent condom use increased slightly over the two years before the last survey performed with regular partners but not with other partners.11 Members of key populations reached met targets during the last year fully reported (FY 2012) but both numbers of target population members reached and VCT figures are under target for the first six months of this financial year 2012 to 2013. (c) Inadequate consideration has been given to data collection by program implementers constraining the ability to determine the extent to which prevention, care and treatment results for the target populations has been achieved. Recently, five ‘signature deliverables’ have been identified12 for project implementers to deliver: (i) Increased case finding per contact; (ii) Reduced cost per case identified; (iii) Increased CD4 cell count on entry to care; (iv) Improved retention in prevention, care, and treatment; and, (v) Improved treatment adherence. Except for reporting on case finding of HIV infection, reporting on these deliverables has not yet begun. The final three are almost exclusively related to the care, support and treatment part of the comprehensive package of for HIV prevention, care, and treatment services. This part requires getting information not only from the individual, but also from health services or ART clinics and will be challenging for community based organizations. Monitoring of performance for these deliverables will be a challenge. Data collection is also compromised by other factors. For example, outreach workers, peer leaders, peer promotion team members, and peer educators (all called peer educators for the purpose of this report) are involved in HIV prevention activities in four cities among one or more of transgendered people, their partners, and men-who-have-sex-with-men. A ‘contact’ is not well defined by implementers and population estimates are guesstimates so that reach and coverage are both difficult to measure. The sexual partners of- and transgender people are currently called ‘men-who-have-sex-with￾men’ by some program implementers. This is not common practice in other countries. Transgender people served by the program are transgendered women, not men. Many of their sexual partners are men who identify themselves as heterosexual and are not being served by the program. Most men-who-have-sex-with-men met by the evaluators self-identify as gay. There is a need to clarify the target population in terminology in order to support better programmatic targeting.13 (d) Results for a number of innovations related to condom/lube social marketing, STI treatment and peer education outreach incentives have been mixed. New ‘Number 1’ grape- 10 Results of Second Round HIV/STI Prevalence and Behavioral Tracking Survey among Male-to-Female Transgenders in Vientiane Capital and Savannakhet Lao PDR, 2012, Slide 20 11 Ibid, slide 25 12 Population Services International, CAP-3D Regional Component, FY2013 Semi-Annual Performance Report, (October 1, 2012 – March 31, 2013) 13 WHO, UNDP, UNAIDS, APTN. Joint technical brief: HIV, sexually transmitted infections and other health needs among transgender people in Asia and the Pacific. Manila: 2013. 32 scented condoms with similarly-scented lubricant sachets in the same package are well known by transgendered beneficiaries of the program. Small tubes of the same lubricant are also now available in drop-in centers. Sales are just beginning so it is too early to assess the impact of this product on the sexual health of transgendered people and men-who-have-sex-with-men. In peer education efforts, the total number of transgendered people or men-who-have-sex-with￾men who are reached by any one of three educators at one time is usually capped at one peer to sixteen clients, limiting both reach and coverage of peer education. While this appears to happen only in centers co-funded by the Global Fund grant, giving peer educators a maximum number of people to reach limits natural networking. Peers are mobile and a common complaint of peer educators is that they are lost to follow up. Special efforts to reach transgendered sex workers, a subpopulation with much higher HIV prevalence, are beginning to be made. Outreach is being informally done through Facebook by some peer educators in their free time. The sexually transmitted infection treatment kits ‘1 STOP’ provide international quality treatment for two common bacterial sexually transmitted infections with high prevalence among transgendered people, men-who-have-sex-with-men, and female sex workers. Sexually transmitted infection prevalence as measured by anal chlamydia among the transgendered population is as high as any recorded in the world and many infections are asymptomatic. Transgendered people with anal symptoms and men-who-have-sex-with-men with anogenital symptoms are usually seen by a doctor in one of the drop-in centers or referred to a local sexually transmitted infection care provider. Laboratory facilities with sensitive and specific methods to diagnose chlamydia and gonorrhea are rarely used. Testing and treatment for syphilis is not freely available. Syphilis prevalence is high among transgendered people in other areas in the region.14 An opportunity to test for and treat this sexually transmitted infection is being lost. While ‘1 STOP’ sexually transmitted infection treatment kits provide gold standard treatment for chlamydia and gonorrhea infections, are prescribed by doctors in the drop-in centers and have been available in pharmacies, challenges to uptake remain. These kits are not currently used for asymptomatic infections and do not adequately treat the usually asymptomatic sexually transmitted infection of syphilis. There is currently a six month national stock out but PSI has plans to purchase more kits this year. A new set of financial incentives to peer educators to bring in more of their peers to reach recently-increased targets have been instituted in the three drop-in centers run by PSI Laos. Peer educators are paid a bonus for each person they bring to the center who tests positive. The outcome of this change in management has not yet been determined. Almost all informants stated that the benefit of testing was reassurance that the person tested was negative. Evidence from a Cochrane meta-analysis of voluntary counseling and testing demonstrates that a negative result has almost no influence on behavior.15 Testing should only be marketed as the gateway to HIV care, not as an effective prevention activity. 14 Republic of Indonesia Country Report on the Follow up to the Declaration of Commitment On HIV/AIDS (UNGASS 2010) Reporting Period 2008 – 2009. 15 Fonner VA, Denison J, Kennedy CE, et al. Voluntary counseling and testing (VCT) for changing HIV related risk behavior in developing countries (Review), Cochrane Database Syst Rev. 2012 Sep 12;9:CD001224 33 Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? (a) CAP-3D funded activities have influenced programming for the overall national HIV response. Quarterly meetings are convened by the Ministry of Health for Ministry officials, USAID staff, PSI staff, and partners to discuss issues of concern. Experience with the models developed by PSI and its partners in implementing CAP-3D were valuable in working with other stakeholders in developing renewal documents so that Global Fund resources could continue to be used for HIV prevention and care in Laos. This includes funding for PSI activities. An example of a change of policy based on PSI efforts is that governmental permission by the Food and Drug Authority to distribute 1 STOP sexually transmitted infection kits to pharmacies was recently given. Mobile testing and innovative approaches to testing are also on the agenda. PSI also supported the national AIDS program by procuring an emergency shipment of 17,500 rapid test kits during a pending national stock out caused by slow national procurement mechanisms financed by the Global Fund. Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? (a) Relationships between PSI and a wide range of partners are, in general, handled professionally and have been well managed. Targets for activities, however, are set by PSI staff in Vientiane and not always discussed with partners before they are set. (b) The project has worked closely with both government and civil society partners to build capacity to provide an effective and sustainable approach to the HIV response. National authors are the first authors on research reports based on PSI-funded research as national authors are now fully responsible for these reports. PSI worked closely with counterparts to provide implementing support to the national program (CHAS) in the integrated bio-behavioural surveys of transgender people conducted in 2010 and 2012. In addition, provincial health authorities complimented the good working relationship and coordination with PSI and implementing partners, in the delivery of services which complements government services. Although it was challenging for local nongovernmental organizations or PSI-run drop-in center staff to point to concrete changes in fundraising or identify how they have changed their modes of operation due to the support given by PACT, all staff consulted were glowing in their reports on the positive influence of this support. 34 Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (a) CAP-3D should be able to achieve critical results at scale related to condoms/lubricant for prevention of HIV. PSI leads in condom marketing and as of 2011, the brand leader in Laos is the donor-subsidized Number 1 condom, with freely distributed generic condoms close behind and very low levels of commercial condom distribution. Other commodities include water-based lubricant and STI treatment kits. PSI is using the “total market approach” to help build healthy and sustainable markets for condoms by coordinating the efforts of all these sectors, especially the government and NGO sector, so that condom distribution will in the future not need to rely on donor subsidies.16 (b) Successful achievement of results at scale for HIV counseling and testing and linkages to care and treatment is constrained by a variety of programmatic and structural factors. These include an inadequate awareness of the benefits of HIV testing as a gateway into early care and antiretroviral treatment. Most key informants interviewed report that the motivation for having an HIV test was for reassurance of a negative status, and to get small incentives. Knowledge around early treatment is lacking amongst peer educators and community members interviewed. In addition, members of key populations who receive a positive test may be accompanied to the antiretroviral treatment center and not followed further, nor contacted as part of peer support. Nearly all informants from managers to peer educators assume that they will be cared for by the care and support activities of self-help groups. Self-help groups indicate that they cater mostly to general population HIV-positive individuals, that participation is voluntary, and that there are few self-disclosed key population members among their number. There is a thus a gap in continued support for members of key populations who are seropositive. Peer educators perceive that the most at-risk within their community are male and transgendered sex workers. Reaching out to them is challenging because of high mobility and refusal of services. In the new drop-in center in Thakek, transgendered peer leaders estimate that they have contacted about half of the transgendered people in that area, and know of individual transgendered sex workers. Much of this contact is through word of mouth, peer to peer face to face contacts, phone networking and social media such as Facebook. The drop-in center does not have a Facebook profile and digital outreach is not measured. It is difficult to confirm if current services are reaching the most at-risk amongst the key populations, whether they are sex workers or individuals with multiple risky sexual contacts. In addition, structural and logistical factors play a role in achieving results at scale. There are reports of stock outs of HIV rapid test kits in drop-in centers and government voluntary counseling and testing centers in the last twelve months. While there are many reasons for this, there is a target-based approach to forecasting. Drop-in centers are given a quota of HIV test kits per annum instead of using needs-based projected forecasting. Technical assistance for this area 16 Building a Sustainable Condom Market for Lao People, 2011-2015, PSI total market approach work plan. 35 has been discussed in meetings of the Country Coordination Mechanism, and currently a procurement process for technical support is in progress. Beneficiaries met in drop-in centers noted that the activities were beneficial to them. There is a leadership vacuum on advocacy on men-who-have-sex-with-men issues as project funding has ended for Burnett Institute, the nongovernmental organization that provided leadership on this issue that led on these issues. Decreased advocacy may undermine the gains made in the prevention response in this key population. There is also an information gap for men-who-have￾sex-with-men as the last studies done were in Vientiane in 2007 where HIV prevalence was 5.6% and in 2009 in Luang Prabang where there were no HIV infections in those surveyed. Sexually transmitted infection rates were high in the Luang Prabang study which demonstrated a rectal chlamydia rate of 8.3%, a rectal gonorrhea rate of 1.7% and either chlamydia and/or gonorrhea infections at 9%. Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (a) There is some use of strategic information to improve programming however use of routine monitoring data is limited by both PSI and partners. There have been a number of behavioural surveys between 2000 to 2012 in key populations (female sex workers, men-who￾have-sex-with-men and transgender people). Specifically PSI has been involved in three surveys on female sex workers (2008) and transgender people (2010, 2012). These have been disseminated nationally with the national HIV program (CHAS). The last men-who-have-sex￾with-men survey was conducted in 2009 by Family Health International (FHI). Condom audit reports are conducted regularly by PSI. The 2012 survey found that transgender sex workers had higher HIV prevalence (7%) compared to non-sex workers (1.7%). As a result of this survey, PSI hired transgender sex workers as project outreach staff in order to improve the networking with peers. It is not clear if data from routine monitoring reports is used at the level of the subcontractor/implementing partners and how this builds into improving local programming. There is little understanding of use of data at the drop-in center level, except in target based programming. (b) At present, indicators are not constructed to track the care cascade from prevention to enrolment in HIV care and treatment. Critical gaps remain in overall strategic information given PSI’s key contributing role in national data. PSI Laos is using a paper-based standardized reporting format which has been aligned with national formats (Global Fund￾funded, Ministry of Health). A system of unique identifiers is under piloting currently to track key population individuals enrolled into the program. Peer educators use reporting formats which lists the persons contacted however does not distinguish between old and new peers contacted. This information (old /new) could be useful in tracking peers who stay on and to inform on mobility and ability to network with new contacts. Some of the indicators such as “aware of nearest Sun Quality Health clinics” will be difficult to measure with routine program data, but may be assessed through the TRAC studies. 36 Individual cohort records are not maintained although the drop-in center maintains a registry of clients using the center, have/had contacts with peer educators, and those who had HIV testing. Current reporting formats track number of contacts, rather than numbers of individuals receiving a longitudinal series of prevention services including HIV testing. Thus, it is unclear how these numbers of contacts are reconciled at country level for reporting as numbers of individuals reached, at regional level. At present, indicators are not constructed to track the care cascade from prevention to enrolment in HIV care and treatment. While there has been work to align reporting formats with that are used for national Global Fund-resourced projects, indicators could also be better aligned to enable reporting towards global needs. One of the critical gaps in overall information is that there is virtually no data on syphilis in key populations. The last survey with data on syphilis was in the FHI 2001 HSS and STI survey which showed 0.2% syphilis in service women.17 Information on men-who-have-sex-with￾men, defined as men who have anal sex with a man in the last six months is inadequate. The 2007 Vientiane study found an HIV prevalence of 5.6% but there has been no recent study in this population. This may change as the review team noted that piloting of dual HIV and syphilis testing was being conducted in drop-in centers, has begun. Of note, penile swabs for sexually transmitted infections were not collected in the 2009 men-who-have-sex-with-men survey (only self-collected anal swabs), and may miss individuals who also have insertive sex, not only receptive anal intercourse. In data quality assurance, verification from source reports is currently not conducted at the local level by managers of the local team. Checks on correct summation of numbers (aggregated data) are performed at the CBO level and transmitted to PSI country office. However, active data quality assurance monitoring is absent. The review team noted that there was a discrepancy of numbers in the total number of HIV testing reported in a month by drop-in centers. This was probably due to poor register design leading to counting errors. The principle of data quality verification could not be articulated by the implementing staff. In general, the monitoring and evaluation system is complex as most of the implementing partners have co-funding from the Global Fund and reporting indicators for these activities are not as specific as those for USAID. PSI and partners have managed this well. Attribution of achievements to any of the funding is difficult for implementers so most use a ‘proportion approach’ which is if the project is funded X % by the USAID, then X % of the achievements are attributed to USAID. This is not optimal however seems to be the current working approach adopted for CAP-3D. In future, when USAID funding shrinks and proportionally other funding increases; this approach may need to be revisited. (c) Strategic information capacity building efforts have largely resulted in increased local capacity. PACT-Thailand’s core organizational capacity building activity in Lao include organizational capacity assessment to identify needs, project cycle management training, M&E and Me training, strengthening management capacities of implementing partners; and specifically New Friends centres to more autonomous functioning. 17 HIV/AIDS assessment: LAO PDR. Doris Seta Mugrditchian and Carol Jenkins, July 2002. USAID/Cambodia/OPH HRN-C-00-99-00005-00. 37 PSI’s implementing partners have been trained by PACT on the logic model, M&E for Me, data analysis and planning as well as use of the new reporting formats. The local organisations reported that this capacity building was useful and helped in understanding the program as well as in management. For example, Lao Red Cross now has its own logic model, and there is increasing delegation of responsibilities by the director as team members’ capacities have increased. Outreach workers have also been trained by PACT and report that the training helped to improving their knowledge, communication and training peer educators. PSI Laos has internal processes to examine data quality using on-site data verification at the level of the local teams conducted by the PSI Monitoring Unit. Internal quality assurance is also done to track and verify the quality of reported data and ensure operational sites implement according to the PSI standards. Quality assurance audits are conducted by PSI quality assurance officers every six months for the first two years and then annually. Audits are shared with program management teams. Data quality verification from source reports is currently not conducted at the local level by managers of the local team. Checks on correct summation of numbers (aggregated data) are performed at the CBO level and transmitted to PSI country office. However active data quality assurance monitoring is absent. During the field visit, there was a discrepancy of numbers compared to the total number of HIV testing reported in a month by drop-in centers. This was probably due to poor register design leading to counting errors. The principle of data quality verification could not be articulated by the implementing staff. Lao PDR: Recommendations The CAP-3D Project should:  Strengthen demand generation and knowledge on HIV testing. HIV testing is the gateway to early care and antiretroviral treatment. Efforts should be made to include knowledge of the availability of antiretroviral treatment and how to live positively.  Review how sexually diverse key populations are defined by program implementers. Consideration should be given to focusing on the key population to be targeted as all people who practice anal sex. .  Relieve the constraints on outreach workers that drive them to cap the number of contacts they make. Since motivated and well-networked outreach workers make more contacts, consideration should be made toward changing the incentive scheme from one that rewards peer educators who bring in peers who test HIV positive to rewarding those who bring in peers who accept HIV testing. Efforts should be made to formalize the use and measurement of text messaging and Facebook to increase contacts.  Consider not including the prevalence of sexually transmitted infections as a goal level indicator for activities. However, further surveys should include sexually transmitted infections. Alternatively, the project should investigate increasing sexually transmitted 38 infection services. This could include regular syphilis testing and periodic presumptive treatment for chlamydia and gonorrhea with 1 STOP.  Consider developing service delivery innovations to reduce dependence on health care staff. For example (a) use of peers to conduct the rapid screening test for HIV and syphilis (b) referral to 'preferred providers' (government and private sector) who treat transgendered people and men-who-have-sex-with-men well.  Develop a method to support people who test positive through the first six months of care or the first six months on antiretroviral treatment. If signature deliverables are to be taken up then it is necessary to be able to track linkages in the prevention, care and treatment cascade. Positive peers or non-discriminating members of self-help groups for people living with HIV should implement this activity. Indicators need to be developed to show outputs at each level of the care cascade. Data quality assurance needs improvement.  Cost out the PSI “one-stop shop” service delivery model through drop-in centers with medical/paramedical staff available for STI treatment and HIV rapid testing for FSW, MSM and TG. While this model may be effective with current donor funding, it not be sustainable given the gradual reduction in resources.  Elaborate the concept of “shared confidentiality”. This principle underlies services linking prevention to care and treatment. For HIV positive key populations, the lack of peer-based open support could be detrimental to their well-being and adherence to care. Reliance on general population PLHIV support group to fulfill the peer needs for HIV-positive MSM and TG is questionable.  Evaluate stigma and discrimination within the at-risk groups toward HIV positive peers. This is an unknown dimension in the peer network and should be assessed. This can be done as part of qualitative studies to improve knowledge around stigma and discrimination within the TG and MSM community, and how to encourage continued peer support for HIV-positive TG and MSM.  Continue support for development of the national TG network and forum to discuss common interests.  Include syphilis and penile swabs, where appropriate, when conducting STI surveillance in key populations.  Consider using TRAC and FoQus studies to monitor, measure and understand factors influencing behavioral change, utilization of services etc. in context of the national program. These studies will be able to complement knowledge on the overall epidemiology and influence national strategic approaches. 39 THAILAND Key Findings Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (a) The approaches used to achieve results have, for the most part, been successful. There have been remarkably high levels of performance for almost all activities except those involving two governmental institutions. There is a wide range of partners, mostly community based organizations, implementing prevention and care services in Bangkok, Pattaya and the province of Chiang Mai. Most organisations show increasing numbers of people who receive services, and are on track to meet or exceed their targets by the end of the financial year. Studies demonstrated a clear positive correlation between being reached with program services and HIV testing. There was a change of regional leadership at PSI, new staff came on board, and new partners were included. Continuing growth in the number of people reached during this period demonstrates that performance did not suffer. All service delivery sites appeared to be ‘MARP-friendly’, as the staff members were almost all members of the communities served and the service delivery sites had positive images of members of the communities. Working hours usually suited the hours available for community members to access services. The notable exceptions are the two public hospitals that are serving as voluntary testing and counseling sites outside of the city of Chiang Mai, in Chiang Mai province. Both Saraphi and Sanpatong are well under target. Most people who undergo HIV testing are from the areas of the hospitals where there may be fewer men-who-have-sex-with-men and are not from nearby Chiang Mai city. Some seropositive migrants were not sent for CD4 cell counts because there was no funding for antiretroviral drugs for them. (b) High performance was supported by sound data collection and management, with the potential to have metrics for the continuum of prevention linked to HIV care. This was most evident with the community based organization SWING but all implementing organizations collected information that could be used to develop measurements for the ‘prevention linked to treatment cascade’ and use this information to improve program outcomes If the goal of HIV testing is to get as many people into earlier treatment as soon as possible, then there needs to be a method of following up these men. Every organization has its own method for follow up and it appears that most of them are in the follow up system. It was also possible to accurately measure program coverage in Pattaya where organizations reaching both male sex workers and transgendered women had clear indications of the total number of people in the population that needed to be reached with services. They made these population estimates through mapping in both the high season and the low season for sex work. 40 Coverage is high. The mobility of transgendered sex workers is being addressed by the ongoing development of a unique identifier system. (c) The CAP-3D funded activities contributed to the national strategic plan calling for increased testing of key populations. It is remarkable that many of newly-tested seropositive were found to have CD4 cell counts above 350 cells/mm3 and so were not eligible for antiretroviral treatment. This could be due to successful targeting of men at risk who were newly infected or high incidence in an epidemic that is out of control, or both. It is worth examining the age of these seropositive individuals as previous Thai studies have noted that older age was associated with HIV-prevalence, while younger age was associated with HIV-incidence.18 If the goal of testing is to get as many people into earlier treatment as soon as possible then there needs to be a method of following up these men. Every organization has its own method for follow up and it appears that most of them are in the follow up system. Research funded by CAP-3D demonstrates that fear of the result of HIV testing is still one of the leading stated reasons for not undergoing HIV testing. The stigma of seropositivity is very high, even in the community of peers, and many people are afraid of being discriminated against by members of their own communities as much as by others. This represents a failure to get the message across to all community members that HIV infection is now a chronic treatable disease and people with controlled HIV infection can and are leading healthy and long lives. (d) CAP-3D activities include several on-going innovations which still need to be evaluated to determine their effectiveness. APMG is supporting initiatives to increase the number of men at risk who are HIV tested as well as giving support to men who test positive. Save the Children in Chiang Mai is currently conducting research among male sex workers, both Thai and migrants from Burma, involving both physical and virtual mapping in order to increase the reach of prevention and care services. Both of these innovative activities have begun too recently to evaluate. Evaluators are concerned that more male sex workers from Burma located in Chiang Mai may undergo HIV testing even though if found seropositive, they may not have access to antiretroviral medications through the Thai public health system. Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? (a) The CAP-3D funded program has not had a major influence on both national public health and community leaders to adequately prioritize and take action on the public health emergency among MSM and transgendered persons. Thailand is the only one of the four countries with a severe and increasing epidemic. The expanding HIV epidemic among men-who￾have-sex-with-men in Thailand, especially among young men in Bangkok, is a public health emergency. Thirty per cent prevalence among men who are not sex workers in Bangkok is higher than adult prevalence among the general population in Swaziland, the country with the highest 18 van Griensven et al. Evidence of an explosive epidemic of HIV infection in a cohort of men-who-have-sex-with￾men in Bangkok, Thailand. AIDS 2012, 26:000–000 41 prevalence in the world. Annual incidence is estimated to be 8% 19 , signifying that one in fifteen men is being infected in a year. The evaluators were very concerned that some public health officials and some staff of community based organizations did not recognize that the epidemic was increasing and was not in control. Although national program leaders recognized the contributions made by the USAID program through PSI work with national working groups, particularly in the development of national MSM guidelines, this did not always translate into sufficient efforts at prioritizing interventions. Some informants thought that ‘treatment as prevention’ or ‘test and treat’ would have a major impact on the trajectory of the epidemic. The scientific evidence that either of these strategies, even in the unlikely case that high treatment coverage were reached, would control this severe an epidemic is not unequivocal. There is inadequate evidence that either or both of these strategies will control this severe epidemic. Neither of these strategies alone will be sufficient to control the epidemic. There are currently no precedents for any community using ART to treat its way out of an HIV epidemic. Most HIV epidemics worldwide have been brought under control through concerted action to increase prevention at the community level. Concerned HIV professionals and young gay leaders must be encouraged to advocate for prioritization and rollout of new and more effective prevention activities, otherwise there is a possibility that the epidemic will continue to be out of control. The need for strong leadership within the affected community to galvanize action is an ongoing issue.20 A second area in which some informants had great hope was changing behavior through digital communication. Although digital communication may increase the number of contacts that men￾who-have-sex-with-men may have with others, there is as yet still little evidence that digital communication is any more effective than another communication channels in to delivering prevention messages. As this is a recent innovative intervention, there is not yet established ways to fully evaluate the outcomes and impact. It is important, however, to balance the There is an opportunity cost if significant resources are used to develop messages using new communication channels, when evidence-informed prevention services have not yet achieved significant impact. (b) For the most part, separating the impact of CAP-3D capacity building from the impact of capacity building by previously funded activities by other USAID-funded projects is difficult to determine. The capacity of most of the implementing organizations to effectively respond to the epidemic is due in most cases to long associations with international nongovernmental organizations who have invested significant resources to building capacity. There are two exceptions. One is in the field of data flow and data management where current capacity building activities have led to clearer reporting of results. The second is in recent and ongoing organizational development in association with PACT. The staff of local implementing organizations all thought that these activities strengthened their capacity to implement activities in a sustainable manner. 19 HIV and Syphilis Infection Among Men-who-have-sex-with-men — Bangkok, Thailand, 2005–2011, MMWR, June 28, 2013, Vol. 62, No. 25, p 518 20 UNGASS Thailand 2012. 42 (c) An enabling environment surrounding local registration of community based organizations may help sustain CAP-3D activities, but continued efforts at developing broader funding bases for some CBOs need to be addressed. It is simple to register a community based organization as a foundation in Thailand, increasing the chances that activities funded by CAP-3D will be sustained by organizations in the future. Most of the community based organizations had knowledge of the registration process. Some of them have already developed diversified funding bases. SWING has Global Fund resources and collects local donations. Glory Hut has resources from the Pattaya Municipality. SISTERS, serving transgendered women in Pattaya, is a foundation, but has not yet developed a broad funding base. Caremat in Chiang Mai also has a narrow funding base. Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? (a) The organization and operations of the CAP-3D project in Thailand, with its many implementing partners, both old and new has the potential to result in the ideal of a truly effective consortium of partners. Under new management, regular meetings between stakeholders and implementing partners are now being held, although it is too early to determine if greater effectiveness will result. There are issues that have been identified and need to be addressed. Some staff in organizations that have in the past focused on HIV prevention feel there is pressure on them to take on care activities; some staff in organizations that have in the past focused on HIV care think there is pressure on them to take on prevention activities. There is a danger of organizations losing focus on the activities that they do most effectively. This has caused tension and competition between the two Chiang Mai organizations involved. Building technical capacities would require much more investment if the initial capacity is not present. (b) There is increasing collaboration with national authorities such as the National AIDS Management Centre (NAMC) of the Ministry of Public Health through national level working groups as well as work on Global Fund activities. At provincial level and with government health facilities, PSI and implementing partners have a close working relationship. Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (a) CAP-3D’s ability to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project are constrained by national policy related to HIV testing. There are issues of HIV testing policy that restrict increased entry into care. National regulations restrict task shifting of HIV testing to non-medical personnel. Current regulations dictate that all HIV testing, including rapid testing, are done by medical personnel, such as a doctor or nurse. 43 Issues with the validation of this point of care approach by the National Department of Medical Sciences remain unresolved. The current HIV testing strategy has several weaknesses:  Reliance on static voluntary counseling and testing sites in government hospitals and the selected community based organizations which employ paramedical personnel/nurses or laboratory technicians (although peer counselors can provide individual support). Service hours in some institutions are confined to regular working hours which are difficult for key populations to access.  While same day testing has been promoted under the national HIV program, uptake and buy￾in from hospital laboratories and practitioners are still limited. Some patients are lost from the cascade of HIV testing as they do not come back for results.  Finger–prick rapid HIV testing is not available. The approach method of one finger prick rapid screening by trained peers as part of mobile outreach and confirmation at a voluntary counseling and testing site has not been adopted.  There is no direct service delivery of CD4 cell count in community based organizations. There are a few specific cases in which the gaps in retention from prevention to HIV testing to enrolment in HIV care are clearly documented. An organization with deep roots in the community, SWING in Pattaya, reported that one quarter of the people tested who received a positive test did not have a CD4 count performed. One of the key issues for follow up of peers who are seropositive is that retrieving detailed retention and adherence information is difficult as these data is kept confidential in the health services. However, HIV-positive individuals should be able to report access to CD4 testing and antiretroviral treatment. (b) Lack of interventions aimed at stigma and discrimination is another area which may negatively affect the project’s ability to achieve critical results. Stigma and discrimination by and experienced by men-who-have-sex-with-men and transgendered women was consistently reported by many stakeholders – in particular stigma and discrimination in health care settings. RTI is working with the national AIDS control program on a measurement framework on discrimination against key affected populations in the healthcare setting. They also conducted a study on human rights violations affecting transgender women It is currently unclear to the review team what specific action at the population level would reduce the level of general stigma and discrimination against sexual minorities. There are few specific activities that have been rigorously evaluated and proven to decrease stigma and discrimination. Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (a) Recent changes in monitoring and evaluation should improve the ability of strategic information to inform planning and implementation. In the draft monitoring and evaluation 44 plan for PSI Thailand there are important elements to streamline processes and research. PSI Thailand is in process of building a monitoring and information system which will track and report data along the full cascade for project partners. This would be a good innovation with the emphasis on linking prevention to care. A unique identifier mechanism has already been implemented by community based organizations, replacing previous systems. It is also important to ensure as much as possible that the core indicators are aligned to the reporting requirements of the country and global needs, since CAP-3D contributes to the national response. In the Thai TRAC studies, key indicators or target behaviors should be comparable to the TRAC of other countries and/or to those usually used in national sentinel surveillance for global reporting needs.21 For example, in the draft Thailand monitoring and evaluation plan, target behaviors to be monitored include “use of condom at last sex” and “use of water-based lubricant at last sex”. This should be consistent with PSI TRAC studies in other countries or national sentinel surveillance such as “consistent use of condoms in the last 1 - or 3 – months’” which is consistent with global reporting for men-who-have-sex-with-men on “use of a condom the last time they had anal sex with a male partner” (Global Indicator 1.12). Current indicators are different from PSI TRAC studies in other countries which uses “consistent use of condoms in the last one month” (Burma TRAC MSM 2008, 2009 and Lao TRAC TG 2010) or “consistent condom use in the past 3 months” (China TRAC MSM 2010). There are other inconsistencies - the Thai TRAC studies uses the indicator “Received HIV test in the past 6 months” 22, while the Thai national IBBS 2010 survey uses the core indicator from the global reporting standards i.e. “having had an HIV test in the last 12 month”. It would be preferable for TRAC studies to use this common core indicator which contributes to global reporting of “percentage of men-who-have-sex-with-men that have received an HIV test in the past 12 months and know their results” (Global Indicator 1.13). (b) Dissemination plans are written into the M&E plan however communicating these needs translation to local language. Implementation partners noted that monitoring and evaluation efforts resulted in collated reports, but these and other research reports were not always explained to them in the language that they could understand in order to take action. In addition, key information from IBBS and other research studies of interest apart from TRAC and FoQus studies could be shared and discussed as part of knowledge assimilation. 21 Global AIDS response progress reporting 2012. Construction of Core Indicators for monitoring the 2011 UN Political Declaration on HIV/AIDS. 22 Both TRAC 2012 in male sex worker; and TRAC 2012 in transgenders use the indicator “Received HIV test in the past 6 months”. 45 Thailand: Recommendations The team recommends the following actions be considered:  Undertake activities to encourage HIV professionals and young gay community leaders to mobilize for increased coverage of effective prevention methods that have been scientifically shown to prevent HIV infection: peer education, condom use, and change in community norms. New interventions should be included only if they have been shown to be effective.  Hold technical discussions involving all partners working with similar target groups to share knowledge and ideas. This will result in less time and effort spent in reinventing the wheel. Some examples where sharing could occur include (i) the APMG orientation workshop for new men-who-have-sex-with-men (ii) the Mplus orientation curriculum for young MSM and TGs (c) SWING’s development of YouTube short videos and tablet apps and games for outreach interventions, and (d) Mplus and Save the Children’s short videos on prevention education for men-who-have-sex-with-men and transgender persons.  Support the development of branding/messaging of HIV as a chronic, manageable disease as part of a national program campaign. Government and local CBO informants report that the message around HIV has been that of a death sentence, which is slowly being changed to one of chronic illness. However there are challenges in evolving this long established branding on HIV. Re-branding will help create a demand for services for key populations, specifically MSM and TGs as well as possibly help reduce stigma and discrimination.  Consider piloting rapid testing by community members with confirmatory testing in VCT centers. This should be piloted by well-established partners so that it can be proven effective to skeptical governmental decision makers.  Encourage peer support activities to enable increased retention of HIV positive community members from HIV testing to having a CD4 test and enrolment into ART services. The first baseline CD4 count could be monitored as part of linkage to care and support, but subsequent CD4 and viral load testing should not be routinely monitored as this area is part of ART treatment and may be difficult to track by community based organizations. Intensified peer support for the initial several months of diagnosis until the individual has established contact and relationship with HIV/ART services, with tapering to more normal peer support could be considered.  Consider contracting out to other partners such as Thai Red Cross to conduct regular ART cohort analysis or special studies on ART retention in MSM/TGs. PSI and implementing partners do not need to not track long term adherence and retention in ART directly or to have these as indicator outputs. This area goes beyond care and support, and steps into treatment which is not the traditional area of strength of PSI. Moreover, there will be logistic, technical and capacity issues by community based organizations which will have to extract data from several government ART services.  Reconsider involvement of both community based organizations in Chiang Mai – Mplus and Caremat – to be involved in both prevention and care. It may not be an efficient use of resources. 46  Consider the ethical implications of demand generation for HIV services where there is little access for migrants, particularly those who are HIV positive. Overall CAP-3D project conclusions Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? CAP-3D has an appropriate regional focus on prevention and improving access and use of HIV testing. There is new focus to ensure continuity and linkages throughout the comprehensive model of prevention, care and treatment. The project is appropriately focused among men-who￾have-sex-with-men, transgendered women, and female sex workers. Interventions for people who inject drugs in China are appropriate and according to the evolution of the country’s HIV epidemic scenario. In Lao, where Global Fund funds the program on female sex workers, CAP￾3D has extended free, open STI and HIV testing services to support improved access for all key populations. Overall, the comprehensive approach to prevention, care and treatment was adapted in each country according to local needs and epidemiology. There remains the challenge, given current funding levels, to obtain a balance between direct service delivery, community capacity building, and providing technical assistance. Sustainability: What progress has been made toward ensuring sustainability of CAP-3D’s approaches? PACT has proven ability in providing assistance in organizational capacity building across the several countries of focus. There is evidence of change in managerial and organizational capacities and practice with improved service delivery ability of local organisations (see detailed country write-up). Direct implementation by PSI of drop-in center- based programs should increasingly transition to customized technical support to community based organizations. The main lesson from China, Myanmar and Thailand in sustainability is that organizational capacity development needs to be initiated as early as possible in the lifetime of the funding grant, since community based organizations vary in their ability to change rapidly. Transition-out requires adequate preparation in terms of the affected community, service providers/organizations and in particular, time to mobilize and diversify funding sources. Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? 47 In all countries, USAID missions, PSI and implementing partners are involved at the national and regional levels for knowledge sharing and technical assistance. Direct partnership with national government varies according to context. There has been evidence of CAP-3D interventions being incorporated into national programming in some countries. Much potential exists for CAP-3D in influencing national and regional policy in HIV prevention through implementation and close working relationships with affected communities. Translating implementing experience and data for advocacy is important and should continue. At all levels – national, provincial and local – PSI and the implementing partners have good coordination with government and other stakeholders. Within the consortium, there is increasing coordination due to new regional PSI management. There have been challenges in operations and relationships in each country related to delayed and reduced funding, structural issues with policy and legal environment, technical issues with service delivery and capacities of local organisations. These have been navigated professionally by each of the PSI country teams. Beneficiaries interviewed in all countries consistently report the benefits and relevance of the CAP-3D interventions. Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? In the three remaining countries that CAP-3D will continue in, policy, legal and technical environments will continue to affect the ability to achieve critical results at scale. There are issues with shared confidentiality among the individual, community organisations providing peer support and health service providing HIV treatment. Stigma and discrimination plays a role in this. Demand generation needs to evolve from HIV counseling and testing as a means of HIV prevention to being an entry point to care. Furthermore, the benefits of earlier antiretroviral treatment both of health and as prevention through reduction of HIV transmission needs to be emphasized and that an HIV diagnosis is not a death sentence. PSI can use its expertise in marketing to create demand for testing as a gateway to treatment for a longer and better life in all three countries. Service delivery models particularly the drop-in centers with medical services need to be costed. Although these models are directly beneficial to key populations, sustainability by government of other funding in the medium and longer term should be considered. Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? Overall, the CAP-3D strategic information systems are robust and there is regular information from TRAC and FoQus studies and good local PSI team capacity in this area. In several countries, PSI is closely working with national programs in strategic information on key populations, with the potential to influence national policies. However, there is limited use of 48 routine reporting and recording information (M&E) in providing more detailed data for programming. It is also difficult to estimate the coverage of services to the overall populations reached as these are not directly reported by partners, but the data may be available at site level in part. Reporting indicators follow that of PEPFAR, but could be more interpretive in order to have better measurement of coverage and outcomes. Indicators used for routine monitoring and reporting should be in alignment with national and global reporting needs. There is variable engagement of stakeholders, implementing partners and affected communities in data dissemination and use for local programming. TRAC and FoQus study results are not being shared and discussed widely with government and other stakeholders. PSI China has performed the best in the area of use and dissemination of strategic information with stakeholders and local partners. TRAC and FoQus studies provide the qualitative component for interventions in key populations. This is valuable data as these studies act as surveillance23 on behaviour change. In many countries, national HIV programs concentrate on quantitative surveys such as sentinel or second generation surveillance which often do not include a robust qualitative component. It would be important to review how to share these results to improve specific interventions whether through branding, marketing and delivery, with government and other stakeholders apart from PSI local implementing partners. A method must be developed to track individuals from testing to treatment so that a 'treatment cascade' can be developed and analyzed to increase and improve entry to care. Current monitoring indicators are not constructed for reporting on the continuum of prevention linked to care cascade. On improving the monitoring of the five ‘signature deliverables’: (i) Increased case finding per contact: Current TRAC studies overall suggests that when clients attend drop-in centers, they are more likely to access the health services (STI treatment and/or HIV testing), and this is intuitive. For people who have “only peer education contact” – they are less likely to have had an HIV test. Segmentation studies may help to define this category of persons so as to design innovations to encourage them to attend services. It may also help to define where the main bulk (largest number of people) of ‘at-risk’ key populations is. This is because large numbers may not be in the “most at-risk” compare to others who have “moderate risk” and who probably contribute more of the numbers of new infection. (ii) Reduced cost per case identified: in many countries, the program has evolved from serving men-who-have-sex-with-men and transgender communities to include serving different key populations such as sex workers, particularly for STI and HIV testing services, even if other key groups are funded differently (e.g. Global Fund). In Lao PDR, the New Friends centers also provide HIV testing to anyone who asks for testing as an approach to extend access to HIV testing. This may reduce the cost per case identified with HIV, but would require costing studies. (iii) Increased CD4 cell count on entry to care: there is inadequate data on CD4 count at the time of HIV diagnosis, and at enrollment into ART services for key populations. In general, the median CD4 count of people starting antiretroviral treatment in Asia Pacific is still low (about 23 Surveillance is routine regular collection of data to inform public health actions. 49 140 cells/mm3 ).24 It will be important to delineate the time point of CD4 count for monitoring by CAP-3D i.e. at HIV diagnosis and/or at starting antiretroviral treatment. This is presuming that all HIV positive individuals will be actively managed, linked to care and treatment services and continuously followed-up until they are started on treatment (which may be in days, months or years). (iv) Improved retention in prevention, care, and treatment: overall this is an excellent objective to have, however retention in care and treatment will be more difficult to monitor by community based organizations. The information has to be retrieved from health services who usually will not share this due to confidentiality issues. Another approach is to have other academic institutions or partners who can conduct regular studies to examine the issue of retention/survival in key populations receiving treatment. (v) Improved treatment adherence: Although antiretroviral drug prescription is usually provided by government services, CAP-3D employed medical personnel could increase support in basic HIV care and track adherence through the usual adherence methods (self- report, pill count, on-time pill pick up at the treatment clinic). This area however is difficult to document objectively. “On-time pill pick up at treatment clinics” is a proxy for level of treatment adherence, and is part of the early warning indicators for prevention of HIV drug resistance. Overall recommendations To USAID/RDMA 1. While the current program is appropriately focused, the HIV epidemic in each country is evolving with reduction of burden in some and increase in others, among the key populations. In addition, there are changes in the number of organisations working with key populations given the reduction in overall HIV funding. CAP-3D and partners need to continue to monitor the overall epidemic situation in each country to optimise response. 2. As country contexts vary widely for engagement with national and regional government, it is important to maximize PSI’s strength in implementation and strategic information in key populations to inform national policy and programming. Strategic use of data for advocacy and technical assistance should continue. 3. Although there is minimal funding for the tuberculosis component in CAP-3D, consider leveraging existing programs to improve TB case detection within current services and networks. This investment will probably be minimal. The ongoing issues with the technical component of case detection (e.g. laboratory technology, clinical algorithm etc.) would be an area which CAP-TB may have major inputs and funding. 24 Zhou et al. Trends in CD4 counts in HIV-infected patients with HIV viral load monitoring while on combination antiretroviral treatment: results from The TREAT Asia HIV Observational Database. BMC Infectious Diseases 2010, 10:361 50 4. Organizational capacity development needs to be initiated as early as possible in the lifetime of the funding grant, since community based organizations vary in their ability to change and diversification of funding including that from the government requires time. As this is the mid-term of the project, it is important to intensify efforts to support this area. To PSI and implementing partners 1. Demand generation for HIV testing as a gateway needs to be strengthened. There continues to be the belief that HIV counseling and testing is a means of HIV prevention instead of an entry point to care, and is a message to rectify in service providers. Positive messaging around HIV being a chronic manageable illness and not a death sentence will be important as part of the education of affected and infected communities. PSI can use its expertise in marketing to support organizations in creating a demand for testing as a gateway to treatment. 2. Segmentation of the key populations may help in tailoring improve interventions, and to inform national policy and program. This is an area which could maximize use of PSI’s traditional strength in branding, marketing and delivery. 3. In view of the reducing resources in HIV, consider costing the various service delivery models to provide information for future sustainability. It would be important to assess which parts of the interventions are essential and additional, and could be funded by government and Global Fund. 4. With the aim of building sustainability, organisation capacity building should be intensified and where possible, implementation transitioned to local organisations. PSI should provide tailored technical assistance. 5. In use and dissemination of strategic information, this should include other relevant studies in the specific area. Results from TRAC and FoQus studies should be disseminated and discussed with local stakeholders including government as these (more qualitative) surveys are useful to complement the traditional IBBS and HIV sentinel surveillance. PSI survey and M&E indicators should be aligned with national and global reporting needs. It is important to have experience-sharing among implementers across the countries and translation or adaptation of innovations according to country context. This aspect could be improved and could be done among key PSI staff and other experts working on research and studies: including methods of studies, results, and common findings across the different target populations; and among local implementers facing similar challenges to share local solutions that may prove helpful for others. 51 Appendix 1 - Evaluation Scope of Work Appendix 2 - Final evaluation design with data collection tools Appendix 3 - Evaluation schedules Appendix 4 - List of people contacted Appendix 5 - References Statement of Work for RDMA CAP-3D Performance Evaluation A. Project to be Evaluated  USAID Regional Development Mission for Asia (RDMA) Control and Prevention of Three Diseases (CAP-3D) Project, Population Services International Prime Cooperating Partner  AID-486-A-11-00004  Total Estimated Cost: $36 million for 5 years Cooperative Agreement contract under USAID/RDMA  The effective date of this Cooperative Agreement is January 15, 2011 through January 14, 2016  Consortium: Pact, Research Triangle Initiative (RTI), Save the Children, and AIDS Program Management Group (APMG) B. Project Background The goal of the project is to reduce morbidity and mortality related to HIV, TB and malaria in the Greater Mekong Sub-region (GMS) by increasing an effective regional response (characterized by stronger country ownership) to prevent and mitigate these diseases. 0 For HIV/AIDS prevention and treatment, target populations include men who have sex with men (MSM), transgendered individuals (TG), female sex workers (FSW) and their clients, injecting drug users (IDU), and people living with HIV/AIDS (PLHA). For tuberculosis control, the target populations include PLHA, the elderly, and slum dwellers. For malaria control, target populations focus on rural populations in endemic areas. The overarching strategy for this project is to increase the coverage, quality, and sustainability of a comprehensive package of for HIV prevention, care, and treatment services (CPP). A major area of emphasis is reducing dependence on external donor resources by strengthening local institutions and securing local government buy-in and diverse funding sources. To achieve this, the objectives of PSI and its partners focus on the scale up of the CPP model, shifting increasingly toward provision of technical assistance to local agencies for CPP implementation (IR1); evaluation and documentation of CPP models and use of strategic information (SI) to advocate for replication and leveraged funding (IR2); and enabling local organizations to scale-up the CPP through provision of technical assistance and capacity building (IR3). Capacity building, behavior change, and greater involvement of most-at-risk populations (MARPs) are all central to the strategy, which is shown on graphic below: In September of 2012, RDMA shared priorities for its own five-year strategic roadmap for HIV/AIDS with the PSI consortium, and encouraged partners to consider ways that they could align their activities in future work-plans to more specific health outcomes under the existing project scope. As part of the broader PEPFAR Asia Regional Program, RDMA envisions enhanced host-country leadership and investments in high-impact, cost-efficient, and sustainable responses to HIV/AIDS, specifically in most at risk populations. Ideally, with RDMA support, Asian countries should realize tangible improvements in their own HIV responses, while also engaging in regional and global collaboration that advances broader implementation. With limited resources, the RDMA aims to realize this vision through targeted investments in: 1) the development and implementation of innovations; 2) rigorous evaluation; 3) advocacy and improved use of data for decision making; and 4) building the capacity of people, institutions, and systems. RDMA aims to track the performance of these investments through routine monitoring and targeted assessments focusing on: a) quality; b) results; c) cost￾efficiency; and d) sustainability, in priority program areas. Persistent challenges in addressing gaps in HIV testing and treatment coverage among key populations – combined with mounting evidence of the opportunities to dramatically advance HIV prevention, care, and treatment by addressing these gaps – have led RDMA to focus on intensified case findings paired with early and sustained access to treatment as a key priority area for its HIV programming. To address this priority RDMA encouraged CAP-3D consortium partners to consider the pursuit of innovations in the following five areas: 1) demand-generation efforts, focusing on promoting the benefits of HIV testing and early access to HIV care and treatment among key populations through interpersonal communication, Internet, mobile phone, and targeted media communication; 2) introduction of cost-efficient systems to incentivize case-finding, providing recognition, rewards, or mobile phone cash transfers or top-ups to members of peer networks and pharmacists who successfully refer individuals most likely to acquire or transmit HIV to high-quality HIV, TB, and STI diagnostic services; 3) support for quality standards and potential branding of public and/or private referral sites meeting clear standards for providing high-quality diagnostic and counseling services and effective linkages to prevention, care, and treatment services; 4) investments in innovative information systems to prevent loss to follow up among high-risk individuals across the continuum of prevention-to-care-to-treatment services, as well as to promote treatment adherence and facilitate the management and monitoring of potential incentive systems to support case finding; and, 5) investments in meaningful engagement of beneficiary populations in the design, implementation, and monitoring of these innovative case-finding efforts, and in mitigating stigma and discrimination and facilitating an enabling environment for service uptake. Illustrative outcomes provided by RDMA for activities in these areas include: a) increased case finding (HIV, STI, and TB) per client contact; b) reduced cost per case identified; c) increased CD4 cell count on entry to care; d) improved retention in prevention, care, and treatment; and e) improved treatment adherence. Country Strategies Recognizing each country’s unique context, the project has worked to align regional activities to address country-level needs, and to implement country-specific activities in ways that foster regional collaboration and advance a regional learning agenda.  To date, the Burma strategy has focused on scaling up the CPP by leveraging existing private sector and NGO service channels, including the Sun Quality social franchise, and Drop in Centers for SWs, MSM and IDUs. Expansion of MARP-managed outreach through the Targeted Outreach Program (TOP) has enhanced community involvement. Due to funding constraints and mission priorities, Burma is currently the only country in which CAP-3D has implemented Malaria and TB control focused activities, in addition to HIV programs.  The focus in Laos has concentrated on bringing together civil society, private and public sector partners to model effective implementation of the CPP, advocate for alignment of the HIV CPP with the national strategy for HIV and AIDS, and leverage other funding to support scale up of effective models for all three diseases. Capitalizing on PSI’s strong record securing Global Fund to Fight AIDS, TB and Malaria (GFATM) funding, PSI worked with local partners across all sectors to secure support to expand their delivery of CPP beyond the project.  The focus in Thailand has been to replicate models for delivering the CPP to MSM and TGs and recruiting other organizations, government offices, hospitals, and donors to use these models. In China, where facilitating the growth of civil society was important and most health services were government-run, CAP-3D worked with Yunnan Association of STI and AIDS Prevention and Control (SAA) to scale-up the CPP using a cascading capacity building approach where public sector partners and CBOs were enabled to provide MARP-friendly services to IDUs, TGs, SWs and the clients of SWs. The partners expanded referral networks to improve links to care and support.  As USAID/RDMA experienced a reduction in funds for HIV in China in year two of the project, CAP-3D’s China program has closed out earlier than expected. The Consortium was also set up to deliver community-based TB and malaria services in Burma and advocate for effective responses to these diseases. In the area of TB control, CAP-3D focuses on the prevention of TB/HIV co-infection by targeting both HIV and TB patients with testing and diagnostic services, expanding case-finding and strengthening treatment and support for TB patients through a private sector network. CAP-3D is also focused on supporting the Ministry of Health’s malaria control program through support for community-based health services and training for national staff in disease prevention and case management. C. Technical Work Purpose and Use of the Evaluation Purpose: This evaluation is being undertaken to analyze the CAP-3D’s project achievements to date in areas related to its performance, ability to achieve critical results, steps made toward sustainability, and its ability to build and maintain critical relationships. The evaluation will comprise a multi country mid-term evaluation for activities in Thailand, Laos and Burma. In addition, there will be an end of project review of the activities in China, as the program is closing out activities in this country. The evaluation conclusions and recommendations are to be used by USAID/RDMA’s Office of Public Health (OPH) and the implementing partners to strengthen project implementation in countries where CAP-3D will continue to work. The evaluation should point out areas that need more focus or should be expanded in order to achieve the project’s expected outcomes. As part of this overarching purpose, the evaluation should analyze gender implications so that gender equity can be achieved in diseases control and prevention. In addition, Burma Mission has indicated the potential use of the findings from the evaluation to inform the development of USAID/Burma’s Health Strategy. Key Evaluation Questions and Proposed Methodology Question 1, Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (35%)1 Data collection and analysis should include (but is not limited to) the following areas:  Behavior change communication (BCC) outreach and uptake of voluntary counseling and testing, quality of care and support services, including home-based care and referral services, (In Laos, look specifically at approaches for reaching beyond TGs), and tuberculosis and malaria prevention and management  Target beneficiary perception of the value of CAP-3D services  Access to MARP user-friendly health services through Sun Quality Health Franchise (SQH) and other providers and retention of patients  The Continuum of Care, including linkages between testing, care, and treatment  Increased case-finding of people living with HIV and utilization of innovations for more effective interventions in all countries.  Incentive schemes and changes in social marketing on the introduction of new packaging for lubricants (Laos)  Adaptability of the program to identify most at risk and vulnerable populations and target/adjust activities accordingly.  Different gender roles to promote demand-generation efforts Methodology should include, but is not limited to, analysis to date on the results noted in the Performance Monitoring System and program reports (i.e. trend analysis of results) and on progress made on desired behavior change based on results of available studies; descriptions of current approaches and challenges in key areas documented from interviews with local implementing NGOs, health providers and other key stakeholders, perceptions of the CAP-3D program, gaps and potential areas for improvement as documented through interviews and/or focus groups with a sample selection of beneficiaries and other stakeholders. The analysis should lead to specific recommendations on approaches for more effective implementation and operations. Specific methodology will be further defined in conjunction with the evaluation team. 1 The percentages listed reflect the approximate proportion of the consultant’s time that should be spent on each of the questions. Question 2, Sustainability: What progress has been made toward ensuring sustainability of CAP￾3D’s approaches? (20%) Data collection and analysis should emphasize, but is not limited to:  Influence of the program on strategy, programming, and policy at the national level  Evidence of organizational and technical capacity built among local implementing partners (in areas such as behavior change communication)  Ability of local partners to leverage other resources as a result of CAP-3D  Adoption of CAP-3D approaches by external stakeholders (such as government counterparts and the Global Fund)  Evidence that the CAP-3D model for prevention, care and treatment will continue to be implemented in China at project end Methodology should include, but is not limited to, analysis to date based on results noted from CAP-3D’s organizational capacity building assessment tools, program reports and interviews with PACT staff, the recipient CBO’s and PSI affiliates who are receiving technical assistance, and other evidence that demonstrates the skills and knowledge gained from this support. Descriptive results should focus on the ability of local partners to leverage resources, adoption of CAP-TB approaches. Analysis of the project’s influence at the national level should rely on interviews with implementing NGOs, government counterparts at the provincial and national levels, and interviews with other key stakeholders, such as the Global Fund, bilateral donors and international partners, as well as USAID and other USG staff, including the US Department of Health and Human Services/Centers for Disease Control and US Embassies. The analysis should lead to recommendations for strengthening approaches to ensure that CAP-3D is sustainable and replicable for the government and other key local and international stakeholders in each focus country at project end. Question 3, Operations and relationships: To what extent does the way that the CAP-3D project is organized, operates, and maintains relationships and communication with key stakeholders, establish an effective and sustainable approach? (15%) Data collection and analysis should emphasize, but is not limited to:  Ability of CAP-3D to function effectively as a consortium (PSI, PACT, RTI, Save the Children and APMG)  Relationship and communication with local implementing partners, community based organizations, other donors/international partners, and with government stakeholders  Challenges and advantages of programming under a multi-infectious disease (HIV, TB, and Malaria) platform (Burma) Methodology should include, but is not limited to, analysis to date based on results noted from project documents such as CAP-3D’s organograms, sub-agreements, and documented reporting, protocol and communications processes within the consortium, and strategic plans. Descriptive information should be based on interviews with consortium partners, government counterparts, international partners, USAID staff in Thailand, Burma, and Laos, and other key stakeholders. Question 4, Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (15%) Data collection and analysis should emphasize, but is not limited to:  Evidence that CAP-3D is set-up to identify, take into account, and respond to emerging needs and priorities at the regional level  Evidence of the ability to influence national programming by the end of the program  Ability to target the hardest to reach and most vulnerable groups, amongst the most at risk populations targeted through CAP-3D Methodology might include analysis to date based on national data (such as the Integrated Biological and Behavioral Survey (IBBS)), UNGASS reports, national strategic plans, Global Fund reports and evaluations, and CAP-3D’s trend data on performance, and quantitative indicators reported for PEPFAR and USAID. Analysis should include a description of the national context and strategy and CAP-3D’s ability to align with those objectives. Descriptive information focused on influence on national programming and ability of the program to take into account emerging needs and priorities should include interviews with government counterparts, project staff, and focus groups/interviews with a sample selection of beneficiaries. Question 5, Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (15%) Data collection and analysis should emphasize, but is not limited to:  Design and use of strategic information in programming for results o Criteria for selection of studies to be undertaken o Quality of study designs and results o Dissemination and use of strategic information by project/Consortium partners for programming/decision making and advocacy o Use of strategic information/data to inform scale-up and decision making (particularly in China)  Gaps in strategic information/data for strategic decision making;  Capacity building related to strategic information collection and its use at different levels. Methodology might include a review of key strategic information study designs, data and results, including TRAC surveys and interviews with PSI and consortium partners and persons in each country (government and implementing partners, M&E Managers or Research Advisers, CBO staff and facilities engaged in strategic information activities). Analysis should determine which strategic information activities they have been engaged in, the level and type of engagement, what information was found to be the most helpful, evidence of increased capacity to collect strategic information, and use of the information obtained from the strategic information activities. Analysis should also focus on the gaps in strategic information that are most relevant for supporting RDMA’s HIV strategy, and should take into consideration the type of SI information, role of the persons, their data needs, and types of strategic information uses (actual and potential). Gender Consideration Evaluation Team should address relevant gender concerns within the project area. Recommendations should outline the most significant gender issues that need to be considered during activity implementation. It should describe how women, men, and transgenders (TG) involved have been affected by the context or the work undertaken. The desk review should include a specific gender analysis. Where applicable, the data in the Evaluation should be disaggregated by sex. D. Technical Team USAID/RDMA requires two international technical specialists for this Evaluation: 1) Evaluation Specialist as the Team Leader and 2) HIV Specialist. In Burma and China, the team will be assisted by an HIV/AIDS Advisor from USAID Headquarters and for Laos and Thailand, a staff member from USAID/RDMA will participate in the Evaluation. In addition, translation support services will be recruited by PSI in all countries. The Evaluation team members should have expertise in conducting program evaluations, and previous work experience with HIV and/or TB and Malaria programming, and community-based approaches. E. Deliverables The Evaluation team leader will be responsible for coordinating and managing the drafting of deliverables, consolidating the individual contributions, and submitting the drafts and final report. Each evaluation team member will be responsible for contributing to the deliverables and writing relevant sections of the documents based on his/her expertise and the tasks assigned by the Team Leader. The required deliverables as a joint output for the Evaluation Team are listed below: 1. Evaluation Design, Tools and Work Plan One week prior to the beginning of the field work and based on their review of existing project and related resources and receipt of the results of the RDMA OPH sampling of sub-partners and their sites to be covered by the Evaluation Team, the Evaluation Team Leader will submit to RDMA OPH a draft work plan for the evaluation, including site visits. In addition, based on inputs of the team members, the Team Leader will provide RDMA OPH with an evaluation design that includes: • Finalization of sub-questions and focal areas that lead to answering the larger evaluation questions. • Date sources (existing data and proposed sources and sampling plans to obtain new information.) • Plan for analyzing quantitative and qualitative information. The plan should be based on obtaining country-level analysis as well as aggregation of data across the countries. The evaluation design document shall include a series of proposed data collection instruments. The Evaluation Team, led by its Team Leader will be expected to finalize the draft work-plan and design including data collection instrument upon receipt of USAID/RDMA’s input and comments. 2. Outline of the Evaluation Report An annotated outline of the evaluation report, including sub-sections of the main body of the report, shall be submitted to RDMA for approval by the end of two weeks of field work. A meeting to discuss initial findings will also be required once the field work has been completed. 3. Debriefings Debriefings on country findings, preliminary conclusions and recommendations will be provided to USAID/Burma and the USAID Laos Office prior to country departure. In these two countries, the team will begin its field work by meeting with a USAID representative who will brief the team. Upon conclusion of the field work, the team will debrief USAID/RDMA on the main evaluation findings, and preliminary conclusions, recommendations and lessons learned. The oral debrief will be accompanied by a written document and/or power point presentation, with electronic copies provided to OPH. For guidance on the presentation for USAID/RDMA, please see Appendix 3. 4. Draft Evaluation Report Following the required structure for final reports and addressing comments from debriefings, a draft of the evaluation report will be submitted to OPH within 6 business days of the RDMA debriefing. The final report should include a section on gender analysis. The draft report will undergo an USAID review process using (but not limited to) the checklist found in Appendix 4. 5. Revised Draft Evaluation Report A revised draft evaluation report will be submitted within 4 business days of receiving consolidated comments from RDMA OPH. The revision will incorporate all feedback provided by USAID Burma, Laos and RDMA reviewing team. The report must comply with the USAID Evaluation Policy’s “Criteria to Ensure the Quality of The Evaluation Report”. (See Appendix 1 of this SOW.)The revised draft will also undergo a quality assurance clearance by USAID/RDMA’s Program Office, as a requirement before acceptance of the deliverable. 6. Final Evaluation Report The Final Evaluation Report will be submitted within 4 business days of receiving comments from RDMA’s Program Office on the second revised draft. The full report must not exceed 40 pages, excluding appendices. The structure of the final report must include: • Executive Summary of the Evaluation, no more than 3 – 5 pages, that concisely states evaluation purpose, methodologies, key findings, and conclusions, recommendations, and lessons learned • Acronyms List • Table of Contents • Main body of the report: Introduction, Background, and Methodology (along with a statement related to methodological limitations); Findings (including a synthesis of findings per country as well as an overall synthesize), Conclusions, and Recommendations sections on each evaluation question and overall Project conclusions, recommendations and lessons learned. • References and List of Persons Contacted • Appendices: At a minimum the appendices will include: Evaluation SOW, Final evaluation design and work plan, any statements of differences, and all data collection tools. The Evaluation team shall ensure that the final report meets USAID required standards for evaluation reports (See ADS 203.3.1.8 and http://transition.usaid.gov/branding/USAID_Graphic_Standards_Manual.pdf) For example, the Evaluation team is expected to insert a high quality photograph representing the project evaluation on the front cover of the final report, with a brief caption and credits included on the inside of the front. Written permission is required from those in photos and from the photographer to use in public document. It is imperative that proper ethical procedures be observed in the use of these photographs – especially when working with vulnerable populations. 7. Electronic Handover of Data and Records The Evaluation team will hand over to RDMA any data and records collected by the Evaluation team (e.g. interview transcripts or summaries) in an electronic file and in an easily readable format as agreed upon with the AOR. The data should be organized and fully documented for use by those not fully familiar with the project or evaluation. USAID will retain ownership of the survey and all datasets development. 8. Submission of the RDMA Approved Report to DEC USAID/RDMA must provide to the Development Experience Clearing house (dec.usaid.gov) an electronic copy of the RDMA approved evaluation report within three months of the evaluation’s conclusion. F. Period of Performance The period of performance for each consultant will vary based on their roles and responsibilities; 1. Evaluation Team Leader is expected to work approximately 58 working days over an elapsed 13 week period. 2. The HIV Specialist is expected to require approximately 56 working days over an elapsed 13 week period. An illustrative schedule and time-frame for deliverables can be found below (Sundays are excluded as free time): Given that this is an illustrative schedule, the offeror is welcome to propose an alternative schedule and overall period of performance to USAID. Description Team Leader HIV Specialist Period (No. of days) (No. of days) During Preparatory phase Reading of background documents & preparation & submission of first deliverable & revisions 5 5 Week 1 Planning meeting & external meetings with PSI & Consortium in Bangkok & travel to China 3 3 Week 2 Field work in China (includes data analysis and initial draft of findings, conclusions for China section) & travel to Burma 10 10 Week 2-4 Field work in Burma (includes briefing & debriefing mission in Burma, data analysis and initial draft of findings, conclusions and recommendations for Burma report section) travel to Laos 9 9 Week 5-6 Field work in Laos (includes briefing & debriefing mission in Laos, data analysis and initial draft of findings, conclusions and recommendations for Laos report section)& travel to Thailand 8 8 Week 6-7 Field work in Thailand (includes briefing & debriefing RDMA, data analysis and draft summary of Thailand section of report 9 9 Week 7-8 Further Data Analysis and Writing including synthesis 3 3 Week 9 of country findings Debriefing with USAID/RDMA (OPH and mission-wide presentation) 1 1 Week 10 Provide first draft report 4 4 Week 11 Respond to USAID Burma/ Laos and RDMA comments & revise draft report and resubmit to RDMA 2 2 Week 12 Finalization and submission of final report 4 2 Week 13 Total 58 56 Note: The above LOE also covers time for responding to comments and questions from Team Leader and RDMA and for the preparation of any documents to be submitted to RDMA OPH, such as summaries of focus group and draft reports. G. Relationships and responsibilities The Team Leader The team leader is solely responsible for ensuring the quality and timeliness of deliverables for USAID, and will coordinate and manage the Evaluation team and will undertake the responsibilities described above. Consortium Partners As the lead for CAP-3D, PSI will be responsible for the following.  For each of the four countries, provide for each geographic area a list of names with titles of key partners, facilities, trainees, and CBO communities engaged in the project to RDMA to transmit to RDMA no later than five weeks prior to commencement of the evaluation.  Provide an electronic copy of all country-level IR data, disaggregated by sub-partner, and if available, its catchment areas (sites), to RDMA no later than four weeks prior to commencement of the evaluation.  Provide to RDMA electronic copies of all semi-annual and annual report, M&E plan, the latest FY implementation plan, special studies, and other documents on CAP-TB.  Send letters to key partners and sub-partners about the upcoming evaluation.  Following guidance from the Team Leader, PSI country program staff member set up appointments with the key stakeholders and sub-partners to be visited.  Provide translation support and in-country transportation support for the Evaluation team. RDMA OPH The Strategic Information and Monitoring and Evaluation Team of RDMA OPH will be responsible for overall management of the Evaluation Team throughout the assignment. This will include:  RDMA OPH will assist with providing criteria for sampling sites or for meetings with partners in each country. The sample will be transmitted to the Team Leader to use to develop the first Deliverable.  RDMA OPH shall serve as the point of contact between PSI and the Team Leader prior to the beginning of the field work stage and after completion of all field work.  RDMA OPH shall ensure that all documents, files and lists mentioned above are obtained from PSI and transmitted to all members of the evaluation team in a timely manner.  To ensure that the field portion of the evaluation begins as scheduled, RDMA OPH shall provide the Team Leader comments and feedback on all deliverables, within the timeframes stipulated below. RDMA OPH, USAID/Burma shall assist the Team Leader with the following:  Provide guidance on recommended secure hotels and methods of in-country travel (i.e., car rental companies and other means of transportation) and if necessary, identify a person to assist with logistics (i.e., visa letters of invitation etc.).  Provide timely review of draft/final reports and approval of the deliverables. H. Required qualifications and application process A consultant bidding on the evaluation should submit a written statement of interest and a CV. He/she should specify clearly the position being applied for. Both Team Leader and the HIV Specialist should submit a succinct description of the proposed methodology, evaluation design and approach. The proposal should not exceed a maximum of 7 pages. The Team Leader will have the primary responsibility as the Point of Contact between the team and the USAID missions (both in RDMA and USAID/Burma). Evaluation team members will have different key role and responsibilities as follows; 1. Team Leader: The Evaluation Team Leader will have the primary responsibility as the Point of Contact between the team and the USAID Missions (both in RDMA and Burma Mission). The Team Leader is also responsible for the overall management and coordination of the team, including detailing individual responsibilities and tracking performance, and ensuring the delivery of high-quality and timely deliverables to USAID. As Team Leader, the consultant will:  Work with the RDMA staff and the team members to finalize evaluation methodology and to refine a plan of action for information gathering, including document review, key informant interviews, as described in section E.  Work with evaluation team to draft and finalize questionnaires for key information interviews and focus groups.  Finalize the team’s overall schedule in consultation with the CAP-3D Project Lead/Chief of Party, as described in section E.  Provide an annotated outline of final evaluation report, discuss with team members, and assign writing responsibilities; ensure timely and quality team contributions of the deliverables described under section E.  Present the debriefing with RDMA’s Office of Public Health and other USAID staff, consolidate all draft sections from team members, and finalize the report for RDMA.  Manage the performance of and ensure that deliverables are met for all (non-USAID) team members.  Act as the primary point of contact with USAID and with other key stakeholders, and as the lead communicator when presenting and debriefing on aspects of the evaluation findings. .  Assume responsibility for the quality and timeliness of all deliverables submitted to USAID throughout the evaluation.  Plan and facilitate team meetings and briefings with USAID. The qualifications of the Team Leader should include:  A graduate or doctorate degree in Public Health, Evaluation, or related field;  Minimum of 10 years of experience in a relevant field related to HIV, public health and infectious diseases programming;  Understanding of HIV, TB transmission, and prevention, treatment and care strategies;  Field experience working with HIV programs;  Experience working in South East Asia;  Experience managing or participating in infectious diseases focused evaluations;  Excellent analytical, writing, and presentation skills;  Experience managing teams, including logistics, planning, and budget management;  Experience in undertaking operational research. 2. HIV Specialist The HIV Specialist will assist the Team Leader to draft relevant assigned sections of the Evaluation Report, assist Team Leader to draft and address comments from USAID on initial and revised draft reports, as well as to prepare presentations for briefing/debriefing with USAID/RDMA and USAID country offices. The HIV Specialist should have comprehensive experience in working on HIV prevention, especially with most-at-risk populations; HIV Testing and Counseling; treatment, care and support. He/she should be familiar with issues connected with strategic information in relation to planning, programming and policy making. It is particularly important that he/she has extensive experience in operational research. The qualifications for the HIV Specialist should include:  Minimum of 5 years experience in a relevant field related to public health and infectious diseases programming;  Extensive experience in monitoring and evaluation and with various evaluation methods and approaches;  Understanding of issues related to drug resistance and the need for accurate diagnosis and effective treatment;  Excellent analytical, writing, and presentation skills. Supporting Document for Preparation Work Necessary supporting document will be supplied to the evaluation team prior to arrival to RDMA. APPENDICES to SOW APPENDIX 1 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT  The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why.  Evaluation reports shall address all evaluation questions included in the scope of work.  The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by the Evaluation’s AOR.  Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report.  Evaluation findings will assess outcomes and impact on males and females.  Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.).  Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions.  Findings should be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex.  Recommendations need to be supported by a specific set of findings.  Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. APPENDIX 2 List of Background and Project Documents to be Provided Team by RDMA The consultant will review relevant documents (including strategies, guidance, policies, project designs, activity designs and other related documents) as identified by OPH. The documents will be provided after the award is made, and will include, but are not be limited to:  Key program/project documents including agreements/contracts, work plans, progress reports, etc. and the project performance monitoring database from each country that shows data from each partner, PSI internal project review.  USAID Evaluation Policy (2011)  USAID/RDMA’s five-year strategic roadmap for HIV/AIDS  HIV/AIDS Next Generation Indicator  HIV/AIDS Annual Performance Report Guideline  Tuberculosis Performance Management Plan  Tracking Results Continuously (TRAC) studies  Integrated Biological and Behavioral Surveillance (IBBS)  Asia Regional Stigma and Discrimination Guidelines (produced by RTI) APPENDIX 3 Guide to USAID/RDMA Evaluation Presentations This document is intended to serve as a guide to evaluation teams in their presentations to the USAID/Asia Mission. Once an evaluation has been completed, the team leader should convene a debriefing meeting with team members, including the appropriate USAID/Asia staff to agree to the terms of the draft final report and formal presentation to the Mission. The evaluation team should debrief the technical team on items to be presented, including but not limited to: evaluation findings, results, prioritization of results, concerns and recommendations. The evaluation team should schedule this debrief no later than one day before the presentation. Below is a suggested format to evaluation presentations. Presentations should include, but are not limited to: PRESENTATION TOPIC TIME (estimated) 1) Introduction of Evaluation Team 5 2) Project Context & Overview  Background  Environment (i.e. context, including challenges)  Project Objectives  Project Main Activities  Stage of Project upon Evaluation 5 3) Purpose/Focus of the Evaluation  Purpose of Evaluation  Evaluation Methodology  Integration of USAID Evaluation Policy Standards 10 4) Evaluation Findings  Summary of input from evaluation respondents  Project Impacts  Implementation Concerns 10 5) Overall Conclusions 5 6) Recommendations to USAID/RDMA, including recommended prioritization in light of overall programming context 10 TOTAL 45 minutes + up to 30 minutes of Q&A/discussion General Tips and Considerations: 1. Presentation should be clear and straight forward. Long text and reading off slides can lose the audience’s attention. Providing the appropriate USAID/Asia staff a draft of the presentation slides at least one day in advance for comment is strongly recommended. 2. The presentation should focus on the evaluation findings and conclusions and specifically link findings to decision-making steps. 3. Consider distributing project and evaluation background to meeting participants prior to presentation. APPENDIX 4 Evaluation Report Checklist Review Factor Yes No Some - what N/A Comments Report Format Does the report include all of the key sections? (cover sheet, table of contents and acronym list/glossary of terms, executive summary, background, evaluation objectives, main evaluation questions, methods, findings, conclusions and lessons learned, recommendations, and any other sections requested) Is the report well -presented? (clear sentences, reasonable length paragraphs with tables & figures where necessary, no typos) Are charts and graphs used to present/summarize data, where relevant? Is there a clear separation between conclusions, findings, and recommendations sections? Report Content Does the introduction adequately describes the project, explain where it is implemented, target group(s), target area (s), include contextual information, and include the ‘theory of change’ or development hypotheses that underlie the project? Are findings specific, concise and supported by strong evidence? Are the conclusions credible, given the findings the report presents? Are recommendations practical, based on the conclusions, and responsive to the purpose of the evaluation? Does the evaluation report explicitly link the evaluation questions to specific future decisions to be made by key stakeholders? Does the report discuss any issues of conflict of interest? (Including lack thereof?) Does the report include statements regarding any significant unresolved differences on opinion on the part of funders, implementers and/or members of the evaluation team? (As applicable) Does the Annex include a copy of the SOW, data collection tools used, and other key supporting materials? Evaluation Fundamentals Is there a clear statement of the purpose, how the evaluation will be used, and who the intended users are? Does the evaluation state the period over which the evaluation was conducted and the project time -span covered by the evaluation? Does the report address each key question around which the evaluation was designed? Technical Information Is there a clear description of the evaluation’s data collection methods? Does the report contain a section describing the limitations associated with the evaluation’s methodology? Does the evaluation scope and methodology section address generalizability of the findings? 1 Draft Evaluation Design, Tools and Work Plan Jamie Uhrig and Polin Chan 30 May 2103 The overall objectives of the evaluation are to analyze the CAP-3D project achievements till date in the following areas: 1. Quality of outputs (Performance) 2. Achievement of critical results: innovations, use of data for decision making and advocacy, and its impact 3. Sustainability including capacity building of people, institutions and systems. Proposed Methodology The general methodology for collection of qualitative and quantitative data across the 4 countries includes: - Interviews with key informants and stakeholders through group-based discussions (eg. focused group discussions with MARPs), individuals and institutions. - Analysis of program reports and routinely collected data: trend analysis of results, local STI and HIV surveys including HIV sentinel surveillance reports, IBBS and special studies including published research studies to examine impact of interventions. Triangulation of local data such as use of the CAP-3D data as well as other available information allows for understanding of the evolution of outcomes and impact. The evaluation report will include descriptions of the implementing approaches, innovations, perceptions of the CAP-3D program, challenges and lessons learned; data analysis to objectively inform on outputs of the interventions, and specific recommendations on approaches to improve effectiveness of implementation and operations. This document consists of three parts: 1) Key evaluation Questions and Proposed Methodology 2) Five lists of questions to guide the evaluators in the field - one for each group of stakeholders - central level, country level, implementing partners, beneficiaries 3) Schedule Key Evaluation Questions and Proposed Methodology This draft of key evaluation questions and proposed methodology is based on the Scope of Work that is Attachment 1 to the Evaluation Specialist's contract. 2 Specific methods for each of the key question for evaluation include: Key Question Areas for review Proposed Methods Remarks Q1: Performance: To what extent have the approaches used for prevention, care and treatment for the most vulnerable populations achieved results to date? (35%) (a) Prevention outputs: BCC, outreach and home-based care, uptake of HIV testing and counseling (b) Linkage and referrals to care and other services: tracking data (c) Care and treatment cascade COPCT and retention: total population, population living with the virus, number of people tested, number of people who know they have HIV, number entering the care system, number begun on treatment, and number continuing treatment at one year. (d) Innovations (e) Incentive schemes and changes in marketing (Laos): increase in sales after the changes was introduced, regularly￾collected sales data. (f) adaptability towards programming for MARPs, gender and access to services, intensified case-finding (number of people found to be living with HIV per quarter) (g) TB and malaria prevention and management (h) Access to MARP user-friendly health This question will be addressed by examining the performance of the implementers with respect to timely production of the targeted outputs. Documents include: - Performance monitoring reports: Annual, semi annual and work plans. Field travel reports - HIV surveillance and IBBS - Other research studies/surveys - Specifics reports if available o VCT, ART data o ART Cohort analysis - Key questions are (a) uptake of services and, (b) retention in services (prevention, care, treatment) (c) if there is impact, for example, on STI and HIV prevalence/incidence; 100% condom use; % utilisation of treatment services; % loss from care Outputs will be compared with targets. Reasons for variations will be noted. For the first intermediate result "Improved access to prevention, care and treatment", there are a few outcome indicators on behaviour change in HIV monitored by TRaC surveys that will be examined to note progress. Contributions toward national impact at the goal level will be examined through examining national level reports on prevalence. 3 services through Sun Quality Health Franchise (SQH) and other providers and retention of patients (output). (i) The Continuum of Care,(CoPCT model) including linkages between testing, care, and treatment Q2: Sustainability (a) Influence of the program on strategy, programming, and policy at the national level and evidence of the ability to influence national programming by the end of the program. Others: evidence that the CAP-3D model will continue to be implemented in China (b) evidence of organisation and technical capacity in local implementing partners (C) ability to leverage other resources (d) Adoption of CAP-3D approaches by other stakeholders (e) Capacity building outputs including training, coaching, and mentoring will be documented and any unexpected impacts As above. Documents include: -CAP-3D assessment tool -program reports - Analysis in this area relies on discussions with key informants on all levels. (a) A special question for China - Is there any written evidence that the CAP-3D model for prevention, care and treatment will continue to be implemented in China at project end. What is it? This question will be asked of all PSI and consortium senior staff and all higher level partners and stakeholders in China. (b) Document in NGO/CBO ability to do new things and the experience of doing new things including ability to change and trouble shoot. (c) Is there any evidence of new financial resources or other new resources that local partners have accessed? What has been the role of project funding in accessing these new resources? (e) Documentation of capacities and adoption of new methods by implementing partners. All PSI and consortium senior staff and all higher level partners and stakeholders is whether national or local practices have changed as a result of the activities or if there are plans to change practice. Evidence in the form of written documents of strategy and policy changes will be sought. The analysis will lead to recommendations for strengthening approaches to ensure that CAP-3D is sustainable and replicable for the government and other key local and international stakeholders in each focus country at project end. 4 will be noted. Q3: Operations and relationships (a) Ability to function as a consortium (b) Relationship and communication with the broad range of partners (c) Burma: challenges and advantage of programming under a multi-infectious disease (HIV-TB-Malaria) platform As above. Documents include: -organograms, sub-agreements, field reports, protocols and communication processes, strategic plans. (a) And (b) Analysis is based mainly on interviews with consortium partners, counterparts from government, international partners, USAID staff and other stakeholders: the strengths and weaknesses of the consortium are, what aspects of consortium functioning have led to effectiveness, and what aspects of consortium functioning can be improved. The question: "What relationship and communication issues have arisen and how have you dealt with them?" will be asked of several partners. (c ) PSI and higher level partners in Myanmar will be asked what the advantages and challenges of three disease funding under CAP￾3D were. What practices should be adopted by other three disease funders and what improvements are recommended. 5 Q4: Ability to achieve critical results: Looking at the HIV programs from a regional perspective, to what extent will CAP-3D be able to achieve critical results, at scale, in HIV prevention, treatment and care by the end of the project? (15%) (a) Evidence that CAP-3D is set-up to identify, account and respond to emerging needs and priorities at the regional level (b) Ability to influence national programming approaches and strategies (c) Ability to target and reach the most vulnerable groups, amongst the MARPs (d) Target beneficiary perception of the value of CAP-3D services. This issue was moved from Question 1. As above. Documents include: -CAP-3D data on performance, reports for PEPFAR and USAID - National strategic plans and GFATM reports and evaluations - National SI and M&E data such as HSS, IBBS and other routine program reports. Descriptive information on influence and ability to account for emerging need and priorities will source from interviews with govt counterparts, project staff and FGDs with beneficiaries. Evaluative questions will be asked at two levels. Questions about how the most vulnerable are chosen and targeted will be asked of all higher level staff at national level. Questions about how the most vulnerable are chosen, targeted, and what adjustments were make will be asked of all local implementing organisations. Focus groups of beneficiaries will be organised in some but not all sites where activities are implemented. In each country there will be four focus groups organised: two in cities where implementing partners think that the activities How CAP-3D activities fit into national strategic plans will be ascertained for HIV, malaria and TB. An assessment of how CAP-3D aligns with the national strategy will be made with senior level informants. 6 have been most successful and two in cities where implementing partners think that the activities have been least successful. There are three main issues to be discussed. They are: How do members of the community use the prevention and care services? What is good about them? And how can they be improved? Q5: Strategic information: To what extent are the strategic information activities serving to inform the planning, implementation and monitoring of the project? (15%) (a) Design and use of SI for decision making and improving programming (b) Gaps in SI (c) Capacity building related to SI and its use at different levels As above. Review key SI study designs, data and results including TRAC surveys. Analysis includes type and nature of SI activities, level of engagement, capacities for SI use including whether the data/formats fulfills the needs for supporting the RDMA strategy and field programming at the various levels. Criteria for selection of studies to be undertaken - Quality of study designs and results - Dissemination and use of strategic information by project/Consortium partners for programming/decision making and advocacy - Use of strategic information/data to inform scale-up and decision making (particularly in China) Studies will be evaluated with respect to the quality of study design and results on a scale of Concrete examples of how planning, implementation, and monitoring of the project have changed as a result of strategic information activities. Evidence of research studies conducted by implementing partners or use of results by implementing partners will be sought. Analysis will determine which strategic information activities they have been engaged in, the level and type of engagement, what information was found to be the most helpful, evidence of increased capacity to collect 7 one to five. The use of the studies and the changes in programming or the reasons that changes were not made in programming will be sought from the consortium partners or implementing partners that are most affected by the studies. A special question will be asked of consortium partners or implementing partners in China to determine what larger scale impacts the studies might have. strategic information, and use of the information obtained from the strategic information activities. Additional: Gender analysis (a) Gender analysis: men, women, transgender – disaggregation of data by sex. (b) Different gender roles to promote demand-generation efforts : assessment of gender-specific marketing activities Gender analysis based on the USAID 2012 publication of “Toward gender equality in Europe and Eurasia :a Toolkit for analysis” Cross-cutting area. 8 Limitations: Data limitations will be described during the evaluation process: on the relevance and reliability of data to answer the specific key questions above. For example, program impact on specific behavioural components of MARPs may not only be due to the CAP-3D program, but also of external conditions and influences, such as other governmental interventions and environmental factors. Timelines: 30th May : Meeting with RDMA 1-2 June : Data analysis and develop specific subset of questions on performance based on trends in the data, including factors lead to overperformance or underperformance and what actions have been taken as a result of an assessment of performance. 3 June: field work starts 9 Interview Guide: List of questions Interview guides were developed for each category of key informant to ensure a consistency of approach in interviews. The questions were based on the questions specified in the Scope of Work and tailored to the categories of key informants. Central level - for PSI and Consortium partners and other higher level stakeholders Q1 - What are the areas where implementers have overperformed and underperformed with respect to meeting output targets? How has this affected your projects? How do you know this? What action has been taken to adjust to it? Q2 - What national or local practices have changed as a result of the activities or are there plans to change practice? Any evidence in the form of written documents of strategy and policy changes? A special question for China - Is there any written evidence that the CAP-3D model for prevention, care and treatment will continue to be implemented in China at project end? What is it? Any special success stories of capacity building or leveraging resources by local implementers that you are aware of? Q3 - What relationship and communication issues have arisen and how have you dealt with them? With respect to Myanmar, what practices should be adopted by other three disease funders and what improvements are recommended? Q4 - What do you think are the regional level health needs ? How does the project fit with national strategic plans and national priorities? How are the most vulnerable of the MARPs chosen and targeted? For TB? For malaria? How well are existing service linkage and referral systems working? Do they work equally well for men and women, or are there some differences? To what extent have TB/HIV and malaria outcomes been improved? To what extent is there a true continuum of prevention to care and treatment? Can you give some examples of how USAID support has facilitated the development of a true CoPCT? What improvements need making? Q5 - How has planning, implementation, and monitoring of the project changed as a result of strategic information activities? What are two key findings from research and what was done in light of them? Any large scale impacts in China? What is unknown, and if it was known, how might it influence programming? Any illustrative stories on how research studies were conducted by implementing partners or about use of results by implementing partners? 10 Country level - for PSI and Consortium partners and other higher level stakeholders Q1 - What are the areas where implementers have overperformed and underperformed with respect to meeting output targets? How do you know this? What action has been taken to adjust to it? What is the contribution of project-funded activities to national goal, outcome, and output targets? Quantified, please. Can progress be shown through national level sentinel surveillance? Are indicators aligned with WHO/UNAIDS and national level indicators? Laos - impact beyond transgendered people? Increased distribution of lube after programme changes? Myanmar - Sun Quality system? What can be shown about access of MARP-friendly services by MARPs? Increased case finding per quarter? Comments on a care cascade? Other care beyond HIV/ART care? Q2 - What national or local practices have changed as a result of the activities or are there plans to change practice? Any evidence in the form of written documents of strategy and policy changes? A special question for China - Is there any written evidence that the CAP-3D model for prevention, care and treatment will continue to be implemented in China at project end? What is it? Any special success stories of capacity building or leveraging resources by local implementers that you are aware of? Q3 - What relationship and communication issues have arisen at country level and how have you dealt with them? With respect to Myanmar, what practices should be adopted by other three disease funders and what improvements are recommended? Q4 – What do you think are the regional level health needs ? How does the project fit with national strategic plans and national priorities? How are the most vulnerable of the MARPs chosen and targeted? For TB? For malaria? What do you think we will hear in focus groups of beneficiaries about their view of activities? How well are existing service linkage and referral systems working? Do they work equally well for men and women, or are there some differences? To what extent have TB/HIV and malaria outcomes been improved? To what extent is there a true continuum of prevention to care and treatment? Q5 - How has planning, implementation, and monitoring of the project changed as a result of strategic information activities? What are two key findings from research and what was done in light of them? Any large scale impacts in China? What is unknown, and if it was known, how might it influence programming? Any illustrative stories on how research studies were conducted by implementing partners or about use of results by implementing partners? 11 Local implementing partners - for all local implementing partners whether governmental or nongovernmental Let's give you a chance to tell us about yourselves. Can you please tell us in five sentences what you do in HIV prevention and care? Q1 - What are the areas where you have overperformed and underperformed with respect to meeting output targets? How do you know this? What action has been taken to adjust to it? What can be shown about access of MARP-friendly services by MARPs? Increased case finding per quarter? Comments on a care cascade? Other care beyond HIV/ART care? Q2 - What local practices have changed as a result of the activities or are there plans to change practice? Any evidence in the form of written documents of strategy and policy changes? Can you give concrete examples of how you have changed your work after training or capacity building activities? Have you been funded by any other organisations after the beginning of this funding? Any special success stories of capacity building or leveraging resources by local implementers that you are aware of? Q3 - What relationship and communication issues have arisen between you and your funder and how have you dealt with them? With respect to Myanmar, what practices should be adopted by other three disease funders and what improvements are recommended? Q4 - Some question about regional level? How does the project fit with national strategic plans and national priorities? How do you know and choose the most vulnerable of the MARPs. How do you provide services for them? For TB? For malaria? What do you think we will hear in focus groups of beneficiaries about their view of your activities? How well are existing service linkage and referral systems working? Do they work equally well for men and women, or are there some differences? To what extent have TB/HIV and malaria outcomes been improved? To what extent is there a true continuum of prevention to care and treatment? Q5 - How has planning, implementation, and monitoring of the project changed as a result of strategic information activities? What research has been conducted during this project? What are two key findings from research and what was done in light of them? Any large scale impacts in China? What is unknown, and if it was known, how might it influence programming? Any illustrative stories on how you conducted research studies or how you used the results? 12 Beneficiaries How do members of your community use the prevention and care services? What is good about them? And how can they be improved? Are there any problems with getting services? Is there stigma and discrimination (community, self, health services, other services)? What is doing done to address this? MARPs questions 1. Which services of this organization do you use? 2. How often do you use these services? 3.Do your sexual partner/s have contact with this service? In what way? What is their opinion of the service? 4. How do you find the staff and volunteers? 5. Would you say you are satisfied or dissatisfied with the services here? Why? Has access to this service made any differences to your life? In what ways? 6. Do you contribute anything to this service to benefit others or help the service? 7. How could the services be improved? 8. Are there any other HIV and STI services you use? Are you satisfied with those services? How do they compare to this service? Gender considerations Within each level, gender considerations will be discussed regarding men, women and transgenders including issues of access, success and lessons learned in addressing this area. The desk review includes sex-disaggregated and where data exists, information on transgender as well. Schedule The attached schedule was drafted by USAID and has been reviewed by the consultants. Tentative Schedule for RDMA CAP-3D Performance Evaluation Update:05/03/13 Date Time Description Place Participants Point of contact Fri-Wed May 20-24, 2013 Preparatory phase Reading of background documents and preparation and submission of first deliverable and revisions Thu May 30, 2013 Afternoon Internal discussion and preparation among 2 international consultants Hotel Fri May 31, 2013 Planing meetings at USAID/RDMA Office USAID/RDMA Office 8.00 - 9.00 am Check-in with Aye Aye Thwin, USAID Thailand/OPH Director 9.00 - 10.00 am Overview and logistics arrangment with Marisa and Ravipa Lunch break 14.00 - 16.00 pm Interview with Project AOR Michael Cassell call-in 16.00 - 17.00 pm SI overview with Marisa and Ravipa Sat- Sun June 1-2, 2013 Internal discussion and preparation among 2 international consultants Mon June 3, 2013 AM/PM External meetings in Bangkok • PSI • PACT • RTI Tue June 4, 2013 AM/PM External meetings in Bangkok • Save the Children • APMG 19.50 pm Travel to Vientiane - Depart 19.50 pm(approx.)/Arrive 21.00 pm(approx.) Stay overnight at Vientiane Wed June 5, 2013 Field working in Laos 8.00 - 10.00 am Check-in with Thomas D'Agnes, USAID Laos/OPH Director USAID Laos Office 10.30 am - 12.30 pmBriefing meeting with PSI Laos PSI Office Lunch break 14.00 - 17.00 pm Visit/drop-in with sex workers, MSM, FSW PSI Drop In Center Thu June 6, 2013 Morning Meeting with MSM: Laos PHA and FSW: PEDA 18.15 pm Travel to Luang Prabang - Depart 18.15 pm(approx)/Arrive 19.00 pm(approx) Stay overnight at Luang Prabang Fri June 7, 2013 AM/PM Meeting with partners in Luang Prabang • PLHIV: Laos Red Cross • FSW: PSI • MSM: PSI • PCCA Sat June 8, 2013 07.40 am Travel to Khammuane - Depart 07.40 am(approx.)/Arrive 13.30 pm(approx.) Luang Prabang - Vientiane-Khummuane (Fly from Luang Prabang to Vientiane and take a bus from Vientiane to Khummuane) Stay overnight at Khammuane Sun June 9, 2013 Free time Mon June 10, 2013 AM/PM Meeting with partners in Khammuane • MSM: PSI • FSW: PEDA • PLHIV: LRC • PCCA Tue June 11, 2013 08.00 am Travel to Vientiane - Depart 08.00 am(approx.)/Arrive 13.00 pm(approx.) Take a bus from Khummuane to Vientiane Afternoon Review and report writing Stay overnight in Vientiane Wed June 12, 2013 AM Debriefing meeting with Thomas D'Agnes 13.50 pm Travel to Rangoon - Depart 13.50 pm(approx.)/Arrive 17.35 pm(approx.) Stay overnight in Rangoon Thu June 13-21, 2013 Field working in Burma Morning Briefing meeting with USAID Burma Afternoon Meeting with partners in Rangoon • PSI • TOP Drop-in Center • Cordia Hand • PACT • Save the Children • Sun Quality • UNAIDS • Government • Travel to selected sites (Bago, Rangoon, Mandalay, Magway) Sat June 22, 2013 14.40 pm Travel to Kunming - Depart 14.40 pm(approx.)/Arrive 18.00 pm(approx.) Stay overnight at Kunming Sun June 23, 2013 Free time Mon June 24, 2013 Meeting with partners in Kunming Morning • PSI China Afternoon • Yunnan Association of STI & AIDS Prevention and Control (SAA) Tue June 25, 2013 Morning • Hu Xiang Hao (HXH) Afternoon • Daguan Community Health Center (CHC) Wed June 26, 2013 Morning • Yunnan MSM Technical Working Group (TWG) Afternoon • Bright Cloud Sky, Daytop Drug Rehabilitation and Recovering Center, Firefly at Xishan District Thu June 27, 2013 08.00 am Travel to Mengzi - Depart 08.00 am(approx.)/Arrive 13.00 pm(approx.) Travel by bus Stay overnight at Mengzi Fri June 28, 2013 Meeting with partners in Mengzi Morning • Sisterhood Health Home (SHH) Afternoon • Kangxin Home Sat June 29, 2013 Review and report writing Sun June 30, 2013 Morning Travel to Gejiu by bus Mon July 1, 2013 08.00 am Meeting with Poplar Tree 10.00 am Travel to Kunming by Bus Stay overnight at Kunming Tue July 2, 2013 08.20 am Travel to Nanning - Depart 08.20 am(approx.)/Arrive 09.35 am(approx.) (Air) 11.00 am Guangxi MSM Technical Working Group (TWG) Lunch break Afternoon Meeting with partners in Nanning • Wangzhou Community Rehabilitation Center • Shuguangjiayuan (SGJY) • Nanning CDC Stay overnight at Nanning Wed July 3, 2013 08.00 am Meeting with Government in Guangxi 14.00 pm Travel to Luzhai by Bus - Depart 14.00 pm(approx.)/Arrive 17.00 pm(approx.) Stay overnight at Luzhai Thu July 4, 2013 Morning Meeting with Chengxi Community Rehabilitation Center Meeting with Government in Luzhai 13.00 pm Travel to Guilin by Bus - Depart 13.00 pm(approx.)/Arrive 15.00 pm (approx.) Travel to Bangkok - Depart 16.30 pm(approx.)/Arrive 20.45 pm(approx.) (Air) Fri July 5, 2013 Meeting with partners in Bangkok • PSI Thailand • NAMC • UNAIDS • Thai Red Cross • SWING Bangkok • The POZ Sat July 6, 2013 14.00 pm Review and report writing and Travel to Pattaya by Bus Stay overnight at Pattaya Sun July 7, 2013 Free time Mon July 8, 2013 Meeting with partners in Pattaya • Sister • SWING Tue July 9, 2013 Morning Meeting with Glory Hut and Chief of prevention/health of Pattaya municipality 13.00 pm Travel to Chiang Mai via Bangkok - Depart 13.00 pm(approx.)/Arrive 18.30 pm(approx.) (Travel By bus from Pattaya to Suvannaphum airport) Stay overnight at Chiang Mai Wed July 10, 2013 Meeting with partners in Chiang Mai • M Plus • CAREMAT Thu July 11, 2013 • Saraphi and Sanpatong Hospital • Provincial Health Office • Save the Children 16.30 pm Travel to Bangkok - Depart 16.30 pm(approx.)/Arrive 17.40 pm(approx.) Stay overnight at Bangkok Fri July 12, 2013 PM Debriefing with USAID/RDMA (OPH and mission-wide presentation) Mon-Tue July 15-16, 2013 Further Data Analysis and Writing including synthesis of country findings Wed-Sat July 16-20, 2013 Provide first draft report Mon-Wed July 22-24, 2013 Respond to USAID Burma/Laos and RDMA comments and revise draft report and resubmit to RDMA Thu-Sat July 25-27, 2013 Finalization and submission of final report Interviewee list Location Organisation Name Job title Thailand/Consortium Bangkok Save the Children Duangkamon Doncha-um HIV project officer Scott McGill Sr HIV/AIDS Officer Thachmach Krairit Health project assistant Ratjai Adjuyutpokin Program manager PSI-Regional office Andrew Boner CAP-3D Chief of Party Gary Mundy Regional M&E/SI advisor Alex Duke PSI Thailand country representative APMG/Thai HIV Foundation Scott Berry Executive director RTI Felicity Young RTI regional director USAID-RDMA Aye Aye Thwin Ravipa Vannakit AID M&E advsior Marisa Michael Cassell Aaron Schubert AOR Panus Na Nakorn PACT Piyachatr Pradubraj PACT Country manager Chatwut Wangwon staff Thirdchai Sattayapani staff Laos Vientenne USAID Laos Thomas D'Agnes USAID Laos PSI Laos Sawkerdas Latthanhol Interpreter Rob Gray Country director Sihamanu RDMA project manager Lattana Sales Manager Saysana Phanalasy Research manager PSI DIC Peuan Mai for MSM/TG and FSW Ketsoda Phantharong Drop in centre manager 13 peer educators and outreach workers Promotion of Education and Development Association (PEDA) Viyada M&E Manixay Outreach worker Santi Douangpraseuth Chairperson Lao PHA Sengphat M&E officer Souliyadeth Coordinator MSM Seneevonesuvang Outreach worker Vieng Akhone Souriyo Director Alounlack Finance officer Luang Prabang PCCA office Dr Thongsavath Director Lao Red Cross Dr Soulany Chancy Director Nitthala Group leader Buaphin Volunteer Xayalath Group Deputy leader Vanpasert Volunteer Darone Volunteer Saudalino Senkeo Provincial Coordinator PSI New Friends DIC (MSM & TG) Somphone DIC manager 8 MSM and TG peer educators Oulaikham Doctor Dao Doctor Sipasewth Assistant PSI New House DIC (FSW) Phetsammone Outreach worker 4 peer educators & 1 peer beneficiary Khammuane PCCA office Dr Odai Soprasert Director HIV/STI Lao Red Cross Dr Soulany Chancy Director and 6 volunteers of self help groups PEDA Phetsammone Outreach worker and 5 FSW peer educators/leader/beneficiaries PSI New Friends DIC (MSM & TG) Souliyo DIC Manager and 7 TG peer educators/leaders and 7 TG community members beneficiaries Myanmar Yangon PSI Myat Min Zaw HS manager (HIV/STI) Hla Myo Kyaw Country program manager Barry Whittle Country director Matt Boxshall HS advisor Tin Aung Strategic information manager NAP Yangon Myo Thant Regional officer NAP USAID Bill Slater Director office of Health Tu van Dinh Health advisor Mya Sapal Ngon Health Program Manager David Leong Deputy Mission Director Yangon Sun Quality clinic Dr Zay Yar Kyaw private practitioner TOP centre Yangon Anne Lancelot Director TOP 4 peer educators Nay Oo Lwin Program manager 2 medical officers 10 MSM and 8 TG community members beneficiaries 20 FSW community members beneficiaries Nay Pyi Daw IHD office, Ministry of Health Dr Htun Nyunt Oo Assistant Director NAP, DOH Naypyidaw Magwe Magwe TOP 6 FSW community members benficiaries 2 TG community members beneficiaries 6 peer educators/staff TOP peer educators/staff Magwe PACT Shae Thot Kyaw Min Htun Program officer CAP-3D Ei Thinzar Min Min Oo Program coordinator TB heroes 10 TB heroes 3 TB patients beneficiaries Sun Quality Health centre Dr Khin Aye Mon private practitioner Mandalay TOP center 9 FSW community members beneficiaries 7 MSM/TG community members and peer workers Dr Thwe Hnin Zin Acting Program officer, Medical officer Mi Mi Kyaw Community worker for FSW Wai Wai Thaung Lab technician Dr May Thu Thu Maw Medical doctor for FSW Dr Zar Ni Nyi Nyi Kyaw Medical doctor for MSM Han Mya Thein Dept assistant/admin assistant Pan Nu Counselor for FSW Casper Counselor for MSM Kyaw Thu Min Khaung Community worker for MSM Sun Quality health clinic DR Win Min Thant private practitioner Yangon UNAIDS Eaamon Murphy UNAIDS Country coordinator UNFPA Dr Hla Hla Aye Assistant representative Dr Ni Ni Khaing National program officer PMTCT/HIV GFATM Barbara Greenwood China Kunming PSI China Andrew Miller Country director PSI Fu Hongyun PSI Wang Kui Provincial Health Bureau Yunan Zeng Zi Yi Kunming AIDS Bureau Zhao Bing PSI Wu Haiyan PSI Wang Kai PSI Shi Zhi Ping PSI Huang Shan Translator SAA Hu Bao Wen Project officer Zhang Chang an Deputy Director SAA Dong Ze yan Project accountant Wang Zhuang Project assistant Hu Xiang Hao (HXH) DIC He Mu Kun Manager of HXH Liu Rong Community outreach coordinator Zhang Jianhong volunteer Li Ling Project assistant Qiu Wei Trainer He Kun Lin Volunteer Pan Li Hua Community outreach coordinator 4 community members beneficiaries Daguan Community Health Service centre Zheng Bauchun Vice-director CHC Yang Xue Feng Director Zhao Jian chun physician Yang Ruyan Head of Lab Rainbow Sky (CBO) Si Lu Volunteer Hua Yun Volunteer Le Yan Head of center Zhang Renchong Doctor CDC, Head of MSM TWG Sa Feng Coordinator/positive care He Ping staff Wu Dezhong staff Daytop Li Xinyue Deputy Director Wang Xiandan staff Dou Chang Jun TC staff Firefly Mo Yundang staff Pu Jiankun volunteer Song Wenqing staff Mengzi Sisterhood Health Home (SHH) Mr Jing Director of CDC Siao Peng Volunteer Ja min Volunteer Min Da Jie staff Wu Chui volunteer Wu Xiang Long volunteer Yang Ching Wei Mengzi CDC Kangxin House, Mengzi Shen member Wu Dan member Feng Ye Head of Kangxin Li Yu member Chao Yun Outreach worker Deng Outreach worker Deng Chi Bien staff for needle syringe program Ching Ker Chang member Chuen Chau Ming volunteer Gejiu Poplar Tree, Gejiu Shin De Ming head of CBO 4 beneficiaries members Kunming PACT China Li Chun Hua Program officer Shue Lin Program officer Guo Miao Senior Program officer RTI China Shicun (Tracy) Cui RTI Yunnan office program manager Zezhi Yang Dali Sea and Moon PLHIV Group Guangxi: Nanning Guangxi MSM Technical working group Wei Shi Guangxi TWG Coordinator Li Bih Green city Rainbow Working Group peer educator Cong Cong MSM TWG Secretariat Xin Cheng Xin (XCX) and Shuguang Jiayuan CBO Zhang Weiqi Coordinator Li Bin Guangxi Shuguanjiayuan Peer educator We Yiaodong client Wangzhou community rehabilitation centre Qiu Shuzhong Peer educator Meng Chun Peer educator Huang Hui client Lu Ping Sheng PSI peer educator Li Junyuan client Deng Zhifang Wangchou Community drug rehabilitation work station Liao Yi Wangchou Community drug rehabilitation work station Nanning CDC Huang zhining Nanning CDC peer educator Sun Heng Nanning CDC official Meeting with Guangxi and Nanning government officials Huang Wenhua Nanning anti-drug office Li Rongjian Guangxi CDC Chen Shirui Nanning Health Bureau Lu Yin xue Nanning CDC MMT Clinic director Sun Heng Nanning CDC Qin Gang Nanning Red Cross MMT Clinic doctor Wei Xiaodong Nanning Xingning district Dongziling community drug treatment(rehabitation) work station Lu Junli Nanning Huaqiang street community rehabitation workstation Dong Feifei Guangxi Jade ribbon director Chen Jianli Xinning Anti-drug office PSI Guangxi Nanning office Wang Jing Manager Huang Ting Ting PSI peer educator Qiu Shuzhong Chen kainan Zhang Hong Lu Pingshan Li Qiu Meng Chun Guangxi: Chengxi and Luzhai Chengxi community rehabilitation centre Madam Qin Party Secretary General, Luzhai Chengxi community office Mr Che Director Luzhai Chengxi community office Mrs Huang Vice director Mr Wei Peer educator Wei Juan Qiang Peer educator Mo Cai Mei Peer educator Mr Ti Peer educator 3 community members beneficiaries Luzhai Provincial health bureau and CDC Mrs Chen Vice director Luzhai county CDC Miss Wei Luzhai county CDC Ms Liang Luzhai county CDC Mr Chen Vice director AIDS Dept Luzhai country CDC Mr Mo director Luzhai town Narcotics control committee Thailand Bangkok PSI Thailand Amornrat Arunmanakul program officer Sanya Umasa Program coordinator Yaawalak Tittakuat research and M&E manager Andrew Boner CAP3D director Siriporn Jarocuchaikun Translator Alex Duke Program manager Siroat Jittjang assistant program manager Thai Research Center Dr Nittaya Phanuphak Senior research physician Somsong Nurse The POZ center foundation Somchai Phremsombut Director Tuss Abdullah Project Manager Raksakul Buajom Program manager SWING Bangkok Saichon Sira-u-rai staff Surang Jauyam Director Chamrong Phangnongyang Deputy director Thai MOPH and partners Patchara Benjara Hanapore Strategic information advisor UNAIDS Petchsri Sirinirund Director, National AIDS Management center Farida Langkafah MSM coordinator CDC Thamongsri Public Health Officer, DDC MOPH Pattaya Pattaya: Banglamung Hospital HIV/ART center Pitak Ketkong kwany Counselor ART clinic Pattaya city clinic Kularb Sandee STI/HIV nurse Sisters Chatchaya Wattijirakart Care and support counselor Thitiyanum Nakpor DIC Manager Glory Hut Foundation Pornsawan Christpirak Director Wichien Bangsiri Referral coordinator SWING Pattaya Manop Project Manager Swing Pattaya Pornpiehit VCT counselor Chiangmai Caremap Patchadej Reunkampoo Project officer Panuwat Palee activity officer Waramorn Promanee M&E officer Jednipat Yiamvaniehpong DIC officer Achanon Boonthep ex=-project manager Nattanop Chaiyarean Finance officer Mplus Eakpol Pichwong Outreach worker Pongprena Patpeerapong Manager Ratthawit Apipultipan Media and webmaster Thanapat Thephawan Outreach worker Jit Srichandorn Outreach worker Pramute Ohsim Outreach worker Ubonrut Poonjemjak accountant Bngthorn Chanleorn Director Save the Children Ratjai Adjayutytin Program manager Wutsitthikorn Suntree Nikos Dacanay researcher Duangkamon Donchaum HIV officer Provincial Health office Chiangmai Chonlisa Chariyalertsak HIV program in-charge Chutima Charuwat AIDS Care program in-charge Kriangkrai Yortrouang HIV surveillance in-charge Sanpatong Hospital Piyaporn Takamtha HIV/AIDS nurse Chaunat Promping Medical technologist Saraphi Hospital Amarin Norchaiwong Chief counseling team List of strategic information documents pertaining to FSW, MSM and TGs No Title Year Agencies/Authors Type Laos 1 Understanding male sexual behaviour in planning HIV prevention programmes: lessons from Laos, a low prevalence country Sex Transm Infect. 2006 April; 82(2): 135–138. M J Toole, B Coghlan, A Xeuatvongsa, W R Holmes, S Pheualavong, and N Chanlivong Research 2 HIV prevalence and risk behaviour among men who have sex with men in Vientiane Capital, Lao People’s Democratic Republic, 2007 AIDS 2009, 23:000– 000 Sarah Sheridan, Chansy Phimphachanh, Niramonh Chanlivong, Sisavath Manivong, Sod Khamsyvolsvong, Phonesay Lattanavong, Thongchanh Sisouk, Carlos Toledo, Martha Scherzer, Mike Toolea and Frits van Griensvenc,d Research 3 LAOS (2008): HIV Prevention Behavior TRaC Study Among Lao Female Sex Workers ROUND 2 2008 PSI, USAID, GFATM TRAC report 4 Behavioral survey among service women and Integrated biological and behavioral surveillance survey among men who have sex with men in Luang Prabang 2009 2009 FHI, GFATM Survey report 5 Integrated Behavioral BiologicalSurveillance among Service Women in Five Provinces of Lao PDR. 2009 CHAS, FHI, USAID, GFATM, WHO, UNAIDS Survey report 6 Lao PDR 2009 Surveillance Surveys: Integrated Biological and Behavioral Surveillance Survey Among Men Who Have Sex with Men in Luang Prabang, 2009 2009 FHI, USAID, CHAS Survey report 7 Prevalence of HIV, Chlamydia and Gonorrhea among Transgender Women in Vientiane and Savannakhet, Laos: findings from a bio-marker survey draft PSI Publication 8 HIV testing and STI screening for transgender women in Laos: a quasi-experimental study of the New Friends program draft PSI Publication 9 First Round HIV/STI Prevalence and Behavioral Tracking Survey among Male-to￾Female Transgenders in Vientiane Capital and Savannakhet, 2010 2010 PSI Survey report 10 Results of Second Round HIV/STI Prevalence and Behavioral Tracking Survey among Male-to-Female Transgenders in Vientiane Capital and Savannakhet Lao PDR, 2012 (2012 report still in draft) PSI, USAID, CHAS Survey report 11 Exploring the sexual networks of men who have sex with men and women in Lao PDR 2011 Burnett Institute, CHAS, MOH Research 12 The people living with HIV stigma index: results from 3 provinces in Lao PDR – Luangprabang, Vientiane capital and Champasak. Draft 2012 UNAIDS, ILO Survey report 13 Defining and redefining harm reduction in the Lao context Harm Reduction Journal 2012, 9:28 Vanphanom Sychareun, Visanou Hansana, Sysavanh Phommachanh, Vathsana Somphet, Phouthong Phommavongsa, Brigitte Tenni, Timothy Moore and Nick Crofts 14 Rapid Assessment of HIV and injecting and non-injecting drug use in Laos Kevin Mulvey, USAID report China 1 China (2010): Routine Behavioral Tracking (RBT) Survey with Female Sex workers in Mengzi, Kunming and Gejiu 2010 PSI report 2 China (2010): Routine Behavioral Tracking (RBT) Survey: Draft summary report of key findings. MSM in Kunming, Nanning and Honghe Prefecture 2010 PSI report 3 China (2012): Routine Behavioral Tracking (RBT) Survey among Community-based Injecting Drug Users (IDUs) in Yunnan and Guangxi 2011 Sept PSI and other partners ppt 4 Do community-based strategies reduce HIV risk among people who inject drugs in China? A quasi-experimental study in Yunnan and Guangxi Provinces 2013 May PSI and other partners report 5 Routine Behavioral Tracking (RBT) Survey among Men who Have Sex with Men (MSM) in Yunnan and Guangxi in 2010: A Comparison of HIV-Related Risk Behaviors among MSMO and MSMW in Southwest China 2012 PSI ppt 6 Routine Behavioral Tracking (RBT) Survey among Community-based Injecting Drug Users (IDUs) in Yunnan and Guangxi in 2012 2012 PSI ppt 7 Hu Xiang Hao ‘Good for you, Good for me’ - A Model of Evidence-based Behavioral Change Communication Interventions for Injecting Drug Uses in Yunnan in Southwestern China 2012 PSI ppt 8 FoQuS- China (2012): Understanding HIV Related Risk Behaviors and Exploring Associated Factors among Men who Have Sex with Men in Kunming & Nanning 2012 PSI report 9 CHINA (2013): ROUTINE BEHAVIORAL TRACKING (RBT) Measuring Consistent Condom Use, Use of VCT, and Use of STI Testing and Treatment Services among Men Who Have Sex with Men (MSM) in Kunming and Nanning Round II 2013 PSI report 10 Social Services Outsourcing to Social Organizations in the HIV Sector in Yunnan Province. Unleashing social innovation or business as usual? Exploratory study 2012 RTI report 11 A Path to Strengthen Funding Efficiency and Mutual Trust between Government and Social Organizations: Bidding Procedures for Social Services Outsourcing to Social Organizations in the HIV Sector 2013 (draft) RTI report 12 Policy Update on Social Services Outsourcing in the HIV Sector 2013 (draft) RTI report 12 Partners in Service Delivery: Pathways to Strengthened Collaboration and Cooperation for Social Services Outsourcing to Social Organizations in the HIV Sector 2013 (draft) RTI report Myanmar 1 Estimation of female sex worker size in Myanmar: how different methods contribute 2011 PSI Report 2 FoQus on Qualitative Segmentation on Determinants of Consistent Condom Use with Clients among FSW Jun-05 PSI ppt 3 FoQus on Qualitative Segmentation on Factors Influencing Consistent Condom Use with Male Partners among MSM 2011 PSI ppt 4 STI prevalence among DiC attendees Jun-05 PSI ppt 5 FoQus on TB barriers to diagnosis 2012 PSI ppt 6 THE ASSESSMENT OF THE PERCEPTION OF TB PATIENTS ON QUALITY OF CARE (1st ROUND, MARCH 2011) 2011 PSI Report 7 THE ASSESSMENT OF THE PERCEPTION OF TB PATIENTS ON QUALITY OF CARE (4th ROUND, MARCH 2012) 2012 PSI Report 8 THE ASSESSMENT OF THE PERCEPTION OF TB PATIENTS ON QUALITY OF CARE (August 2010) 2010 PSI Report 9 THE ASSESSMENT OF THE PERCEPTION OF TB PATIENTS ON QUALITY OF CARE (March 2010) 2010 PSI Report 10 FSW size assessment in selected cities 10.8.2012 2012 PSI ppt 11 Myanmar 2011 HIV sentinel surveillance 2011 WHO, NAP Report 12 HIV sentinel surveillance report 2012 2012 NAP Report 13 The Cultural Queasiness Factor: Intersections of Gender, Sexuality and HIV Prevention in Burma/Myanmar 2011 Asian Studies Review, 35: 2, 189-207 Gillian Fletcher publication 14 Using and Joining a Franchised Private Sector Provider Network in Myanmar PLoS ONE 6 (12): e28364 PSI: Kathryn O’Connell, Mo Hom, Tin Aung, Marc Theuss, Dale Huntington publication 15 Equity and the Sun Quality Health Private Provider Social Franchise: comparative analysis of patient survey data and a nationally representative TB prevalence survey. Int J Equity Health. 2013 Jan 10;12:5 PSI: Montagu D, Sudhinaraset M, Lwin T, Onozaki I, Win Z, Aung T. publication 16 Validation of a new method for testing provider clinical quality in rural settings in low and middle income countries: the observed simulated patient PlosOne 7 (1) PSI: Tin Aung et al publication 17 Reaching Men Who Have Sex with Men in Myanmar: Population Characteristics, Risk and Preventive Behavior, Exposure to Health Programs AIDS Behav 2012 PSI Tin Aung, Willi McFarland, Ehthi Paw & John Hetherington publication 18 Coverage of HIV Prevention Services for Female Sex, Workers in Seven Cities of Myanmar AIDS Behav 2013 Tin Aung, Ethi Paw, Nyo Me Aye, Willi McFarland publication Thailand 1 Reducing HIV risk among transgender women in Thailand: a quasi-experimental evaluation of the Sisters program PlosOne PSI Publication 2 THAILAND (2012): Baseline HIV/AIDS TRaC Study among Male Sex Workers in Bangkok and Pattaya TRAC PSI report 3 THAILAND (2011): Baseline HIV/AIDS TRaC Study among Transgenders in Pattaya, Sattahip and Sriracha TRAC PSI report 4 FOQus 2011: Factors Associated with Sexual Risk Behaviors among Transgenders A Qualitative Study among TGs in Pattaya, Sattahip and Sriracha, Chonburi Province foQus PSI report