EVALUATION THE REGIONAL HIV/AIDS PREVENTION AND CARE PROJECT MID-TERM PERFORMANCE EVALUATION REPORT November 2015 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by USAID/WEST AFRICA THE REGIONAL HIV/AIDS PREVENTION AND CARE PROJECT MID-TERM PERFORMANCE EVALUATION REPORT November 25, 2015 AID-624-C-15-00001 Cover Photo: MSM in Togo Credit: Stephane d’Almeida DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGEMENTS IBTCI extends a special thank you to Sheila Mensah, Contracting Officer’s Representative (COR) for Evidence for Development (E4D) and Abdul Bala, her Alternate in the United States Agency for International Development (USAID)/ West Africa (WA) Regional Health Office in Accra, Ghana for their unyielding support and guidance. We also are grateful for the assistance of Laurent Kapesa, PACTE VIH the Agreement Officer’s Representative. We appreciate the PACTE-VIH’s Implementing Partners cooperation, particularly the FHI 360 team in Accra, as well as in Togo and Burkina Faso, for their valuable assistance with logistics and coordination with the evaluation respondents. We also express gratitude to all the stakeholders for their interest and participation. Finally, we are thankful for the opportunity to speak with health facility staff, community members and representatives among the key populations in all the sites we visited. Without the cooperation of the respondents, this evaluation would not be possible. i TABLE OF CONTENTS EXECUTIVE SUMMARY.................................................................................................................................................. 1 BACKGROUND .......................................................................................................................................................... 1 EVALUATION PURPOSE .......................................................................................................................................... 2 EVALUATION QUESTIONS..................................................................................................................................... 3 EVALUATION METHODOLOGY.......................................................................................................................... 3 KEY FINDINGS AND CONCLUSIONS................................................................................................................ 4 Assessing Access...................................................................................................................................................... 4 Evaluation of the Effectiveness.............................................................................................................................. 4 Assessing Enabling Environment........................................................................................................................... 5 Assessment of the Health Seeking Behavior: Behavior Change Communications.................................. 5 Evaluation of Perceptions...................................................................................................................................... 6 Assessment of Ability to Scale-up ........................................................................................................................ 6 RECOMMENDATIONS.............................................................................................................................................. 7 EVALUATION PURPOSE AND EVALUATION QUESTIONS............................................................................. 8 EVALUATION PURPOSE .......................................................................................................................................... 8 EVALUATION QUESTIONS................................................................................................................................... 10 PROJECT BACKGROUND.......................................................................................................................................... 12 DESCRIPTION OF PACTE-VIH ............................................................................................................................. 12 IR1- Increased access to high-quality/comprehensive prevention and care programs for MARP ..... 13 IR2- Improved enabling environment for evidence-based public health interventions targeting MARP........................................................................................................................................................................ 13 IR3- Improved quality of Global Fund HIV prevention interventions that focus on MARP ................ 13 IR4- Increased access to quality and timely information/data focused on MARP interventions......... 13 EVALUATION METHODS AND LIMITATIONS................................................................................................... 14 EVALUATION TEAM................................................................................................................................................ 14 EVALUATION DESIGN ........................................................................................................................................... 14 DATA SOURCES ....................................................................................................................................................... 14 Secondary data ....................................................................................................................................................... 14 Primary data............................................................................................................................................................ 14 SAMPLING STRATEGY............................................................................................................................................ 15 LIMITATIONS ............................................................................................................................................................. 17 DATA ANALYSIS....................................................................................................................................................... 18 ETHICAL CONSIDERATIONS .............................................................................................................................. 18 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ............................................................................ 22 ASSESSING ACCESS ................................................................................................................................................. 22 ii Burkina Faso............................................................................................................................................................ 22 Togo.......................................................................................................................................................................... 24 EVALUATION OF THE EFFECTIVENESS ........................................................................................................... 25 ASSESSING ENABLING ENVIRONMENT..................................................................................................... 30 ASSESSMENT OF THE HEALTH SEEKING BEHAVIOR: BEHAVIOR CHANGE COMMUNICATIONS.......................................................................................................................................... 33 ASSESSMENT OF ABILITY TO SCALE-UP .................................................................................................... 39 DISCUSSION AND CONCLUSIONS....................................................................................................................... 43 EVALUATION OF ACCESS .................................................................................................................................... 43 EVALUATION OF THE EFFECTIVENESS ........................................................................................................... 44 ASSESSING ENABLING ENVIRONMENT.......................................................................................................... 45 EVALUATION OF PERCEPTIONS ....................................................................................................................... 46 ASSESSMENT OF ABILITY TO SCALE-UP ......................................................................................................... 46 RECOMMENDATIONS................................................................................................................................................. 47 BIBLIOGRAPHY .............................................................................................................................................................. 49 ANNEX 1: CONFLICT OF INTEREST DISCLOSURE.......................................................................................... 51 ANNEX 1 A: PACTE-VIH POST-MID-TERM EVALUATION RECOMMENDATIONS ACTION REVIEW TABLE................................................................................................................................................................................. 52 ANNEX 3: PACTE-VIH TARGET SITES IN TOGO AND BURKINA FASO.................................................. 56 ANNEX 4: STATEMENT OF WORK........................................................................................................................ 58 ANNEX 5: DOCUMENTS FOR REVIEW................................................................................................................. 74 ANNEX 6: QUESTIONNAIRE FOR MSM INTERVIEW....................................................................................... 75 ANNEX 7: QUESTIONNAIRE FOR FSW INTERVIEW ....................................................................................... 75 ANNEX 8: IN-DEPTH INTERVIEW GUIDE............................................................................................................ 75 ANNEX 9: FOCUS GROUP GUIDELINE................................................................................................................. 75 i ANNEXES Annex 1: Conflict of Interest Disclosure Annex 2: Abridged Bios of the Evaluation Team Members Annex 3: PACTE-VIH Target Sites in Togo and Burkina Faso Annex 4: Statement of Work Annex 5: Documents for Review Annex 6: Questionnaire for MSM Interview Annex 7: Questionnaire for FSW Interview Annex 8: In-Depth Interview Guide Annex 9: Focus Group Guideline LIST OF TABLES Table 1: Evaluation questions and their respective analytical domain Table 2: Description of key population by selected socio-demographic characteristics in Burkina Faso and Togo Table 3: Number of In-Depth Interviews (IDI) and Focus Group Discussions Conducted in Burkina Faso and Togo Table 4: Design/Data Analysis Matrix Table 5: Number of people tested and counseled for HIV in Burkina Faso (Year 1 – Year 3) Table 6: Number of people tested and counseled for HIV in Togo (Year 1 – Year 3) Table 7: Aggregate Achievements at Point of Midterm Evaluation Table 8: Proportion of KP who have benefited from PACTE-VIH activities in Burkina Faso Table 9: Proportion of KP who have benefited from PACTE-VIH activities in Togo Table 10: Achievements of key activities planned to enable the environment for optimized HIV Prevention Table 11: Sexual behavior of FSWs and MSM in Burkina Faso Table 12: HIV/AIDS and Health behavior among KP in Burkina Faso Table 13: Sexual behavior of FSWs and MSM in Togo Table 14: HIV/AIDS and Health behavior among KP in Togo LIST OF FIGURES Figure 1: The PACTE-VIH components and intermediate results (IR) Figure 2: Evaluation Questions Figure 3: The regional HIV/AIDS prevention and care project results framework Figure 4: Number of condoms and lubricants distributed Year 1-Year 3 (Quarter 1) in Burkina Faso Figure 5: Number of condoms and lubricants distributed Year 1-Year 3 (Quarter 1) in Togo Figure 6: Participation of Key Population to the PACTE-VIH activities in Burkina Faso Figure 7: Participation of Key Population to PACTE-VIH activities in Togo Figure 8: Proportion of Key Population by circumstances leading to unprotect sexual intercourses in Burkina Faso Figure 9: Distribution of Key Populations by circumstances leading to unprotected sexual intercourse in Togo ii ACRONYMS AAB Alternatives Burkina Faso AAS Association Africaine Solidarité ACI Africa Consultants International ADS Automated Directives System AIDSETI AIDS Empowerment and Treatment International AJDRB Association des Jeunes pour le Développement de Bittou AMMIE Association Appui Moral Matériel et Intellectuel à l’Enfant AOR Agreement Officer’s Representative ART Anti-retroviral Therapy ASD Association SOS Santé et Dévelopment AWARE II Action for West Africa Region II BCC Behavior Change Communication CBO Community Based Organization CCM Country Coordinating Mechanism CDC Centers for Disease Control and Prevention CNLS National Council for the Fight Against AIDS (Conseil National de Lutte Contre le Sida) COPC Continuum of Prevention and Care CPHDA Center for the Promotion of Human Rights and Development in Africa CSO Civil Society Organization DIC Drop-in Center DQA Data Quality Assessments DQI Data Quality Improvement ECOWAS Economic Community of West Africa States EVT Espoir Vie Togo FAMME Forces in Action for Mother and Child Well Being FC Female Condom FGDs Focus Group Discussions FHI360 Family Health International FP Family Planning FSW Female Sex Worker GBV Gender-Based Violence GFATM Global Fund to Fight AIDS, TB and Malaria GRADE Grades of Recommendation, Assessment, Development and Evaluation HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome HPP Health Policy Project HTC HIV Testing and Counseling IBTCI International Business Technical Consultants, Inc ICT Information and Communication Technology IDI In-depth interview IDU Intravenous Drug Use iii IP Implementing Partner IST/ STI Infections sexuellement Transmissibles/ Sexually Transmitted Infections KIS Key Informant Survey KP Key Population LB Live birth LDTD Long Distance Truck Drivers LNGO Local Non-Governmental Organization MARP Most-at-Risk Population MOH Ministry of Health MSM Men Who Have Sex with Men NAC National AIDS Commission CNLS National Council Against AIDS NETCAT Network Technical Capacity Assessment Tool NGO Non-Governmental Organization NSP National Strategic Plan PACTE-VIH The Regional HIV/AIDS Prevention and Care Project PE Peer Educator PLHIV People Living with HIV PMP Performance Monitoring Plan PMTCT Prevention of Mother-to Child Transmission of HIV PR Principal Recipient PRB Population Reference Bureau PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses PSI Population Services International PWUD Persons Who Use Drugs QI Quality Improvement REVS+ Responsabilité Espoir Vie Solidarité RHO Regional Health Office SBC Strategic Behavior Change SME Subject Matter Expert SOW Statement of Work SR Sub-Recipients SSA Sub-Saharan Africa STI Sexually Transmitted Infections TAB Technical Advisory Board TFR Total Fertility Rate TPM Team Planning Meetings TWG Technical Working Group UIC Unique Identifying Code UNAIDS Joint United Nations Program on HIV/AIDS UNDP United Nations Development Programme UNESCO United Nations Organization for Education, Science and Culture UNFPA United Nations Population Fund iv UNICEF United Nations Children’s Fund USAID United States Agency for International Development USAID/WA United States Agency for International Development/West Africa Mission VCT Voluntary Counseling and Testing WAHO West African Health Organization 1 EXECUTIVE SUMMARY BACKGROUND The West African region has a low HIV prevalence (2.5%) in the general population compared to other Sub-Saharan Africa (SSA) regions. However, while HIV prevalence is low in the general population, (3.2% in Togo, 1% in Burkina Faso) it is quite high among female sex workers (FSWs) and men who have sex with men (MSM). The prevalence of HIV is estimated at 28.5% among FSWs in Lomé (Togo), 14.4% in Ouagadougou and 32,7% in Bobo-Dioulasso (Burkina Faso). Studies also revealed that 5% of MSM in Ouagadougou and 32.7% in Bobo-Dioulasso (Burkina Faso), and 18.5% in Lomé (Togo) live with HIV. The combination of three factors drives the spread of HIV and sexually transmitted infections (STIs) and HIV/AIDS among FSWs and MSM in Togo and Burkina Faso: (1) low condom use, (2) weak health systems and poor quality health services, and (3) stigmatization and discrimination of the most-at-risk populations (MARP) in these countries. These problems are further exacerbated by the lack of qualified staff to support the commodities and logistics system, and delayed, inconsistent, and/or suspended disbursement of funding from the Global Fund to fight AIDS, Tuberculosis and Malaria (Global Fund) due to governance and transparency issues. Against this backdrop, the United States Agency for International Development (USAID) funded the Regional HIV/AIDS Prevention and Care activity (PACTE-VIH). This is a five-year cooperative agreement (August 2012 – August 2017) between the USAID/West Africa Mission (USAID/WA) and Family Health International (FHI 360) (AID-624-A-12-00006) with a total budget of $13,500,000. This regional activity aimed to address the critical gaps in key population (KP) programming across the region by strategically targeting interventions that build political will, and by allocating resources to engage key stakeholders to support programs that are evidence-based, adaptable, replicable and transferable throughout the region. The activity initially focused on four sites in Burkina Faso (Ouagadougou, Bobo-Dioulasso, Bittou, and Niangologo) and in five sites in Togo (Lomé, Kpalimé, Kara, Cinkassé, and Dapaong). USAID/WA’s E4D activity was tasked to conduct an independent, external mid-term performance evaluation of PACTE-VIH to measure achievements of the activity in regard to the sexual behavior of KP, attitudes toward FSWs and MSM, and access of KP to HIV/AIDS services. Lessons learned from this research have the potential to inform HIV/AIDS prevention and care implementation policies in other countries of the West African region, as well as other regions, such as SSA. PACTE-VIH has four major components and targets four intermediate results (IRs) (see Figure 1). 2 Figure 1 – The PACTE-VIH components and intermediate results (IR) Component Intermediate Result (IR) Develop a scalable, replicable model for Burkina Faso and Togo that can be adapted in the region for the provision of essential and comprehensive prevention and support services for MARP (FSWs, their clients, and MSM) in mixed epidemic environments with high levels of stigma and discrimination. IR1- Increased access to high￾quality/comprehensive prevention and care programs for MARP Create an enabling environment for optimized HIV prevention among MARP in Burkina Faso and Togo, especially FSWs, their clients and MSM, by reducing stigma and discrimination and by building the capacity of community health facilities, national governments, and regional partners to sustain quality, public health HIV/AIDS MARP programming. IR2 - Improved enabling environment for evidence-based public health interventions targeting MARP Provide technical assistance to selected country coordinating mechanisms (CCM) and Global Fund principal applicants and sub-applicants, in order to strengthen application submissions and improve the quality of prevention programming respectively, with a specific emphasis on MARP and in particular, FSWs, their clients, and MSM. IR3 - Improved quality of GFATM HIV prevention interventions that focus on MARP Develop a communication strategy to disseminate program data to key stakeholders. IR4 - Increased access to quality and timely information/data focused on MARP interventions. EVALUATION PURPOSE This PACTE-VIH mid-term performance evaluation has three objectives: 1. Examine the implementation of the model service package; 2. Determine what is working well and what is not working; and 3. Gain some understanding of how the model can be replicated in other countries in West Africa. The target audience for the mid-term evaluation is decision-makers and managers in the USAID/WA Regional Health Office (RHO), Regional Office of Acquisition & Assistance, the Mission Director, Deputy Mission Director, PACTE-VIH Agreement Officer’s Representative (AOR), PACTE-VIH management team, and Ministry of Health (MOH) and National Council for the Fight against AIDS (CNLS) officials in Togo and Burkina Faso. Funding allocated specifically for this performance evaluation amounted to $156, 980. The evaluation findings will be used to inform on-going implementation of the PACTE-VIH project and implementing partners (IPs), as well as the design of future regional HIV/AIDS projects. The evaluation team was comprised of an expatriate Team Leader, Thomas Park; an expatriate Evaluation Specialist, Amadou Moreau; a Burkinabe Senior Subject Matter Expert (SME), Issa Zongo; a Togolese Senior SME, Stephane D’Almeida; four research assistants, two located in Burkina Faso and two in Togo; two note takers (for Burkina Faso and Togo); and a Logistician. Our team members possessed a mix of expertise in evaluation and HIV/AIDS programming. Sam Ngambo, Executive Director of the Center for the Promotion of Human Rights and Development in Africa (CPHDA) provided management and administrative oversight of the local consultants, research assistants and logistician. IBTCI E4D staff, Jacques Emina, Salima Mutima, and Annette Bongiovanni led the data analysis and writing of the report, with the assistance of the evaluation team members (IBTCI, 2015). Please see Annex 1 for Conflict of Interest Disclosure forms. 3 EVALUATION QUESTIONS The study framework relies on nine questions corresponding to six analytical domains (See Figure 2): Figure 2 – Evaluation Questions Assessing Access ● Evaluation Question 1: How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? Evaluation of the Effectiveness ● Evaluation Question 2: What is working well and what is not working well for the KP programming in Togo and Burkina Faso? Assessing Enabling environment ● Evaluation Question 3: How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso? Assessment of the Health seeking behavior ● Evaluation Question 4: How has this HIV prevention package influenced health-seeking behavior among KP? Evaluation of Perceptions ● Evaluation Question 5: How do KP perceive the services and providers? ● Evaluation Question 6: How do service providers perceive the services they provide to KP? ● Evaluation Question 7: How do other stakeholders Assessment of ability to Scale-up ● Evaluation Question 8: What conditions are necessary to scale-up this model in other countries in the region? ● Evaluation Question 9: How might the elements of the model be adapted in other West African countries? EVALUATION METHODOLOGY This mid-term performance evaluation used a non-experimental cross-sectional design. It focused on the implementation processes employing mixed methods. The analyses rely on secondary and primary data. The secondary data gathered includes existing surveys and activity monitoring reports, the PACTE Monitoring and Evaluation (M&E) Plan indicators, and Ministry of Health (MOH) data. The primary data consists of key population informant surveys (KPs), focus group discussions (FGDs) and in-depth interviews (IDIs). Table 3 presents the sample size for KPs and FGDs in the two countries. The sample includes 101 KPs in Burkina Faso (51 FSWs and 50 MSM), 105 KPs in Togo (53 FSWs and 52 MSM), 10 FGDs in Burkina Faso among which seven were with FSWs, and eight FGDs in Togo (5 FSWs and 3 MSM). The IDIs’ database includes 12 key stakeholders in Burkina Faso and 15 in Togo. More than 65% of participants in KP surveys in the two countries were aged less than 30 years old (65% or above) and/or have never been in union. The greater proportion of surveyed FSWs in the two countries (54% or above) had at least a child. This proportion is higher in Burkina (72%) compared to Togo (54%). The minority of surveyed MSM (18% in Burkina Faso and 8% in Togo) had reported having a child. Furthermore, less than 50% of interviewed FSWs have secondary education in the two countries, whereas more than 90% of studied MSM had attended at least secondary school. Data analysis relied on analyses of proportion for quantitative data and content and thematic techniques using Atlas.ti software for qualitative data. The main limitation of the methodology is that the KP surveys used a very small sample size, which undermines the reliability of estimates. In addition, this study selected KP sampling from the PACTE-VIH health users. These populations might have different characteristics (current or past health status, 4 acceptability of the KP status, residence distance to the PACTE-VIH health facilities, etc.) compared to other KP (who did not visit health facilities). KEY FINDINGS AND CONCLUSIONS Our findings revealed both similarities and differences between samples of FSWs and MSM within the country as well as similarities and differences between countries depending on the analytical domain or the indicator considered. Assessing Access How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? PACTE-VIH has supported 23 partner clinics in Burkina Faso and Togo in collaboration with the national coordinating committees. These clinics employ 201 health-care workers and 110 peer educators. Data from all sources reported greater access to condoms and lubricants and STI/HIV services among KP enrolled at the PACTE-VIH program. The program has distributed over one million male condoms (487,937 in Burkina Faso and 563 in Togo), 124,231 female condoms (93,763 in Burkina Faso and 30,468 in Togo), and 461, 964 lubricants (224,085 in Burkina Faso and 237,879 in Togo). The large majority of KP interviewed in Burkina Faso (88% of FSWs and 98% of MSM) and in Togo (84% of FSWs and 60% of MSM) reported having received condoms and lubricants from PACTE-VIH program. The program has conducted HIV-test for 6,940 KP in Burkina Faso, and 7,014 HIV tests in Togo during the first two years of the program. FSWs represented the majority of tested KP (50%) in the two countries compared to MSM (around 20% or less). Findings from KP surveys indicated that 47% of FSWs in Burkina Faso and 57% of FSWs in Togo have received HIV testing in PACTE-VIH facilities, whereas the corresponding figures for MSM are estimated at 22% in Burkina Faso and 16% in Togo. Evaluation of the Effectiveness What is working well and what is not working well for the KP programming in Togo and Burkina Faso? Overall, the activities successfully reached the targets (sometimes over 100%). A total of 38,218 KP (4,705 MSM, 16,656 FSWs and 16,857 clients of FSWs) were reached (in Year 1 and Year 2) in both countries through one-on-one or group level peer education sessions, an average of 113% achievement over the project life. Data from FGDs and IDIs have reported that the PACTE-VIH partner clinics and drop-in centers (DICs) in Burkina Faso and Togo were fully operational in December 2014. They offer a standardized package of Continuum of Care and Prevention (COPC) services, including HTC and STI services, referrals, peer educator (PE) outreach activities, social events for KP, psychosocial counseling and support groups, distribution of condoms and lubricants, human rights and gender-based violence (GBV) awareness and support services, as well as income generating activities. The program increased targets and achievements every year. The number of KP reached increased from 7,871 in Year 1 to 30,347 in Year 2. Likewise, the program has distributed over one million condoms and lubricants in Year 2 compared to 378,000 in Year 1. In Year 2, 9,741 KP received HIV testing, compared to 4,532 KP in Year 1. However, the program experienced low performance regarding the number of KP who completed referrals (especially in Burkina Faso), the number of condoms and lubricants distributed (Years 1 and 2), and the number of KP reached with stigma message. In Year 2, the percentage of individuals with completed referrals was estimated at 50% in Burkina Faso, compared to 55% targeted and 81% in Togo. The number of condoms and lubricants distributed (Years 1 and 2) and the numbers 5 of KP reached with stigma message were below the targets. Nevertheless, the percentage of achievements for the last indicator (the number of KP reached with stigma message) has increased in Year 3. Serious shortages and stock-outs of HIV/STI Prevention Commodities such as condoms, lubricants, HIV test kits, and STI and HIV drugs experienced throughout Year 1 and the beginning of Year 2 explained low performance regarding numbers of condoms and lubricants. Low referral rates and a high level of stigma and discrimination in Burkina Faso were listed as key challenges in the program implementation. Assessing Enabling Environment How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso? PACTE-VIH has organized several advocacy and capacity building activities to improve the skills of KP and stakeholders to engage in policy dialogue and improve service delivery. More than 600 healthcare workers successfully completed an in-service training program for HIV related service delivery. More than 200 MARP-friendly clinic staff received regular supportive supervision and mentoring. In 2014, 30 journalists and 55 Senior Editors and Media Managers (20 in Togo and 35 in Burkina Faso) attended an advocacy consultative workshop on issues relating to KP. In addition, the program has provided technical assistance to IPs in the area of M&E, and through mentorship and support supervision from the PACTE-VIH senior management team to the IPs. The program has also organized training on proctology. Data from KP surveys reported a high level of sensitization activities and condom distribution in the two countries. About 70% of FSWs in Burkina Faso and 84% of FSWs in Togo reported having benefits from PACTE-VIH sensitization activities. However, most of the FSW in both countries reported that efforts should be taken regarding interventions that explicitly address GBV and stigmatization from the security force and media. The large majority of FSWs and MSM in the two countries feel stigmatized though the program has contributed to their capacity strengthening and to their access to sexual and reproductive health services. Nevertheless, all KP have never been denied access to health facilities, especially the PACTE-VIH affiliated facilities. Assessment of the Health Seeking Behavior: Behavior Change Communications How has this HIV prevention package influenced health-seeking behavior among KP? Analyses of health seeking behavior using qualitative and quantitative suggested mixed findings. Whereas more than 90% of surveyed KP reported having received a HIV test in the two countries, 10% or less have received HIV testing after unprotected sexual intercourse. Only a few respondents (8% of FSWs and 10% of MSM) had a HIV test because of unprotected sexual intercourse in Burkina Faso. These proportions are estimated at 4% for FSWs and 10% for MSM in Togo. Furthermore, whereas in Burkina Faso, 72% of MSM reported using condoms systematically during sexual intercourse, the proportion is estimated at 47% for FSWs in the same country. Less than 40% of interviewed FSWs in Burkina Faso reported using condom with their main partners compared to 78% for surveyed MSM. The large majority of surveyed FSWs (>80%) in Togo reported systematic use of condoms during sexual intercourse, except with the usual partner (45%). Fifty-five percent have reported unprotected sexual intercourse with the regular partner. 6 Evaluation of Perceptions How do KP perceive the services and providers? How do service providers perceive the services they provide to KP? Findings from KP surveys and FGDs suggested that the majority of KP value the PACTE-VIH STI and HIV/AIDS services, including the availability of health staff. They reported that they have never been denied health services at the PACTE-VIH affiliated facilities. The large majority of the KP survey participants in Burkina Faso (94% of FSWs and 98% of MSM) prefer to visit the PACTE-VIH affiliated centers for STI and HIV/AIDS information and care. All stakeholders in the two countries reported benefits to the PACTE-VIH collaboration. They mentioned financial and logistic (inputs, including antiretroviral treatment) support, and individual and institutional capacity building. KP studied in Togo consider the PACTE-VIH program as a capacity strengthening opportunity: human right strategies to avoid STI/HIV/AIDS. The majority of stakeholders’ institutions, including the joined United Nations program on HIV/AIDS and the national HIV/AIDS programs, revealed that PACTE-VIH program has improved KP access to sexual and reproductive health services. Condoms, lubricants, STIs/HIV tests and treatments are available at the DICs. Furthermore, the DICs are referring patients to ACS services, whereas PEs cadres are mobilizing and following up KP. However, in both countries participants (stakeholders and KP) raised the issue of sustainability of activities due to financial problems and lack of capacity strengthening. Assessment of Ability to Scale-up What conditions are necessary to scale-up this model in other countries in the region? How might the elements of the model be adapted in other West African countries? Findings from IDIs suggest four key prerequisites to scaling up the PACTE-VIH model: (1) rely on the evidence-based approach; (2) strengthen individual and institutional capacity; (3) create integrated sub￾regional structures based on existing institutions; and (4) create or improve the enabling the environment, especially garnering political will. Stakeholders, including United Nations Population Fund (UNFPA), United Nations Development Programme (UNDP) and Population Services International (PSI) support the need for the MOH structures to be more involved in the PACTE-VIH activities (Country Coordinating Mechanism (CCM)) and the necessity for the program to be developed and implemented within the national health policy framework (CNLS2). It is also important to identify key partners for effective and sustainable intervention (media, police, justice, etc.). Some stakeholders suggested that the PACTE-VIH package should integrate family planning (FP) services in the future because all FSWs are aged 15-49. In conclusion, the PACTE-VIH program constitutes an opportunity and hope for fighting HIV/AIDS, as well as promoting human rights in Burkina Faso and Togo through KP. The program has achieved most targets through sensitization, distribution of condoms and lubricants, and providing HIV services. Furthermore, the program has organized several activities to better enable the environment through workshops and the media. However, efforts should be made to implement interventions that explicitly address GBV and stigmatization from the security force and media staff. The mixed sexual behavior of KP in Burkina Faso and Togo, the persistent, though declining, internal and external barriers due to traditions, and the lack of sustainable financial and human resources depicts a truly complex situation of the dynamics of HIV in the studied communities. 7 RECOMMENDATIONS • Findings suggested low access to HIV testing in PACTE-VIH facilities. The program should increase sensitization and activities to encourage HIV testing, especially after unprotected sexual intercourse. The program should provide regularly required inputs to achieve this objective. • Distribution of condoms was low during Year 1 and Year 2. About 44% of male condoms were distributed in the first quarter of the third year. The program should make an effort to maintain the availability and distribution of condoms over the project life. • The activities successfully reached the targets (sometimes over 100%). Achievements above 100% suggest the need to develop more realistic targets, and avoiding underestimation of targets. • Efforts should be made regarding interventions that explicitly address GBV and stigmatization from the security force and media staff. • Assess the effectiveness of the DICs and other KP-specialized facilities through a trends analysis of users, the user profiles, and the activity’s attraction strength (number and proportion of new visitors). This could be the topic of operations research. • KP and stakeholders valued the PACTE-VIH program in providing financial and logistic (inputs, including antiretroviral treatment) supports, as well as individual and capacity building. The program should maintain dialogue with stakeholders, and support to IPs. 8 EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE West Africa is an immense and diverse geographic region of 21 countries facing some of the most significant health development challenges in the world. About 339 million people live in this region (Population Reference Bureau (PRB), 2014), which has the highest total fertility rate (TFR) worldwide (PRB, 2014). On average, the TFR in West Africa is estimated at 5.4 children per woman, ranging from 2.1 children per woman in Cape Verde to 7.6 children per woman in Niger (PRB, 2014). This high fertility, coupled with a low modern contraception use (11% compared to the global average of 56% use among women of reproductive health (WRH) age) and declining infant mortality (from 152 deaths per 1,000 live births (lb) in 1970 to 66 deaths per 1,000 lb in 2013), means that West Africa's population is expected to double by 2050, yielding approximately 784 million people. Following the high fertility, this region has the world’s highest maternal mortality rate (the regional average is in the mid-to high range of 483-888/100,000) due to persistent inadequate health options for women (PRB, 2014). The West African region has a low HIV prevalence (2.5%) in the general population compared to other regions of Sub-Saharan Africa (SSA). However, West Africa contributes a significant number of new infections to the global burden due to its large population. Furthermore, the prevalence of HIV/AIDS, while low in the general population, (3.2% in Togo, 1% in Burkina Faso) is quite high in key populations (KPs), notably female sex workers (FSWs) and men-who-have-sex-with-men (MSM), with rates ranging from 19 to 30 times higher than in the general population. Results of size estimation studies in Burkina Faso and Togo show an HIV prevalence rate of 28.5% among FSWs in Lomé, 14.4% among FSWs in Ouagadougou and 32,7% in Bobo-Dioulasso. HIV prevalence among MSM is estimated at 5% in Ouagadougou and Bobo-Dioulasso, and 18.5% in Lomé (Papworth, et al., 2014). Data from an epidemiologic study conducted in 24 West and Central African countries estimated HIV prevalence at 34.9% among FSWs, 7.3% among their clients, and 17.7% among MSM (Papworth, et al., 2013). The majority of FSW (more than 85%) who underwent HIV testing and tested positive in Lomé, Togo were never told that they had the HIV infection (International Business and Technical Consultants, Inc. (IBTCI), 2015). They continued to have unprotected sex (10 to 30 clients per night) with minimal condom use. A combination of behavioral, structural, and environmental determinants drive the spread of HIV and sexually transmitted infections (STIs) in both Togo and Burkina Faso: low condom use; limited and low quality health information, and stigma and discrimination against most-at-risk-populations (MARP), thereby preventing vulnerable populations from seeking and accessing healthcare services (IBTCI, 2015; Kellerman, et al., 2011). Weak health systems, and poor governance and accountability are critical constraints to the delivery of quality health services in West Africa. In addition, barriers to accessing health services remain a major constraint, particularly for marginalized populations including women, children and other KP. These barriers are perpetuated by a policy environment, which often is not inclusive of human rights and not grounded in empirical data but rather, uninformed attitudes. One of the ramifications recently manifested as a result of these weak or non-functioning health systems has been unprecedented stock￾outs of life-saving HIV/AIDS medications such as anti-retroviral treatment (ART) and opportunistic infection drugs. These problems are further exacerbated by the lack of qualified staff to support the commodities and logistics system, and delayed, inconsistent, and/or suspended disbursement of funding from the Global Fund to fight AIDS, Tuberculosis and Malaria (Global Fund) due to governance and transparency issues. 9 Against this backdrop, the United States Agency for International Development (USAID) funded an HIV/AIDS prevention and care program in the West Africa region. The Regional HIV/AIDS Prevention and Care Project (PACTE-VIH) is a five-year cooperative agreement (August 2012 – August 2017) between the USAID/West Africa Mission (USAID/WA) and Family Health International (FHI 360) (AID￾624-A-12-00006) with a total budget of $13,500,000. This regional activity aims to address the critical gaps in KP programming across the region by strategically targeting interventions that build political will and by allocating resources to engage key stakeholders to support programs that are evidence-based, adaptable, replicable and transferrable throughout the region. The activities initially focus on Burkina Faso and Togo, two West-African countries, with mixed HIV epidemics. During Year1, activities were implemented in four sites in Burkina Faso (Ouagadougou, Bobo-Dioulasso, Bittou, and Niangologo) and in five sites in Togo (Lomé, Kpalimé, Kara, Cinkassé, and Dapaong). A development activity’s effectiveness should be measured by scientific, rigorous methods, in order to inform programming and position decision-makers to formulate appropriate, long-term recommendations. Hence, the USAID project Evidence for Development (E4D) performed an independent, external mid-term performance evaluation of PACTE-VIH to measure achievements of the activity in regard to the sexual behavior of KP, attitudes toward FSWs and MSM, and access of KP to HIV/AIDS services. Findings from this research have the potential to inform HIV/AIDS prevention and care implementation policy in other countries of the West African region, as well as other regions, such as SSA. This PACTE-VIH mid-term performance evaluation has three objectives: 1. Examine the implementation of the model service package; 2. Determine what is working well and what is not working; and 3. Gain some understanding of how the model can be replicated in other countries in West Africa. The target audience for the mid-term evaluation is decision-makers and managers in the USAID/WA Regional Health Office (RHO), Regional Office of Acquisition & Assistance, the Mission Director and Deputy Mission Director. Other target audiences include the PACTE-VIH Agreement Officer’s Representative (AOR), PACTE-VIH management team, and Ministry of Health (MOH) and National Council for the Fight against AIDS (CNLS) officials in Togo and Burkina Faso. Funding allocated specifically for this performance evaluation amounted $156, 980; however, this figure does not account for IBTCI technical assistance and management oversight which is factored into the overall E4D budget. Evaluation findings will be used to inform on-going implementation of the PACTE-VIH project and its IPs, as well as the design of future regional HIV/AIDS projects. USAID/WA presented IBTCI with a draft Statement of Work (SOW) which was further developed by IBTCI and finalized in unison; there were no modifications to the original SOW. (See Annex 4 Scope of Work.) 10 EVALUATION QUESTIONS The mid-term evaluation framework relies on six analytical domains. Table 1 presents the nine evaluation questions by analytical domain. Table 1 – Evaluation questions and their respective analytical domain Analytical Domain Evaluation Questions Access 1. How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? Effectiveness 2. What is working well and what is not working well for the KP programming in Togo and Burkina Faso? Enabling Environment 3. How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso? Health Seeking Behavior 4. How has this HIV prevention package influenced health-seeking behavior among KP? Perceptions 5. How do KP perceive the services and providers? 6. How do service providers perceive the services they provide to KP? 7. How do other stakeholders (e.g. Conseil National de Lutte contre le Sida (CNLS), Global Fund, United Nations Fund for Population (UNFPA), West African Health Organization (WAHO), etc.) perceive services and technical assistance provided? Scale-up 8. What conditions are necessary to scale-up this model in other countries in the region? 9. How might the elements of the model be adapted in other West African countries? 11 Figure 3 - The Regional HIV/AIDS Prevention and Care Project Results Framework Source: Ettiegne-Traore & Traore Toure (2013) 12 PROJECT BACKGROUND DESCRIPTION OF PACTE-VIH PACTE-VIH was initiated to address the critical gaps in most-at-risk-populations (MARP) programming across the region. The activity targets were FSWs and their clients, and MSM (Ettiegne-Traore & Traore Toure, 2013). PACTE-VIH has four major components (Ettiegne-Traore & Traore Toure, 2013): 1. Develop a scalable, replicable model for Burkina Faso and Togo that can be adapted in the region for the provision of essential and comprehensive prevention and support services for MARP (FSWs, their clients, and MSM) in mixed epidemic environments with high levels of stigma and discrimination. 2. Create an enabling environment for optimized HIV prevention among MARP in Burkina Faso and Togo, especially FSWs, their clients and MSM, by reducing stigma and discrimination and by building the capacity of community health facilities, national governments, and regional partners to sustain quality, public health HIV/AIDS MARP programming. 3. Provide technical assistance to selected country coordinating mechanisms (CCM) and Global Fund principal applicants and sub-applicants, in order to strengthen application submissions and improve the quality of prevention programming respectively, with a specific emphasis on MARP and in particular, FSWs, their clients, and MSM. 4. Develop a communication strategy to disseminate program data to key stakeholders. PACTE-VIH includes the four operational objectives: 1. To deliver a combination prevention approach of behavioral, biomedical and structural interventions with the aim of improving the health knowledge and well-being of MARP; 2. To increase use of health services through active two-way referral networks; 3. To strengthen the human and institutional capacity of regional, national and community-based organizations; and 4. To promote an environment conducive to KP programming and scale-up. To achieve these purposes, the project has four key guiding principles: 1. Using evidence-based and cost-effective approaches to maximize effective and efficient responses; 2. Building capacity for long-term sustainability; 3. Promoting gender awareness and mitigating the impact of gender-based discrimination and violence; 4. Fostering partnership and collaboration. FHI360 leads the implementation of PACTE-VIH in collaboration with implementing partners (IPs) in four sites in Burkina Faso (Ouagadougou, Bobo-Dioulasso, Bittou, and Niangologo) and five sites in Togo (Lomé, Kpalimé, Kara, Cinkassé, and Dapaong). Annex 3 illustrates maps of the PACTE-VIH targeted sites in each country. IPs in Burkina Faso include Africa Consultants International (ACI), AIDS Empowerment and Treatment International (AIDSETI); Alternative Burkina (AAB), Association African Solidarité (AAS); Association Appui Moral Materiel et Intellectuel à l’Enfant (AMMIE), Association des Jeunes pour le Développement de Bittou (AJDRB); Association Solidarité Défis (ASD), Responsabilité, 13 Espoir, Vie, Solidarité (REVS+); and YERELON, Clinique Yerelon. In Togo, the IPs include EVT and Forces in Action for the Well-Being of Mother and Child (FAMME). PACTE-VIH is planning to achieve four intermediate results (IRs) through a number of evidence-based activities and state-of-the art approaches. Figure 1 depicts the relations between the different components of the project, including the four components, activities implemented and expected IRs. IR1- Increased access to high-quality/comprehensive prevention and care programs for MARP The project team proposes a combination of activities including Strategic Behavioral Communication (SBC) using peer outreach, Information and Communication Technology (ICT), and specifically tailored events for reaching this particularly hard-to-reach population; provision of MARP-friendly services in Drop-in Centers (DICs), with strong referral linkages for increasing service accessibility; and distribution of HIV prevention products (condoms and lubricants). IR2- Improved enabling environment for evidence-based public health interventions targeting MARP To be achieved through activities involving policy development, community advocacy at various levels (including health providers, police and media), and capacity building of local, national, and regional organizations. IR3- Improved quality of Global Fund HIV prevention interventions that focus on MARP To be achieved through technical assistance to CCMs and Global Fund recipients and sub-recipients in the two selected countries. IR4- Increased access to quality and timely information/data focused on MARP interventions The project team proposes communication strategies and a data dissemination plan for sharing program results with all stakeholders. PACTE-VIH’s result framework relies on five key assumptions, which are monitored and reported on a quarterly basis, as follows: 1. Social, political and legal environments that remain favorable to intervention implementation. 2. Continued commitment of ministries of health to support intervention implementation. 3. Donors, including Global Fund continue to provide support to countries for health service provision, commodities, equipment and program support funds. 4. Partner institutions’ and civil society organizations’ (CSOs) technical competencies and good governance systems. 5. Same approaches successfully tested in other countries will work to address service delivery gaps in this region. 14 EVALUATION METHODS AND LIMITATIONS EVALUATION TEAM The evaluation team was comprised of an expatriate Team Leader, Thomas Park; an expatriate Evaluation Specialist, Amadou Moreau; a Burkinabe Senior Subject Matter Expert (SME), Issa Zongo; a Togolese Senior SME, Stephane D’Almeida; four research assistants, two located in Burkina Faso and two in Togo; two note takers (for Burkina Faso and Togo); and a Logistician. Our team members possessed a mix of expertise in evaluation and HIV/AIDS programming. Sam Ngambo, Executive Director of the Center for the Promotion of Human Rights and Development in Africa (CPHDA) provided management and administrative oversight of the local consultants, research assistants and logistician. IBTCI E4D staff, Jacques Emina, Salima Mutima, and Annette Bongiovanni led the data analysis and writing of the report, with the assistance of the evaluation team members (IBTCI, 2015). Please see Annex 1 for Conflict of Interest Disclosure forms. EVALUATION DESIGN The purpose of this evaluation was to examine the implementation of the PACTE-VIH model service package, determine what is working well and what is not working well, and to gain an understanding of how the model can be taken to scale regionally and replicated in other countries in West Africa. The study used a non-experimental design because the PACTE-VIH intervention does not have a comparison/control group. It employed a mixed method, including quantitative and qualitative methods. Quantitative data encompassed secondary data (from the PACTE-VIH program) and primary data (structured interviews of FSWs and MSM in the four selected sites), whereas qualitative data contained primary information from Focus Group Discussions (FGDs) and In-depth Interviews (IDIs) with key stakeholders. DATA SOURCES Data sources included secondary data and primary data collected in Burkina Faso (Ouagadougou and Bobo-Dioulasso) and in Togo (Lomé and Kara). The period of performance for this evaluation was June 15—October 31, 2015. The data collection phase was between June 22, 2015 and July1-18, 2015. The three-year reference period for which PACTE VIH was reviewed covered August 2012 through July 2015. Secondary data Secondary data gathered included existing surveys and activity monitoring reports, the PACTE-VIH Monitoring and Evaluation (M&E) Plan indicators, and MOH data. Annex 5 lists the key documents reviewed. Primary data Primary data consisted of KP surveys, FGDs and IDIs. Overall, the evaluation team organized 10 FGDs in Burkina Faso (seven FGDs with FSWs and three with MSM). In Togo, the team conducted eight FGDs (five with FSWs and three with MSM). Furthermore, the primary data were obtained through IDIs with program implementers (FHI 360 and its partners) and program beneficiaries (FSWs, their partners, and 15 MSM) living in the participating cities of Ouagadougou and Bob-Dioulasso (Burkina Faso) as well as Lomé and Kara (Togo). SAMPLING STRATEGY Quantitative survey (KP survey) This approach targeted FSWs and MSM using structured questionnaires. The quantitative data comprised of data collected from 101 KPs in Burkina Faso (51 FSW and 50 MSM) and 105 KPs in Togo (53 FSWs and 52 MSM). KPs were selected among the PACTE-VIH clients who had attended the health facilities during the study period in four sites (Ouagadougou and Bobo-Dioulasso in Burkina Faso, Lomé and Kara in Togo). This is a simple random sampling because all KPs who visited the PACTE-VIH health facilities during the study period (between June 22, 2015 and July1-18, 2015) had equal chance of being selected. The budget and time constraints have influenced the small simple size. The team used a structured questionnaire designed according to the evaluation analytical domains and questions. Indeed, though the determination of sample size depends on the statistical parameters, including the prevalence, the precision and the desired power; the budget limitation and the study timeframe also influenced the estimation of the sample (Lenth, 2001). Health providers supported the recruitment process, especially to identify KPs. The inclusion criteria for KP participants included the following five characteristics: (1) being 18 years or older; (2) able to provide informed consent, (3) visited the PACTE-VIH associated health programs, (4) living in Burkina Faso or Togo; and (5) the person considered him/her (self) as a FSW or MSM. A FSW is defined as female who reported having transactional sexual activities as the principal source of income over the past 12 months. MSM are men who reported anal sex with another man at least once in the past 12 months. The exclusion criteria encompassed being under 18 years old; being unable to understand or provide informed consent; being under the influence of alcohol and drugs, which is likely to impair judgment and behavior; or being a duplicate recruit. Table 2 reports the characteristics of the KP sample in the two countries. In total the team conducted 101 KPs in Burkina Faso (51 for FSWs and 50 for MSM). In Togo, the total sample size was 105, which included 53 FSWs and 52 MSM. The KPs utilized a semi-structured questionnaire (Annexes 6 and 7). Table 2 – Description of key population by selected socio-demographic characteristics in Burkina Faso and Togo Burkina Faso Togo Selected Demographic Characteristics FSWs MSM FSWs MSM N Mean N Mean N Mean N Mean Age: average 51 27.9 50 27.7 53 27.5 52 24.4 % has a child 51 72.5 50 18.0 52 53.8 52 7.7 Number of children 51 1.1 50 0.2 51 1.2 52 0.1 % in union 51 9.8 50 10.0 51 11.8 52 3.8 % Never in union 51 72.6 50 90.0 51 82.4 52 92.3 % No longer in union 51 17.6 - - 51 5.9 52 3.8 % with secondary education 51 49.0 50 90.0 53 49.1 52 94.2 On average, participants FSWs were aged 27.9 years. The distribution of participants FSWs by age in Burkina Faso is consistent with findings from the baseline report (Papworth, et al., 2014), which estimated that 65% of FSWs in Burkina Faso were aged less than 30. The large majority of participants 16 FSWs had at least one child (range 0-5), and they had 1.1 children on average. Table 2 also shows also that 72.6% of FSWs who participated have never been in union, whereas 10% were in union during the study period. The baseline survey estimated this proportion was at 1% in Ouagadougou and 4% in Bobo￾Dioulasso. The majority of participants have attended school: 33% have attended primary school, whereas 49% have secondary education or higher. The baseline survey (Papworth, et al., 2014) estimated about 30% of FSWs with secondary education in Burkina Faso (39% in Ouagadougou compared to 21% in Bobo-Dioulasso). Table 2 shows also that MSM participants in Burkina Faso were 27.7 years old on average. This result supports the baseline survey which reported about 94% of MSM aged less than 30 years in Ouagadougou. Contrary to FSWs who participated, but consistent with findings from the baseline survey (Papworth, et al., 2014), the large majority of MSM participants (90%) have never been in union. Only 18% among them reported having a child. The vast majority of MSM participants (90%) in Burkina Faso have attended at least secondary school. This proportion is consistent with findings from the baseline survey (Papworth, et al., 2014), which reported 94.6% of MSM who had at least a secondary level education. The distribution of the FSW respondents by age in Togo (Table 2) shows that, on average, FSWs were 27.5 years old (range 18-56). The majority (54%) of interviewees reported having at least one child (average 1.2 children per woman). About 12% of FSW participants were in union, whereas about 82% have never been married. Similar to Burkina Faso, 49% of FSW participant have a secondary education or higher. FSWs without formal education represented 13% of the FSWs studied. These findings support estimates from the baseline survey, which reported that about 80% of FSWs were aged less than 30 years old, about 50% had at least secondary education, and more than 50% of FSWs who never been in union (Papworth, et al., 2014). Furthermore, Table 2 shows that MSM respondents were 24 years old on average. Only 8% declared having a child. A mere 4% of MSM sampled in Togo were in union, whereas the large majority (92%) have never been in union. The proportion of the sample with secondary education or higher is estimated at 94%. The baseline survey (Papworth, et al., 2014) reported that about 80% of MSM in Togo were aged less than 30 years old, 50% had at least secondary education, and more than 90% have never been in union. Qualitative surveys Qualitative data included information from 18 FGDs and 43 IDIs with stakeholders in the two countries (Table 3). Table 3 – Number of IDIs and FGDs conducted in Burkina Faso and Togo Method Burkina Faso Togo Stakeholders In-depth interview 12 17 Health workers In-depth interview 3 11 Total In-depth interview 15 28 Female Sex Workers (FSWs) FGDs 7 5 Men who have sex with Men (MSM) FGDs 3 3 Total FGDs 10 8 The objective of IDIs with stakeholders and health workers was to ascertain information related to their perceptions about the impact of the PACTE-VIH activity and to discuss the scaling up the process. In both countries, the evaluation team interviewed one representative from the MOH, CNLS, Global Fund, 17 United UNFPA, West African Health Organization (WAHO) community leaders, media, and police. This was done to gain an understanding of how each group perceived the PACTE-VIH approaches currently being implemented, methodologies and limitations, and contextual barriers and opportunities. In total, the database includes 15 IDIs in Burkina Faso and 28 in Togo. In parallel, the evaluation team conducted 10 FGDs in Burkina Faso and eight FGDs in Togo. Annexes 6-9 present the different tools (the key informant survey questionnaires, as well as the IDIs and FGD guidelines.) In the two countries, Peer Educators (PE) used their network to identify candidates (FSWs and MSM) to participate in the FGDs. Potential participants were contacted by phone (SMS or call) and were informed about the study objectives and the FGD schedule (date, time, and place). Only KP (FSW sand MSM) who ever benefited from the PACTE-VIH project were eligible to participate in the FGDs. An individual was not eligible to participate in both quantitative and qualitative surveys. The evaluation team conducted the FGDs in various venues: schools, hotels, DICs. In Burkina Faso, all FGDs and IDIs were conducted out of the PACTE-VIH affiliated associations for security reasons. The health provider’ interviewees were people (men or women) who were present at the study￾selected sites and were responsible for the coordination of site activities and/or were directly involved in the management of KPs. The evaluation team used the literature review to identify key stakeholders taking into account their availability and the representativeness of the technical and financial partners, the MOH, and community leaders and beneficiaries. FGDs are particularly valuable for gathering information, giving rise to divergent opinions, consensus, or results that involve complex issues that require in-depth exploration. The evaluation team conducted seven FGDs with FSWs in Burkina Faso and three FGDs with five FSWs in Togo. In parallel, we conducted three FGDs with MSM in each country. FGDs relied on a semi-structured focus group guideline (See Annex 8). Each group included 8-10 individuals selected based on meeting qualification criteria for participation. The team did not pay group participants for participation, but provided a modest amount for travel to and from the FGD location. In Togo, the evaluation team spent three hundred and twenty-six dollars (326$) for FGDs; forty dollars (40$) per FGD on average. The cost of FGDs was estimated at two hundred and twenty dollars (220$) in Burkina Faso. These funds were used for transportation. To decrease the likelihood that clients would give biased responses about the health services they receive from a facility because of the fear of being overheard, FGDs were scheduled to take place in the community as opposed to the health facility being discussed. LIMITATIONS This study presents few methodological limitations. First, KP surveys used a very small sample size, which undermines the reliability of estimates. Second, this study selected KP survey sampling from the PACTE-VIH health users. These populations might have different characteristics (current or past health status, acceptability of the KP status, residence distance to the PACTE-VIH health facilities, etc.) compared to other KPs (who did not visited health facilities). Third, there are potential recall biases on the part of the respondent due to the use of retrospective questions. The study might also face interviewer bias among the different evaluation team members who invariably posited the semi￾structured and unstructured questions slightly differently (this is the nature of gathering rich qualitative data). Furthermore, some interpretation errors might occur during the transcription and/or translation. FGDs and IDIs were conducted in local languages and/or in French, whereas the analysis was performed in English. 18 Last but not least, the use of trends analysis should be done with caution because of the methodological difference between the baseline and the current evaluation study. The baseline survey (Papworth, et al., 2014) used the respondent driven sampling (RDS), whereas respondents for the evaluation study were selected at the PACTE-VIH health facilities. Although the respondent-driven sampling has proven effective for collecting large and diverse samples in a wide range of settings, involving respondents in the sampling process means that the RDS data-generating process is largely outside of the control, and even the view, of researchers (Heckathorn, 2011). Therefore, the method can generate unbiased estimates. The bias in recruitment by socioeconomic status is likely to be generalizable to most, if not all, respondent-driven-sampling studies because different sub-groups of the target population are likely to be differentially motivated by whatever incentives are offered. Against this background, RDS should be regarded as a (potentially superior) form of convenience sampling method, and caution is required when interpreting respondent-driven sampling study findings. Furthermore, although findings from the KPs and FGDs will give insight about the views of participants, we cannot generalize them to the entire population of beneficiaries of PACTE-VIH. DATA ANALYSIS Data analysis relies on the triangulation of the various data sources used (the PACTE-VIH baseline and process reports, quantitative data from KP surveys, FGD and IDI data), which constitutes the basis of the interpretation of our findings. We used content and thematic analysis for qualitative data (FGD and IDI data). In reviewing data from IDIs along with field notes, we used inductive analysis to identify themes and patterns, and to construct typologies. Our qualitative analysis provides an overview on how, and to what extent, existing models are applicable to target countries on the one hand, while on the other, contributes to defining models for extension/replication of PACTE-VIH related strategies in other settings. The team conducted IDIs in French, whereas the FGDs were performed in either one of the local languages or in French, depending on the preferred language of the group. All qualitative data were transcribed in French. The team applied thematic analysis using the Atlas.ti software. Quantitative technique (interpretation of proportion) relies on data from the KP surveys (FSWs and MSM). The evaluation team performed all the analyses (quantitative and qualitative) by evaluation question. Table 4 summarizes the analysis plan, including the outcome indicators associated with each evaluation question and the corresponding analysis technique. The team then summarized the discussions in both French and English, in standardized transcripts, which were then coded using Atlas.ti® version 7.5.6. ETHICAL CONSIDERATIONS Data were collected to meet the objectives of the evaluation. No out-of-context data and/or findings were released, and maximum confidentiality was assured. Outcomes from the evaluation will not be associated with any one person, or the name of any informants or people indirectly involved in the selected countries. Prior to interviewing key informants, surveyors obtained written consent from all KPs. This form consisted of an introduction, the objectives of the PACTE-VIH mid-term evaluation, the process of data collection, confidentiality of all information to be collected, the advantages and risks that lie in 19 participation, etc. The team insisted on voluntary participation for all respondents. The consent form indicated that participants could stop the interview or participation in the FGD at any time. Moreover, in those cases, the evaluators documented the reasons. These ethical procedures are consistent with IBTCI’s standard operating protocol: IBTCI Ethical Standards and Protocols for Field Research, and are based on international policies and protocols. The protocol seeks to ensure: informed consent, confidentiality, voluntariness, anticipation and mitigation of adverse effects, avoidance of harm (psychological and physical), avoidance of undue intrusion, and respect for participants’ property rights. The survey questionnaire and the transcript did not include individual names to ensure anonymity. Furthermore, the interviews took place in private places so their responses could not be heard. 20 Table 4: Design/Data Analysis Matrix Analytical Domain Evaluation Questions Outcome Indicator Data Source Analysis Technique Access 
 1. How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? ● Heard about PACTE-VIH ● Minimum package services content ● Sources of access to condom, lubricant and place of HIV test. ● Geographical accessibility ● KP surveys ● FGD with MSM and FSW. ● Analysis of proportion ● Content analysis Effectiveness 2. What is working well and what is not working well for the KP programming in Togo and Burkina Faso? ● Obstacles to use PACTE HIV services ● Sources of information ● Participation to activities ● Sensitization themes ● Services offered by association ● Ever benefit from the association services ● Satisfy about the services ● Reasons for not using condom. ● Reasons for inaccessibility to lubricant. ● KP surveys ● FGD ● In-depth interview of stakeholders ● Project reports ● Analysis of proportion ● Content analysis Enabling Environment 
 3. How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso? ● Percent denied access to a public service (health care, assistance, church/mosque, work place, social event, leisure). ● Percent victims of violence. ● KP surveys, , project reports. ● In-depth-Interview with key stakeholders, FGD with MSM and FSW. ● Analysis of proportion ● Quantitative: Chi￾square, T-test ● Content analysis Health seeking Behavior 4. How has this HIV prevention package influenced health￾seeking behavior among KP? ● Prevalence of unsafe sex among KP ● Number of partners over the last six months, last month and the last week ● Condom use (frequency, type of partners, etc.) ● Convince partners to use condom ● Use of lubricant ● HIV test (frequency, place, reasons…) ● STI test. ● KP surveys, ● Project reports ● Project reports ● In-depth-Interview with key stakeholders, FGD with MSM and FSW ● Analysis of proportion ● Quantitative: Chi￾square, T-test ● Content analysis Perceptions 5. How do KP perceive the ● ● FGD with MSM ● Content analysis 21 Analytical Domain Evaluation Questions Outcome Indicator Data Source Analysis Technique services and providers? and FSW ● IDI with stakeholders 6. How do service providers perceive the services they provide to KP? ● Service providers’ perception of KP (MSM and FSW) ● Service providers’ perception of services to KP. ● In-depth-Interview with Health facility staff. ● Content analysis 7. How do other stakeholders (e.g. Conseil National de Lutte contre le Sida (CNLS), Global Fund, United Nations Fund for Population (UNFPA), West African Health Organization (WAHO), etc.) perceive services and technical assistance provided? ● Stakeholders’ perception of KP (MSM and FSW) ● Stakeholders’ perception of services to KP. ● In-depth-Interview with key stakeholders ● Content analysis Scale-up 
 8. What conditions are necessary to scale-up this model in other countries in the region? ● Other categories of the population to be included in PACTE-VIH in the future ● Recommended activities in future interventions. ● The most cited conditions for scaling up ● In-depth-Interview with key stakeholders. ● Project reports. ● Content analysis 9. How might the elements of the model be adapted in other West African countries? ● Reasons for scaling up the PACTE￾VIH activities in other West African countries ● In-depth-Interview with key stakeholders ● Project reports ● Content analysis 22 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ASSESSING ACCESS This section addresses evaluation question 1: “How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso?” The key indicators include: 1) assessment of proportion of participants who ever heard about PACTE￾VIH, 2) the content of the minimum package services, and 3) sources of access to condoms and lubricants, as well as the HIV testing site. The PACTE-VIH has twofold objective: (1) to provide essential and comprehensive prevention and support services to KPs; and (2) to create an enabling environment for optimized KP programming by strengthening the human and institutional capacity of national and community-based organizations (CBOs) to plan, coordinate, deliver and monitor service delivery for the targeted populations. PACTE-VIH has supported 23 partner clinics in Burkina Faso and Togo in collaboration with the national coordinating committees. These clinics cumulatively employ 201 health-care workers, and 110 PEs. Furthermore, PACTE-VIH sponsored five DICs in the two countries. There are two DICS (one for FSWs and one for MSM) in Lomé, Togo, and three are established in Burkina Faso, among which one is for FSWs in Bobo-Dioulasso, and two are for both FSWs and MSM in Ouagadougou. Burkina Faso Overall, the PACTE-VIH program has distributed about 488,000 male condoms, 94,000 female condoms and 224,000 lubricants over the evaluation period in Burkina Faso. Figure 4 displays these figures per year. Year 3 data refer to the first quarter only (October – December 2014) Figure 4 – Number of condoms and lubricants distributed Year 1-Year 3 (Quarter 1) in Burkina Faso 136083 90 59310 134812 31445 69621 217042 62228 95154 0 50000 100000 150000 200000 250000 Male condom Female condom Lubricant Year 1 Year 2 Year 3 23 Currently we are out of stock of condoms and we do not benefit from the distribution of hygiene kits such as (soap, bleach, etc. ...). Currently, the program provides these inputs only to FSWs [MSM, Burkina Faso]. These data show that 44% of male condoms were distributed in the first quarter of the third year. Figure 4 also shows progress in the distribution of female condoms and lubricants over the project life. The number of distributed female condoms increased from 90 in Year 1 to more than 62,000 in the first quarter of Year 3. Likewise, the program distributed more than 95,000 lubricants in the first quarter of Year 3 compared to 59,310 in Year 1 (60% increased). These data raised the question about the implementation barriers in Years 1 and 2. Data from FGDs confirmed greater access to condoms and lubricants among KPs enrolled at the PACTE-VIH program. Findings from KP surveys indicated that the large majority of KPs in Burkina Faso have access to condoms. Only a few people (less than five) reported difficulty in getting condoms and lubricants in Burkina Faso. PACTE-VIH provides free consultations, condoms and lubricants. The large majority of interviewed KPs (88% of FSWs and 98% of MSM) reported having received condoms and lubricants from PACTE-VIH program. We do not have accessibility constraints to the PACTE-VIH services. The PACTE-VIH affiliated clinics and peer educators provide free condoms and lubricants though sometimes the quantity is not enough for us. Furthermore, the consultations to the clinics and generic drugs are free. The unique accessibility issues are transportation if you live far from the PACTE￾VIH affiliated facilities and cost of medications in case of stock-out [FSW, Burkina Faso]. However, sometimes there are stock-out and logistic problems: From Year 1 to Year 2, the PACTE-VIH program tested and counseled 6,940 KPs with HIV (FSWs, MSM and clients of FSWs). Table 5 reports the distribution of KPs tested and counseled for HIV during the study period per year and by category of KP. The number of KPs tested increased from 2,449 in Year 1 to 4,712 in Year 2. FSWs represent 54% of tested KPs over the evaluation period compared to 25% of their clients and 21% of MSM. The prevalence of HIV increased slightly from 1.0% to 1.3%. HIV prevalence among tested KPs in the first quarter of Year 3 is estimated at 3.4%. Table 5 – Number of people tested and counseled for HIV in Burkina Faso (Year 1 – Year 2) Year 1 Year 2 Total Female Sex workers (FSWs) 1,393 2,361 3,754 FSWs Clients 551 1,211 1,762 Men who have Sex with Men (MSM) 505 919 1,424 Total tested 2,449 4,491 6,940 HIV Prevalence 1.0% 1.3% Sources: PACTE-VIH annual reports (2012-2014). A review of positivity rates by KP sub-groups shows relatively high HIV prevalence among FSWs (2.3% in Year 1 and 1.9% in Year 2) compared to other sub-groups. HIV prevalence was estimated at 1% and 24 0.8% among the tested MSM in year 1 and year 2 respectively. The prevalence of HIV was estimated at 0.8% in Year 2 for FSW clients. Findings from the KP surveys reported that the large majority of KPs (more than 95%) in Burkina Faso have reported a HIV test. Consistent with findings from the PACTE-VIH report, proportion of KP who received HIV test in the PACTE-VIH facilities was high among FSWs (47%) compared to MSM (22%). Togo Overall, the PACTE-VIH program distributed about 563,177 male condoms, 30,468 female condoms and 237,879 lubricants over the evaluation periods. Figure 5 displays these figures per Year. Like for Burkina Faso, Year 3 reports the first quarter data only (October – December 2014). Figure 5 reveals progress over time in the program distribution of condoms and lubricants in Togo. Figure 5 – Number of condoms and lubricants distributed Year 1-Year 3 (Quarter 1) in Togo However, the trend in male condoms seems atypical compared to trends in the distribution of female condoms and lubricants. Like in Burkina Faso, findings from FGDs and KP surveys confirm greater KP access to the PACTE-VIH services. About 84% of FSWs and 60% of MSM reported having received condoms and lubricants from the PACTE-VIH program. However, transportation (for those living far from the PACTE-VIH health facilities) and extra-fees in case of stock-outs are the principal access challenges. HIV testing is the foundation for both prevention and care efforts. The project reports indicated that a total of 7,014 people received HIV services. Table 6 depicts the distribution of HIV services by year and sub-group. Table 6– Number of people tested and counseled for HIV in Togo (Year 1 – Year 3*) Year 1 Year 2 Total Female Sex workers (FSWs) 1,064 415 1,479 FSWs Clients 754 2,358 3,112 Men who have Sex with Men (MSM) 265 2,158 2,423 Total tested 2,083 4,931 7,014 136083 9066 48069 273092 7184 90040 154002 14218 99770 0 50000 100000 150000 200000 250000 300000 Male condom Female condom Lubricant Year 1 Year 2 Year 3 25 Year 1 Year 2 Total HIV Prevalence 3.9% 5.2% Like in Burkina Faso, the number of HIV beneficiaries doubled from 2,083 in Year 1 to 4,931 in Year 2. At the end of the Year 3 first quarter, PACTE-VIH has tested and counseled 1,778 KPs. Like in Burkina Faso, the majority of tested KPs in the PACTE-VIH facilities were FSWs (43%) followed by their clients (32%) and then MSM (25%). The prevalence of HIV was estimated at 5.2% in Year 2 and 4.6% in Year 3. Overall, MSM displayed the highest positivity rate (10.4 % in Year 2 and 6.1% in Year 3), followed by FSWs (7.4 % in Year 2 and 5.3% in Year 3) and FSW clients (1.7 % in Year 2 and 2.2% in Year 3). Findings from the KP survey indicated that more than 90% of KPs reported a HIV test in Togo. The proportion of the KP who received a HIV test from the PACTE-VIH facilities varied sub-group. Fifty percent of FSWs reported having been tested in the PACTE-VIH facilities, whereas the corresponding proportion is estimated at 16% for MSM. Furthermore, in Togo, the MOH, in collaboration with the PACTE-VIH program, has developed an “advanced strategy” to reach FSWs using mobile clinic, Short Message Service (SMS) through cell phones, reminding KPs of treatment regimes, testing schedules, safe sex practices, etc. Furthermore, PACTE- VIH is using a Unique Identification Code (UIC) to facilitate KP record tracking, expedite treatment, and to maintain confidentiality. EVALUATION OF THE EFFECTIVENESS This section addresses the evaluation question 2: What is working well and what is not working well for the KP programming in Togo and Burkina Faso? The responses to this question rely on document review, analysis of performance indicators from the project reports and qualitative data from FGDs and IDIs. Table 7 summarizes key achievements compared to the targets. Overall, the activities reached indicator targets (sometimes over 100%). A total of 38,218 KPs (4,705 MSM, 16,656 FSWs and 16,857clients of FSW) were reached (in Year 1 and Year 2) in both countries through one-on-one or group level peer education sessions; an average of 113% achievement over the project life. Likewise, the program targets increased targets and achievements every year. The number of KPs reached increased from 7,871 in Year 1 to 30,347 in Year 2. Likewise, the program has distributed over one million condoms and lubricants in Year 2 compared to 378,000 in Year 1. In Year 2, 9,741 KPs received HIV testing compared to 4,532 in Year 1. Table 7 – Aggregate Achievements at Point of Midterm Evaluation Year 1 Year 2 Indicator Targets Achievement (%) Target Achievement (%) No. of KP reached 7,607 7,871 (103%) 28,960 30,347 (105%) No. of KP tested with results 1,650 4,532 (275%) 11,590 9,741 (84%) No. of KP who received STI services 500 834 (167%) 5,110 3,933 (77%) No of KP with completed referrals 1,508 5,631 No of condoms and gels distributed 592,400 377,954 (64%) 2.9 M 1,607,786 (55%) No. reached with stigma messages 7,607 2,375 (31%) 28,960 10,941 (38%) Sources: PACTE-VIH annual reports (2012-2014). However, the performance was low regarding the number of KP who completed referrals during Year 1, especially in Burkina Faso. Only 65 KPs completed referrals in Burkina Faso compared to 1,443 in 26 Togo in Year 1. In Year 2, the percentage of individuals with completed referrals was estimated at 50% in Burkina Faso compared to 55% targeted and 81% in Togo. Furthermore, the number of condoms and lubricants distributed (Years 1 and 2) and the number of KPs reached with stigma messages were below the targets. Nevertheless, the percentage of achievements for the last indicator (the number of KPs reached with stigma message) increased in Year 3. During the first quarter of Year 3, the program has reached 14,780, whereas the activity targeted 12,156 people (121% of achievement) Serious shortages and stock-outs of HIV/STI Prevention Commodities such as condoms, lubricants, HIV test kits, and STI and HIV drugs were experienced throughout Year 1 and the beginning of Year 2, which explained low performance regarding number of condoms and lubricants. REVS+, one of the stakeholders, attributes the success of the PACTE-VIH in Burkina Faso to the fact that the beneficiaries (KPs) are themselves actors. FSWs and MSM were involved at the early stage of the activities planning. In parallel, analysis of quantitative (KP) and qualitative (FGDs and IDIs) data reported that the MSM fear of sexual orientation disclosure in their neighborhood was the major barrier to the project success. This is due to the persistent high level of stigma and discrimination towards MSM in Burkina Faso. For instance, in Year 2, PACTE-VIH postponed the implementation of activities (training sessions and social media initiatives) and scaled down outreach activities due to hostility to homosexuality. Obstacles arise in the activities undertaken in the community: MSM are attacked. There is lack of strong leadership to talk about MSM status in public. It is difficult to identify yourself as MSM in Burkina Faso context. MSM are forced to hide, regardless of their socioeconomic status (head of department, officials, married and fathers of families). Sometimes even the family does not want face this problem. We are currently on standby in our activities within the community due to social resistance against MSM related activities in Bobo-Dioulasso. Fortunately, the well-trained peer educators are working hard to identify KP and provide psycho-social advices and referring them to the PACTE-VIH affiliated facilities. [REV+]. Data from FGDs and IDIs show that the PACTE-VIH partner clinics and DICs in Burkina Faso and Togo were fully operational in December 2014. They offer standardized package of Continuum of Prevention and Care (COPC) services, including HTC and STI services, referrals, PE outreach activities, social events for KPs, psychosocial counseling and support groups, distribution of condoms and lubricants, human rights and gender-based violence (GBV) awareness and support services as well as income generating activities. AAS (a PACTE-VIH affiliate facility) provides condoms of high quality compared to those from the shops. PACTE-VIH associated facilities provide free consultations and medication (if available). Otherwise, the KP should pay medication at the pharmacy [FSW, Burkina Faso]. We frequently receive Peer educators visit for sensitization and advices [FSW, Togo]. We are working with KPs. Confidentiality is a key element of our work to protect KPs. Before the implementation of this activity (PACTE), we used to perform STI test. Now in addition, we are providing treatment of STIs and HIV/AIDS (Medical Doctor, AAS, Burkina Faso). 27 Since the implementation of this activity, we have improved the reception, our staff are regularly trained, inputs, including lubricants, condoms and ARV are available. We organize every month sensitization conferences and workshops. All visitors receive systematically 30 condoms and 30 lubricants (Medical Doctor, ACS, Togo). Findings from KP surveys are consistent with data from qualitative analyses and the program reports. Burkina Faso Participation in the PACTE-VIH Activities Figure 6 depicts the distribution of KP by type of PACTE- VIH activities in Burkina Faso using data from KP surveys. Figure 6 – Participation of Key Population to the PACTE-VIH activities in Burkina Faso Female Sex Workers Men who have Sex with Men About 57% of surveyed FSWs have participated in sensitization, or prevention sensitization, whereas 45% have participated in activities promoting knowledge on STIs and HIV/AIDS. Furthermore, more than one in five FSWs has been involved in sensitizing the community in voluntary testing (22%). Eighteen percent of FSWs have sensitized the community on treatment in case of infection. However, only few participants have participated in activities regarding support to people living with HIV. About 16% of surveyed FSWs reported ever participating in other activities, including screening fibroid and other gynecological problems as well as GBV. Figure 6 reveals also that large majority of MSM surveyed have been involved in activities promoting knowledge (80%), prevention (82%), treatment (65%) and voluntary testing (62%). However, less than half of MSM reported their participation in counseling or support to people living with HIV. More than 30% of MSM have participated to the activities promoting the PACTE-VIH program. In addition, both FSWs and MSM listed other activities, such as the fight against stigmatization and GBV, and how to use condom. The majority of surveyed KPs have received HIV/AIDS commodities from the following facilities : Association African Solidarité, Clinique Yerelon - Association Keego, Danayasso, Espoir et Vie and Niangologo. Nevertheless, 63% of participants benefited from services from at least one STI/HIV/AIDS health facility in the country. Table 8 reports the repartition of KPs by service received in Burkina Faso. 45.1 56.9 21.6 17.6 15.7 0.0 20.0 40.0 60.0 Knowledge Prevention Voluntary Testing Treatment Other 80.0 82.5 62.5 37.5 65.0 20.0 32.5 15.0 0.0 20.0 40.0 60.0 80.0 100.0 Knowledge Prevention Voluntary Testing PACTE association Treatment Counseling Support to People… Other activities 28 Table 8 – Proportion of MSM and FSW who received services from the PACTE-VIH activities in Burkina Faso Female Sex Workers Men who have Sex with Men Proportion of FSWs and MSM who received services from the PACTE-VIH activities in Burkina Faso N Mean N Mean Sensitization 32 68.8 41 87.8 Received Condoms/ lubricants 32 87.5 41 97.6 STI diagnosis and treatment 32 46.9 41 22.0 HIV care and support 32 12.5 41 4.9 Source: E4D KP survey (2015). The large majority of surveyed FSWs reported free condoms (88%) and sensitization to preventative behavior (69%) as the most received services, followed by STI diagnosis and treatment (47%). Likewise, the proportions of MSM studied that received free condoms and/or benefited from sensitization to preventative behavior are estimated at 98% and 88% respectively. However, only 22% of MSM have benefited from free STI diagnosis and treatment. Furthermore, few KPs reported HIV care and support. Nevertheless, 56% of FSWs and 81% of MSM surveyed reported challenges in using the PACTE services. Within the studied KPs, 38% of FSWs and 50% of MSM indicated stigmatization. For MSM, stigmatization refers to fear of sexual orientation and HIV-status disclosure, in the context where the society does not accept sexual intercourse between men. FSWs however, report fear of sexual transaction activities and HIV-status disclosure. One health provider reported that such attitudes prevent some KPs from visiting health facilities and is mostly observed from new PACTE-VIH clients. Furthermore, MSM reported: (1) accessibility problems, including lack of information on the package being offered, long distance to the health facilities and lack of financial resources to pay the services; and (2) poor services, including drug stock-outs, long waiting time, and poor welcome at the reception. Other concerns indicated by FSWs included a lack of reliable information, lack of financial support, and poor services in some health facilities. Togo Participation in the PACTE-VIH Activities Figure 7 summarizes KP participation in the PACTE-VIH activities in Togo using data from the KP survey. More than 70% of FSWs have participated in activities that promote STI/HIV/AIDS knowledge and prevention. The proportion of surveyed FSWs who participated in the voluntary testing activities is estimated at 53% (Figure 7). 29 Figure 7 – Participation of Key Population to PACTE-VIH activities in Togo Female Sex Workers Men who have Sex with Men The proportion of participants who reported treatment is estimated at 28%, whereas less than 15% participated in activities involving facilities which provide STIs and HIV/AIDS care. Surveyed FSWs (9%) reported blood testing to screen other infections such as malaria. Likewise, more than 90% of MSM have been involved in prevention and knowledge-promotion activities. The proportions of MSM who participated in the voluntary testing activities, treatment, and promotion of institution providing care are estimated at 32%, 24% and 12% respectively. Table 9 reports the proportion of KPs who have benefited from PACTE-VIH activities, by type of service, in Togo using data from the KP surveys. Like in Burkina Faso, more than 80% of FSWs in Togo have benefited from free condoms/lubricants and sensitization to preventative and/or to access to STI and HIV/AIDS care. Contrary to Burkina Faso, more than half (57%) of FSWs in Togo have been diagnosed and/or received treatment. Table 9– Proportion of MSM and FSW who received services from the PACTE-VIH activities in Togo Proportion of FSWs and MSM who received services from the PACTE￾VIH activities in Togo Female Sex Workers Men who have Sex with Men N Mean N Mean Sensitization 44 79.5 45 62.2 Received Condoms/ lubricants 44 84.1 45 60.0 STI diagnosis and treatment 44 56.8 45 15.6 HIV care and support 44 18.2 45 28.9 Source: E4D KP survey (2015). More than 60% of MSM have been sensitized against STI and HIV/AIDS, whereas 60% have received condoms or lubricants during the reference period. Only 29% and 16% of MSM have reported STI diagnosis and HIV care benefits respectively. Like in Burkina Faso, surveyed KPs have listed major challenges in participating to the PACTE-VIH program, including stigmatization, distance to the service providers, lack of financial support, and poor management of some PACTE-VIH affiliated associations. These findings suggest that globally, the PACTE-VIH program has provided STIs/HIV/AIDS commodities to KPs in the selected sites of Burkina Faso and Togo. In parallel, findings from the KP surveys indicate also that PACTE-VIH program is not the principal venue for HIV testing, especially for MSM. Indeed, among the KP surveys participants who have reported having received HIV test, only 47% of FSWs and 71.7 69.8 52.8 13.2 28.3 9.4 0.0 20.0 40.0 60.0 80.0 Knowledge Prevention Voluntary Testing Structure of care Treatment Other activities 94.1 91.2 32.4 11.8 23.5 5.9 2.9 38.2 0.0 20.0 40.0 60.0 80.0 100.0 Knowledge Prevention Voluntary Testing PACTE- Association Treatment Counseling Support to People… Other activities 30 22% of MSM in Burkina Faso have reported the PACTE-VIH program facilities as the venue. Likewise, in Togo, the proportions of KPs that reported getting HIV testing in the PACTE-VIH facilities are estimated at 57% among FSWs and 16% among MSM. Other sources of condoms and lubricants include shops, other Associations/NGOs facilities, peer￾FSWs/MSM, pharmacies, and health facilities. KP listed also other associations and NGO facilities as well as public health facilities as the venue of HIV test. Overall, PACTE-VIH activities faced four major challenges: ● Shortages and stock-outs of HIV/STI Prevention Commodities, such as condoms, lubricants, HIV test kits, and STI and HIV drugs in Years 1 and 2. Two factors could explain this situation. First, partners failed to reach a timely agreement, which in turn delayed the procurement process. Secondly, the government of Togo cut the CNLS budget, causing country-wide shortages of STI drugs. This situation influenced the PACTE performance in Year 2. In response to this challenge, PACTE-VIH increased efforts to secure stocks of HIV Prevention Commodities through the launch of small grants. PACTE-VIH also obtained emergency supplies of condoms and lubricants through the USAID DELIVER Project with USAID/WA’s assistance. ● The referral rates were low in Burkina Faso. ● The high level of stigma and discrimination towards MSM in Burkina Faso during Year 2 postponed the implementation of activities (training sessions and social media initiatives) and scaling down (outreach activities). ● In Togo, stakeholder interviews reported the loss of PEs from one program because they thought that the PACTE-VIH was compensating the PEs more than in other health facilities. ASSESSING ENABLING ENVIRONMENT The objective of this section is to answer the Evaluation Question 3: “How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso”? PACTE-VIH has organized several advocacy and capacity building activities to improve the skills of KPs to engage in policy dialogue and improve service delivery. In addition, the program has provided technical assistance to IPs in the areas of M&E, and through mentorship and support supervision from PACTE-VIH senior management team to the IPs. The program has also organized training on proctology to improve care support to MSM. Table 10 reports the achievements of key activities planned to improve the enabling environment for optimized HIV prevention among MARP. Overall the program has achieved its indicator targets. 31 Table 10– Activities organized to enable environment INDICATORS Year 1 Year 2 Target Achievement % achieved Target Achievement % achieved No. of healthcare workers who successfully complete an in-service training program for HIV related service delivery * 217 266 123% 260 365 140% Stigma reduction and advocacy skills building with the media No. of MARP friendly clinic staff who received regular supportive supervision and mentoring 47 21 45% 70 201 287% No. of peer educators who received regular supportive supervision and mentoring 80 80 100% 80 110 138% No. of MARP service delivery settings that have successfully replicated the project models [WA2.1.3] 2 0 0% 4 3 75% No. of supervision visits to new sites applying new models [WA2.1.4] 8 0 0% 12 12 100% (0 in Burkina Faso) No. of individuals (MARPs, health care providers, local media, police) reached with intervention that explicitly addresses gender‐based violence and coercion related to HIV/AIDS [WA2.1.5] 7,607 0 0% 28960 5974 21% No. of local organizations provided with technical assistance for HIV related sustainable capacity development *** 19 15 79% 24 27 113% Number of local organizations following defined guidelines for a monitoring and evaluation plan [WA2.1.10] 7 5 71% 9 24 267% No. of MARP TWG meetings held 2 0 0% 4 3 75% 32 More than 600 healthcare workers successfully completed an in-service training program for HIV related service delivery. More than 200 MARP friendly clinic staff received regular supportive supervision and mentoring. In 2014, 30 journalists and 55 Senior Editors and Media Managers (20 in Togo and 35 in Burkina Faso) have attended an advocacy consultative workshop on issues relating to KP. Furthermore, a total of 82 key stakeholders and MSM and FSW leaders (40 in Togo and 42 in Burkina Faso) have attended a two-day workshop to share experience and explore ways to better coordinate their activities. The program also trained more than 130 PEs (more than 60 in each country) to provide accurate information on HIV/AIDS and STIs to their peers. Various advocacy tools, including the advocacy kits, two stigma reduction tools, a 10-point Media Charter to guide reportage on KP and the advocacy newsletter were also finalized and will soon be disseminated in both countries. However, according to KP survey more effort should be made regarding interventions that explicitly addresses gender-based violence and stigmatization from the security force and verbal violence. Burkina Faso In Burkina Faso, 30 journalists (including eight women) attended a three-day workshop for the training in the gathering and processing of information on key populations (Ouagadougou, 17th – 19th September, 2014). Qualitative and quantitative data (E4D surveys) do not report stigmatization in access to health facilities. However, one MSM has reported stigmatization in one health facility without mentioning whether it was a PACTE-VIH affiliated or not during FGD in Burkina Faso. One of us has been humiliated and stigmatized in one health facility. He was bleeding after sexual intercourse, and the doctor delayed providing care. Instead, he invited his colleagues to come and see how a homosexual was bleeding [MSM, Burkina Faso]. Such a situation is fortunately isolated in health facilities, particularly in the PACTE-VIH affiliated facilities (See accessibility). Therefore, some MSM fell ashamed and avoid attending health facilities because they might be victims of various forms of discrimination. Findings from FGDs with FSWs and MSM indicate stigmatization and discrimination by the police. “ We are frequently arrested by the polices. The penalty amount in case you are arrest varies between 5,000 and 15,000 CFA Francs. The major problem is that we cannot anticipate the police reaction because they are also our clients. We do not know whether they come for sexual business or for arresting us” [FSW, Burkina Faso]. Therefore, the large majority of MSM do not like disclosing their sexual orientation in their neighborhood. They still feel stigmatized though the program has contributed to their capacity strengthening and to the accessibility to sexual and reproductive health service. PACTE has improved the prevention and care services in health facilities. Before the implementation of this program, our health facilities used to focus on STI screening. Now we are able to screen and provide treatment. 33 Furthermore, we have received cauterization device and anoscope [Medical Doctor, AAS, Burkina Faso]. Togo In Togo, a training workshop was organized for 15 MSM leaders, 5 FSW leaders and 16 resource persons to improve the capacity of KP to engage in advocacy and policy dialogue in Year 1. Though all KP participants to FGDs did not mention stigmatization as a challenge, findings from IDIs with stakeholders reported that the context of stigma and discrimination was a major obstacle to achieve the overall goal of PACTE-VIH (ACS, CNLS, FAMME, HPP, MO-Justice, UNAIDS, UNDP). “Currently the parliament and the government are preparing a law which to increase penalties against homosexuality. This might influence the PACTE-VIH program” [HPP, Togo]. IDIs with stakeholders have raised the following recommendations to improve enabling environment: Sensitization of leaders, the media and the police. There is a need to provide training to journalists and media managers and continue to strengthen the capacity of KP on their security issues. Findings from the literature review and qualitative data are consistent with the results of quantitative analyses (KP surveys). Overall there are no barriers to health services, particularly to the PACTE-VIH program. However, the large majority of KPs have reported external and internal barriers, which could indirectly influence access to public facilities, including health facilities. External barriers include stigmatization, discrimination, cost of services and distance to service providers. The unwillingness of KPs to reveal their sexual practices to health providers constitutes an important internal barrier. External barriers include stigmatization, discrimination, the relationship with the police, media attitude, cost of services and distance to service providers (FGDs and IDIs data). The unwillingness of KPs to reveal their sexual practices to health providers constitutes an important internal barrier (FGDs). Though the law in Burkina Faso does not, specifically, prohibit prostitution, soliciting and pimping are illegal. ASSESSMENT OF THE HEALTH SEEKING BEHAVIOR: BEHAVIOR CHANGE COMMUNICATIONS This section discusses the Evaluation Question 4: “How has this HIV prevention package (PACTE-VIH) influenced health-seeking behavior among KP”? The section uses data from PACTE-VIH report, quantitative data from the KP surveys and qualitative data from FGDs and IDIs. Burkina Faso Table 11 summarizes sexual behavior of FSWs and MSM in Burkina Faso. Though large proportion of FSWs reported that they could always use a condom or convince their partner to use a condom during sexual intercourse, more than 65% had unprotected sexual intercourse with their usual partner over the last six months. Surveyed FSWs in Burkina Faso had, on average, about four clients per week, and had 15 during the month that preceded data collection. Furthermore, 47% of FSWs reported using condoms systematically, whereas 39% used condoms with their usual partner. Table 11 – FSW and MSM Sexual behavior in Burkina Faso 34 FSW and MSM Sexual behavior in Burkina Faso Female Sex Workers Men who have Sex with Men N Mea n N Mean Unsafe sex with the regular partner 40 65.0 50 - Had unsafe sex the last 6 months 51 66.7 50 44.0 Number of clients/ partners last month 44 14.8 50 1.9 Number of clients/partners last week 44 3.7 50 0.5 Use condom systematically 51 47.1 50 72.0 Can always use condom 49 83.7 50 86.0 Convince partners to use condom 49 89.8 50 92.0 Use condom with usual partner 49 38.8 50 78.0 Use condom with casual partners 49 73.5 50 92.0 Use condom with clients/ sex workers 49 77.6 50 20.0 Source: E4D-KP surveys (2015) The large majority of surveyed FSWs (>=74%) reported the use of condoms with casual partners and/or with clients. Table 11 shows that the majority of MSM (more than 70%) were using condoms. All MSM reported the use of water-based lubricant. They had, on average, two partners the month which preceded data collection. During the baseline survey, 94% of FSWs reported using condom with their main partners. The percentage of FSW who reported using condom with new clients was estimated at 90%. In parallel, 77% of MSM reported that they were using condom with their main partner, whereas 51% were systematically using condom during sexual intercourse. The PACTE-VIH activities also promoted HIV testing among FSWs and MSM. Table 12 reveals that 96% of FSWs in Burkina Faso who have had an HIV test mostly to know their status (76%). In addition, FSWs’ usual partners have also an HIV test. Table 122– HIV/AIDS and Health behavior among KP in Burkina Faso HIV/AIDS and Health behavior among KP in Burkina Faso Female Sex Workers Men who have Sex with Men N Mean N Mean Had a HIV-test 51 96.1 50 98.0 HIV-test to know the status 49 75.5 49 77.6 HIV-test during illness 49 8.2 49 2.0 HIV-test after unprotected sex 49 8.2 49 10.2 Usual partner got HIV-test 51 43.1 50 56.0 Source: E4D-KP surveys (2015) Table 12 shows that 98% of MSM and 56% of their partners had an HIV test in Burkina Faso. Like FSWs, the main motivation for having the HIV test was to know their status. Only few people (8% of FSWs and 10% of MSM) had an HIV test because of unprotected sexual intercourse. Data from the baseline reported 44% of MSM who get HIV test compared to 55% among interviewed FSWs (Papworth, et al., 2014). 35 One of the strategies to assess prevention behavior among KP is to analyze their attitude vis-à-vis unprotected sexual intercourse (Figure 8). The large majority of surveyed FSWs could accept unprotected sexual intercourse with a usual partner (boyfriend or spouse). This proportion did not exceed 25% for other circumstances, including HIV-negative partner. Figure 8 – Proportion of Key Population by circumstances leading to unprotect sexual intercourses in Burkina Faso (KP survey, 2015) Female Sex Workers Men who have Sex with Men By contrast, only 36% of surveyed MSM in Burkina Faso might accept unprotected sexual intercourse with a usual partner. This proportion is estimated at 20% if the partner is HIV-negative, and does not exceed 10% under other circumstances. Both FSWs and MSM mentioned other circumstances that could warrant unprotected sexual intercourse, including intercourse with a spouse, love, lack of a condom, and trust in the partner. Togo Table 13 reports FSWs and MSM sexual behavior in Togo using data from the KP surveys. FSWs had 20 clients on average in the month preceding the survey, and on average 7 clients during the week, which preceded the data collection. Overall the large majority of surveyed FSWs (>80%) used a condom during sexual intercourse, except with the usual partner (45%). Fifty-five percent have reported unprotected sexual intercourse with the regular partner. During the baseline, the prevalence of condom use was estimated at 89% with the regular partner and 90% with new client(s). Table 13 –FSW and MSM Sexual behavior in Togo FSW and MSM Sexual behavior in Togo Female Sex Workers Men who have Sex with Men N Mea n N Mean Unsafe sex with the regular partner 38 55.3 - - Had unsafe sex the last 6 months 53 43.4 52 25.0 Number of clients/ partners last month 51 19.6 52 1.5 Number of clients/ partners last week 50 6.7 52 0.4 Use condom systematically 53 79.2 52 69.2 Can always use condom 53 86.8 51 84.3 Convince partners to use condom 53 98.1 52 73.1 67.3 12.2 12.2 24.5 0.0 20.0 40.0 60.0 80.0 Usual partner Know partner for long Know own HIV status Partner is HIV negative 36.0 10.0 6.0 20.0 0.0 10.0 20.0 30.0 40.0 Usual partner Know partner for long Know own HIV status Partner is HIV negative 36 Table 13 –FSW and MSM Sexual behavior in Togo FSW and MSM Sexual behavior in Togo Female Sex Workers Men who have Sex with Men N Mea n N Mean Use condom with usual partner 53 45.3 52 78.8 Use condom with casual partners 53 84.9 52 65.4 Use condom with clients/ Sex workers 53 81.1 52 7.7 Source: E4D-KP surveys (2015) On average, MSM in Togo had 1.5 sexual partners in the month, which preceded data collection. The majority of MSM (>=65%) had protected sexual intercourse, even with the regular partner. Only 25% of MSM had unprotected sexual intercourse over the six months, which preceded the survey. Furthermore, only 8% of MSM reported an unprotected sexual intercourse with sexual workers. Like in Burkina Faso, more than 90% of FSWs and MSM had an HIV test in Togo, mainly to know their HIV status (Table 14). The proportion of KP whose partners got an HIV test is estimated at 60%. The baseline reported that 48% of FSW and 43.7% of MSM got HIV test. Table 14 - HIV/AIDS and Health behavior among KP in Togo HIV/AIDS and Health behavior among KP in Togo Female Sex Workers Men who have Sex with Men N Mean N Mean Had HIV-test 53 94.3 52 98.1 HIV-test to know the status 50 88.0 51 86.3 HIV-test during illness 50 10.0 51 5.9 HIV-test after unprotected sex 50 4.0 51 9.8 Usual partner had HIV-test 48 60.4 52 71.2 Table 14 shows also that 71% of MSM partners had an HIV test. Furthermore, like in Burkina Faso only few people (less than 10%) had an HIV test because of unprotected sexual intercourse. Figure 9 depicts circumstances, which could lead to risky sexual behavior in Togo using data from the KP surveys. Thirty-six percent of FSWs might have sexual intercourse without condom with a usual partner. This proportion does not exceed 8% under other circumstances. 37 “Here, we use condoms with all of our clients without exception and regardless of the among you want to pay. However, with our boy-friends or spouses, we abandon condoms only Figure 9 – Distribution of Key Populations by circumstances leading to unprotected sexual intercourse in Togo Female Sex Workers Men who have Sex with Men Likewise, about 31% of surveyed MSM indicated the possibility of participating in unprotected sexual intercourse with a usual partner. But the proportion is estimated below 4% for other circumstances. However, both FSWs and MSM indicated that they might have unprotected sexual intercourse with spouse and/or trusted partner; if (s)he is under influence of alcohol; or to avoid problems getting an erection. Data from both quantitative (KP) and qualitative sources (FGDs) highlight the high and growing use of COPC services, as well as the rate of KP HIV testing. The large majority of KP are systematically using condoms (more than 65% on average) during sexual intercourse, especially with clients and casual partners. For instance, one FSW in Bobo￾Dioulasso (Burkina Faso) stated that: Findings from FGDs support the results of KP survey, suggesting that all surveyed MSM are using lubricants systematically. "Prior to my participation to the PACTE-VIH activities, I used to have unprotected sexual intercourse because I was ashamed to buy condoms and lubricants at pharmacy (MSM, Togo)."I would like to use this opportunity to thank the PACTE-VIH programme because I have access to condoms and lubricants (MSM, Togo). Evaluation of Perceptions The PACTE-VIH activity aims to strengthen the capacity of facilities and associations, to provide good quality information and services. This section addresses three Evaluation Questions: Evaluation Question 5: How do KP perceive the services and providers. Evaluation Question 6: How do service providers perceive the services they provide to KP? 36.5 7.7 7.7 3.8 0.0 10.0 20.0 30.0 40.0 Usual partner Know partner for long Know own HIV status Partner is HIV negative 31.4 3.9 2.0 3.9 2.0 2.0 15.7 0.0 10.0 20.0 30.0 40.0 Usual partner Know partner for long Know own HIV status Partner refuses… Partner pay Partner is HIV negative Other circumstances 38 The team used the following indicators to assess the Evaluation Question 6: Service providers’ perception of KP (MSM and FSW); Service providers’ perception of services to KP. Evaluation Question 7: How do other stakeholders perceive the PACTE- VIH services? We analyzed stakeholders’ perception of KP (MSM and FSW) and their perception of services to KP to address this question. Overall, health workers identify KP during the consultation. The majority of health workers recommended more specific trainings to strengthen their capacity in sensitization, mobilization and testing. They also suggested strengthening the minimum package, particularly the availability of drugs. They cited a persistent lack of STI testing kits. Burkina Faso Findings from FGDs revealed that all FSWs (7) are ashamed of their occupation. They do not like it to be disclosed though they are aware that people, including their family, might be informed. They reported that only foreigners could openly perform this occupation. Such attitude might have negative influence on their health seeking behavior. However, one FSW from Ouagadougou stated that doing a job without liking and enjoying it is a major barrier to achieving their life goal. Likewise, the large majority of MSM do not like disclosing their sexual orientation in their neighborhood. However, the majority of KP value the PACTE-VIH STI and HIV/AIDS services, including the availability of health staff. They reported having never been denied health services at the PACTE-VIH affiliated facilities. One FSW reported that she used to attend public health institution. But now, she goes to YERELON clinics to seek treatment. Findings from KPSs showed that the large majority of the study participants (94% of FSWs and 98% of MSM) prefer visiting the PACTE-VIH affiliate centres for getting STI and HIV/AIDS information and care. For more than 60% of FSWs, medical staff constitute the best source of information on STI and HIV/AIDS, whereas the same proportion of MSM prefer to discuss with male medical staff. All stakeholders reported benefits to the PACTE-VIH collaboration. They mentioned financial and logistic (inputs, including antiretroviral treatment) supports, and individual and institutional capacity building. PACTE-VIH arrived at the right time. We have an excellent collaboration with PACTE_HIV. The program provides free antiretroviral treatment (ARVs) and other inputs. Previously, our activities were based on research and providing care. This program has enabled us to compute indicators as well as the establishment of the cascade of care for people living with HIV. In addition, the minimum package of services is good and suitable. Service quality and the implementation of the program are satisfactory [YERELON Clinic, Bobo￾Dioulasso]. We value the collaboration with PACTE-VIH. The program does not impose its vision. Rather, it contributes to our capacity strengthening through meetings and the monitoring systems [REV+, Bobo-Dioulasso]. Togo 39 Like in Burkina Faso, the large majority of FSWs and MSM considered visiting a health facility for STI/HIV/AIDS. Also, medical staff constitutes the best sources of information on STI and HIV/AIDS for more than 70% of KP. The KP also reported other reliable sources of information, including peer FSWs/MSM and their partners. MSM also reported school as source of reliable information on STI/HIV/AIDS and the best venue for discussion. KP studied in Togo consider the PACTE-VIH program has a capacity strengthening opportunity: human right, strategies to avoid STI/HIV/AIDS. The majority of KP value the PACTE-VIH STI and HIV/AIDS services, including the availability of health staff. For me, the PACTE-VIH training has gave me opportunity to perform other activities and accept myself as FSW. I can now introduce myself as FSW. I am not ashamed of my profession because those who insult me in the morning are my clients in the evening (FSW, Togo). I used to be ashamed of my status. I was avoiding to attend hospital even if I have infection or uncomfortable with my sexual health. The PACTE-VIH program has enabled me to claim my sexual orientation and benefit health services. The program has built my capacity and increased my knowledge and gave the joy of life. I am not longer ashamed and do not fear stigmatization. PACTE-VIH is a supportive program! [MSM, Togo] The majority of these institutions, including the joined United Nations program on HIV/AIDS and the national HIV/AIDS programs appreciated the PACTE-VIH activities. The discussion with major stakeholders revealed that PACTE-VIH program has improved KP access to sexual and reproductive health services. Condoms, lubricants, STIs/HIV tests and treatments are available at the DICs. Furthermore, the DICs are referring patients to ACS services, whereas PEs cadres are mobilizing and following up KP [Stakeholders, Togo]. In addition, this program is complementary to the Global Fund and other USAID activities (HPP). However, Stakeholders (IDIs) raised the issue of sustainability of activities due to financial problems and lack of capacity strengthening. ASSESSMENT OF ABILITY TO SCALE-UP This section includes two Evaluation Questions: ● Evaluation Question 8: What conditions are necessary to scale-up this model in other countries in the region? This question relies on the most cited conditions for scaling up PACTE-VIH in other regions. ● Evaluation Question 9: How might the elements of the model be adapted in other West African countries? This evaluation question focuses on reasons for scaling up the PACTE-VIH activities in other West African countries. Burkina Faso In Burkina Faso, Stakeholders (UNAIDS, UNFPA and NGOs) listed several conditions necessary for scaling up the PACTE-VIH model, which could be grouped into three categories: (1) rely on the evidence-based approach; (2) individual and institutional capacity strengthening; (3) and creation of integrated sub-regional structures based on existing institutions. 40 1. Evidence-based conditions to assess the environment and planning actions The stakeholders suggest that scale-up is less likely if the intervention is not supported by data, past￾experiences in the context (success and challenges) and published implementation science literature. Therefore, they recommend the following activities as key steps in scaling up an effective and sustainable PACTE-VIH model: situational analysis, sites mapping, numbering and identification of KP and their network, creation of specialized association clinics for KP. Scaling up should rely on reliable data, which inform the magnitude of the problem in relation to the target. FHI360 could produce the required data, which will guide the process [CCM]. Include the intervention’ activities into the national HIV and Health Strategic Framework (2016-2020) taking into account environmental challenges [UNAIDS]. This step includes situational analysis to identify needs in human and material resources as well as assess the environment for most effective and sustainable planned actions. 2. Individual and institutional capacity strengthening According to the interviewed stakeholders, successful scaling up of the PACTE-VIH model requires investment to build individual and institutional capacity while expanding an innovation. Institutions should test capacity strengthening before wide-scale expansion. Strengthen the capacity of staff of Ministries involved in implementation of the PACTE-VIH model - Strengthen the technical and logistic resources (CD4 counter and other inputs, etc.) - Strengthen the capacity of various stakeholders (management team, Key populations, etc.) [FHI360]. Identify and train Peer Educator, build and equip the center from the existent institutions. The project must build on the existent institution, which should be innovated. The program will strengthen service providers and strengthen health structures to improve the management of KP [YERELON Clinic] Stakeholder from PAMAC highlighted the importance of building the capacity of medical staff, including physicians on the key populations specific pathologies. Likewise, peer educators training and their mobility are key conditions to expand successfully the model. 3. Enabling the environment The environments within which expansion and institutionalization of innovations must occur are the foundation of realistic expectations about the pace and scope for scaling up. Given spatial mobility of KP in West Africa, it is important to have sub-regional institutions: • Develop a sub-regional program, which will allocate intervention areas and action to the existing implementation structures [REVS]. • Create a sub-regional coordination framework. • Create a sub-regional Forum for sharing experiences • Create working groups within existing implementation structures for resources mobilization [AB]. 41 Furthermore, to allow sustainability, AAS recommends the integration of the KP and PACTE-VIH model within the national policy, initiation of discussions withal institutions involved in this thematic and train all stakeholders prior to scaling up the model. Availability of funding, involvement of government, involvement of local NGOs, the sensitization and participation of local leaders, and the sensitization of the population. FSWs and MSM listed several conditions necessary for scaling up the PACTE-VIH model, which we grouped into five categories: Availability of funding, involvement of government, involvement of local NGOs, the sensitization and participation of local leaders, and the sensitization of the population. FSWs did not mention particular countries. However, they advocated for the implementation of these activities in other cities in Burkina Faso, especially in touristic and mining regions. Furthermore, FSWs recommended the extension of PACTE-VIH activities to adolescent and young people, unmarried women and women working in restaurants, bars, hotels and shops. According to FSWs, these women are exposed to, and involved in, transactional sexual activities. There are fewer consensuses around answers to Question 9: How might the elements of the model be adapted in other West African countries. Some stakeholders recommended to expand the PACTE-VIH model to Hounde and Dédougou areas, South-West (Gaoua, Dedougou, Koudougou) where many FSW and MSM are living (AB and REV). In parallel, SPCNLS representative did not support the scaling up activities because of human resources and materials challenges: “We must not speak about scaling up. PACTE-VIH has only one staff (the coordinator) working in two cities. This is enough and you are thinking about expanding activities? There are several associations, which have several staff in the field. You must first, increase number of staff and improve the logistic conditions. This is possible with a decentralized system. Can you realize that it is very difficult for a PACTE-VIH colleague to participate to some activities because of long and complex administrative processes? We cannot rely on you because most of the time the authorization is issued after the activity has been completed” Furthermore, other stakeholders think that only discussion with KP could provide suitable information about their health needs and how to scale up the PACTE-VIH model. What is missing and what is not necessary [AB & AAS]. MSM and FSWs leaders should be involved at the starting point [AAS]. Some students are FSWs to pay their education fees, particularly if parents are poor. You can introduce a man as your boyfriend to your family just to hide the source of your revenues. This enables you being absent from home (three days or four days), pretending be at the boyfriend house (FSW, Ouagadougou). Togo Discussions with stakeholders revealed also that PACTE-VIH had established a KP organization accreditation program, which includes a standard operating procedure manual for KP service delivery. Such strategy will facilitate scaling up and replication. The analyses of IDI allow identifying three key prerequisites to scaling up the PACTE-VIH model. 1. Enabling the environment, especially get the political will. Stakeholders, including UNFPA, UNDP and PSI support the need for the MOH structures 42 Our principle concern is the transfer of skills to manage the program. We are worried about institutional capacity. We are also worried how such a program would be financed [CCM]. to be more involved in the PACTE-VIH activities (CCM) and the necessity for the program to be developed and implemented within the national health policy framework (CNLS2). It is also important to identify key partners for effective and sustainable intervention (Medias, police, justice…). 2. Identifying gaps (Evidence-based). Like in Burkina Faso stakeholders suggest that reliable data and evidence from past-experiences (success and challenges) and published implementation science literature support successful interventions. Therefore, they recommend identification of financial, logistical, human resources and environmental gaps before scaling up the model. Such process will enable clear planning and development of tools for effective and sustainable PACTE-VIH interventions; develop a strong Resource Mobilization and Capacity building plan to improve the coordination and the management. The elements of the COC model to be adapted should depend on a gap analysis at the country. Scaling up should be based on filling the gaps in the model. Whether or not DICs should be included depends on the gap analysis [PNLS]. Mapping KP and surveys on key population should guide the process of scaling up PACTE-VIH. The Maritime region and Kétao should be the next target given high the potential high proportion of KP in the area. Stakeholders also, mentioned interest in strengthening and integrate the peer education strategy into the package of activities of public structures to maintain good services and attendances. According to UNAIDS, the COC is good model that can be adapted to other countries, particularly the use of PE. 3. Capacity strengthening and resource mobilization Like in Burkina Faso, successful scaling up of the PACTE-VIH model requires investment to build individual and institutional capacity: identification and training of Peer Educators, look for additional funding to ensure availability of inputs and medical staff training. Findings revealed the importance of sensitization and diffusion facilities providing prevention and care treatments. Furthermor e, the majority of stakeholders raised the issue of running out of financial support, drugs and other inputs. Also, the scaling up and replication should take into account the overall lack of well-trained MOH health personnel in the country, as well as their unequal distribution in the urban areas. In our KISs with health workers, we found that health workers support the expansion of KP services to other health facilities in other areas because FSWs tend to be mobile. Therefore, the COPC needs to expand in order to reach them. In 43 addition, health workers support the development of a network to share experiences and challenges within the country, as well as between the two countries (Burkina Faso and Togo). Findings from quantitative analyses are consistent with the qualitative results in the two countries. KP and Stakeholders cited availability of funding, involvement of the government, involvement of local NGOs and building their management capacity, the sensitization of local leaders, and the sensitization of the population, as fundamental for scaling-up the PACTE-VIH activities and ensuring sustainability. Indeed, cultural barriers, lack of knowledge and lack of financial resources constitute the major challenges to scale up the PACTE-VIH activities. DISCUSSION AND CONCLUSIONS Targeted interventions and evidence-based prevention programs have been advocated as a cost-effective strategy to combat HIV/AIDS, especially in concentrated HIV/AIDS epidemics. Such a strategy reduces the levels of vulnerability and risk as well as allowing HIV interventions to optimize coverage, reduce costs and lower the number of new infections (Baral, et al., 2012; UNAIDS, 2011). In countries with concentrated HIV/AIDS epidemics, such as Burkina Faso and Togo, Key Populations and the most-at-risk populations disproportionately affected by HIV include CSWs, MSM and long-distance truck drivers (Baral, et al., 2012). This study aimed to measure the achievements of the PACTE-VIH activity implemented among FSWs and their clients, and MSM in Burkina Faso and Togo. The study framework includes six analytical domains: access, effectiveness, enabling environment, health seeking behavior, perceptions, and ability to scale-up. Analyses relied on data from the project reports, KP quantitative survey, FGDs and IDI. PACTE-VIH mid-term evaluation aimed to measure the achievements of the program in terms of accessibility, enabling environment, Key Populations (KP) health behavior, effectiveness and perception of the activities among KP in Burkina Faso and Togo. The study applied mixed methods using data from the project report, KP surveys, FGDs and IDI. Findings suggest mixed results depending on the considered analytical domain, the category of KP and the country. EVALUATION OF ACCESS Evaluation Question 1: How has the PACTE-VIH minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? Overall PACTE-VIH has supported 23 partner clinics and has sponsored five drop-in centers (DIC) in the two countries. The program has distributed 487,937 male condoms, 93,763 female condoms, and 224,085 lubricants over the evaluation period in Burkina Faso. In addition, the program has conducted HIV-test for 6,418 KP, among which 59% were FSWs. MSM represented only 17% of KP tested in the PACTE-VIH facilities, whereas 24% of KP tested were FSW clients. The corresponding achievements for Togo include 563,177 male condoms, 30,468 female condoms, and 237,879 lubricants distributed. Overall 3,704 HIV tests were conducted in Togo. FSWs represented the majority of tested KP (50%), followed by their clients (30%). MSM represented only 20% of KP who received HIV test in the PACTE￾VIH affiliated facilities. These findings support results from KP surveys, which reported that 58% of FSWs received STI/HIV diagnosis test from the PACTE-VIH facilities compared to only 16% for MSM. Likewise, 46% of FSWs have reported HIV test from PACTE-VIH compared to 22% among MSM in Burkina Faso. 44 The large majority of surveyed FSWs reported having received free condoms (88%) from PACTE-VIH facilities and sensitization to preventative behavior (69%) as the most received services, followed by STI diagnosis and treatment (47%). Likewise, the proportions of MSM studied that received free condoms and/or benefited from sensitization to preventative behavior are estimated at 98% and 88% respectively. However, only 22% of MSM have benefited from free STI diagnosis and treatment. The KP surveys reported that the large majority of surveyed MSM (84% in Burkina Faso and 65% in Togo) received HIV test from PACTE-VIH affiliate facilities compared to FSWs (39% in Burkina Faso and 25% in Togo). Likewise, KP surveys suggested that PACTE-VIH health facilities constituted the principal source of condoms and lubricants compared to FSWs. The majority of FSWs has reported receiving condoms, lubricants as well as HIV test from the public health facilities. The difference should be interpreted with caution for two reasons. First, the structure effect due to potential large number of FSWs in the population and their clients compared to MSM (minority in the population). Last, the KP survey sample is not statistically representative of their population. Findings from FGDs and IDI suggest greater access of KP (FSWs and MSM) to PACTE-VIH services. However, analyses of KP surveys and FGDs revealed geographical and financial challenges in KP access to the PACTE-VIH services due to the PACTE-VIH facilities in some neighborhoods; and lack of financial support in case HIV/STI prevention commodities shortages and stock-outs. Few KP also have reported the fear of being identified as sex workers or MSM as barriers to visit the PACTE-VIH facilities. The program has developed strong strategies to improve the access of KP to HIV/AIDS prevention and care commodities, encompassing specialized DICs, the mobile clinic and the distribution of the voucher system. EVALUATION OF THE EFFECTIVENESS Evaluation Question 2: What is working well and what is not working well for the KP programming in Togo and Burkina Faso? Overall, the activities successfully reached the targets (in some cases over 100%). Findings from activity reports and qualitative data (IDIs and FGDs) suggest that the DICs are fully operational in the two countries. They are the main sources of care and prevention services for the KP. They offer standardized package of COC services, including HTC and STI services, referrals, PE outreach activities, social events for KP, psychosocial counseling and support groups, distribution of condoms and lubricants, human rights and GBV awareness and support services as well as income generating activities. These findings suggest that globally, the PACTE-VIH program has been successfully providing STIs/HIV/AIDS commodities to the KP in the selected sites of Burkina Faso and Togo. In parallel, findings from the KP surveys indicated also that PACTE-VIH program is not the principal venue for HIV test, especially for MSM. Indeed, whereas more than the KP surveys participants have reported having received HIV test, only 47% of FSWs and 22% of MSM in Burkina Faso have reported the PACTE-VIH program facilities as the venue. Likewise, in Togo, the proportions of KP who reported HIV test in the PACTE-VIH facilities are estimated at 57% among FSWs and 16% among MSM. However, the program’s performance was below the target in the following indicators: the number of KP who completed referrals in Togo, number of condoms and lubricants distributed (Years 1 and 2), and the number of KP reached with stigma message. In addition, the interventions that explicitly address gender-based violence are in need of further effort. Furthermore, though achieving above 100% of target could be interpreted as an excellent performance indicator, it could also suggest that the targets were not realistic or were underestimated. 45 The activity has specifically faces the following challenges in the process of implementation: ● Poor services are offered by the PACTE-supported facilities, encompassing shortages and stock-outs of HIV/STI prevention commodities, such as condoms, lubricants, HIV test kits, and STI and HIV drugs, long waiting times, and bad reception at the front desk. ● Accessibility problems include: 1) need to travel long distance to the PACTE-VIH health facilities; 2) and lack of financial resources to pay for services, particularly in case of drugs stock-out. The large majority of FSWs in the two countries raised this concern compared to MSM. ASSESSING ENABLING ENVIRONMENT Evaluation Question 3: How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo and Burkina Faso? PACTE-VIH has organized several advocacy and capacity building activities to improve the skills of KP in to engage in policy dialogue and improve service delivery. The program has organized an advocacy consultative workshop for 55 Senior Editors and Media Managers (20 in Togo and 35 in Burkina Faso). Likewise, in Year 2, 82 key stakeholders and MSM and FSW leaders (40 in Togo and 42 in Burkina Faso) attended a two-day workshop to share experience and explore ways to better coordinate their activities. Furthermore, 49 health care providers (25 in Burkina Faso and 24 in Togo) were trained to provide efficient and quality services to MSM and FSWs. In addition, the program has provided technical assistance to implementing partners (IP) in the areas of M&E (DQA conducted this quarter), and through mentorship and support supervision from PACTE-VIH senior management team to the implementing partners. The program has also organized training on proctology to improve care support to MSM. In the two countries, KP are not facing discrimination to access the PACTE-VIH health facilities. However, DGDs and IDIs have raised the necessity of improving interventions that explicitly addresses gender-based violence and stigmatization from the security force and verbal violence. Stigmatization of KP is one of the key factors preventing FSWs and MSM from visiting health services. The high level of stigma and discrimination towards MSM in Burkina Faso during Year 2 postponed the implementation of activities (training sessions and social media initiatives) and scaling down (outreach activities). Evaluation Question 4: How has this HIV prevention package (PACTE-VIH) influenced health-seeking behavior among KP? Findings from the PACTE-VIH reports, the PACTE-VIH KP surveys and FGDs highlight the high proportion of KP using health services (including HIV testing) compared to results from Papworth and colleagues (Papworth, et al., 2013). Our findings revealed that 98% of MSM in Burkina Faso and Togo have had an HIV test compared to less than 80% of the MSMs reported in 2013 (Papworth, et al., 2013). Likewise, whereas 96% and 94% for FSWs in Burkina Faso and Togo reported HIV testing in our study, in 2013 the prevalence of HIV test among FSWs was estimated at 80% in Burkina Faso and about 70% in Togo. These findings also revealed a high proportion of KP who are systematically using condoms, particularly with casual partners and clients. Nevertheless, the prevalence of risky sexual behavior 46 remains high among KP because the majority of KP (especially FSWs) had unprotected sexual intercourse with their regular partners over the last six months prior to our survey. Findings from FGDs suggest also that PACTE-VIH program has improved KP health seeking behavior considering high proportion of KP who reported having received condoms and lubricants from the PACTE-VIH program as well as the proportion of those have participated to sensitization activities. EVALUATION OF PERCEPTIONS Evaluation Question 5: How do KP perceive the services and providers? Evaluation Question 6: How do service providers perceive the services they provide to KP? Evaluation Question 7: How do other stakeholders (e.g. Conseil National de Lutte contre le Sida (CNLS), Global Fund, United Nations Fund for Population (UNFPA), West African Health Organization (WAHO), etc.) perceive services and technical assistance provided? Overall, the large majority of KP and stakeholders in the two countries valued PACTE-VIH’s STI and HIV/AIDS services, including the availability of health staff. They reported that they have never been denied health services at the PACTE-VIH affiliated facilities. Findings from KP surveys are consistent with qualitative data (FGDs and IDIs). Indeed, in the two countries, the large majority of the study participants (FSWs and MSM) prefer to visit the PACTE-VIH affiliate centres for getting STI and HIV/AIDS information and care. All stakeholders reported benefits to the PACTE-VIH collaboration, particularly in financial and logistic (inputs, including antiretroviral treatment) support as well as in terms of individual and capacity building. ASSESSMENT OF ABILITY TO SCALE-UP Evaluation Question 8: What conditions are necessary to scale-up this model in other countries in the region? Evaluation Question 9: How might the elements of the model be adapted in other West African countries? Findings from IDI suggest four key prerequisites to scaling up the PACTE-VIH model: (1) rely on the evidence-based approach; (2) individual and institutional capacity strengthening; (3) and creation of integrated sub-regional structures based on existing institutions; and (4) enabling the environment, especially get the political will. Stakeholders, including UNFPA, UNDP and PSI support the need for the MOH structures to be more involved in the PACTE-VIH activities (CCM) and the necessity for the program to be developed and implemented within the national health policy framework (CNLS2). It is also important to identify key partners for effective and sustainable intervention (Medias, police, justice…). Some stakeholders suggested that PACTE-VIH package integrate family planning services in the future because all FSWs are aged 15-49 (reproductive age). In conclusion, PACTE-VIH program constitutes an opportunity and hope for fighting HIV/AIDS as well as promoting human right in Burkina Faso and Togo through KP. The program has achieved most targets through sensitization as well as availability of condoms and lubricants. However, efforts should be made to implement interventions that explicitly address gender-based violence and stigmatization from the security force and media staff. The mixed sexual behavior of KP in Burkina Faso and Togo, the persistent, though declining, internal and external barriers due to traditions, and the lack of sustainable financial and human resources depicts a truly the complex situation of the dynamics of HIV in the studied communities. 47 RECOMMENDATIONS Our findings raise a number of programmatic and policy-relevant recommendations for addressing the questions involved in promoting sexual and reproductive health among KP and the population in Burkina Faso and Togo, in terms of accessibility, effectiveness, enabling environment, health seeking behavior, perceptions, and ability to scale-up. • Findings suggested low access to HIV test in PACTE-VIH facilities. Sensitization and increased activities to encourage HIV testing, especially after unprotected sexual intercourses. The program should provide regularly required inputs to achieve this objective. • Distribution of condoms was low during Years 1 and Year 2. About 44% of male condoms were distributed in the first quarter of the third year. The program should make effort to maintain availability and the distribution of condoms over the project life. • The activities reached successfully the targets (in some cases over 100%) Achievements above 100% suggest a need to develop more realistic targets and avoid underestimation of targets. • Effort should be done regarding interventions that explicitly addresses gender-based violence and stigmatization from the security force and media staff. • Assess the effectiveness of the DICs and other KP-specialized facilities through a trends analysis of users, the user profiles, and the activity’s attraction strength (number and proportion of new visitors). This could be the topic of operations research. KP and stakeholders valued the PACTE-VIH program in providing financial and logistic (inputs, including antiretroviral treatment) supports, as well as individual and capacity building. The program should maintain dialogue with stakeholders as well as supports to implementation partners. • Scale-up efforts should take into account the overall lack of local resources to maintain and expand the PACTE-VIH activities. This includes low funding opportunities, lack of well-trained MOH health personnel in the country, as well as their unequal distribution in the urban areas. There is a need to develop a network with other stakeholders working on sexual and reproductive health with the same KP to share experiences, lessons learned and challenges within the country, as well as between the two focus countries. • Lessons and experiences with the PACTE-VIH model should inform national health and development policy. In particular, national programs should address societal causes of, and intolerance of, FSWs and MSMs. • The MOH should lobby internally to get more funding for the minimum package of services to prevent and provide care to the most vulnerable population and ensure sustainability of the program. • Voluntary school-based medical visits and periodic public health campaign for youth (regardless of sexual orientation and occupation) could be the strategies promoting regular HIV testing and sensitization, especially of informal “FSW” and hiding MSM”. General Comments from Tamara 1. Try to avoid writing in first person, e.g. “we assume”, “we interviewed”, “our study” and etc. Rather, elaborate on arguments that justify the assumptions, and study methods chosen and try to deliver sentences that present an objective view, rather than a subjective one. 2. The limitation section needs to be expanded. There are a number of limitations for qualitative studies that are not addressed here. Also, please provide a sentence or two on how the limitations were addressed. 48 3. Provide linkages of the findings with PACTE-VIH programming – you can’t tell from the study whether the achievements are attributed to the program, or whether they’re a “given”. Is there a baseline that they could be assessed against? 4. The study reads a bit descriptive and lack an analytic component to it, that is, what do these findings tell us and how they fit within the general context of PACTE-VIH and situations in Togo and Burkina Faso. Consider adding a DISCUSSION section to the study. 5. Avoid generalization of the findings. That is, when stating an outcome, ALWAYS clarify that this was found among the FGD participants, or people interviewed. Otherwise, it sounds like we are making the statement about MSMs and FSWs in general, and the sample size is too small to make that statement. In fact, generalization (or the lack of one) should definitely be stated as one of the limitations of the study. 6. I would go “easy” on the quantitative component of the study - the sample size is just not big enough. It may be better to refer to it as a “tool that support in presenting qualitative findings.” This study is mostly qualitative, and the quantitative portion seems to be supplementary to it. 7. The report needs “restructuring” to make sure that the content has a logical flows and everything is included in its relevant sections. 8. The line between “access” and “enabling environment” seems a bit blurry in the “Key Findings” section. We should try to make clearer distinction or merge them. 49 BIBLIOGRAPHY Baral, S., Beyrer, C., Muessig, K., Poteat, T., Wirtz, A. L., Decker, M. R., et al. (2012). Burden of HIV among female sex workers in low-income and middle-income countries: a systematic review and meta-analysis. . Lancet Infectious Diseases , 538-549. Beyrer, C., Wirtz, A., Walker, D., Johns, B., Sifakis, F., & Baral, S. (2011). The Global HIV Epidemics among Men Who Have Sex with Men: Epidemiology, Prevention, Access to Care and Human Rights. . Washington, D.C.: World Bank . Ettiegne-Traore, V., & Traore Toure , F. (2013). Regional HIV/AIDS Prevention and Care Project in West Africa Region. Performance Monitoring Plan. Unpublished document. Accra, Ghana: FHI360. International Business and Technical Consultants, Inc. (IBTCI). (2015). USAID/West Africa Evidence for Development AID-624-C-15-00001 Statement of Work PACTE-VIH Mid-term Performance Evaluation. Accra, Ghana: USAID/West Africa. Kellerman, S., Holtz, S., Dutta, A., Aliou, S., Diallo, I., Redding, S., et al. (2011). The Epidemiology of HIV epidemics in the 21-country West Africa Region: The impact of most at risk populations (MARP). Action for West Africa region II(AWARE II). Accra, Ghana: USAID West Africa. Heckathorn, D. (2011). Snowball versus respondent-driven sampling. Sociol Methodol. , 355–366. Lenth, R. V. (2001). Some Practical Guidelines for Effective Sample-Size Determination. The American Statistician , 187-193. Papworth, E., Ceesay, N., An, L., Thiam-Niangoin, M., Ky-Zerbo, O., Holland, C., et al. (2013). Epidemiology of HIV among female sex workers, their clients, men who have sex with men, and people who inject drugs in West and Central Africa. Journal of the International Aids Society , 10.7448/IAS.16.4.18751. Papworth, E., Grosso, A., Ketende, S., Wirtz, A., Cange, C., Kennedy, C., et al. (2014). Examining Risk Factors for HIV and Access to Services among KP in West Africa. The Research to Prevention (R2P) Project Report. Baltimore, Maryland, USA : Johns Hopkins University (JHU). Tanser, F., Barnighausen, T., Grapsa, E., Zaidi, J., & Newell, M. L. (2013). High coverage of ART associated with decline in risk of HIV acquisition in rural KwaZulu-Natal, South Africa. Science , 966-971. Population Reference Bureau (PRB). (2014). The World's Youth 2014 Data Sheet. Washington DC: PRB. UNAIDS. (2015, July 14). UNAIDS announces that the goal of 15 million people on life-saving HIV treatment by 2015 has been met nine months ahead of schedule. Retrieved October 14, 2015, from UNAIDS web site:http://www.unaids.org/en/resources/presscentre/pressreleaseandstatementarchive/2015/july/201507 14_PR_MDG6report UNAIDS. (2011). World AIDS day report 2011. How to get to zero: faster, smarter, Better. Geneva: UNAIDS. UNDP. (2015, 09 25). Sustainable Development Goals (SDGs). Retrieved 10 14, 2015, from United Nations Development Programme web site: http://www.undp.org/content/undp/en/home/mdgoverview/post￾2015-development-agenda.html United Nations. (2012). The millennium development goals. 2012 report. New York: United Nations. Ettiegne-Traore, V., & Traore Toure , F. (2013). Regional HIV/AIDS Prevention and Care Project in West Africa Region. Performance Monitoring Plan . Accra, Ghana: FHI360. 50 Papworth, E., Ceesay, N., An, L., Thiam-Niangoin, M., Ky-Zerbo, O., Holland, C., et al. (2013). Epidemiology of HIV among female sex workers, their clients, men who have sex with men, and people who inject drugs in West and Central Africa. Journal of the International Aids Society , 10.7448/IAS.16.4.18751. Papworth, E., Grosso, A., Ketende, S., Wirtz, A., Cange, C., Kennedy, C., et al. (2014). Examining Risk Factors for HIV and Access to Services among KP in West Africa. The Research to Prevention (R2P) Project Report. Baltimore, Maryland, USA: Johns Hopkins University (JHU). 51 ANNEX 1: CONFLICT OF INTEREST DISCLOSURE UNDER SEPARATE COVER 52 ANNEX 1 A: PACTE-VIH POST-MID-TERM EVALUATION RECOMMENDATIONS ACTION REVIEW TABLE Evaluation Recommendation Acceptance Status (Accept/Reject) If not accepted, reason(s) for rejection Responsibility for Action (USAID, Implementing Partner, Other￾specify) Deadline for Implementation Implementation Status Findings suggested low access to HIV testing in PACTE-VIH facilities. The program should increase sensitization and activities to encourage HIV testing, especially after unprotected sexual intercourse. The program should provide regularly required inputs to achieve this objective. Accept This recommendation is accepted as identifies the need for increased sensitization around HIV testing. PACTE-VIH needs a strategy to increase both HIV testing and site yield by reaching those at highest HIV risks such as people with recent history of unprotected sex and other high risk groups. This is particularly important for Burkina Faso in both KP groups and MSM in both countries. PACTE-VIH September 2016 USAID/WA and PACTE-VIH staff in Accra and held a technical meeting on February 23, 2016 and an action plan was defined, which will be monitored quarterly. Distribution of condoms was low Reject PACTE-VIH has maintained an 53 Evaluation Recommendation Acceptance Status (Accept/Reject) If not accepted, reason(s) for rejection Responsibility for Action (USAID, Implementing Partner, Other￾specify) Deadline for Implementation Implementation Status during Year 1 and Year 2. About 44% of male condoms were distributed in the first quarter of the third year. The program should make an effort to maintain the availability and distribution of condoms over the project life. increased supply of condoms and lubricants in Year 2 which the project received free from USAID/ W/OHA The first year shortfall was due to the project order delay. The activities successfully reached the targets (sometimes over 100%). Achievements above 100% suggest the need to develop more realistic targets, and avoiding underestimation of targets. Efforts should be made regarding interventions that explicitly address GBV and stigmatization from the Reject PACTE-VIH uses size estimates that were made by the Research to Prevention (R2P) project (JHU) for two cities in Togo and two in Burkina Faso. PACTE set targets for each country based on those estimates and made projection for all project sites to set August 2017 For SGBV: A gender assessment and strategy was completed in in August 2015. The current Peer Educator education toolkits and messages were revised and adapted to address gaps related to GBV. A study was completed in November 2015 to 54 Evaluation Recommendation Acceptance Status (Accept/Reject) If not accepted, reason(s) for rejection Responsibility for Action (USAID, Implementing Partner, Other￾specify) Deadline for Implementation Implementation Status security force and media staff. a denominator for the key population sizes. The project and AOR set realistic and reasonable targets. The project accelerated its efforts to reach KP in Year 2 to ensure that it meets its life of project targets. Therefore, over achievement here is not due to poor target setting. In addition, results are within the acceptable range of +/-10% assess internal and KP led violence and violence perpetrated by the police to inform advocacy efforts with the police and reduce gender based violence among KP and from security officers. Assess the effectiveness of the DICs and other KP￾specialized facilities through a trends Accept See comments. PACTE-VIH January 2017 PACTE is conducting a program review and analysis of the project database 55 Evaluation Recommendation Acceptance Status (Accept/Reject) If not accepted, reason(s) for rejection Responsibility for Action (USAID, Implementing Partner, Other￾specify) Deadline for Implementation Implementation Status analysis of users, the user profiles, and the activity’s attraction strength (number and proportion of new visitors). This could be the topic of operations research. for trend analysis. The Evidence for Development in Health (E4D) Project will conduct operations research on the PACTE platform 2016 - 2017. KP and stakeholders valued the PACTE￾VIH program in providing financial and logistic (inputs, including antiretroviral treatment) supports, as well as individual and capacity building. The program should maintain dialogue with stakeholders, and support to IPs. Reject PACTE-VIH and the Regional Health Office dialogues with stakeholders on a regular basis. PACTE has led two regional KP stakeholder meetings in the last two years. The KP regional platform works and is providing needed support to community- based organizations on service delivery and advocacy. 56 ANNEX 2: ABRIDGED BIOS OF THE EVALUATION TEAM MEMBERS ANNEX 3: PACTE-VIH TARGET SITES IN TOGO AND BURKINA FASO 57 58 ANNEX 4: STATEMENT OF WORK DESCRIPTION OF PACTE-VIH ACTIVITY The Regional HIV/AIDS Prevention and Care Project (PACTE-VIH) is a cooperative agreement between the United States Agency for International Development/West Africa Mission (USAID/WA) and Family Health International (FHI360) for the period August 2012 through August 2017. The project Award Number is: AID-624-A-12-00006. The total five-year budget for the project is: $13,500,000. The Agreement Officer’s Representative (AOR) is Dr. Laurent Kapesa. FHI360 leads PACTE-VIH, with 12 implementing partners (IPs) including Africa Consultants International (ACI), a Dakar-based NGO leading the advocacy and policy development activities; EVT, Forces in Action for the Well-Being of Mother and Child (FAMME), the two IPs in Togo; AIDS Empowerment and Treatment International (AIDSETI); Alternative Burkina (AAB), Association African Solidarité (AAS); Association Appui Moral Materiel et Intellectuel a l’Enfant (AMMIE), Association des Jeunes pour le Développement de Bittou (AJDRB); Association Solidarite Defis (ASD), Responsabilité, Espoir, Vie, Solidarité (REVS+); and YERELON, Clinique Yerelon, the nine IPs in Burkina Faso. TARGET POPULATIONS AND INTERVENTION SITES PACTE-VIH targets Female Sex Workers (FSWs) and their clients, and Men having Sex with Men (MSM). Initially PACTE-VIH focused on Burkina Faso and Togo, two West-African countries with mixed HIV epidemics. In Burkina Faso, the FHI360 implemented project activities in four sites: two cities (Ouagadougou and Bobo Dioulasso) and two transit points that have become hotspots for transactional sex (Bittou, and Niangologo). In Togo, FHI360 implemented the project in five sites including Lomé, Kpalimé, Kara, Cinkassé, and Dapaong. BACKGROUND STATEMENT OF THE PROBLEM West Africa is an immense and diverse geographic region of 21 countries facing some of the most significant health development challenges in the world. West Africa is home to 365 million people. With a Total Fertility Rate (TFR) as high as 6.89 in Niger and 6.16 in Mali, West Africa's population is expected to double by 2050. The looming demographic changes are likely to increase instability in a region that already has some of the world’s poorest health indicators. West Africa has the world’s highest maternal mortality rate (the regional average is in the mid-to high range of 483-888/100,000) due to persistent inadequate health options for women. The region has a low use of modern contraceptives (regional average of nine percent), and a high unmet need for contraception (regional average of 30 percent). Vitamin A deficiency is pervasive in the region, contributing to high perinatal death rates; and the prevalence of diarrheal disease is closely linked to unreliable access to clean water and is a leading cause of child mortality under five (regional average of 156/1,000 live births). Despite the low HIV prevalence in the general population relative to other regions of Sub-Saharan Africa, West Africa contributes a significant number of new infections to the global burden due to its large population. In addition, the prevalence of HIV/AIDS, while low in the general population, (3.2 59 percent in Togo, 1 percent in Burkina Faso) is quite high in key populations (KP), notably FSW and MSM, with rates being 19 to 30 times higher than in the general population. Results of size estimation studies in Burkina Faso and Togo show an HIV rate of 28.5 percent among FSW in Lomé and 21.4 percent among MSM. Other studies state that the rate is 10 to 20 times higher among two primary high risk groups in both countries: MSM and FSW1. An epidemiologic study conducted in 24 West and Central African countries showed that the pooled of HIV prevalence among FSWs was 34.9 percent, among their clients was 7.3 percent, among MSM was 17.7 percent2 The majority of FSW (more than 85 percent) who underwent HIV testing and tested positive in Lomé were never told that they had the HIV infection. They continued to have unprotected sex (10 to 30 clients per night) as condom use was minimal. A combination of behavioral, structural, and environmental determinants drive the spread of HIV and Sexually Transmitted Infections (STIs) in both countries: low condom use; limited and low quality health information, and stigma and discrimination against Most At Risk Populations (MARPs), preventing them from seeking and accessing healthcare services, among others. A systematic review of existing data showed that clients of FSW had a HIV/AIDS prevalence nearly three to five times higher than the other adult men in the general population. The clients of FSW include mainly long distance truck drivers (LDTD), crossing countries which are well connected by roads. They are bridge populations which will likely bring HIV infection in their households. Drug use is becoming an extensive and growing issue in the region. The prevalence of intravenous drug use (IDU) in Nigeria is 3.8 percent (three times the prevalence among general population) and 19.2 percent in Ghana. In West Africa, weak health systems, and poor governance and accountability are critical constraints to the delivery of quality health services. In addition, barriers to accessing health services remain a major constraint, particularly to marginalized populations including women, children and members of KP. These barriers are perpetuated by a weak health system, a policy environment that is not inclusive of the rights of the marginalized population, and a general lack of knowledge that is the manifestation of uninformed attitudes. As a result of weak or non-functioning health systems, lately the region has been experiencing unprecedented issues of stock-out of life saving HIV/AIDS drugs such as anti-retroviral treatment (ART) and opportunistic infection drugs. These problems are further exacerbated by the lack of qualified staff to support the commodities and logistics system, and delayed, inconsistent or suspended disbursement of funding from the Global Fund to fight AIDS, Tuberculosis and Malaria (GFATM) due to governance and transparency issues. HYPOTHESIS We expect that by achieving these three Intermediate Results (IRs): IR1: Increased access to high￾quality/comprehensive prevention and care programs for MARPs (FSW, their clients, and MSM); IR2: Improved enabling environment for evidence-based public health interventions targeting MARPs; and IR3: Improved quality of GFATM HIV prevention interventions that focus on MARPs in select countries, that PACTE-VIH will address the critical gaps in KP programming across the region, build political will, strengthen the human and institutional capacity of national coordination bodies, and gather resources to engage key stakeholders including donors and civil society organizations to support programs that are evidence-based, adaptable, replicable and transferrable throughout the region. 1 Togo SP/NAC, 2010, PAMAC – UNDP, 2010 2 “Epidemiology of HIV among female sex workers, their clients, men who have sex with men, and people 60 who inject drugs in West and Central Africa”, Journal of the International Aids Society, 2013, by Papworth E. and al”. 61 PURPOSE OF THE EVALUATION The purpose of the mid-term performance evaluation is to examine the implementation of the model service package, determine what is working well and what is not working well, and to gain an understanding of how the model can be taken to scale regionally and replicated in other countries in West Africa. The information from the mid-term performance evaluation will be used by the decision makers and managers in the USAID/WA Regional Health Office (RHO), Regional Office of Acquisition & Assistance, the Mission Director, and Deputy Mission Director. In addition, FHI360 and its IPs will use the evaluation findings to inform on-going implementation of the PACTE-VIH project and USAID will use the findings to inform the design of future regional HIV/AIDS projects, as well as implementation of projects that PACTE- HIV coordinates, such as those with GFATM. We will involve USAID/WA, as the requester of this mid-term performance evaluation, throughout the process from development of the Scope of Work (SOW) starting with the Team Planning Meetings (TPM) until the final report. International Business and Technical Consultants Incorporated (IBTCI) prefers to have USAID/WA staff join the performance evaluation team during data collection. PACTE￾VIH’s IPs will help support the logistic preparation and facilitate contacts with respondents to streamline field operations. IBTCI expects the IPs to make a presentation during the TPMs to provide background on the activity to the performance evaluation team. The evaluation team will also interview some of its staff as key informants. The team will visit local partner organizations and see their opinion through key informant surveys (KISs). The Ministry of Health (MOH) will contribute information through data sharing, and employees serve as key informants. Beneficiaries will participate in KISs and members of focus group discussions (FGDs). We will also approach other international donors for interviews. EVALUATION PERIOD This mid-term performance evaluation will cover activities from August 2012 to March 31, 2015. The start date of the performance evaluation is on/about June 15, 2015 and it will end on/about September 17, 2015, including all evaluation tasks, debriefs, and submission of the final performance evaluation report. EVALUATION QUESTIONS This performance evaluation will address the following questions, which are listed in order of priority: 1. What is working well and what is not working well for the KP programming in the Togo, Burkina Faso? [Analytical Domain: Effectiveness] 2. How has the minimum package of services increased access to HIV/AIDS prevention, care, support, and treatment services for KP in Togo, Burkina Faso? [Analytical Domain: Access] 3. What are the key stakeholders’ perceptions of the services and technical assistance provided through the PACTE-VIH activity; specifically: [Analytical Domain: Perceptions] a. How do KP perceive the services and providers? b. How do service providers perceive the services they provide to KP? c. How do other stakeholders (e.g. Conseil National de Lutte contre le Sida (CNLS), 62 GFATM, United Nations Fund for Population (UNFPA), West African Health Organization (WAHO), etc.) perceive services and technical assistance provided? 4. How has this HIV prevention package influenced health seeking behavior among KP? [Analytical domain: (Health seeking) Behavior} 5. How, if at all, has community involvement (KP, media, police, health workers and other community members) affected the environment for implementing evidenced based interventions for KP in Togo, Burkina Faso? [Analytical Domain: Enabling Environment] 6. How can this model for HIV/AIDS prevention, care, support and treatment be replicated in other countries in West Africa? [Analytical Domain: Scale-up] a. What conditions are necessary to scale-up this model in other countries in the region? b. How might the elements of the model be adapted in other West African countries? EVALUATION METHODS EVALUATION DESIGN AND METHODOLOGY This mid-term performance evaluation will be a non-experimental cross-sectional design as it will focus on implementation processes. It will employ mixed methods. We will conduct multi-level analysis using quantitative methods at some levels and qualitative at others. The revised draft SOW will contain the penultimate design and methodology. After the evaluation team has commenced the study, they will revise the design and methodology based on their document review and input from the TPMs. the Inception Report will present the final design and methodology and will also include the Data Analysis Matrix and draft data collection instruments. The performance evaluation will meet the criteria of a quality evaluation as defined in the USAID Evaluation Policy. The performance evaluation team will commence the study at their home base and conduct document review for one week before traveling to Accra to meet with USAID/WA, convene the TPMs, and produce the Inception Report. The team will then divide into two sub-teams, A and B, to travel to Burkina Faso and Togo. There, they will meet up with the research assistants and train them before commencing to collect data, first in the capital cities, followed by one week of data collection in the regions. After the data collection period ends, the teams will conduct any needed debriefs in-country before the sub-teams travel back to Accra to meet with USAID/WA, manage and analyze the data, and prepare the first draft of the report. The full team will meet with USAID/WA to discuss the preliminary draft. SAMPLING The evaluation will draw a purposive sample, given that the data collection in the regions will be up to three weeks in Togo and Burkina Faso. The performance evaluation team will define the sampling frame, site selection criteria, and unit(s) of analysis after they have completed their document review about PACTE-VIH. We will purposively select Key Informants, who will participate in KISs, and FGD participants. Although findings from the FGDs and KISs will give insight about the views of participants, IBTCI cannot generalize them to the entire population of beneficiaries of PACTE-VIH because the study design does not allow for randomization. At a minimum, the inclusion criteria for participants include: being 18 years or older; able to provide informed consent, and being associated with PACTE-VIH or health programs. Exclusion Criteria include being unable to understand or provide informed consent; being 63 under the influence of alcohol and drugs, which is likely to impair judgment and behavior; being under 18 years; or being a duplicate recruit. DATA SOURCES IBTCI will collect both primary and secondary data. The performance evaluation team will commence the study no later than 30 days after USAID/WA has finalized and approved the SOW. The team will gather information from various stakeholders including Representatives of the MOH, CNLS, GFATM, UNFPA, UNAIDS, United Nations Development Programme (UNDP), WAHO, community leaders, MSM, FSW, media, and the police, as well as from intervention site service statistics. The performance evaluation team will answer the evaluation questions using several sources of information, namely: Document Review The performance evaluation team will conduct an extensive desk review of data and reports from the MOH, USAID and PACTE-VIH, as well as materials from other donors and independent research organizations. Please see Annex B for a list of the key documents included in this review. We will use the Grades of Recommendation, Assessment, Development and Evaluation (GRADE) criteria, which evaluate strength of evidence for program effectiveness, when selecting documents to include in the review. To ensure that reviewers are methodical in the review process, we will develop a data extraction form using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for reporting. A data extraction form is a standard instrument used to systematically collect data from official documents or scientific reports. Using the PRISMA checklist, we will divide the form into sections that will include: the title; the abstract or structured summary; the rationale and objectives; the methods used to collect information, including the protocol, eligibility criteria, information sources, data collection process, risk of bias; and results. IBTCI will pre-test the data extraction forms. We will ask two reviewers to review one document using the form. The evaluation team will review each completed form w to ensure that the two reviewers captured the same information. The desk review will allow the team to answer evaluation questions 1 and 2. Primary Data Key Informant Interviews Through IDIs with representatives of MOH, CNLS, GFATM, UNFPA, -, WAHO, community leaders, media, and police, IBTCI will seek to understand how each group perceives the PACTE-VIH approaches currently being implemented, methodologies and limitations, and contextual barriers and opportunities. KISs can be done in-person where possible or by phone / Skype or other cost￾effective means. The evaluation team will conduct one interview with a representative of each international organization listed above, one community leader and a representative of police in each city, as well as two representatives of the media in big cities only. 64 The evaluation team will also conduct KISs with program implementers (FHI360 and partners) and beneficiaries. Program beneficiaries will include FSW, their partners, and MSM living in participating cities in Burkina Faso and Togo. Through interviews with program implementers and beneficiaries, we will ascertain information related to their perceptions about the impact of PACTE-VIH. In the beginning of the data collection period, the evaluation team will complete KISs before FGDs begin, so they can use information from interviews to finalize the question guide for FGDs. Due to logistical considerations and availability of respondents, the schedule might not always allow for the KISs to be convened prior to the FDS. IBTCI will not pay key informants for interviews, but may provide a modest amount for travel to and from the interview location, if the location is other than the key informant’s normal place of business. One evaluator can conduct three and a half KISs per day, while in the capital cities. Two evaluation team members will conduct a total of 28 KISs (3.5 x4 days x 2) in each country. In the regions, although evaluators have one week, the team will plan on four days of actual data collection due to travel and unexpected delays. Limitations Given the limited timeline for the mid-term evaluation, we will not cross-check information collected through interviews (our primary data source), which may affect the data quality. Qualitative studies are prone to biases, such as recall bias on the part of the respondent, and interviewer bias among the different evaluation team members who will invariably posit the semi-structured questions slightly differently. This is the nature of gathering rich qualitative data. To mitigate intra-interviewer bias, the evaluation team members will remain in close communication by telephone and email on a daily basis to allow them to compare their experiences. During the TPM, the team members will also be designing the study and its instruments jointly so they will share the same interpretation of the intent behind each question posed in the instruments. We will pre-test the instruments once the teams are in Burkina Faso and Togo, then revise them based on the findings from the pre-test. Note that we aim to have standardized instruments for both countries. However, this will be open for discussion for circumstances whereby certain questions or wording of a question might be particular to only one country and not the other. The KISs will allow the team to answer evaluation questions 1- 6. Focus Group Discussions The evaluation team will conduct FGDs separately with MSM, FSW, community leaders, media, and the police. FGDs are particularly valuable for gathering information, which gives rise to divergent opinions, consensus or results that involve complex issues that require in-depth exploration. Whenever relevant and feasible, the team will ensure gender balance among the FGD participants to give a holistic picture. The evaluation team will hold FGDs in one city and one rural area for both countries. They will select the city and rural location with the highest volume of KP reached by PACTE-VIH. Groups will be made up of 8-10 individuals and selected based on meeting qualification criteria for participation in the FGD. IBTCI will not pay group participants for participation, but may provide a modest amount for travel to and from the FGD location. 65 The evaluation team will facilitate the FGDs according to semi-structured focus group guidelines. To decrease the likelihood that clients will give biased responses about the health services they receive from a facility because of the fear of being overheard, research assistants will schedule FGDs to take place in the community as opposed to in the health facility being discussed. They will also conduct discussions in the language preference of the group. The team will record FGDs, and type summaries of the discussions in English and enter them in the computer (Excel or Word) for coding by the International Evaluation Specialist. IBTCI's Evaluation and Research Advisor will review the coded data sets for validity and reliability. LIMITATIONS The team will employ a purposive sampling strategy for selecting FGD participants and key informants at participating sites, which may affect the quality of data. The differences in the level of education and the mother tongues between the interviewees and the interviewers may also affect the quality of communication and ipso facto the quality of the information collected. Furthermore, the note taker may not capture the full intent or meaning of some statements. This omission may also affect the quality of the data collected. We anticipate that some interviews may be conducted in project sites where at least one or more outside observers, including project staff, are present, and that interview responses could be affected by the presence of these observers because of respondent’s fear of being overheard. The evaluation team will make efforts to mitigate these limitations through recruitment and training of note takers and interviewers who are native of the areas where the interviews will take place. As much as possible, we will exclude the presence of observers, especially staff members and service providers, to allow for free and open communication. The FGD will allow the team to answer evaluation questions 3, 4 and 5. Secondary Data The evaluation team will also collect secondary quantitative assessment trends in project performance. The senior evaluation team members will work closely with the research assistants who will extrapolate data from existing surveys and monitoring reports, the PACTE Monitoring and Evaluation Plan indicators (preferably outcome indicators), MOH data, and other sources of relevant data for the purpose of triangulating and substantiating the primary data collected in this study. The secondary data will allow the team to answer evaluation question 1, 2, and 4. DATA COLLECTION INSTRUMENTS Prior to carrying out data collection, the performance evaluation team will submit the data collection instruments in the Inception Report for USAID/WA’s approval. These instruments will include a semi￾structured KIS questionnaire, an FGD guide with modules tailored to the various FGD respondent groups, and an FGD Facilitator’s Guide that will detail how we will conduct focus group interviews and how we will transcribe and analyze the data. IBTCI will translate all data collection instruments into French and local languages and test the instruments prior to using them to collect data from the target population. Trained research assistants will conduct FGD data collection using the different discussion guides. A facilitator will 66 moderate the FGD sessions while a note taker takes notes and records sessions using the audio recorders, which the note taker will later transcribe. Only the senior evaluation team members, who will remain in close virtual contact during the course of the data collection period to share any findings which might have implications (e.g., discovering new information which would lead to adding additional questions to the KIS semi-structured questionnaire), will conduct KISs. Due to the nature of collecting qualitative data during FGDs and KISs, interviewers will not necessarily posit all of the questions to each respondent or group. This will allow the interviewer to delve into new information which inevitably will arise during the course of the interviews – information which could not have been anticipated when developing the instruments. Nonetheless, the team will note which questions were asked of each respondent and group in order to keep track of the true denominator for each question. Interviews will last approximately one and a half hours. DATA MANAGEMENT The information collected through KISs and FGDs will be qualitative in nature. The evaluation team will audio-record some of the information and transcribe all notes verbatim. We will extract and code information from the document review, as appropriate. We will keep data in a password-protected space accessible only to the evaluation team members. We will store and manage all data, regardless of the source, using NVIVO or Atlas.ti. We will analyze data using the content analysis approach. The evaluation team will code three interviews and discuss the codes to ensure consistency in the coding approach. The team will use the developed code book to code the rest of the interviews and they will group the codes into broad categories from which themes will be extracted. To ensure confidentiality of information, the evaluation team will remove all identifiable information from quotes as needed. We will also select typical quotes for inclusion in the reports in order to emphasize the response given without losing the original context of the meaning. DATA DISAGGREGATION The evaluation team will analyze the information collected in the course of this evaluation to identify major trends and issues. The team will disaggregate primary data by KP, sex, age, site location and country to identify how project inputs are benefiting these groups. We will use the findings to assess outcomes and impact on males and females. DATA QUALITY Data quality must meet USAID’s five quality standards: validity, integrity, precision, reliability and timeliness. Data collected must be free of bias such as interviewer bias, unrepresentative sampling, or transcription bias and should clearly and adequately represent the intended results. The evaluation team will assess data quality throughout the data collection, and the evaluation report will provide a summary of these assessments. IBTCI has a series of protocols and procedures for conducting evaluations to ensure the primary data collected through the course of its evaluations meet USAID’s five data quality standards. IBTCI provides performance evaluation team members with guidance such as our Evaluation Team Manual which is closely linked to the requirements of USAID’s Evaluation Policy (ADS 203), IBTCI Ethical Standards and Protocols for Field Research which protects human subjects and contributes to the validity of the data. During data collection, IBTCI conducts random checks on approximately 10-20 67 percent of the data collected to ensure reliability and precision of the data. Also during data collection, IBTCI expects the sub-teams to meet each evening to debrief to improve intra-interviewer reliability so the questions are being posited in the same fashion and to add new probes as needed. The team leader will collect written KIS and FGD transcripts from each team member within 48 hours of completion of the interviews to minimize recall bias and improve the validity and precision of the data. IBTCI standard operating procedures also dictate that a senior IBTCI technical expert reviews all major deliverables well ahead of the due date to USAID to ensure consistency in the data collected which improves the reliability and to a lesser extent the validity. As well, this internal quality assurance procedure ensures compliance with the ADS 203 and the SOW. The research team will ensure that data collected is used only for the purpose it was collected and not be manipulated for other use. The evaluation team will ensure that progress toward performance targets reflects real changes rather than variations in data collection methods. The data collection tools and methods must be reliable. Finally, we will collect and analyze data in a timely manner to allow management decision-making at the appropriate levels. METHODOLOGICAL STRENGTHS AND LIMITATIONS IBTCI designed the different measures, including checking quality of data collected, triangulation of secondary data using different sources, and verifying the database validation check and control mechanism for ensuring data quality, and applied them to the methodology of this cross-sectional study to mitigate limitations due to sample selection, recall bias, interviewer bias, and note taker deficiency in FGDs. These measures will allow a high level of reliability to the information that will be produced and will give a fair image of the overall project performance. We cannot generalize findings from this study to the general population for lack of representativeness of respondents, but the most objectively and scientifically rigorously produced information will inform decision makings, and the recommendations will guide necessary changes and project improvements. DATA ANALYSIS PLAN The Inception Report will include a preliminary data analysis plan for both quantitative and qualitative analysis. The plan will describe how we will transcribe and analyze focus group discussions and key informant interviews. It will also indicate the method of analysis for each evaluation question including the data sources and describe how we will integrate qualitative and quantitative data to reach final conclusions and recommendations. IBTCI will organize data according to analytical domains which correspond to the evaluation questions. Referring back to the evaluation questions, those domains include: achievements; challenges; effectiveness; access; perceptions; evidence-based interventions; and scale-up. Similarly, we will organize the FGD and KIS transcript templates according to these analytical domains in order to pre-code them before the data analysis phase. Triangulation will be integral to our analysis of the various data sources that will aid in the interpretation of the findings. To assure the validity and reliability of evaluation findings, the evaluation design will gather information from different sources and involve a research team whose members have different educational background and experiences. The final report will include the 68 data analysis procedures. CRITICAL ASSUMPTIONS For the successful implementation of this project, IBTCI made the following assumptions: ● Social, political and legal environments remain favorable for implementation ● MOH remains committed ● Continued support from other donors, including GFATM ● IPs demonstrate technical competencies and good governance EXISITNG INFORMATION SOURCES IBTCI will make all work plans, quarterly reports, Monitoring and Evaluation Plan, annual reports, and baseline data for the project start date to present available to evaluators in a folder on docs.google.com. You can find the list of documents in Annex B. DELIVERABLES AND TIMELINE OF THE SOW We have summarized the important activities to be undertaken to meet agreed upon deliverables in the attached timeline in Annex A (Deliverables are specified in bold this timeline). The duration of these activities is as follows: Activity Start and end date IR 1.2 Evaluation of health projects conducted Identify HIV/AIDS projects to be evaluated and evaluation questions coordinating with the COR April 1 – June 30 Recruit evaluation team based on evaluation SOW for the regional HIV/AIDS project, PACTE-VIH (Togo. Burkina Faso) May 11 – June 1 Desk review of background documents June 15 – June 19 Develop PACTE-VIH Desk Review Report to outline findings from desk review related to the evaluation methodology and questions (Deliverable) June 19 Evaluation TPM June 22 – June 24 Finalize Inception Report which is a continuation of the SOW and based on findings from the Desk Review Report; evaluation implementation plan and draft data collection instruments will be included (Deliverable) June 22 – June 27 69 Activity Start and end date USAID reviews the Inception Report June 29 – July 03 Translation/back translation of data collection tools June 26 – June 27 Train data collection team of local partner pilot testing instruments, revise data collection instruments, final translation of instruments June 29 – July 02 Implement data collection activities according to approved SOW KIS at central level capital cities in Burkina Faso and Togo. Data collection in the field (provinces) Data Management: transcription, cleaning, coding, data entry, data quality control audits June 29 – July 03 July 06 – July 11 July 03 – July 05 Provide weekly updates to COR on evaluation progress June 29 – July 10 Finalize data analysis July 06 – July 08 Hold debrief with USAID and stakeholders and submit draft evaluation report to USAID Debriefing Partners (Deliverable) July 20 July 21 First draft report July 20 July 29 IBTCI internal quality assurance review of the report July 27 – July 29 Copy editing and formatting of first draft of evaluation report for submission to USAID July 29 -– August 1 USAID review draft report (Deliverable) August 03 – August 14 IBTCI prepares presentation for Dissemination Meetings in Burkina Faso and Togo (Deliverable) July 27 – July 29 Evaluation team incorporate USAID edits August 17 – August 19 IBTCI internal quality assurance review of report August 18 – August 19 IBTCI submits final report (Deliverable) August 22 USAID reviews final report August 24 – August 28 IBTCI incorporates USAID edits and finalizes report August 31 – September 02 Close out (upload of report to DEC, submit data sets, etc) Evaluation raw and analyzed data transferred to USAID in a machine-readable format application program interface. All data collection tools transferred to USAID. (Deliverable) September 04 The evaluation must comply with the January 2011 USAID Evaluation Policy, available at 70 http://www.usaid.gov/evaluation/policy. The final evaluation report must follow the guidelines stated in the USAID Evaluation Policy. Specifically, the report must adhere to the following quality criteria for the preparation of evaluation reports, as outlined in Appendix I of the Evaluation Policy: ● The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. ● Evaluation reports should address all evaluation questions included in the statement of work. ● The evaluation report should include the statement of work as an annex. All modifications to the statement of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology, or timeline need to be agreed upon in writing by the technical officer. ● Evaluation methodology should be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. ● Evaluation findings will assess outcomes and impact on males and females. ● Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, conjecture, hearsay or the compilation of people‘s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. ● Sources of information need to be properly identified and listed in an annex. ● Recommendations need to be supported by a specific set of findings. Recommendations should be action-oriented, practical, and specific, with defined responsibility for the action. The final report with (at minimum) should include the following: ● An executive summary 3-5 pages in length that summarizes key points (project purpose and background, evaluation questions, methods, findings, conclusions) ● The Evaluation SOW in an annex Any “statements of differences” regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team in an annex ● All tools used such as questionnaires, checklists, survey instruments, and discussion guides in an annex ● All sources of information, properly identified and listed in an annex. These performance evaluation reports will be written in English and French. DISSEMINATION PLAN We will share findings of the mid-term evaluation with various stakeholders including USAID, MOH, 71 WAHO, GFATM, UNFPA etc. The performance evaluation team will organize a PowerPoint Presentation for USAID/WA. We will organize another debriefing for other stakeholders to discuss the major findings of the evaluation and produce and forward to USAID/WA hard and electronic copies of the final report. We will determine the total number of the final reports based on USAID/WA recommendation. EVALUATION TEAM COMPOSITION Four core members, both international and local, with a mix of expertise in evaluation and HIV/AIDS programming, will compose the evaluation team. One team member will be an evaluation specialist. The evaluation team will consist of: an expatriate Team Leader, an expatriate Evaluation Specialist, a Burkinabe Senior Subject Matter Expert, a Togolese Senior Subject Matter Expert; two Research Assistants located in Burkina Faso and Togo, two note takers (for Burkina Faso and Togo), a Logistician and Editor. We will divide the team into two sub-teams in order to collect data in the two countries (two major cities, two rural areas) in which FHI360 implemented PACTE-VIH. The Team Leader, who will be a Health Evaluation Expert, will lead sub-team A, which will also include a Burkinabe HIV/AIDS Subject Matter Expert, two Burkinabe research assistants and a note taker. The Evaluation Expert will lead sub-team B, which will also include a Togolese HIV/AIDS Subject Matter Expert, two Togolese research assistants and a note taker. The roles, responsibilities and qualifications of the evaluation team are defined below: EVALUATION TEAM LEADER Responsibilities The Evaluation Team Leader will be responsible for: ● Overall management of the evaluation team; ● Desk review of documents, development of Inception Report, consisting of draft methodology, detailed work plan; ● Coordination of evaluation activities including training of data collectors, data collection, implementation, data management and quality assurance and other related tasks; ● Conduct debriefing with USAID and IBTCI on the methodology; ● Conduct debriefing with implementers on evaluation findings; ● Conduct field visit to the pilot project site and interviews with stakeholders (Key Informants, Key populations etc. ● Development and submission of the mid-term evaluation draft report; ● Finalization and submission of the final evaluation report to IBTCI after incorporating suggestions received on the draft report; ● Throughout the evaluation period, be able to exercise strong communication, organizational, team leadership and interpersonal skills; periodically coordinate/update IBTCI’s Senior Monitoring and Evaluation Advisor and as requested. ● Disseminate the evaluation findings. Qualifications ● An advanced degree in public health, social science or a related field ( preferred); 72 ● Extensive (a minimum of 10 years) experience in conducting HIV/AIDS program evaluations with both quantitative and qualitative methods for data collection and analysis; (highly desired) ● Previous experience leading evaluation teams is required; ● Prior evaluation experience in Sub-Saharan Africa is required; ● Excellent oral and written skills in French and English are required; ● Previous experience preparing high-quality evaluation reports; ● Previous experience with USAID-funded projects and knowledge of USAID’s ADS2013 policy, standards, guidance and protocols (highly desired). SENIOR EVALUATION SPECIALIST Responsibilities The Evaluation Expert's responsibilities include, but are not limited to, the following: ● Develop evaluation design, methodology, sampling strategy, and data collection instruments; ● Coordinate evaluation activities including training of data collectors, data collection, implementation, data management and quality assurance and other related tasks; ● Develop data analysis plan and conduct qualitative and /or quantitative data analysis, as required ● Actively participate with other team members during data triangulation, presentations and report writing. ● Assist the Team Leader in completion of the Inception Report and the writing of the evaluation report in conformance with the statement of work; ● Develop final evaluation report with quality assurance and timeliness of all deliverables to USAID and be responsive to all comments. Qualifications ● A Master’s degree in social sciences, public health, statistics, or a related area; ● At least 7 years of evaluation experience with both qualitative and quantitative methods for data collection and analysis; (highly desired) ● Prior evaluation experience in West Africa is required; ● Previous experience with USAID-funded projects and knowledge of USAID’s ADS2013 policy, standards, guidance and protocols (highly desired); ● Experience in using SPSS, STATA and/or other analytical software packages including qualitative analytical software packages such as NVIVO; ● Strong oral and written communication skills in French and English is required; ● Ability to effectively work in teams and embrace participatory approaches. SENIOR LOCAL SUBJECT MATTER EXPERT (BURKINA FASO AND TOGO) Responsibilities The Evaluation Expert's responsibilities include, but are not limited to the following: ● Develop evaluation design, methodology, sampling strategy, and data collection instruments; 73 ● Coordinate evaluation activities including training of data collectors, data collection, implementation, data management and quality assurance and other related tasks; ● Develop data analysis plan and conduct qualitative and /or quantitative data analysis, as required ● Actively participate with other team members during data triangulation, presentations and report writing. ● Assist the Team Leader in completion of the inception report and the writing of the evaluation report in conformance with the statement of work; ● Develop final evaluation report with quality assurance and timeliness of all deliverables to USAID and be responsive to all comments. Qualifications ● A Master’s Degree from an accredited institution in public health, medicine, nursing, or similar discipline is required. Formal training and experience in HIV/AIDS is required; ● A minimum of seven years of progressive responsibilities in program management for HIV/AIDS programs is required; ● Experience evaluating international public health programs is highly desired. ● Knowledge of West and Central African health institutions as well as familiarity with and sensitivity to socio-cultural factors affecting development in the region is required;; ● Excellent quantitative and qualitative analytical skills; experience in using SPSS, STATA and/or other analytical software packages including qualitative analytical software packages such as NVIVO is desired; ● Strong oral and written communication skills in French and English is required; ● Ability to effectively work in teams and embrace participatory approaches; and ● Resident of Burkina Faso for the evaluation team assigned to Burkina Faso or Togo for the evaluation team assigned to Togo. Team members will be required to provide a written disclosure of conflicts of interest. SCHEDULING, LOGISTICS AND BUDGET SCHEDULING To allow the evaluation team members to move freely in Togo and Burkina Faso to conduct interviews, transcribe, and conduct preliminary analyses, IBTCI along with our subcontractor CPHDA, will provide the necessary logistics, including space and means of transportation (e.g. a car for each sub team) and portable computers for data entry and analyses. CPHDA will provide the logistics support for in-country scheduling in Burkina and Togo. The CPHDA Project Coordinator/Logistician will maintain and update daily a scheduling log to keep track and communicate appointments for the evaluation team. The attached detailed budget in Annex C summarizes the costs which will be incurred to meet agreed upon deliverables. 74 ANNEX 5: DOCUMENTS FOR REVIEW 1. FHI360. The Regional HIV/AIDS Prevention and Care Project in West Africa (PACTE-VIH). Performance Monitoring Plan. 2. Draft SOW for PACTE-VIH Mid-term Evaluation 3. Project Year 2 Quarterly Report 1: October 1 2013-December 31 2013 4. Project Year 2 Quarterly Report 3, April June 2014 5. JHU-Baseline-Baral study Togo and Burkina Faso FSW and MSM indicators.xlsx 6. JHU-Baseline-Baral_R2_Togo_BurkFaso_FSW_MSM_ NOV2013.xlsx 7. Revised Work Plan for WAHO 8. Key indicators for PACTE-VIH_Final.xlsx 9. PACTE VIH Burkina pop size estimates- Data from JHU.pdf 10. PACTE-VIH FY14 QRT4 and Annual Summary Report_Final_SM.pdf 11. PACTE-VIH FY14Q2 Report_30_4_2014_Final_SM.pdf 12. PACTE-VIH FY15Q1 Report_10-2-2015_SM.pdf 13. PACTE-VIH QRT 2 Report_Final Revised_Feb 2014_SM.pdf 14. PACTE-VIH QRT3 Report_FinalRevised_Feb 2014_SM.pdf 15. PACTE_VIH Year 2 Workplan Oct 23 FINAL_SM.pdf 16. Prévalence VIH dans les régions du Togo (1).pdf 17. Q1FY14Report Annex.xlsx 18. Q2FY14 Report_Annexes_30-04-2014_Final_.xlsx 19. Q2FY14 Report_Annexes_30-04-2014_Final_.xlsx 20. Rapport FHI_Revu_Togo_Version 1.pdf 21. Revised PMP_November2014_Final.pdfvaluation_Policy_0.pdf 22. USAID_Evaluation_Policy_0.pdf 23. Workplan Final version Dec 24_SM.pdf 24. Year 3 Workplan (Nov 25)_SM.pdf 25. PACTE-VIH ANNUAL REPORT ANNEXES 26. FHI360. The Regional HIV/AIDS Prevention and Care Project in West Africa (PACTE-VIH mid Term Evaluation 2015. 27. Jean Paul Tchupu et al. Pacte-VIH utilization of the HIV treatment cascade model for program impact in Togo and Burkina Faso. 28. Checklist for Assessing USAID Evaluation Reports 29. Checklist for Reviewing Statement of Work for Performance Evaluation 30. Rapport Final de l'Analyze Situationnelle Rapide au Burkina Faso, Mai-Juillet 2013 31. FY 2015 APR Site Level Data Collection PACTE-VIH_10-2-2015.xlsx Togo Demographic & Health Survey, 2013-14. Final Report. Burkina Faso Demographic & Health Survey, 2010. Final Report 32. Other documents and sample tools as they become available 75 ANNEX 6: QUESTIONNAIRE FOR MSM INTERVIEW UNDER SEPARATE COVER ANNEX 7: QUESTIONNAIRE FOR FSW INTERVIEW UNDER SEPARATE COVER ANNEX 8: IN-DEPTH INTERVIEW GUIDE UNDER SEPARATE COVER ANNEX 9: FOCUS GROUP GUIDELINE UNDER SEPARATE COVER