EVALUATION Mid-Term Performance Evaluation of the USAID/Nepal Saath Saath Project (SSP) September 2014 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mendez, England & Associates. MID-TERM PERFORMANCE EVALUATION OF THE USAID/NEPAL SAATH SAATH PROJECT (SSP) Final Report September 2014 Prepared under RFTOP No. SOL-367-14-000011 Submitted to: USAID/Nepal Submitted by: Rose Schneider, Team Leader Dr. Alice Morton Dr. Dil Prasad Shrestha Ms. Mahima Malla Contractor: Mendez England & Associates 4300 Montgomery Avenue, Suite 103 Bethesda, MD 20814 Tel: 301- 652 -4334 www.mendezengland.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government CONTENTS EXECUTIVE SUMMARY................................................................................................................ I Evaluation Purpose............................................................................................................. i Project Background............................................................................................................ i Evaluation Design, Methods and Limitations ................................................................. ii Findings, Conclusions and Recommendations...............................................................iii Question 1: Relevance of SSP Model ......................................................................................... iii Question 2: Effectiveness of Service Delivery .......................................................................... v Question 3: Capacity Development........................................................................................... vi Question 4: Best Practices...........................................................................................................ix 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS ....................................... 1 1.1 Evaluation Purpose ............................................................................................................................... 1 1.2 Evaluation Questions........................................................................................................................... 1 2.0 PROJECT BACKGROUND ............................................................................................. 2 2.1 Implementation Approach.................................................................................................................. 3 3.0 EVALUATION METHODS & LIMITATIONS .............................................................. 4 3.1 Evaluation Methodology...................................................................................................................... 4 3.2 Limitations.............................................................................................................................................. 6 4.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS ..................................... 6 4.1 Question 1: Relevance of SSP Model ..................................................................... 6 4.1.1 Question 1a: Extent to Which SSP Model has Optimized Service Delivery to Key Populations.................................................................................................................. 6 4.1.1.1 Findings................................................................................................................ 6 4.1.1.2 Conclusions ...................................................................................................... 16 4.1.1.3 Recommendations.......................................................................................... 16 4.1.2 Question 1b: Relevance of SSP Model in Light of New Paradigm of Test, Treat and Retain (TTR).................................................................................................. 16 4.1.2.1 Findings.............................................................................................................. 17 4.1.2.2 Conclusions...................................................................................................... 18 4.1.2.3 Recommendations.......................................................................................... 19 4.2 Question 2: Effectiveness of Service Delivery.....................................................19 4.2.1 Findings ............................................................................................................................... 19 4.2.2 Conclusions........................................................................................................................ 21 4.2.3 Recommendations............................................................................................................ 22 4.3 Question 3: Effectiveness of Capacity Development Model .............................22 4.3.1 Question 3a: Activities Targeting SSP Subgrantees.................................................. 22 4.3.1.1 Findings.............................................................................................................. 22 4.3.1.2 Conclusions ...................................................................................................... 25 4.3.1.3 Recommendations.......................................................................................... 26 4.3.2 Question 3b: Activities Targeting SSP Subgrantees.................................................. 26 4.3.2.1 Findings.............................................................................................................. 26 4.3.2.2 Conclusions ...................................................................................................... 27 4.3.2.3 Recommendations.......................................................................................... 27 4.3.3 Question 3c: Activities Targeting the GON .............................................................. 27 4.3.3.1 Findings.............................................................................................................. 27 4.3.3.2 Conclusions ...................................................................................................... 30 4.3.3.3 Recommendations.......................................................................................... 30 4.4 Question 4: Best Practices ....................................................................................30 4.4.1 Findings ............................................................................................................................... 31 4.4.2 Conclusions ........................................................................................................................ 32 4.4.3 Recommendations............................................................................................................ 32 Annexes Annex 1: Evaluation Scope of Work Annex 2: Program Budget vs. Expenditure of Implementing Agencies Annex 3: Achievement Against SSP M&E Plan Indicator Targets Annex 4: List of Districts for Field Visit and Mini-Survey Questionnaire Annex 5: Summary of IAs – TOCAT Principal Capacity Domains Phase II Annex 6: Quantitative Analysis of FSW FGD Responses to Questions #9 and #10 Annex 7: Evaluation Responses from DPHOs and IAs Annex 8: TOCAT Rollout Process and Steps Annex 9: Saath Saath Project Data Flowchart Annex 10: Saath Saath Project Data Management Information System Framework Annex 11: Map of SSP Coverage Annex 12: District-Wide Targets vs. Achievement on Key Indicators Annex 13: Implementing Agency Survey Data Annex 14: District Officials Survey Data Tables Table 1: Number of FSW Reached Through Community Outreach That Promotes HIV Prevention Table 2: Number of FSW Who Received HIV/STI Test Results After Post-Test Counseling Table 3: Number of FSW Diagnosed and Treated for STI Table 4: Number of Services to FSWs in Saath Saath Project, October 2011 – June 2014 Table 5: Current Expenditure Status of SSP Budget Table 6: TOCAT Areas, Domains, and Indicators Figures Figure 1: How would you rate/assess the quality of the services that you have received in the past two years? Figure 2: How would you rate the following aspects of the services at the clinic/DIC that you go to? Figure 3: TOCAT Rollout Process and Steps ACRONYMS ABC Abstinence, Be faithful and Correct and consistent condom use AHF AIDS Healthcare Foundation AIDS Acquired Immunodeficiency Syndrome AKP Asha Kiran Pratisthan AMDA/Nepal Association of Medical Doctors of Asia/Nepal ANC Ante-Natal Car ARSH Adolescent Sexual and Reproductive Health ART Anti-Retroviral Therapy ARV Anti-Retroviral ASHA Advancing Surveillance, Policies, Prevention, Care and Support to Fight HIV/AIDS Project AWP Annual Work Plan BCC Behavior Change Communication CAC Community Action Center CBO Community Based Organization CCC Community Care Centre CCM Country Coordinating Mechanism CD4 Cluster of Differentiation 4 CHBC Community-Based Health Care CHW Community Health Worker CIP Community Information Point CM Community Mobilizer CPR Contraceptive Prevalence Rate CSO Civil Society Organization CTTR Community Test, Treat and Retain DAC Department of AIDS Control DACC District AIDS Coordinating Committee DDC District Development Committee DIC Drop-in Center DOHS Department of Health Services DPHO District Public Health Office DQA Data Quality Assessment EDP External Development Partner EIHS Expanded Integrated Health Services EPC Essential Package of Care EPC Expanded Program of Care FCHV Female Community Health Volunteer FCHW Female Community Health Worker FGD Focus Group Discussion FHD Family Health Division FHI 360 Family Health International FP Family Planning FSW Female Sex Workers GESI Gender Equity and Social Inclusion GIS Geographic Information System GON Government of Nepal GWP General Welfare Pratisthan HIV Human Immunodeficiency Virus HMIS Health Management Information System HP Health Post HSCB HIV/AIDS and STI Control Board HSS Health System Strengthening I/NGO International Non-Governmental Organization IA Implementing Agency IBBS Integrated Bio-Behavioral Survey IDU Injecting Drug User IEC Information, Education and Communication IG Income Generating JMMS Jagriti Mahila Maha Sangh KII Key Informant Interview M&E Monitoring and Evaluation MARP Most At-Risk Population ME&A Mendez England and Associates MIS Management Information System MOHP Ministry of Health and Population MSM Men who have Sex with Men NAP National AIDS Action Plan NCASC National Centre for AIDS and STD Control NDHS Nepal Demographic and Health Survey NFHP-II Nepal Family Health Program NGO Non-Governmental Organization NHRC National Health Research Council OE Outreach Educator PE Peer Educator PHC Public Health Center PLHIV People Living with HIV PLWD Persons Living with Disability PMTCT Prevention of Mother-to-Child Transmission of HIV/AIDS PO Program Officer RFTOP Request for Task Order Proposal RH Reproductive Health S&D Stigma and Discrimination SDP Service Delivery Point SI Strategic Information Unit SOP Standard Operating Procedures SOW Scope of Work SSP Saath Saath Project STI Sexually Transmitted Infection TA Technical Assistance TB Tuberculosis TFR Total Fertility Rate TOCAT Technical and Organization Capacity Assessment Tool TTR Test, Treat and Retain UNAIDS Joint United Nations Programme on HIV/AIDS USAID United States Agency for International Development USG United States Government VCT Voluntary Counseling and Testing WHO World Health Organization EXECUTIVE SUMMARY EVALUATION PURPOSE This is a report on the Mid-Term Performance Evaluation of the Saath Saath Project (SSP) funded by the United States Agency for International Development (USAID) Mission in Nepal. SSP is being implemented by Family Health International (FHI 360) between September 2011 and September 2016. The evaluation was conducted during the period July – September 2014, by a team assembled by Mendez England and Associates (ME&A), with headquarters in Bethesda, Maryland. The purpose of the evaluation was to: 1) to assess the performance of SSP from its inception; and 2) provide recommendations for the direction of the remaining period of the project. Further, the evaluation was to analyze the relevance of the SSP model, the effectiveness of the SSP implementers’ service delivery, and FHI 360’s ability to institutionalize evidence-based practices in providing HIV/AIDS and family planning (FP) services in its network of sub-grantees. The evaluation covered the period of September 2011 – March 2014. The information uncovered by this evaluation will be used by USAID to inform any necessary changes to improve SSP’s implementation and capacity building efforts, both within this project and across the Mission’s portfolio. PROJECT BACKGROUND As the flagship USAID Nepal HIV/AIDS project, SSP contributes to the achievement of the USAID/Nepal Country Assistance Objective, which aims to improve the survival and quality of life of Nepalese through equity and well-governed health systems. The total program cost will be $30.3 million, including $27.5 million in USAID funding, and a non-federal cost share of $2.8 million. Working to achieve sustained reductions in HIV transmission and improve reproductive health among selected key populations, SSP was designed to improve access to, demand for, and quality of tightly targeted, cost-effective, and integrated HIV/Reproductive Health (RH) services that are anticipated to be fully transitioned to country platforms by 2016. The project’s main goal is to reduce the transmission and impact of HIV/AIDS and improve reproductive health among most-at-risk populations (MARPS). MARPs include female sex workers (FSWs), clients of FSWs, injecting drug users (IDUs), migrant workers and their wives, and men who have sex with men (MSM). To reach this goal, SSP was designed to achieve five outcomes: • Outcome 1. Decreased HIV prevalence among selected MARPs • Outcome 2. Increased use of FP services among MARPs • Outcome 3. Increased Government of Nepal (GON) capacity to plan, commission, and use strategic information • Outcome 4. Increased quality and use of HIV services • Outcome 5. Strengthened coordination among all HIV/AIDS partners The SSP model adopts four relevant principles and approaches to implement its program, including: 1) strengthening country and local ownership of the GON’s Ministry of Health and i Population (MOHP), National Center for AIDS and Sexually Transmitted Diseases (STD) Control (NCASC), and district and local health, development, and HIV/AIDS coordinating committees; 2) increasing effectiveness through surveillance, research, technical, monitoring and evaluation (M&E), and other guidance; 3) facilitating institutional coordination on joint work plans with partner organizations, and participation in HIV working groups responsible for the National AIDS Plan (NAP), Annual Work Plan, etc.; and 4) building local capacity of 43 implementing agencies (IAs) for strengthened technical, organization, and management skills. EVALUATION DESIGN, METHODS AND LIMITATIONS The evaluation was conducted using a participatory approach and consulting with USAID, SSP, SSP’s sub-agreement partners, 43 IAs, MOHP, and NCASC in several in-depth meetings. The Evaluation Team carried out site visits in seven districts, as well as extensive discussions and key informant interviews (KIIs) with selected IA boards and staff, SSP regional representatives, district public health offices (DPHOs), district development committees (DDCs), and district AIDS coordinating committees (DACC) coordinators. The Team also used a participatory approach to elicit information and opinions of FSWs who are the beneficiaries of the program. The evaluation approach of soliciting information, opinions, and reactions from many stakeholders during interviews as part of the data collection was complemented by a series of listening sessions to solicit feedback and further input on evaluation findings. These methods helped the Team address the complex issue of possible changes to the SSP model in light of the anticipated GON paradigm of “Test, Treat and Retain (TTR)”. In addition to KIIs, the Team also conducted mini-surveys, focus groups discussions (FGDs) with FSW beneficiaries, interactive presentations to USAID and SSP, and listening sessions. Throughout the evaluation, the Team adjusted its data collection methodologies to address challenges and limitations. The Team cross-validated and triangulated data and asked similar questions to different stakeholders involved in the same issue, including Pooled Fund and Global Fund donors and Fund implementing partners. Survey/Sampling Methodology To select 15 representative samples of 33 total districts and FSW-focused programs, the Team first used the comprehensive mappings and listings of FSW services and activities supported by SSP. Since FSWs were the only beneficiaries who were defined by USAID as the focus of the evaluation, the Team selected districts and IAs by the types of services and activities provided to them. Districts were selected by using purposive sampling to ensure key program characteristics were covered by the evaluation. A mix of convenience and criterion sampling was applied: i) Convenience sampling: Seven districts were included based on their accessibility to the Evaluation Team. FSWs were then selected in the locations where the Team conducted site visits. In these districts, Evaluation Team Members conducted in-person KIIs with IAs, DPHOs, DACCs, and DDCs. ii) Criterion sampling: Fifteen additional districts were selected for mini-survey. In these districts, selected stakeholders were surveyed remotely (via email, or hard copy distribution). These districts were selected based on the number of relevant SSP￾ii supported services for FSWs that were similar to those in the seven districts that the Team selected to visit. FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Question 1: Relevance of SSP Model Question 1a: Extent to which SSP model has optimized service delivery to key populations Findings SSP provides services to FSWs, a high at-risk population in Nepal’s concentrated HIV/AIDS epidemic, in order to reduce the group’s transmission of HIV and the overall impact of AIDS. FSW services are implemented through defined IA service delivery points (SDPs) located at drop-in centers (DICs) and include expanded and integrated high impact packages of HIV and health services care. SSP’s model of service delivery reaches and supports tens of thousands of FSWs through community mobilizers (CMs), outreach educators (OEs), and peer educators (PEs) who provide information, supplies, promotions, and referrals for testing. SSP’s model also includes pre and post-test counseling, edutainment (entertainment that is designed to be educational) activities, promotion of positive behavior change, treatment of sexually transmitted infections (STIs) and HIV at IA service delivery sites, expanded care, and community-based home care for people living with HIV (PLHIV). Counseling plays a very important role in the SSP model. Counselors use SSP and government guidelines and in-depth checklists when providing services, which include testing client knowledge about HIV and STIs by discussing personal and medical histories. When a client is diagnosed as HIV-positive, the counselor provides additional psychological support to meet her needs, and instruct her about maintaining health through proper nutrition and taking medication on time. Clients are then referred for expanded package of care (EPC) services. All HIV-positive individuals are immediately enrolled into pre-anti-retroviral treatment (ART) services. During site visits, the Evaluation Team observed that clinical ART providers in MOHP facilities were aware of and using GON ART protocol. Previously, only PLHIV with CD4 counts of 350 or below were referred for ART treatment. Recently, however, the protocol has changed and PLHIV with CD4 counts of 500 or below are now referred. All ART referrals are made according to the World Health Organization (WHO) Clinical Assessment Guidance. PLHIV who are very ill, rejected by their families, and have more complex physical and social needs are identified to receive services from the Community Care Center (CCC). Currently, 54 clinics under SSP provide expanded integrated health services (EIHS) in 26 districts, 33 of which are satellite sites. EIHS includes STI case management, HIV testing and counseling, EPC for PLHIV, and laboratory services for HIV and STI tests. SSP began EIH services and EPC for PLHIV and, since then, EIHS sites reported conducting over 13,600 HIV tests and 13,868 STI tests, and providing pre-ART services for 1,894 PLHIV. iii SSP’s project design in its Cooperative Agreement included a stratified approach to address overlapping risk groups of FSWs, which include: 1) FSWs in the first six months of sex work; 2) FSWs returning from Mumbai and other parts of Maharashtra State; 3) FSW IDUs and; 4) females who have occasional transactional sex. To identify these groups, in 2012 SSP re￾invigorated the USAID/Advancing Surveillance, Policies, Prevention, Care and Support to Fight HIV/AIDS Project (ASHA) model of micro-planning. Although SSP extensively collects data on the overall FSW beneficiaries, data on the four high risk groups from 2011 and 2012 are not yet available and SSP did not routinely report information on the outreach and services provided to these overlapping risk groups. Previously, SSP’s reporting mainly focused on providing information on street-, home-, and establishment-based FSWs. The project started collecting and reporting data on the overlapping risk groups only in 2013 and that information is now included in SSP reports. Analysis of SSP expenditures vs. amount budgeted indicates that the burn rate used to achieve the five project outcomes tracked closely with the budgeted figures, as did the status of the expenditures of the 43 IAs. Activity achievements, such as training IA and MOHP staff in HIV outreach and referral services, surpassed targets. However, the data on specific services to FSWs, including STI treatment and testing, could not be found in routine SSP reporting. Conclusions The SSP model includes a major focus on quality outreach, training, and supporting OEs and CMs, among others. The clinical management of PLHIV has been institutionalized and is updated with HIV clinical guidelines, standard operating procedures (SOPs), and training manuals. Through its model, SSP reaches and supports tens of thousands of FSWs. For quality assurance, SSP has developed guidelines and SOPs covering each service delivery protocol, and SOP checklists for each clinical activity for service providers. The overlapping risk group within the larger FSWs population is more vulnerable for STI and HIV infection. Although outreach capacity building has been implemented, SSP continues to provide most outreach, referral, testing, and care to FSWs as a group rather than clearly tracking, serving, and reporting on the overlapping risk groups according to their high risk profile. MOHP has yet to develop protocols addressing overlapping risk groups in general and SSP has yet to develop guidelines addressing all of the groups. SSP did state, however, that in 2013 they developed the guidelines for two groups: FSWs IDUs and FSWs with less than six months of sex work. Question 1b: Relevance of SSP model in light of the new paradigm of test, treat and retain (TTR) Findings SSP prepared a brief draft document, Approaches of Saath Saath Project for Community Testing and Counseling and Test, Treat and Retain, which presents guiding principles for the expansion of TTR and outlines the current role of SSP’s HIV program structures that could be used for community TTR, including: continued community outreach, community and home-based care with positive prevention, expanded and integrated health centers, and laboratory services. However, the document did not discuss in any depth a process for actually empowering the iv very communities of HIV high-risk vulnerable groups who are unidentified and untreated and who are intended to carry out the “in-reach” that is at the core of community TTR. Conclusion Literature review, field visits, discussions, and observations of SSP’s approaches indicate that SSP’s current role, model, and approach would need to be adapted to future needs to optimally support the GON in carrying out an effective community TTR initiative. Recommendations for Questions 1a and 1b To fully use the structure, expertise, and capacity of SSP to support the GON in carrying out a high quality, expanded TTR paradigm, SSP should adapt and expand its approach to more truly empower the HIV high risk groups, especially overlapping risk groups, to assume leadership for the “in-reach" into their communities to identify others, motivate their changes in behavior, and encourage them to actively seek testing and treatment. SSP should expand and strengthen its linkages with the MOHP at the central and district levels, as well as linkages with other GON agencies (Ministry of Federal Affairs and Local Development and Ministry of Women, Children and Social Welfare), and multilateral and bilateral organizations for increased options for HIV affected and infected persons for livelihood and income generating opportunities. Question 2: Effectiveness of Service Delivery Findings Participants from 13 FGDs in seven districts overwhelmingly expressed high levels of satisfaction with the services they have received related to information, counseling, testing, treatment for STIs, pre-ART services, and referrals. Specific positive details of their experiences include: 1) outreach staff is easily accessible for discussions whenever they need information or support; 2) CMs regularly visit or call to remind them of the date of service or testing; 3) service providers treat them well, are always helpful, and provide all needed information; 4) they receive quality care and medication from clinics that have “cured” their STIs; 5) they are reminded about their medication and checkups; and 7) some FSWs have been linked with income generating programs and provided with seed money to raise hens, start small businesses, and carry out other income generating activities. Further, FSWs reported that they have been receiving improved services from DICs and clinics in the past two years. FSWs that participated in FGDs discussed several additional services that they would like to see at DICs and clinics, such as group blood testing, general check-up services, permanent FP methods including sterilization, increased number of medical checkups, more referrals especially for male friends, child care facilities, and linkages to income generating activities. Conclusion Qualitative and quantitative data shows that beneficiaries are largely very satisfied with the preventative and treatment services related to HIV and STIs that they receive from SSP￾supported DICs and clinics. FSWs interviewed by the Evaluation Team reported that DICs and clinic services have improved since the start of SSP in 2011; however, they still request additional services. v Recommendation IAs should continue to provide quality services, through DICs and clinics, for FSWs and other clients. IAs should encourage FSWs to increasingly promote these services to others who have not yet accessed DIC and clinic information and services. Question 3: Capacity Development Question 3a: Effectiveness of Capacity Development Model – Activities Targeting SSP Subgrantees Findings The Technical and Organizational Assessment Tool (TOCAT) was developed by FHI 360 and used by SSP to build the capacity of IAs. The 242 indicators, five areas, and 29 domains in TOCAT’s checklist present a model system for institutional performance. The indicators represent the “standard of performance” or model of an idealized organizational status that the IAs could achieve. The roll-out process was developed in two phases – 24 IAs in phase 1 and 17 IAs in phase 2 – with four key steps: 1) Preparation; 2) Conduct Assessment; 3) Implementation; and 4) Follow-ups. The TOCAT instrument was used as the baseline for IAs’ capacity development. The Team’s review found that the TOCAT baseline scores that could reflect the starting point for the two roll-out groups of IAs were not developed as a group but rather were developed by each of the 43 IAs individually. The lack of a compiled baseline report detailing the two roll-out IA groups made it difficult to develop a complete picture of the baseline status of each cohort for capacity development plans. The TOCAT team and other SSP staff that supported the Action Plans developed activities generally related to the indicators. The Team reviewed various Action Plans with specific activities, persons responsible in the IAs, and SSP for these activities and timelines. Reporting on these plans included progress but did not state the type or amount of SSP support. Conclusion The TOCAT instrument is a large, detailed, 242-indicator assessment tool that may benefit from some simplification. SSP’s TOCAT analysis is conducted by individual IAs; however, group analysis and scoring would be helpful to provide technical assistance (TA) and assess progress by the two roll-out groups. The Action Plan, fairly aligned with a sub-set of indicators, is a tool used for IA actions, SSP TA, and monitoring progress. Many IAs had completed their first Annual Action Plan and reported increased interest of boards and actions, such as development of Strategic Action Plans, documentation of procedures, improvement of financial recording, etc. Recommendation TOCAT should be streamlined, simplified, and focused somewhat more on the capacity development needs and tasks directly related to the IAs’ work in HIV. Defining, implementing, and tracking the capacity development provided by various units and sources within SSP vi support, as well as the overall progress of capacity development of IAs, should be coordinated and tracked by a single entity within SSP. TOCAT capacity development should fit within SSP’s larger and more coordinated capacity development strategy and approach. Question 3.b. Capacity development of sub grantees Findings SSP’s achievements have led to significant gains in the capacity development of IA staff, resulting in the majority of targeted interventions being met. Due to the efforts of its technical capacity building for IAs, SSP has reached more than 22,142 FSWs (from August 2011 – January 2014). Training received by counselors, lab personnel, and other clinical related service providers in STI/HIV/AIDs resulted in a high level of satisfaction and use of the services by FSWs. The training and supportive supervision of the three types of outreach workers inform FSWs to seek services and to reach out to other FSWs. Micro planning is being taught to OEs and CMs. This process trains staff to differentiate the categories of overlapping risk and provide a specialized approach to reaching those groups of FSWs that are hard to find, and approach and encourage them to be tested and treated. FSWs’ training and support to acquire the skills and resources to maintain income generating activities have not yet been successful. The linkages with experienced income generating non￾governmental organizations (NGOs) with special skills that can help interested and committed FSWs with income generating work have largely not yet been connected to the FSWs being served by IAs. Conclusion The multiple and high quality trainings provided by SSP to their supported IAs have resulted in the increase in staff skills and improvements in the quality of services. At the same time, the Evaluation Team did not find a comprehensive vision or approach to capacity development across SSP. Recommendation SSP should continue to provide quality capacity development. It should clarify its vision and approach to capacity development and strengthen the central point of coordination that guides, directs and coordinates its capacity development activities. SSP should also decentralize its capacity development with strong regional coordinators and program officers who are responsible to coordinate, train, and support capacity development of IA, Department of AIDS Control (DAC), and DPHO staff at the district level. Questions 3.c. Effectiveness of Capacity Development Model – Targeting the GON Findings vii SSP invests significant resources in strengthening the technical capacity of the MOHP’s NCASC, and other MOHP units. Key activities include: Improve MOHP capacity to conduct second generation surveillance. SSP conducts secondary surveillance through its robust recording and reporting system, the constant updating of its management information system (MIS), and use of its geographic information system (GIS) capability. With NCASC, SSP developed the interactive GIS map to identify HIV high risk areas and hot spots, and establish planning to mitigate them. GIS is now used to identify hotspots where both FSW and their potential clients congregate. The map has further helped SSP reach the maximum number of FSWs in clusters based on their location, increase OEs and CMs, and open additional clinics and satellite clinics. SSP has helped build the technical capacity of NSCAC to use these methods and technologies to improve, expand, and pursue second generation surveillance. NCASC now has the capacity, hardware, and software to replicate some of SSP’s second generation surveillance activities. Improve the capacity of MOHP and DACCs to analyze and use strategic information. At the district level, SSP has supported 50 DACCs through training, supervision, and software and other resources. SSP also supports the roll-out of trainings on how to analyze and use data for trainings, workshops, meeting, interactions, and provides support in preparing presentable data through the use of PowerPoint. SSP revised the curriculum and conducted trainings for 23 DACC coordinators and 23 HIV focal persons from 33 districts. These trainings have helped DACC coordinators support other DACC members to participate in these tasks. SSP has provided training and support to 26 districts in the use of strategic information. Support National Research Agenda. SSP has supported research and works with NCASC on the development of a national research agenda and strategy. SSP supported the GON in developing research on FSW networks, sexual behaviors, and new at-risk groups. Guideline Development. M&E is an important part of SSP’s technical support to GON staff at NCASC. SSP developed M&E guidelines to be used by GON staff, the distribution of which supports district level MOHP and IA M&E staff. Conclusions NCASC and SSP have a good working relationship. However, despite SSP’s TA, capacity building, and M&E support, results are uneven, due partially to frequent changes in NCASC’s leadership. GON and SSP collaboration at the district level for improved reporting and data analysis that supports quality program planning has room for improvement. GON and SSP collaboration at the district level would also include development of joint reporting systems. Recommendations SSP’s Strategic Information (SI) Unit should continue to collaborate with counterparts at NCASC to implement their joint work plan. SSP should increase efforts to build capacity at the district level by working directly with the DPHO HIV focal person and other DACC members. viii Question 4: Best Practices Findings Interaction to attract and retain FSWs. FGDs and KIIs with all levels of IAs staff and their boards emphasized the critical importance of friendly, warm communications, as well as the respectful interactions and supportive, non-judgmental approach used by IA and SSP staff in all their interactions with FSWs. Guidelines and checklists for quality care. SSP developed and updated detailed guidelines and checklists to guide the quality of care, reflecting updated state-of-the-art clinical and other technical information to guide SSP’s services provision. Records show that SSP’s guidelines have been adopted by other IAs. Strong reporting, review, and supervision. Responses from the IAs and District Officials (Chief DPHO, HIV focal person, and DAC coordinators) in surveys and interviews with the Team stated that SSP’s reporting and reviews in Quarterly and Semi Annual Review Meetings, as well as the supportive supervision/monitoring, strengthened IA management and technical capacity. These practices supported regular monitoring and supervision of IA projects, improved tracking of financial resources, and provided good feedback mechanisms and communication on program changes. GIS Mapping. GIS capability allows DACCs to map FSWs in the areas identified by the outreach workers and CMs, thus potentially identifying hotspots and other areas of concentration of key populations in their geographic areas. GIS also supports DACC coordinators to compile HIV data from IAs, DPHOs, and others to construct the District AIDS Profile that supports planning and monitoring of HIV activities. Data Quality Assessments (DQAs). SSP has extended training and capacity building in DQA at the district level. The SSP SI unit has trained DACC coordinators, DPHOs, HIV focal persons, FP officers, and others at the district level, as well as IA staff such as the IA MIS M&E officers, project managers, and others. Conclusions SSP’s development of a number of effective and exemplary practices, tools, checklists, processes, and approaches have supported the improvement and consistency of the quality of outreach information and clinical HIV services provided in facilities. These tools have not only been used by SSP but have also been adopted by other NGOs. Recommendations SSP should continue its broad support to NCASC and other MOHP entities. However, as NCASC becomes more capable of implementing its agenda, SSP should focus more on strengthening the capacity of DACCs, DPHOs and other GON entities to collect, analyze, and use data for improved HIV/AIDS programming at the district level. ix 1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS 1.1 EVALUATION PURPOSE This is a report on the Mid-Term Performance Evaluation of the Saath Saath Project (SSP) funded by the United States Agency for International Development (USAID) Mission in Nepal. SSP is being implemented by Family Health International (FHI 360) between September 2011 and September 2016, for a total estimated cost of $27.5 million. The evaluation was conducted between July and September 2014 by a technical team assembled by Mendez England and Associates (ME&A), which consisted of: Ms. Rose Schneider (Team Leader), Dr. Alice Morton (International Specialist), Dr. Dil Prasad Shrestha (Local Specialist), and Ms. Mahima Malla (Local Specialist). The team was supported by two Interpreters, Ms. Trishna Shaw, and Mr. Sirjan Adhikari. The main goal of the evaluation was to assess the performance of SSP from its inception in September 2011 through March 2014, and provide recommendations for the direction of the remaining period of the project. Further, the evaluation was to analyze the relevance of the SSP model, the effectiveness of the SSP implementers’ service delivery, and the implementer’s ability to institutionalize evidence-based practices in providing HIV/AIDS and family planning (FP) services in its network of sub-grantees. The results of this evaluation may be used by USAID to inform any necessary changes to improve SSP implementation and capacity building efforts both within this project and across the Mission’s portfolio. Further, they will be used to: 1) provide information that will enable improvements in project performance among the implementer and sub-awardees for the remaining period of the project; 2) increase buy-in to the project by the Government of Nepal (GON); 3) empower project beneficiaries to provide greater input to project staff and management to continually improve SSP implementation; and 4) improve USAID’s and partners’ understanding of how to best utilize the Technical and Organizational Assessment Tool (TOCAT) as a capacity development tool. The USAID/Nepal Mission will be the primary audience for this evaluation. Other audiences include: FHI 360, all 43 sub-grantees, project beneficiaries, and the GON. 1.2 EVALUATION QUESTIONS The initial scope of work (SOW) evaluation questions were discussed by the Evaluation Team in in-briefing meetings with USAID/Nepal. USAID directed the Team to make several changes, which were then approved by the Monitoring and Evaluation (M&E) Officer and served as the guidance for the Team’s evaluation work plan and data collection. The following questions include the approved changes: 1. Relevance of SSP Model a. To what extent has the SSP model optimized service delivery to female sex workers (FSWs) as the key population for this evaluation in the context of a concentrated epidemic? 1 b. SSP’s model of building the technical, financial, and management capacity of more than 40 local non-governmental organizations (NGOs) has clearly resulted in a stronger civil society in HIV/AIDS. However, as Nepal moves to adopt a test and treat approach, is this model effective in light of this new paradigm? 2. Effectiveness of Service Delivery a. How do FSWs rate the quality of SSP services, including: 1) information; 2) counseling; 3) testing; 4) treatment [sexually transmitted infections (STIs) and pre anti-retroviral treatment (ART) services]; 5) referral from approximately 50 SSP￾supported clinics; and 6) FP? 3. Effectiveness of Capacity Development : Activities targeting SSP sub-grantees: a. To what extent has SSP strengthened the capacity of SSP sub-grantees? In particular: i. In what ways have the TOCAT instrument and TOCAT corrective action plans been effective tools for assessing and developing an organization’s financial, administrative, management, and service delivery capacity? ii. What has worked, what has not worked, and how can capacity building activities be improved? Activities targeting the GON: i. In what ways has SSP been able to build the technical capacity of the GON, especially in the areas of HIV surveillance and M&E? 4. Best Practices a. What are SSP’s (both from the prime and among sub-awardees) most effective and exemplary practices with respect to service delivery and capacity development of GON, subgrantees and Departments of AIDS Control (DACs) that, if shared, would improve the results of SSP sub-grantees and other organizations operating in the HIV sector? 2.0 PROJECT BACKGROUND Nepal has a strong foundation for integrating and increasing the effectiveness and sustainability of targeted HIV and FP/Reproductive Health (RH) services for most-at-risk populations (MARPs). The GON provides leadership, especially under the Ministry of Health and Population (MOHP) and the National Centre for AIDS and STD Control (NCASC). The MOHP provides strategic direction under the National Health Sector Strategy-II, the National Policy on HIV and STIs, and the National HIV and AIDS Strategy. The MOHP also provides a negotiated and country-led coordination framework under the National HIV and AIDS Action Plan (NAP) with harmonization of external development partner (EDP) resources for HIV. The country has multiple sources of strategic information on the HIV epidemic through the National HIV Surveillance System, program performance data, research, the Health Management Information System (HMIS), and Nepal Demographic and Health Survey (NDHS). There is also a strong evidence base of what works in Nepal, including targeted programming for MARPs. At the same time, Nepal faces significant challenges in moving towards full country ownership of the national HIV response under the shared leadership of the GON and civil society groups. 2 Over the past 15 years, Nepal has significantly reduced the total fertility rate (TFR) and increased the contraceptive prevalence rate (CPR). However, TFR is still high (2.9 children per woman) and the CPR has stalled over the past few years. In 2006 there were major disparities in access to FP services and methods of choice across the country, with significantly higher TFR in rural areas (3.3) and in the mountains of the Far West (4.1). The World Health Organization (WHO) estimated that the unmet FP need in 2006 was 25%, with higher rates of unmet need in marginalized populations [FSW, people living with HIV (PLHIV), and female injecting drug users (IDU)], migrants, and people in remote areas. Anecdotal reports indicate a high rate of unwanted pregnancy and abortion among FSWs. NDHS and Nepal Family Health Program (NFHP-II) data also indicate low uptake of contraception among wives of migrants who work in India and return for short periods during the year, as well as poor community acceptance of long-acting contraceptive methods for these women. As Nepal awaits a new constitution to define its socio-political and social order, the transitional peace process has been prolonged, creating an unstable environment and division in the country along ethnic and geographical lines. This has negatively impacted the overall functioning of the government and caused major hurdles in the system’s development arena across different sectors, including health. SSP, as the flagship USAID Nepal HIV/AIDS project, contributes to the achievement of the USAID/Nepal Country Assistance Objective, which aims to improve the survival and quality of life of Nepalese through equity and well-governed health systems. The total program cost will be $30.3 million, including $27.5 million in USAID funding and a non-federal cost share of $2.8 million. The period of performance is from October 2011 to September 2016. Working to achieve sustained reductions in HIV transmission and to improve RH among selected key populations, SSP was designed to improve access to, demand for, and quality of tightly targeted, cost-effective, and integrated HIV/RH services that are anticipated to be fully transitioned to country platforms by 2016. The goal of SSP is to reduce the transmission and impact of HIV/AIDS and improve RH among MARPS. MARPs (which are now referred to as “key populations” both throughout the global HIV community and in this document) include FSWs, clients of FSWs, IDUs, migrant workers and their wives, and men who have sex with men (MSM). To reach this goal, SSP was designed to achieve five outcomes: • Outcome 1. Decreased HIV prevalence among selected MARPs • Outcome 2. Increased use of family planning services among MARPs • Outcome 3. Increased GON capacity to plan, commission and use strategic information • Outcome 4. Increased quality and use of HIV services • Outcome 5. Strengthened coordination among all HIV/AIDS partners 2.1 IMPLEMENTATION APPROACH SSP was designed and built on the achievements of Advancing Surveillance, Policies, Prevention, Care and Support to Fight HIV/AIDS in Nepal (ASHA) Project that showed tremendous successes working through local organizations. Maintaining coverage while increasing effectiveness and sustainability of the overall approach was the key challenge 3 identified for ASHA; therefore, SSP continued to work with the experienced partner Implementing Agencies (IAs) under ASHA with increased focus on local capacity building. By strengthening the capacity of local civil society organizations (CSOs) and the GON, SSP works to improve the overall national HIV/AIDS response. The overall SSP guiding principles are listed below: 1. Strengthening country and local ownership: SSP supports the leadership and ownership of GON, civil society groups, and MARP networks in reducing transmission and impact of HIV/AIDS and improving reproductive health among MARPs in Nepal. 2. Increasing effectiveness: SSP works with national stakeholders to use unit cost data to improve cost-effectiveness and sustainability of integrated HIV/RH services for MARPs, surveillance and essential data collection activities, and community-based prevention of mother to child transmission of HIV/AIDS (PMTCT). 3. Institutionalizing coordination and collaboration: SSP coordinates all activities under the National AIDS Plan (NAP) and Annual Work Plan (AWP), and ensures close coordination with Global Fund and Pooled Fund IAs and with other United States Government (USG)-supported programs to maximize population and geographic coverage. 4. Building local capacity: SSP works with selected IAs to deliver structured financial, technical, and managerial capacity building for local NGOs. The program is providing targeted needs-based capacity building for the GON in the delivery of strategic information (SI) and second generation surveillance. 3.0 EVALUATION METHODS & LIMITATIONS 3.1 EVALUATION METHODOLOGY The evaluation methodology assessed the performance and progress made by SSP from September 2011 to June 2014. As per USAID’s request, the methodology did not address all the SSP objectives but rather responded to USAID’s defined sub-set of objectives covered in the evaluation questions. To conduct the evaluation, the Team used a participatory approach which entailed consulting with USAID in several in-depth meetings, and collaborating via in-depth meetings with SSP, SSP’s sub- agreement partners and 45 IAs, MOHP, and NCASC. The Team carried out site visits in 7 districts where it conducted interviews and in-depth discussions with key informants of selected IA boards and staff, SSP regional representatives, district public health offices (DPHOs), district development committees (DDCs), and district AIDS coordinating committee (DACC) coordinators. The Team also used a participatory approach to elicit information from and opinions of FSW beneficiaries. Solicitation of information, opinions, and reactions from stakeholders was complemented by a series of listening sessions, focus group discussions (FGDs), interactive presentations to USAID and SSP, and surveys. These techniques helped capture information and diverse opinions of USAID Mission staff and management, other donors, MOHP and other GON officials, SSP project and IA staff, and beneficiaries. 4 Throughout the evaluation, the Team adjusted its data collection methodologies to address challenges and limitations, or take advantage of presented opportunities. For example, the Team added an additional district and a different IA when a previously identified IA was found to have only one FSW service at the time of the field study. Survey Sampling Methodology To reach a representative sample, the Evaluation Team selected districts by using a combination of sampling techniques, including multi-stage, purposive, and random. Given the relatively small number of districts (25 for FSWs), it was concluded that random sampling would not reliably deliver a representative sample of the variety of services and clients of interest to the evaluation. Accordingly, districts were selected by using purposive sampling to ensure that key program characteristics were covered by the evaluation. The rationale changed due to the directive to focus data collection on one type of beneficiary, FSWs. Districts may be quite diverse in terms of: 1) their share of the 45 IAs; 2) utilization of service delivery sites by FSWs; 3) the presence of expanded integrated health services (EIHS) model sites; 4) the presence of positive prevention activities; and 6) the presence of SSP-supported sites. The Team purposively selected those districts that fit the evaluation purpose and where the Team could collect the maximum amount of data on FSWs. To select representative samples of districts and FSW-focused programs, the Team first used the comprehensive SSP mappings and listings of FSW services/activities supported by SSP to select 15 of the 33 districts. Since FSWs were the only beneficiaries to be interviewed, the Team selected districts and IAs by the types of services/activities provided to FSWs. Seven districts (Kathmandu, Bhaktapur, Chitwan, Bara, Parsa, Morang, and Jhapa) were chosen for direct site visits based on feasibility of travel, geographic spread, and presence of an adequate number of FSW services provided by the IAs. Next, in each of the 15 purposively selected districts, the Team randomly selected two SSP-supported sites, for a total of 30 sites. The Team conducted surveys by phone, email, or distribution of hard copy in 15 districts. In each selected district there were two sample populations: 1) beneficiaries; and 2) selected stakeholders, such as service providers (NGOs), DPHOs, and IA staff. Beneficiaries. Data from FSW beneficiaries were collected through FGDs. At each of the two randomly selected sites in the 7 districts visited by the Evaluation Team, FGDs were conducted with 7-15 beneficiaries each, for a total of 14 FGDs. FGD participants were selected by using convenience sampling. In total, information and data was collected from 153 FSW beneficiaries. Stakeholders. The Evaluation Team conducted key informant interviews (KIIs) of DPHOs, relevant HIV/AIDS focal persons, and DACC coordinators, IA project directors and coordinators, and other staff and selected board members in SSP-supported sites (2 per district) in the 7 districts where the Team conducted fieldwork. A summary of the information from KIIs is included in Annex 7. In addition to KIIs, the Team also conducted a number of mini-surveys of similar groups of stakeholders in 15 districts to collect quantitative data. The summary data is included in Annex 13: Implementing Agency Survey Data. 5 3.2 LIMITATIONS The Evaluation Team experienced the following limitations inherent to the evaluation: 1. Some key informants were reluctant to be interviewed in-person or to complete the mini-survey. However, the Evaluation Team does not consider this an issue for the evaluation. 2. Since the SSP was considered a follow-on to the ASHA project, some staff, stakeholders, and beneficiaries found it somewhat difficult to provide information and perceptions on SSP as separate and distinct from ASHA. To address this limitation, the Team clarified the start date of SSP and elicited information specific to SSP’s activities, allowing them to be confident in the accuracy of information attributed to SSP. 3. Some data that were to be reported in semi-annual reports to assess the services to FSWs were not reported. For example: number of FSWs receiving STI treatment by age and number of FSWs who have accessed HIV testing services and received their results in the last 12 months were not found in SSP’s routine reports. This made it difficult for the Evaluation Team to verify these services delivered by SSP. 4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS 4.1 QUESTION 1: RELEVANCE OF SSP MODEL To what extent has the SSP model optimized service delivery to key populations in the context of a concentrated epidemic? 4.1.1 Question 1a: Extent to Which SSP Model has Optimized Service Delivery to Key Populations 4.1.1.1 Findings SSP focuses support on FSWs, one of its high risk populations, via a number of information, outreach, testing, and other services in order to reduce the group’s transmission of HIV and the overall impact of AIDS. SSP implements prevention outreach to FSWs through defined IA service delivery points (SDPs) located at drop-in centers (DICs) by providing expanded and integrated services as a high impact package of HIV and health services, including: outreach, voluntary counseling and testing (VCT), essential packages of care (EPCs), counseling, information, screening for STIs, and HIV testing. To scale-up effective prevention, SSP outreach educators (OEs) and community mobilizers (CMs) work in high-risk communities to inform FSWs about risky behaviors and provide information on and referrals to SSP-supported IA services, especially those promoting VCT. In addition, community information points (CIPs) are one of the major methods of reaching hard-to-reach populations in the SSP model. A total of 364 volunteer peer educators (PEs) including CIP operators were mobilized to support CMs and OEs for outreach activities, especially to identify new hotspots and serve new FSWs. In additionl to CIPs, a national network of FSWs – Jagriti Mahila Maha Sangh (JMMS) – also helps CMs and OEs identify and reach new FSWs. SSP’s model is designed to reach coverage levels of 90% of FSWs in the 25 districts it supports through active outreach and promotion of sustained positive behavior change (condom negotiation, correct and consistent condom use, reduction of partners, and regular STI and HIV 6 testing and STI treatment). As per SSP’s Third Semi Annual Report, August 2013-January 2014, the project has reached more than 90% coverage levels. SSP’s model and strategy use a number of community based IAs, NGOs, and CBOs to provide FSWs with interventions such as: 1) the scale-up of outreach, testing, counseling, referral, and treatment of STIs and pre-ART treatment; 2) micro-planning approaches to define specific outreach and community mobilization activities for specific groups of FSWs; 3) mapping, surveillance and data analysis for the identification and response to changing hot spots of FSW activities; 4) training in condom negotiation, and correct and consistent condom use; and 5) DIC information activities and VCT, EPC and other facility based-services. In addition, the SSP model supports services for FSW and other PLHIV with referral to ART, community care centers (CCCs) and other services provided by public sector and other providers. The model expands IA services to PLHIV such as community based health care to support PLHIV with monthly home visits as well as interventions to inform communities on HIV and reduced discrimination. SSP’s model and its support to optimize service delivery to FSWs Outreach – community mobilization and outreach education to FSWs. SSP's community outreach activity is one of the major strengths of its model of support to FSWs. SSP has trained and supported IAs in the mobilization of outreach staff such as OEs, PEs, and CMs. Under this model adopted by IAs, PEs, and CMs come from the same key HIV-vulnerable population groups as those with whom they work, enabling them to gain access to key populations including FSWs, clients of FSWs, and migrants and their spouses. All three groups have a familiarity of and are in regular contact with FSWs in their communities, helping them identify and bring FSWs into DIC for testing services. SSP's model includes other IA outreach activities that also result in referrals and follow-ups for STI diagnosis and treatment, VCT, and FP counseling and services. OEs and CMs continually assess, identify, and reach new hotspots of FSWs as they emerge. SSP's Program Officer conducts monthly supervision visits and provides onsite coaching and mentoring support to outreach staff, FP educators, supervisors, DIC operators, and project coordinators on the outreach process. SSP supports a total of 220 CMs and OEs, who were trained and mobilized in 2013, for HIV prevention activities among FSWs and their clients, migrant workers, and the spouses of migrant workers. OEs and CMs were trained and are regularly oriented on HIV prevention approaches and basic principles of HIV testing, including confidentiality, stigma, and discrimination. IAs organize quarterly orientation and review meetings and refresher trainings with PEs and CIP operators to strengthen their knowledge of and skills on HIV and FP information, including dual method promotion. During its field visit to Kathmandu, Bhaktapur, Chitwan, Bara, Parsa, Morang, and Jhapa, the Evaluation Team observed that OEs and CMs play a major role in reaching FSWs and other key populations to provide information, referrals, and support for getting tested, and often even accompany FSWs for clinical services. Since the start of the project, SSP’s model has included OEs and CMs using micro plans and daily dairies, and has supported field visits to reach the one overlapping high-risk FSW groups. A single CM reaches and refers an average of 75 FSWs per month including the repeat visits. With SSP's technical assistance (TA) and financial support, 191 community health workers 7 (CHWs) and 315 female community health volunteers (FCHVs) in Bara were trained in integrating FP/HIV message, HIV prevention, and referrals for STI testing. In reviewing Integrated Bio-Behavioral Survey (IBBS) survey data, the Evaluation Team found that more than 53% of FSWs has met, discussed, or interacted with PEs or OEs in the last 12 months.1 During the period from October 2011 to June 2014, SSP-supported IAs tested a total of 17,970 FSWs for HIV, of whom 133 were diagnosed HIV-positive and sent for treatment. Table 1, below, shows the number of FSWs reached through community outreach that promotes HIV prevention in the 7 districts that the Evaluation Team visited. Table 1: Number of FSW reached through community outreach that promotes HIV prevention FSW Bara Bhaktapur Chitwan Jhapa Lalitpur Morang Parsa T A T A T A T A T A T A T A 861 846 4,279 4,006 1,279 1,394 1,207 1,317 3,859 3,547 698 754 861 1,057 At 4,006, Bhaktapur had the highest number of FSWs reached through community outreach, followed by 3,547 in Lalitpur, and 1,394 in Chitwan. DIC information, edutainment, enabling environment. Fifty-one DIC sites are operational across SSP’s 33 districts, out of which 49 were co-located with EIHS. Branded with the BISHWAS sign, these DIC sites help provide a safe and comfortable place for FSWs and other beneficiaries to access FP, HIV, and STI-related services. In the sites visited by the Evaluation Team, SSP had over 80 types of information, education, and communication (IEC) materials, although some were developed during the previous ASHA project. DICs follow guidelines on the use of IEC materials which benefit key populations and support their ability to provide different edutainment (entertainment that is designed to be educational) activities. DICs organize creative activities, such as arts and crafts, quiz contests, beautification events (henna, nail art, green bangle distribution), cloth doll making, and mini dramas on the importance of using the dual method. Such activities help motivate FSWs, their clients, and others to make frequent DIC visits, and promote positive health behavior change, regular STI checkup, and HIV testing. In FGDs with FSWs, the Evaluation Team found that of FSWs who visit DICs for STI/HIV testing and FP services, many had already been seeking STI and other service at BISHWAS sites for 8-9 years. Most people who visit regularly have STIs that re-occur about two to three times a year, while some are treated two to four times per year for STIs. Further, FSWs reported visiting DICs for blood tests on a monthly basis and for HIV tests on a quarterly basis. Apart from STI and HIV tests, FSWs visit DICs for information on HIV, to receive STI testing and treatment, and to take part in the edutainment games and makeup activities. FSWs told the Evaluation Team that they pass on information received during their visits to neighbors, 1 IBBS: 2012 8 relatives, and friends. According to IBBS data, 53.3% of FSWs in 6 districts, and 43.8% in 16 districts had visited DICs only once in 12 months.2 Counseling – pre-test and post- test lab facilities. Counseling plays a very important role in the SSP model. After OEs and CMs refer clients, they are registered and given a Saath-Saath ID which preserves their confidentiality and privacy and then sent to a counseling room for testing service. Trained clinical staff, including nurses and health assistants, provides FP counseling and referrals in addition to pre-test counseling for HIV. After HIV pre-counseling, clients are referred for HIV and syphilis blood tests, and then on to a DIC for edutainment activities while they await test results. Sometimes counselors provide pre-test counseling individually and at times this is preceded by group information sharing due to the number of clients. Post-test counseling is always provided individually as are referrals to EPC (pre-ART) services. When diagnosed HIV-positive, a FSW is referred for pre-ART service where she receives her STI report and medicine. SSP's clinical service delivery,established through EIHS, was found to be of good quality. The laboratory room is set up and medical procedures, which are conducted by a trained lab assistant and nurse, are provided according to WHO standards. The clinical helper, post training on medical waste management, uses technical guidelines for waste management to avoid the infection. One lab asssitant usually tests between 9 and 11 clients per day. Counselors routinely use SSP and government standard operating procedure (SOP) guidelines for providing services. SSP developed guidelines and checklists for each service category, including: information, counseling, testing, treatment (STI and pre-ART services), referral, and FP. The in-depth checklists and guidelines, which include testing client knowledge about HIV and STIs by discussing their personal and medical histories, maintain quality service assurance and improvement. When a client is HIV-positive, the counselor provides additional psychological support to meet her needs, and instructs her about maintaining health through proper nutrition and taking medicine on time. The client is then referred to EPC services. During field visits, the Evaluation Team observed the recording and reporting system used by IAs, and IAs staff commented on the numerous types and quantities of checklists and forms they are required to use. For example, OE/CM guidelines are extensive and include a number of required checklists. Staff mentioned that such reporting requirements are excessive and repetitive and may require 2.5-3.5 hours per day of OCs and CMs’ time, which, in their opinions, could be better used for service delivery. IAs staff also informed the Evaluation Team that reporting took up to 25% of the time for senior IAs staff, and that sometimes they received no feedback from SSP (see Annex 9). SSP provides feedbacks through the quarterly reporting meetings, and through monthly PO visits. Some feedbacks are provided immediately if the performance of the IAs is questionable. For electronic reporting, SSP provides feedbacks for all problems identified in the electronic reporting and the reports are only accepted once they are error-free. 2 IBBS 2012 9 Table 2, below, shows the number of FSWs who received HIV/STI test results after post-test counseling in the 7 districts visited by the Evaluation Team. At 2,817, the highest number of FSWs was counseled in Bhaktapur, followed by 779 in Lalitpur, and 602 in Chitwan. Table 2: Number of FSW who received HIV test results after post-test counseling Districts Bara Bhaktapur Chitwan Jhapa Lalitpur Morang Parsa T A T A T A T A T A T A T A FSW 282 283 2,251 2,817 598 602 561 595 648 799 419 381 452 468 STI and HIV testing. SSP-supported sites provide services in STI case management, HIV testing and counseling, and EPC. The EIHS serve as sites for enrolling HIV-positive clients into care before they become eligible for ART. This service has a strong connection to community outreach and community-based healthcare (CHBC), which helps with retention. According to IBBS data, 45% of FSWs visited any STI clinic in the last 12 months and, of the roughly 49% of FSWs who were examined for STIs, 38% were diagnosed with one. Table 3, below, shows the number of FSWs who were diagnosed and treated for STIs in the 7 districts that the Evaluation Team visited. The urban Bhaktapur district, which adjoins Kathmandu and has high levels of street-based and establishment-based FSW activities, and treated the highest number of STI cases among FSWs. Table 3: Number of FSWs diagnosed and treated for STI Districts Bara Bhaktapur Chitwan Jhapa Lalitpur Morang Parsa T A T A T A T A T A T A T A FSW 283 282 2,107 2,791 619 602 567 596 758 847 431 384 447 447 Referral to ART centers. All HIV-positive individuals are immediately enrolled into pre-ART services. During field visits, the Evaluation Team observed that clinical ART providers in MOHP facilities were aware of and using the GON ART protocol. Previously, only PLHIV with CD4 counts of 350 or below were referred for ART treatment. Recently, however, the protocol has changed and PLHIV with CD4 counts of 500 or below will be reported once the protocol is endorsed. Referrals are made according to the WHO Clinical Assessment Guidance. Clients with a tuberculosis (TB)-HIV and hepatitis co-infection are put on medication for two to eight weeks before ART is initiated. HIV-positive pregnant women begin treatment at 14 weeks of pregnancy. In Chitwan, the Evaluation Team reviewed records and found that of 1,120 cases of PLHIV, 515 are on ART services linked with a MOHP hospital for ART services. Monthly clinical follow-ups are provided by SSP clinical staff using their referral guidelines. PLHIV who are very ill, rejected by their families, and have more complex physical and social needs are identified to receive services from the CCC supported by GFATM. During fieldwork, CCC staff from GFATM program in Birganj, Bara district, told the Evaluation Team that since 2008, a total of 592 PLHIV were admitted, 76 of whom were from the Rauthat district and had died. Data show that, since 2011, 138 persons from Rauthat were diagnosed as HIV-positive. PLHIV from Rauthat are reported to have come from one small area of the district and to be migrant workers who had engaged in risky sexual behavior while working as migrants in India or who were IDUs. Although a high concentration of PLHIV came from one small area, there has been a limited response from the DPHO and IAs to move services closer to the area. 10 Further, no satellite or mobile clinics for ART under GFATM program were assigned to this area and clients from here reported having to travel 3.5 hours per week to get medicine. Due to limited ART medicine and the fact that the CD4 testing machine has not worked in the Birganj District Hospital for about six months, these PLHIV from Rauthat and others are facing difficulties in receiving ART services. EIHS services. Currently, 54 clinics under SSP provide EIHS in 26 districts, 33 of which are satellite sites. EIHS components include STI case management, HIV testing and counseling, EPC for PLHIV, and laboratory services for HIV and STI tests. All EIHS sites have a co-located DIC which is utilized as a community support center and is frequently visited by members of key population groups. Laboratories, whose services are an essential part of EIHS, are staffed by trained laboratory assistants and their tests performed are compliant with national algorithms and supported by an external quality assurance system. Tests are conducted only after proper counseling and the results and identity of tested person are always kept confidential. According to the Saath Saath Project Third Semi Annual Report August 2013-January 2014, EIHS sites reported conducting over 13,600 HIV tests, of which 54% were female; 13,868 STI tests, of which 6,088 people were treated for STIs; and providing pre-ART services for 1,894 PLHIV. General Welfare Pratisthan (GWP) static site staff in Bara district informed the Evaluation Team that they serve about 5-6 clients per day, while nearly 15-20 clients are served a day in satellite sites during the 3-4 days per month that they operate. The Evaluation Team discussed with staff the rationale for increasing the number of satellite sites and days of operation to reach more disperse populations with testing and counseling. Even though the number of patients seen per day at satellite clinics is three times that at static clinics, staff hesitated to reduce the number of static site days although those using static sites could be notified of a change of schedules. Although the lab assistant at the static clinic in Chitwan can test 25-30 clients a day, the clinic only serves 9-11 people a day because pre-counseling requires significant time according to protocols and limits the total number of clients who can be tested in any given day. According to SSP, currently, there are close to 1.5 times the number of satellite clinics than static clinics in SSP. This proportion has constantly increased over the life of SSP and is aimed at more efficient use of the resources. Community home based care. SSP built upon ASHA’s model for integrated care with CHBC services for PLHIV and their families for 18 IAs in 20 districts. Integration of CHBC services with positive prevention interventions has increased the availability and accessibility of HIV and FP-related care and support services to PLHIV to improve their health and well-being. During the last semi-annual reported period, 3,358 PLHIV (1,538 male and 1,820 female) were reached by positive prevention activities. In 2013, CHBC teams reached 4,654 people, out of which 1,003 were followed-up with after receiving pre-ART services from EIHS sites.3 CHBC workers motivate discordant couples to be tested because the HIV-negative partner is directly at risk of contracting HIV from his/her HIV-positive partner. CHBC can also support screened individuals to get confirmatory tests and encourage newly diagnosed HIV-positive individuals to seek support from family members and proper care. PLHIV, during a meeting organized by Chahari Mahila Samuha (CMS) in Chitwan, mentioned to the Evaluation Team that they were 3 Saath Saath project, Third Semi Annual Report August 2012-Jan2014 11 initially discriminated against by their families and neighbors but that following a stigma discrimination reduction orientation and training, the scenario has changed. They reported that they are now engaging in some livelihood work, staying in their homes, and being treated well by family and friends. One female PLHIV shared that, upon finding out about her HIV-positive diagnosis, her landlord threw her and her son out, forcing them to spend two nights on the street. After coming in contact with CMs, she was able to settle and re-establish her life and is presently a staff member at CMS. The Team’s review of data, interviews of IA and SSP key informants, and document review, confirmed that SSP provides many quality services and support mechanisms to reach FSWs. It was also noted that SSP and IAs have focused more on street-based, home-based, and establishment-based FSWs. As part of the original SSP design in the Cooperative Agreement, the stratified approach to reach the overlapping high risk categories of FSWs was defined as Strategy 1.1.1 (No AID 367-A-11-0000x) and was to address overlapping risk behaviors and increase coverage of especially vulnerable, hard to reach FSWs. Under this strategy SSP’s approach was to support and train IA CMs and OEs to better stratify FSWs and then to use targeted approaches within the strata, focusing on: 1) FSWs in the first six months of sex work: 2) FSWs returning from Mumbai and other parts of Maharashtra State; 3) FSWs who inject drugs; and 4) females who have occasional transactional sex. In order for the IAs to reach these overlapping high risk FSW groups, the SSP model was to include a strategic approach to cooperate with networks of FSWs and FSW IDUs and with established NGOs working in anti￾trafficking and cross-border migration to reach, identify, test, and treat these complex, sub￾categories of FSWs. SSP defined this targeted approach in its Cooperative Agreement but did not launch this approach early in the project. Relatively recently, SSP has started to provide a specific focus on stratifying the overlapping risk categories of FSWs who are identified as more vulnerable sub￾groups. Data from SSP’s recent work with these overlapping risk groups is only available for the period from August 2013 to June 2014, given the delayed start of SSPs work to reach the overlapping risk groups. Achievements against SSP outcomes and indicator targets Achievement against SSP M&E plan-specific FSW indicator targets. The current SSP model/results framework consists of five major outcome areas, each of which has several outputs and indicators. A detailed table with the full list of outcome and output indicators, targets vs. achievements, means of verification, and justification of progress is provided in Annex 3. The Evaluation Team observed that the project exceeded some targets, e.g., number of individuals trained in HIV/AIDS prevention, FP/RH, SI (including M&E, surveillance, and HMIS), health care (in-service training for GON staff), and HIV-related stigma and discrimination reduction. However, the Team found almost no output targets under the fifth outcome. For Output 5.1 “Improved Process for joint planning and harmonization of HIV/AIDS strategic planning and review,” no baseline was recorded and, although targets were set as 1, the achievements were recorded as “not applicable.” Likewise for Output 5.2 “Improved systems 12 within the GON established for tracking and monitoring HIV/AIDS activities within all GON ministries,” no baseline was recorded and targets and accomplishments recorded only “yes” or “no.” To make progress on these outputs, GON would have needed more and stronger SSP support across a number of ministries to create better processes and systems. Relevance of the model for current SSP services delivery In addition to the information provided by SSP, the Evaluation Team, with help from SSP team, assembled as much information as possible from various sources. To create Table 4, below, the Evaluation Team asked SSP to provide the number of: existing FSWs reached, new FSWs reached, STI referrals made, VCT referrals made, and other services provided specifically to FSWs and new FSWs during STI and HIV testing. The table presents data on specific services provided to FSWs by SSP to show the types of services and numbers of FSWs served. Table 4: Number of services to FSWs in Saath Saath Project- October 2011- June 2014 Prevention Number of FSWs outreached by prevention 41,230 Number of New FSWs outreached by prevention 26,818 Number of STI referrals made to FSWs by prevention * 350,475 Number of VCT referrals made to FSWs by prevention* 262,722 STI Number of FSWs examined for STI 18,364 Number of new FSWs examined for STI 15,165 Number of FSWs referred for HIV testing by STI 5,523 Number of FSWs referred for other STI by STI 42 Number of FSWs referred for other services by STI 73 VCT Number of FSWs who were provided post-test counseling with results 17,970 Number of new FSWs who were provided post-test counseling with results 15,018 Number of FSWs referred for STI by VCT 12,617 Number of FSWs referred for CST by VCT 190 Number of FSWs referred to PLHA support group by VCT 19 EPC Number of PLHIV receiving Basic Health Care (EPC Care) 4,789 Number of New PLHIV receiving Basic Health Care (EPC Care) 2,324 Number of PLHIV referred from EPC to ART 779 Number of PLHIV referred from EPC to CD4 2,682 CHBC Number of PLHIV receiving home/community-based care 6,159 Number of new PLHIV receiving home/community-based care 2,998 Number of PLHIV referred from CHBC to ART 1,601 Number of PLHIV referred from CHBC to CD4 4,902 * Indicators refer to number of times of referral and it is not head count of people ** The data disaggregated by the four groups of overlapping risk categories of FSWs are not yet available. The data indicates that during the period covered by the evaluation, the project reached 41,230 FSWs; of this number, according to SSP, 65% (26,818) were FSWs new to SSP services. During the same period, the project made 350,475 STI referral activities which are those that advise/send beneficiaries to a service (a test, counseling, DIC center visit, etc.). According to 13 SSP’s definition, the number of referrals is based on “activities” and not a head count of FSWs referred for STI prevention. Of the above referrals, 18,364 FSWs were reported to have been examined, indicating that for each FSW STI examination, 19 referral activities (such as referring FSWs by OEs and CMs to DICs for services) were carried out during the time period. The Evaluation Team questions the large number of referral activities that are done to result in a single FSW STI examination. If the training and supervision of OEs and CMs become more focused on reaching overlapping high risk groups, perhaps a smaller number of more focused referral activities would result in more FSWs tested in these four high risk groups. Table 4 also shows that there were 262,722 FSW referrals for VCT during the same period. Of these referrals (not head counts), SSP data shows that 15,165 were examined for STIs, meaning an average of 17.3 STI referral activities were carried out for each STI examination performed. The relationship between the large number of referrals and the resulting numbers of FSWs examined was assessed by the Team as needing further analysis by SSP to verify the focus and quality of referral results in an appropriate number of STI examinations. The large number of referral activities reported could indicate a need to focus SSP referrals on the high risk overlapping FSW groups. The number of FSWs examined for STIs during the period was 18,364, of which 82.5% were new FSWs. “New” FSWs are those considered new to the SSP program clinics and not new to (less than six months in) sex work or newly reached by the outreach activity. This indicates that these FSWs are not “carry overs” from the previous ASHA project. Regarding the VCT post-test counseling, the table shows that 17,970 FSWs were provided with post-test counseling and that of that number, 83.5% were new to SSP. The number of those FSWs who were provided with post-test counseling with results was 6,159 of which 2,998, or 48.7%, reportedly were FSWs new to the program. These particular figures were provided by SSP but their accuracy could not be verified by the Evaluation Team as they could not be triangulated with data available in other SSP reports. The table also contains data on two other services: EPC and CHBC services provided to PLHIV. The data provided was not disaggregated to be able to assess the number of FSWs within the number of PLHIV receiving these two types of services. It is possible that data associated with the Unique Numerical Identifier system would be able to provide these data. The Evaluation Team sought to track the overlapping high risk FSW groups in routine reporting and in other SSP sources. The Project Strategy 1.1.1 of the SSP Cooperative Agreement 367-A￾11-0000 states “establish stratified approaches to address overlapping risk behaviors and increase coverage of hard to reach FSWs.” Because this is a key SSP strategy that was to be developed as a key part of the project, the SSP M&E plan should have included indicators and tracked the progress of its implementation from SSP’s start up as a key part of the M&E plan. According to data provided by SSP during the period of August 2013 to June 2014, the number of FSW beneficiaries included 4,934 FSWs in the first six months of sex work, 156 FSWs returning from India, and 50 FSWs who inject drugs. However, the data on females who engage in occasional transactional sex was not available. 14 The Evaluation Team attempted to analyze the relevance of SSP activities related to the four overlapping risk categories of FSWs but data are not yet available for the years 2012 and 2013. These FSW overlapping risk categories are priorities as they are a part of the widely estimated 30% of unidentified and untested PLHIV in Nepal as noted in the Nepal HIV Investment Plan 2014-2016. It is estimated that there are currently around 49,000 PLHIV in Nepal. The Team addressed many prevention activities but the SOW did not direct the Team to address the issue of prevention of HIV vs. value for money. However, tracking service delivery point (SDP) cost information to strengthen cost effectiveness was part of SSP’s performance indicators in the Cooperative Agreement. Effectiveness of the Current SSP Model Current Expenditure Status of SSP Budget The Evaluation Team tracked the total allocated SSP budget from October 2011 to June 2016, against the yearly expenditure during October 2011 to June 2014. This information presented in the Table 5, below. Table 5: Current Expenditure Status of SSP Budget Outcomes Budget Amount (US$) Outcome-wise Share (%) Expenditure (as of June 2014) Total Expenditure (%) Yr 1 : Oct 11 - Jul 12 Yr 2 : Aug 12 - Jul 13 Yr 3: Aug 13 - Jun 14 Total Expenditure: Oct 11 - Jun 1. Decreased HIV prevalence among selected MARPs 6,875,000 25 835,105 1,387,457 1,191,706 3,414,268 50% 2. Increased use of family planning services among MARPs 6,875,000 25 742,316 1,233,293 1,059,293 3,034,902 44% 3. Increased GON capacity to plan, commission and use SI 5,500,000 20 494,877 822,197 706,197 2,023,271 37% 4. Increased quality and use of HIV services 6,875,000 25 866,036 1,438,843 1,235,843 3,540,722 52% 5. Strengthened coordination among all HIV/AIDS partners 1,375,000 5 154,649 256,936 220,686 632,271 46% Total 27,500,000 100 3,092,981 5,138,725 4,413,724 12,645,430 46% Source: SSP, June 2014 The above table shows that SSP appears to have managed its expenditures and has largely been efficient and effective in achieving its stated major outcomes. Of the five major outcomes, only the third, “increase GON capacity to plan, commission and use SI,” seems to be behind with a total expenditure of 37% against the total planned budget. However, SSP and IAs staff reported that SSP was unable to spend its allocated budget around the expected level (50%) due to a delay in government agencies (NCASC, DPHO) that resulted from frequent changes in leadership during a two year time span and ultimately slowed the implementation of those activities that required SSP’s technical assistance. 15 Current expenditure status of IAs’ budget. In addition to the above, the Evaluation Team analyzed IAs’ expenditures vs. budgeted amounts. Currently, 43 SSP-supported IAs are implementing HIV prevention and service activities in 33 districts. Their agreements with SSP have been amended several times (in one case up to eight times) due to many reasons, including changes in the foreign exchange rate. The total average budget expenditure of all IAs (from October 2011 to June 2014) is 52%. While most IAs are on track, a few have already completed one off – training or other assignments with 100% budget release as per the nature of their agreement with SSP. A detailed list of all IAs, their project titles, agreement budget, year-wise expenditures, and percentage of total expenditures is shown in Annex 2: Program Budget vs. Expenditure of Implementing Agencies (October 2011 to June 2014). 4.1.1.2 Conclusions The SSP model includes a major focus on quality outreach, training, and support to IAs, OEs and CMs, counseling, laboratory, and other staff. The clinical management of PLHIV has been institutionalized and is updated with HIV clinical guidelines, SOPs, and training manuals. For quality assurance, SSP developed guidelines and SOPs covering each service delivery protocol. For each clinical activity, SSP developed SOP checklists and guidelines for service providers. The Evaluation Team observed that clinical service delivery was very strong and satisfactorily managed by trained and skilled clinical staff. As part of community-wide stigma and discrimination (S&D) reduction, SSP IAs conducted orientation and sensitization sessions on S&D reduction using a standard reduction toolkit for nearly 4,889 different beneficiary groups and influential people such as local community leaders, health workers, civil society representatives, and community groups. The Evaluation Team saw evidence of some specialized efforts to identify, reach, and refer for testing FSWs based on their overlapping risk groups toward the mid-point of the project. The overlapping risk groups within the larger FSW population are more vulnerable to STI and HIV infection. Although outreach capacity building has been implemented, SSP continues to provide most outreach, referral, testing, and care to FSWs as a group rather than clearly tracking, serving, and reporting on the overlapping risk groups according to their high risk profile. 4.1.1.3 Recommendations SSP should continue to provide support to quality services by IAs to FSWs with increased emphasis on especially vulnerable overlapping high risk groups. Review of referrals to assure more effective links between outreach activities and effective testing and treatment should be done by SSP. Increased and strengthened district level support by SSP to IAs and DPHO, DAC and DDCs should also be done to strengthen collaboration and linkages between IAs and district health and HIV entities. SSP needs to plan and provide more adequate technical and institutional support to assist the GON in improving joint planning processes and systems for tracking within the GON. 16 4.1.2 Question 1b: Relevance of SSP Model in Light of New Paradigm of Test, Treat and Retain SSP's model of building the technical, financial, and management capacity of more than 40 local NGOs has clearly resulted in a stronger civil society in HIV/AIDS. However, as Nepal moves to adopt a test and treat approach, is this model effective in light of this new paradigm? 4.1.2.1 Findings Government Policy and Paradigm Shift in the Health Sector The new National Health Policy of 2014 sets broad-based policies to guide, plan, and implement health programs, such as mainstreaming health in each GON policy by strengthening the collaboration of multi-sectoral stakeholders related to health. TTR is a new approach that the GON intends to implement to manage HIV as a concentrated epidemic. The GON developed the National HIV/AIDS Strategy (2011 – 2016), which includes facilitating the gradual uptake of HIV testing and counselling by the public health system and which will be expanded to public health center (PHC) and health post (HP) levels by 2015 as an integral part of government healthcare services. The GON also prepared the Nepal HIV Investment Plan for 2014 – 2016. During discussions with MOHP and Department of Health Services (DOHS) officials, the Evaluation Team learned that MOHP is going to develop a new strategic plan for the next five years (2016-2020), which will be centered on empowering communities to assume a strong role in their health and decentralizing funding to promote health and development. Hopefully this plan will more fully describe the community empowerment modality and the changed roles of the government, community, and international NGOs. Senior MOHP officials discussed with the Evaluation Team needed overall changes that they and DOHS believe would provide better quality health services, including: new structural arrangements, bottom-up planning, adopting cross-cutting issues [e.g. gender equity and social inclusion (GESI), climate change, etc.], reforming periodic planning, improving quality of curative services, introducing the new health policy, rehabilitating certain target groups [such as accident victims, persons suffering from serious diseases, person living with disability (PLWD), etc.], collaborative partnership between government-to-government (G2G), public and private sectors, cooperative development, multi-sectoral involvement, and so on. Officials confirmed to the Evaluation Team that EDPs and international NGOs will still play a vital role in serving communities that the government has not reached, especially marginalized and vulnerable communities and target groups. TTR Implementation at the Community Level The new Nepal Community Test and Treat approach, at the core of the GON’s Nepal HIV Investment Plan, focuses on an community-led HIV testing expansion. The Director of NCASC officially acknowledges that this approach is a carefully crafted economic model that embraces innovations such as the rapid scale-up of HIV testing and ART for key affected populations and is identified as Nepal’s new TTR paradigm. The paradigm emphasizes the innovation of the rollout of Nepal’s Community TTR paradigm as the foundation for community-led HIV testing and treatment. The approach, carefully designed to be cost effective, is focused on maximizing 17 the support to PLHIV as leaders of Community Test Treat and Retain (CTTR) with efficient and results-based adapted/transformed support of DICs, OEs, CMs, and PEs operating within a framework of public-private partnerships. The community approach involves empowering PLHIV and high risk HIV groups in these communities to actively “reach in” to others in their community who are vulnerable to HIV by identifying, motivating, urging, and supporting them to be tested, treated and to continue treatment. Empowering high risk FSWs, IDUs, migrants, and others within their communities will help them play a proactive role in identifying people that are in need of services informing them of the needed behavior changes, and promoting testing. Although active outreach by IAs staff (OEs, CMs, DICs, and others) have had positive results in increasing testing of vulnerable high risk groups, the Community TTR paradigm centers around empowering PLHIV and high risk groups to reach in to their communities to affect these changes. The Team discussed the TTR approach with SSP, IA, and MOHP staff, and reviewed the SSP draft document, Approaches of Saath Saath Project for Community Testing and Counseling and Test, Treat and Retain. This document presents guiding principles for the expansion of TTR and presents the current role of SSP’s HIV program structures that could be used for community TTR, including: continued community outreach, community- and home-based care with positive prevention, expanded and integrated health centers, and laboratory services. However, the Team found that the document did not discuss in any depth a process for actually empowering the very communities of HIV high-risk vulnerable groups who are unidentified, untreated, and who are intended to carry out the in-reach that is at the core of community TTR. Early identification of PLHIV, timely initiation of ART, and life-long care are key elements of the strategy of achieving universal access to HIV treatment and care, and are increasingly recognized as the means to ending the epidemic. Expansion of community testing of highly vulnerable, especially overlapping risk groups, is critical to the success of HIV treatment. Retention of ART is essential for individual patient outcomes and also has public health consequences. Although the Nepal Investment Plan, which discusses the critical issues for Nepal’s national HIV response, has been published, it has not been sufficiently defined in order to implement a government response. However, the Evaluation Team’s discussions with MOHP’s central leadership, district-level HIV CAC coordinators, members of the DPHOs, IAs’ technical and program staff, and FSWs, identified interest and conviction in the real potential to actively involve and empower HIV high-risk and vulnerable overlapping groups in playing an active role in expanding testing within their communities. 4.1.2.2 Conclusion Literature review, field visits, discussions about, and observations of SSP’s approaches indicate that its current role, model, and approach would need to be adapted and strengthened to meet the needs to optimally support the GON in carrying out an effective community TTR initiative. 4.1.2.3 Recommendations To fully use the respected structure, expertise, and capacity of SSP to support the GON in carrying out a high quality, expanded TTR paradigm, SSP should adapt and expand its approach 18 to more truly empower the HIV high-risk, especially overlapping risk, groups to assume leadership for the in-reach into their communities to identify others, motivate their changes in behavior, and encourage them to actively seek testing and treatment. SSP should expand and strengthen its linkages with the MOHP at the central and district levels, as well as linkages with other GON agencies (Ministry of Federal Affairs and Local Development, Ministry of Women and Social Welfare) and multilateral and bilateral organizations for increased and expanded options for HIV affected and infected persons for livelihood and income generating opportunities. 4.2 QUESTION 2: EFFECTIVENESS OF SERVICE DELIVERY How do FSWs rate the quality of SSP services, including: 1) information; 2) counseling; 3) testing; 4) treatment (STIs and pre ART services); 5) referral from approximately 50 SSP-supported clinics; and 6) FP? 4.2.1 Findings To gather information about the effectiveness of SSP’s service delivery through various IAs, the Evaluation Team conducted 13 FGDs with a total of 153 FSWs. In response to the FGD question "Why do you come to this drop-in-center/clinic?" FSWs mentioned that they tend to come to the DIC for information about the prevention, transmission, and treatment of STIs and HIV; counseling (both pre- and post-test); HIV and STI testing; STI and pre-ART treatment; referrals; and to pick up medicine. Beneficiaries have received a wide range of information on topics such as the prevalence and transmission of HIV/AIDS; prevention and treatment of HIV; prevention and treatment of STIs; FP counseling and services; types of contraceptives available; proper usage and efficiency of contraceptives including Depo-Provera injections, implants, and condoms; services available at DICs and clinics; information about referrals to other clinics for treatment; etc. When describing pre-test counseling, beneficiaries mentioned that a counselor fills out their personal information and medical history, asks about their knowledge of HIV and STIs, and informs them about the types of procedures or tests needed. Beneficiaries are then counseled about the possible test results and asked to sign an informed consent form. Similarly, in post￾test counseling, depending on the test results provided by the clinical staff, a counselor informs beneficiaries about treatment and medication options, and discusses how to inform their spouses and relatives of the results. After counseling, beneficiaries decide whether to opt for HIV and STI tests. If they do opt for the tests, beneficiaries are asked to come for STI tests every month and for HIV tests every three months. FSW beneficiaries mentioned that they had STIs, were treated, and felt much better with the medications they received. Only PLHIV mentioned that they had been referred for pre-ART services. Most beneficiaries are not referred to other clinics for treatment because the clinic at which they are tested is often able to provide the required medication and treatment for their STI. They do, however, request referrals to other clinics when they need treatments not offered at theirs. 19 FSWs almost unanimously expressed high levels of satisfaction with the services they have been receiving from DICs and clinics, including: 1) outreach staff are easily accessible for discussions whenever they need information or support; 2) CMs regularly visit or call to remind them of the date of service or testing; 3) service providers treat them well, are always helpful, and provide all the needed information; 4) they have received quality care and medication from DICs and clinics that have “cured” their STIs; 5) they are reminded about their medication and checkups; and 7) some have been linked with income generating programs and provided with seed money to raise hens. One beneficiary mentioned that she received a free birth control implant that will last 5 years. FSWs reported that they have been receiving improved services from DICs and clinics in the past two years such as additional interactive tools and games and FP measures. During FGDs, FSWs mentioned additional services that they would like to see at DICs and clinics, such as blood group testing, general check-up services, permanent FP methods including sterilization, more referrals especially to male friends, child care facilities for their children during work hours, linkages to income generating activities, and an increased numbers of medical checkups. FSWs discussed very few cases of referrals except for those related to ART. All referrals are directed to government hospitals. During FGDs, when discussing whether they had previously been asked about the quality of the services provided, some FSWs stated that they had been asked about health problems, quality of service, reasons for joining the profession, and desired alternative jobs. At the end of each FGD with FSWs, the Evaluation Team asked the women to rate on a scale of 1 to 3, with 1 being “poor” and 3 being “very good”: 1) the quality of the services received in the past two years, i.e. the services provided by a SSP- supported DIC or clinic and other staff; and 2) specific aspects of the services at the DIC clinic they attend. Responses to these two questions are presented in Figures 1 and 2, below. 0% 20% 40% 60% 80% 100% Information Counseling Testing Treatment - STI and pre￾ART Referral Other Service Figure 1: How would you rate/assess the quality of the services that you have received in the past two years? Very Good Good Poor 20 The above figures show that services and aspects more under the control of SSP- supported IA clinics and DICs were consistently rated very high, reflecting FSWs’ high level of satisfaction with these services. Scores were also high for the aspects of the services (i.e. providing information, counseling, testing, and treatment) that were directly under the control of IAs. For those service and aspects not fully within the control of IAs, FSWs’ scores were somewhat lower. For example, referrals and follow-up after treatment scores were high but lower than other scores, possibly indicating that FSWs felt the need for additional follow-up support after treatment. Data collected show that 30.8 % of FSWs rated other aspects of services as very good. This is supported by interviews, during which some FSWs discussed an interest in knowing their blood type, receiving ultra sounds, pregnancy and other general health tests, and accessing income generating activities. The analysis of the data collected through FGDs conducted with IAs indicated that the answers from one FGD scored consistently lower than those of other FGDs. For example, the scores for three services – counseling, testing and treatment - were ranked “good.” The same score was given for the question on aspects of service, i.e. waiting time and adequacy of medication and supplies. The Evaluation Team found that a number of members of that FGD were relatively new to IA services and most had migrated from Kathmandu. It is possible that these factors have affected their rating of the quality of services and the aspect of services. It is also possible that the new FSW beneficiaries need additional orientation and information on IA services during the first year and that they have different expectations on waiting time and availability of medication. 4.2.2 Conclusions Qualitative and quantitative data shows that beneficiaries are largely very satisfied with the preventative and treatment services related to HIV and STI that they received from SSP￾supported DICs and clinics. FSWs interviewed by the Evaluation Team reported that DIC and clinic services have, in their opinions, improved since the start of SSP in 2011. Many FSWs, although satisfied with the current services, request additional services. 0% 20% 40% 60% 80% 100% Short waiting time?  Friendly and warm communications by staff? Respectful interaction with you by staff? Adequate medicine and treatment supplies? Follow up after treatment? Other aspects? Figure 2: How would you rate the following aspects of the services at the clinic/DIC you go to? Very Good Good Poor 21 4.2.3 Recommendations IAs should continue to provide quality services, through DICs and clinics and active outreach, for FSWs and other clients. IAs should encourage and motivate FSWs to take an even more proactive role in promoting these services to others who have not yet accessed DIC and clinic information and services. 4.3 QUESTION 3: EFFECTIVENESS OF CAPACITY DEVELOPMENT MODEL 4.3.1 Question 3a: Activities Targeting SSP Subgrantees In particular, in what ways have the TOCAT instrument and TOCAT corrective action plans been effective tools for assessing and developing an organization’s financial, administrative, management, and service delivery capacity? 4.3.1.1 Findings TOCAT instrument. SSP has strengthened the capacity of its IAs with TOCAT and other capacity building mechanisms. Although FHI 360 provided capacity development support to IAs before the start of SSP, TOCAT is considered FHI 360’s global tool and the primary tool for developing and carrying out SSP’s capacity building plan. The global TOCAT instrument was adapted to Nepal and developed as a tool for SSP to build the capacity of IAs to function effectively, efficiently, and sustainably. TOCAT was pre￾tested before its Phase I launch in late 2012 with the first set of 24 IAs. According to SSP’s documentation, the TOCAT instrument is divided into five areas, 29 domains, and 242 indicators (see Table 6 at right). Before the launch of Phase 1, SSP staff initiated pre-assessment communications and meetings with IAs to orient their leadership on the tool and process, secure organizational buy-in, and gather needed documents. Following these meetings, SSP facilitated a series of sessions for the first set of 24 IAs on TOCAT checklists and other TOCAT formats, as well as a staff group self-assessment and consensus exercise on the IAs’ level of performance. The 242 indicators in TOCAT’s checklist present a model system across the five functional areas and were used to assess the IAs’ baseline level of performance. The indicators represent the standard of performance or model of an idealized organizational status that IAs can achieve. TOCAT process of implementation. Figure 3, below, shows the TOCAT rollout process and steps of implementation. It should be noted that SSP adjusted these steps based on their experience and lessons learned. TOCAT roll-out. The active, participatory TOCAT process, facilitated by SSP staff, engaged IA boards, project directors, and a number of key staff. These intense TOCAT rollout process sessions covered individuals completing the checklists, (before the workshop, a sharing of individual checklists, team consensus building based on discussions, and addressing concerns and comments). A reflection discussion followed where IA participants were given time to realize Table 6: TOCAT Areas, Domains, and Indicators Functional Areas Domains Indicators Technical 10 66 Organizational 11 113 Financial 3 254 Procurement 2 9 Monitoring for Management 3 28 TOTAL 29 242 22 the importance of the TOCAT process and to define their IAs’ strengths as well as areas that needed improvement in some of the TOCAT’s 242 indicators. Based on these discussions, participants were charged with establishing priorities for a two-year strategic plan and one-year action plan. According to SSP documentation, the process included four modules: • Module 1, pre-assessment: pre-meetings and readiness workshops/orientations regarding the checklist with individual scoring. • Module 2, assessment, prioritization, strategic planning, and action: scoring individual checklists, conducting joint assessments, building team consensus, reflection discussions, setting priorities, developing IA strategic plans and comprehensive action plans in a series of sessions that took different IAs 6 to 30 days and was included in SSP Program Officer’s follow-up to assure completeness and coherence. Completed and verified checklists were sent to the SSP TOCAT Committee. • Module 3, workshop to finalize TOCAT assessment, triangulation and validation, and consensus building: SSP Program Officer facilitates the presentation of reviewed checklists, strategic plans, and action plans. • Module 4, a series of follow-ups included in the quarterly Action Plan, reflection on the status of the earlier Action Plan and second prioritized areas for improvement of the Strategic Plan, and details for the next quarter’s Action Plan: SSP’s guidance states that the entire process should be completed in 5-6 weeks depending on time availability. As part of SSP’s review and support to TOCAT, program staff, the TOCAT Review Committee, and several other groups perform reviews and provide comments. After the first rollout, SSP assembled lessons learned which were incorporated into the TOCAT II rollout for the remaining 17 IAs. One major change reported was the use of a concentrated two-day workshop for each IA orientation workshop to provide TA to the IAs, verify justifications on the ratings of the 242 indicators, and create a draft annual work plan to address identified gaps. Members of the SSP Capacity Building Program, HR, Finance, and M&E participated in at least one of the 17 different TOCAT assessment workshops for each of the IAs in the second rollout. Analysis of TOCAT rollout. The Evaluation Team’s extensive document review of guidelines, tools, action plans, quarterly narrative reports, quarterly review meetings, quarterly 23 work plans, rollout of TOCAT, experiences from TOCAT II, and discussions with the SSP Capacity Development Unit, found that the TOCAT baseline scores for starting points exist for all 41 individual IAs. Instead, each individual IA received a baseline score for its own unit for SSP’s capacity development plan. SSP used the IA action plans that they had reviewed during the TOCAT process as the starting point for support to build IA capacity individually and in groups. Per the Evaluation Team’s request, SSP developed a composite report detailing the scores of the two rollouts (see Annex 5). The Evaluation Team discussed with SSP the potential for TOCAT IA scores on the 242 indicators to be generally optimistic (as self-assessed by IAs). SSP stated that the workshop justification process may have tempered some ratings but that baseline scores were generally high at TOCAT’s start. TOCAT action plans. The Evaluation Team found that IAs developed their action plans based on a subset of the 242 indicators and generally categorized them by the TOCAT’s five areas and 29 domains. IAs established priorities in their action plans but used different numbering systems based on priorities that were generally related to the indicators. The action plans provided IAs with a set of actions, especially for the first year. According to IAs, action plans have helped ensure that boards are more active, documentation procedures are completed, and financial systems are improved. Using the action plans, IAs were energized and took actions on a subset of priorities, including NGO management, good governance, best practices, report writing, documentation, proposal writing, leadership development, waste management, procurement, and strengthening and increasing the involvement of boards of directors. Action plans were detailed for the first year, with a number of the actions marked as completed. Some action plans that the Evaluation Team reviewed did not record accomplishments but rather SSP stated that they had held reviews after the first year to assess and document progress against activities included in them. The Evaluation Team noted that progress against a subset of indicators from the action plans measured progress or completion of those specific actions. At the same time, the action plan itself did not allow the IAs or SSP to be aware of or track progress against the larger TOCAT baseline instrument’s scores or to assess progress across the 242 TOCAT indicators. Action plan: data analysis and follow-up. As part of SSP’s data analysis and follow-up support to IAs, the CB Unit presented the detailed, individual IA scores (not grouped scores) to SSP program and other staff. These individual scores did not allow SSP or the team to develop an overall picture of the two TOCAT IA groups’ capacity baseline scores, so each IA was the unit for future support. Even without the group-level baseline information, SSP staff compiled IAs subsets of actions from their individual action plans into an SSP consolidated report/group action plan and then developed SSP’s TA Plan to support capacity development of individual IAs and groups of IAs. Trainings, workshops, mentoring, specific TA, and other support were provided by various units, specialists, and consultants, and were reportedly coordinated by the Program Officer. SSP’s Consolidated Action Plan for Phases 1 and 2 grouped similar actions identified within each of the two IA groups. SSP then divided the individual plans’ designated responsibilities into three categories: 1) self/IA; 2) SSP TA; and 3) other IAs/TA. The Evaluation Team noted that, although the Consolidated Action Plan defined the TA that SSP would provide to the individual IA or group, the resulting report did not mention the TA that was provided by SSP. This 24 missing information on the report of SSP’s TA and the mismatch between the Consolidated Action Plan and its Report Out document did not allow the Evaluation Team to track and evaluate the effectiveness of SSP’s follow-up capacity development TA. TOCAT staff stated that SSP TA for this follow-up to the TOCAT workshops is recorded in the larger SSP TA plan but that tracking the effectiveness of capacity development by TOCAT was not done. After the IAs’ first Annual Action Plan, SSP reported that the TOCAT Action Plan is incorporated into the IAs overall quarterly work plan. Survey responses on TOCAT. KIIs with IAs in 7 districts and the results of the survey conducted in an additional 15 districts uniformly show interest in and positive reactions to TOCAT tools and training experience. Many IAs have completed their first annual action plan and report increased interest of boards and actions, such as development of strategic action plans, documentation of procedures, improvement of financial recording, etc. SSP reported providing a number of different training programs and TA from different units to the IAs but the tracking was not routinely conducted by the Capacity Development Unit and the Evaluation Team found it difficult to establish the level of capacity development provided. In the 7 districts visited, DPHOs generally did not know about TOCAT. In summary, the Evaluation Team assessed that TOCAT is a large, detailed, 242-indicator assessment tool used to start IA assessments, with scores self-assessed by IAs and jointly justified. SSP’s TOCAT analysis is implemented by individual IAs; however, a group analysis/score to enable TA and the assessment of progress by the two groups was not completed. The Action Plan (fairly aligned with a subset of indicators) is the tool used for IA actions, SSP TA, and monitoring progress, with “actions completed or in progress” being the measures of progress. Repeated TOCAT assessments have not yet been implemented to document progress against baseline scores. 4.3.1.2 Conclusions TOCAT is a large instrument with many complex indicators. The time and expertise needed to adequately understand the multiple indicators would be difficult for IAs at start up, especially during the short timeframe. During the process, the IAs’ self-assessment was considered too difficult to have been completely impartial. However, although the self-scoring was complemented with SSP’s support during the justification process, it seems that the result was many dubious high scores. The Team’s review of the content noted that TOCAT indicators can be ranked high at a point in time but can change rapidly and repeatedly. At the same time, TOCAT indicators provide a standard for a well-functioning IA that can be reviewed and used over time by the IAs. To date, the full TOCAT instrument has not been used again for IAs assessments. The SSP and IAs focus has rather been on a subset of actions identified as priorities. The use of the full TOCAT as a guide could help IAs and SSP monitor overall progress of their capacity. However, SSP and IA focus is on tracking actions in the action plans. The IAs action plans are a subset of activities generally related to the TOCAT indicators. SSP and IAs focus on completing actions identified as priorities that are somewhat aligned with the indicators. IAs report improvements more frequently in governance, management 25 strengthening, increased board involvement, improved HR management, etc. Although SSP’s capacity building support/TA from various staff is planned and provided, the reporting format does not track or report it, but rather reports the progress of the IA. Multiple planning and reporting formats and responsibilities for TA make tracking the progress of action plans diffuse across different units in SSP. 4.3.1.3 Recommendations TOCAT should be streamlined, simplified, and focused somewhat more clearly on the capacity development needs and tasks more directly related to the IAs’ work in HIV. Defining, implementing, and tracking the capacity development provided by various units and sources within SSP support, as well as the overall progress of capacity development of IAs, should be coordinated and tracked by a single entity within SSP. TOCAT capacity development should fit within a larger SSP capacity and more coordinated capacity development strategy and approach. SSP should consider simplifying the tracking of the consolidated progress of the two rollout groups of IAs as well as the gradual devolution of responsibility for this task to one or two senior IAs within each group. 4.3.2 Question 3b: Activities Targeting SSP Subgrantees What has worked, what has not worked, and how can capacity building activities be improved? 4.3.2.1 Findings SSP has significantly developed the capacity of IA staff, resulting in the majority of targeted interventions being met. Further, these efforts have enabled SSP to reach more than 22,142 FSWs from August 2011 to January 2014, and 38,348 FSWs through 25 project districts. What has worked SSP’s work to increase the technical capacity of IAs, through multiple and various trainings, monitoring, and follow-up with central office and field coordinator staff has directly and indirectly supported the quality of services provided to FSWs, including improved knowledge, awareness, and skills. FSWs are reached by capacitated OE, CM, and PE outreach workers charged with identifying and referring FSWs. The trainings for counselors, lab personnel, and related clinical service providers in STI/HIV/AIDs have resulted in FSWs frequently using these services for testing, STI treatment, and other clinical services with high levels of satisfaction. The training and supervision of OEs, CMs, and PEs has resulted in FSWs having increased skills and knowledge, empowering them, and helping them reach out to other FSWs. DICs provide a decent platform where FSWs are capacitated through edutainment activities to expand their knowledge of the behaviors and risk factors associated with STI/HIV/AIDs. Edutainment activities also motivate FSWs and their clients to frequently visit DICs and promote positive health behavior changes. Creative events with thematic titles generate awareness among and increase the participation of FSWs. Events are followed by facilitated discussions on HIV/AIDS, STIs, condom use and condom negotiation skills, risk behavior, importance of regular STI checkups and HIV testing, and S&D reduction. 26 SSP and IAs more recently trained a number of outreach staff in the micro planning process which is currently being used by OEs and CMs and is considered helpful. This training is one of the few that capacitates staff to differentiate the categories of overlapping risk. In addition, it provides a specialized approach to reach these hard to reach groups of FSWs to encourage testing and treatment and to orient them to special overlapping risks and vulnerabilities and encourage behavior change. As a result of SSP’s capacity building support, IAs have increased their knowledge and awareness and are now able to reach more FSWs for testing, treating, and counseling. What has not worked Although some FSWs have received income generating training, SSP reported that most leave without completing it. Further, many of those who received seed money to launch an income generating program have continued working in the sex industry. Due to time limitations, the Team was unable to assess SSP’s collaboration with organizations with income generating expertise to strengthen SSP’s capacity for its remaining time. 4.3.2.2 Conclusion SSP’s many high quality trainings and their support for IAs have increased the skills of staff and improved the quality of services provided. However, the Evaluation Team did not find a comprehensive vision or approach to capacity development across SSP units and training and capacity development activities. 4.3.2.3 Recommendation SSP should continue to provide quality capacity development. SSP should clarify its vision and approach to capacity development and strengthen a central point of coordination that guides, directs, and coordinates the multiple, varied capacity development activities. 4.3.3 Question 3c: Activities Targeting the GON In what ways has SSP been able to build the technical capacity of the GON, especially in the areas of HIV surveillance and M&E? 4.3.3.1 Findings SSP has supported strengthening the technical capacity of the MOHP‘s NCASC and other MOPH units. To accomplish this, SSP’s three strategic unit teams – Surveillance, M&E, and Research – work closely with their relevant counterpart units at NCASC and other divisions. The annual SSP/SI work plan is “nested within” NCASC’s work plan for the same period, making it easier to plan TA and capacity building that is both coherent and timely. This planning period also takes into account the possibility that multilateral external donors may bring resources and priorities to NCASC projects that require expertise it does not possess but which can be supplemented by SSP. NCASC tends to call upon SSP to fill in gaps when they arise since SSP has the flexibility to add tasks to its work plan and funding to facilitate certain outputs. SSP’s key SI responsibilities are to: • Improve MOHP capacity to conduct second generation surveillance. 27 • Improve the capacity of MOHP and DACCs to analyze and use strategic information. • Periodically update the MOHP research agenda. Improve the MOHP capacity to conduct secondary surveillance. SSP emphasizes secondary surveillance through its robust recording and reporting system, the constant updating of its management information system (MIS), and use of its geographic information system (GIS) capability. The project has supported NCASC in developing the capacity to undertake these approaches. NCASC’s secondary surveillance responsibilities are to monitor the concentrated HIV/AIDS epidemic and to analyze the risk among key populations. SSP’s mandate to deal with MARPs, including FSWs, is to employ a number of methods to measure and analyze risk and then to organize innovative ways to reach these populations through sub￾groups. NCASC now has the capacity to utilize the same methods in a spatial dimension. Upon NCASC’s request, SSP and NCASC developed an interactive GIS map to assess HIV high risk areas and hot spots. Both organizations use the interactive GIS map to first show hotspots where FSWs are dominant among a population, and where their potential clients congregate, such as truck drivers along highways, and then to map existing and potential service sites. The map has helped SSP reach the maximum number of FSWs by increasing the number of OEs and CMs and opening additional clinics in hotspot areas, as well as by closing or moving sites and personnel to other areas as needed, based on changing hotspots. Further, the maps spatially show the spheres of IA action and identify gaps in support by outreach workers that need to be filled. Essentially, any data in the SSP MIS can be displayed in one or a variety of maps generated by the GIS system to guide program decisions. Over the past two years, SSP has provided TA and built the technical capacity of NSCAC to use these methods and technologies to improve and expand second generation surveillance. With its recently acquired second generation surveillance skills, hardware, and software, NCASC now has the capacity to replicate SSP’s second generation surveillance activities in consultation with SSP/SI. As a member of technical working groups at NCASC, SSP/SI is able to collaborate with other donors and NCASC to conduct HIV estimates, population estimates, and other types of second generation surveillance. Improve the capacity of the MOHP and DACCS to analyze and use strategic information. At the district level, there are currently 50 DACCs, each with 16 members that consist of representatives of MOHP, civil society and PLHIV. SSP has supported DACCs by providing training, supportive supervision, and software to DACC coordinators and other DACC members. DACCs enjoy latitude in their planning and activities compared to civil servants at the district level. Since they report directly to NCASC, they are accessible to SSP for training and capacity building. SSP supported NCASC to develop a training module for the district level, Data Analysis and Use, which was rolled-out for DACC coordinators who receive, report, and use HIV data on a regular basis. SSP also supports trainings on how to analyze and use data. Based on recommendations from a prior SSP data analysis and use assessment, under the leadership of NCASC, SSP revised the training curriculum. Also, under NCASC’s leadership, SSP revised the curriculum and conducted training for all DACC coordinators and 23 HIV focal persons from 28 the project’s 33 districts. SSP has planned a second training for DACC coordinators in data analysis and use. SSP/SI, program officers, and IAs supported DACC coordinators from 33 project districts to collect, compile, analyze, and prepare a presentation for the annual regional review meeting, as well as to prepare district fact sheets and construct district profiles. The coordinators were able to review a presentation for a meeting of the AIDS Country Coordinating Mechanism (CCM) based on a DQA conducted on ART services in regional hospitals. DACC coordinators have also served on numerous joint monitoring visits to clinical facilities with IA staff. Micro-planning is an approach used by IA OEs and CMs to better identify FSWs in their areas of responsibility. Situation analysis allows outreach staff to distinguish between FSWs who have engaged in sex work for six months or less, FSW returning from India, and FSW who are IDUs, and also helps identify FSWs as street-, home-, and establishment-based, thus improving the relevance of new behavior change communication and other outreach strategies. The same kind of approach could be adopted by the GON if it decides to mobilize female community health volunteers (FCHVs) in order to reach more FSWs at the community level, possibly under a TTR approach. National HIV research agenda (July 2013-2016) outlines the process for creating a national research agenda and was supported by SSP. It identifies a considerable variety of research projects currently scheduled or being carried out and, as a priority, three broad research themes: 1) prevention of HIV and STIs; 2) treatment, care, and support of infected and affected individuals; and 3) health and community system strengthening. IBBS. NCASC has recently taken over the responsibility of managing and carrying out the IBBS from SSP, which designed, funded, implemented, and analyzed the survey results on FSWs in different parts of Nepal, following on its predecessor project. NCASC may sub-contract the survey implementation but will manage the process. SSP/SI will be involved through the first round to support NCASC throughout the process of developing a curriculum for data collectors and providing training to the researchers who will carry out the survey. Development of guidelines on M&E of HIV. SSP has been instrumental in the funding, drafting, publication, and dissemination of some of NCASC’s most significant documents, such as M&E guidelines, which present the division of labor for M&E coordination at the national, regional, and district levels. A national-level technical working group and a number of other institutionalized bodies are responsible for organizing and funding the M&E agenda. NCASC sent an HIV specialist to strengthen capacity at the regional level. At the district level, DACCs provide M&E support to service sites and seek M&E TA from NCASC. A multi-sectoral function, M&E is seen as the responsibility of all organizations working in HIV. 4.3.3.2 Conclusions The results of SSP’s efforts to build the technical capacity of the MOHP have been uneven. Like many organizations that have a very broad mandate but not enough staff, NCASC takes 29 advantage of all available TA and capacity building support but is still not necessarily able to implement its work without that support. When, as with SSP, capacity building and TA are combined with financial support, it is difficult for local agencies to turn down. MOHP’s capacity to conduct second generation surveillance has improved as a result of SSP’s support. An example is the development and use of the GIS interactive map. Although SSP and NCASC continue to collaborate, NCASC is now able to use GIS without direct external support. Another example is the creation of the HMIS, which now includes information on HIV/AIDS that was formerly part of a separate MIS system. The capacity of NCASC and the DACCs to analyze and use strategic information has improved. SSP IAs provided training to DACC coordinators on a variety of data and information related skills, such as data analysis and use, allowing them to participate more successfully in a number of repeated reporting activities at the district, regional, and national levels. MOHP’s research agenda has been updated and the related National HIV Research Agenda in Nepal, 2013, is clear and provides appropriate guidance for all aspects of carrying out HIV￾related research by sub-topic. The National Guidelines on M&E of HIV Response in Nepal, 2012, presents a professional assessment of what needs to occur in M&E at the national, regional, and district levels. It also provides the guiding principles for M&E of the epidemic and activities designed to reduce prevalence rates. As the guidelines were prepared with support from an FHI360 consultant, it remains to be seen whether or not capacity exists at each level to implement them and whether implementation will take place. 4.3.3.3 Recommendations SSP should continue to provide TA and capacity building to NCASC and funds as needed when activities could not otherwise take place. SSP should assist NCASC to better link planning of key activities with budget allocations. SSP should build additional capacity of the IAs it supports to further strengthen the DACC capacity for data collection and analysis of strategic information at the district level. SSP should strengthen the capacity and further empower DACCs to significantly engage line ministry members of DACCs to collaborate and support HIV prevention in their agencies at the district level. SSP’s SI unit should continue to collaborate with its counterparts at NCASC to implement their joint work plan to build the long-term capacity and systems at NCASC and to decrease the time and effort spent in responding to less productive ad hoc requests for additional activities. 4.4 QUESTION 4: BEST PRACTICES What are SSP’s (both from the prime and among sub-awardees) most effective and exemplary practices with respect to service delivery and capacity development of both GON and 30 subgrantees that, if shared, would improve the results of SSP sub grantees and other organizations operating in the HIV/FP sector? 4.4.1 Findings Interactions attract and retain FSWs. During KIIs, all levels of IA staff and board members provided statements of the friendly and warm communications, respectful interactions, and supportive, non-judgmental approaches used by IA and SSP staff with FSWs during outreach activities in the field and at clinics. FSWs consistently confirmed this in FGDs when they responded to questions regarding why they visited a specific clinic or DIC, their satisfaction with services provided, and their opinion about the improvement in services (see their responses to specific questions in Annex 6). Further, FSWs discussed with the Evaluation Team their appreciation for the confidentiality and privacy maintained at IA locations. There is potential for these best practices to be expanded in public sector facilities through SSP-MOHP collaboration and staff interchanges in pilot efforts. Micro planning. SSP re-invigorated micro-planning as a bottom up process used by OEs and CMs to identify and differentiate characteristics and behaviors in order to reach high risk and overlapping risk groups of FSWs for prevention, testing, and treatment activities. Recently, 14 IAs in 25 districts reportedly began to reach the four differentiated overlapping FSW high risk groups. Although this practice is promising, SSP admits that, due to the illiteracy of many CMs, data is incomplete. However, strengthening the micro-planning approach and expanding its use has great potential even though it would require vigorous supervision and support by IA managers and data analysis. Adding empowerment of overlapping groups to the micro planning methodology would link it more closely to the approach of the Community TTR paradigm. Guidelines and checklists for quality care. Detailed guidelines and checklists have been developed and/or updated to guide the quality of services provided by IAs. These reflect updated state-of-the-art clinical and other technical information. Records show that SSP guidelines have been adopted by other IAs. In addition to IAs using SSP guidelines for specific clinical services, they carry out client satisfaction surveys using a quality assurance checklist provided by SSP. Strong reporting, review, and supervision. In survey responses and interviews with the Evaluation Team, IAs stated that SSP’s reporting and reviews in quarterly and semi-annual review meetings, technical supportive supervision, and monitoring, strengthened and supported their management and technical capacity as well as regular monitoring and supervision of their projects, improved tracking of their financial resources, and provided good feedback mechanisms and communication on program changes. GIS mapping. SSP’s SI Unit works with NCASC at the national level; the SI Unit support has been extended to the district level for GIS mapping of hot spots and related information. The SI Unit trained IA MIS staff to, in turn, train OEs and CMs to collect data for GIS mapping. GIS mapping introduced a spatial dimension to support the work of IAs, DACCs and DPHO staff. This GIS capability supports DACCs to map their position in relationship to FSWs identified by the outreach workers and CMs in the area. This potentially supports identifying hotspots and other areas of concentration of key populations in their geographic areas. GIS technology also 31 supports DACC coordinators to compile HIV data from IAs, DPHOs, and others to construct the District AIDS Profile that supports planning and monitoring of HIV activities. DQAs. The SSP/SI unit provided training and capacity building for DACC coordinators, DPHOs, HIV focal persons, FP officers, IA MIS/M&E officers, IA outreach workers, project managers, DIC staff, lab assistants, and other staff in the IA to implement DQAs at the district level. Usually semi-annually, IAs support the implementation of DQAs by coordinating with DACCs to identify a team comprised of members from different HIV organizations to assess the quality of a particular service delivery method or outreach site. At the end of each visit, the DQA team documents the assessment. This practice has significantly improved the quality of data on service delivery and outreach sites. 4.4.2 Conclusions SSP developed a number of effective and exemplary practices including tools, checklists, processes, and approaches that have improved the consistency and quality of outreach information and services provided. These tools have not only been used by SSP but have also been adopted by other NGOs. SSP’s warm and supportive approach is recognized by thousands of FSWs and other beneficiaries who mention it as a key factor in the using of HIV services. 4.4.3 Recommendations SSP should continue its broad support to NCASC and other MOHP entities. However, as NCASC at the central level becomes more capable of implementing its agenda, SSP should significantly increase its support to the DACCs and other GON entities at the district level to build capacity and adapt lessons learned to strengthen district level systems. Micro-planning should be continued as it allows outreach workers to identify the FSWs most at-risk. Micro-planning reports are now an intrinsic part of the data that are included in the MIS and are thus accessible to all SSP staff. SSP’s flexibility in working with NCASC on a number of complex projects has allowed collaboration leading to beneficial products and systems. Attempts by SSP’s SI unit and other headquarters staff to bring the recording and reporting systems down to the district level have served DACCs and AIDS focal persons well by informing them of the latest technologies, and helping them practice their use. 32 ANNEXES 33 ANNEX 1: EVALUATION SCOPE OF WORK 34 STATEMENT OF WORK The primary objective of this midterm performance evaluation is to assess the performance of SSP from its inception in September 2011 through March 2014 and provide recommendations for the direction of the remaining period of the project. Rather than addressing all of the SSP objectives included in Figure 1, the evaluation will focus the subset of SSP objectives covered by the Evaluation Questions listed in section C.3.2 below. C.3.1 Intended Audience and Uses The audience for this evaluation includes: USAID, FHI 360 (the Prime), all 43 subgrantees4 , project beneficiaries, and the GON. USAID intends that the evaluation be used to: 1) provide information that will enable improvements in project performance among the Prime and subawardees for the remaining period of the project; 2) increase buy-in to the project by the GON and willingness to replicate those aspects that work well; 3) empower project beneficiaries to provide greater input to Project staff and management to continually improve SSP implementation; and 4) improve USAID and partner understanding of how to best utilize the TOCAT as a capacity development tool. C.3.2 Evaluation Questions The Contractor will answer the following questions in this midterm performance evaluation. (The figures in parenthesis indicate the relative level of effort USAID anticipates is required to address each question.) a.Relevance of SSP Model (30%) To what extent has the SSP model optimized service delivery to female sex workers as the key population for this evaluation in the context of a concentrated epidemic? SSP’s model of building the technical, financial, and management capacity of more than 40 local NGOs has clearly resulted in a stronger civil society in HIV/AIDS. However, as Nepal moves to adopt a test and treat approach, is this model effective in light of this new paradigm? In answering this question, the evaluation team must use available data on targets and achievements reflected in the project’s PMP, as well as other data sources such as the Integrated Bio-Behavioral Survey (IBBS), FP/HIV assessment, key informant interviews, etc. b.Effectiveness of Service Delivery (30%) How do FSW rate the quality of SSP services, including: 1) information, 2) counseling, 3) testing, 4) treatment (Sexually Transmitted Infection (STI) and pre Anti-retroviral Treatment (ART) services), 5) referral from approximately 50 SSP-supported clinics, and 6) family planning? c.Effectiveness of Capacity Development (30%)Activities targeting SSP subgrantees: To what extent has SSP strengthened the capacity of SSP sub-grantees? In particular: 4 As of December, 2013 there are 43 sub-awardees. One organization, New Era, was hired for research activities which are now complete. One organization, JHPIEGO, was originally a sub￾awardee but is no longer one. 35 1. In what ways have the TOCAT instrument and TOCAT corrective action plans been effective tools for assessing and developing an organization’s financial, administrative, management, and service delivery capacity? 2. What has worked, what has not worked, and how can capacity building activities be improved in general and especially as related to working with the focus on female sex workers? Activitiestargetingthe GON: 3. In what ways has SSP been able to build the technical capacity of the GON, especially in the areas of HIV surveillance and M&E with a focus on FSW but including other key populations? d.Best Practices (10%) What are SSP’s (both from the prime and among sub-awardees) most effective and exemplary practices with respect to service delivery and capacity development of GON, subgrantees and DACs that, if shared, would improve the results of SSP subgrantees and other organizations operating in the HIV sector? In addition the Contractor must consult with the COR to conclude the following as part of the finalization of the Work Plan and Final Evaluation Design Matrix: 1. Total number of KIIs, FGDs conducted per district, and identification of the primary respondents in these. 2. For FGDs what characteristics will determine the “focus”? Will the same “focus “be applied to all the FGDs? Or will there be more than one focus? For the FDGs, if so what would these be? 3. Delete 4. Delete 5. travel proposed, the Contractor must plan for geographic reach in consultation with the COR during the final Work Plan approval process. 6. The approach for measuring the effectiveness of the capacity building of local organizations needs to be more specific during the Work Plan approval process. 7. The Contractor must identify, in consultation with the COR during the work planning process, very specific indicators that will measure the effectiveness of capacity building efforts. 36 ANNEX 2: PROGRAM BUDGET VS. EXPENDITURE OF IMPLEMENTING AGENCIES Program Budget Vs Expenditure of Implementing Agencies (October 2011 to June 2014) S.N FCO/ID# Implementing Partners Project Title Period of Performance LOP Budget Expenditure Total Expendi ture (%) Yr 1 Yr 2 Yr 3 Total Begin End Oct 11- Jul 12 Aug 12- Jul 13 Aug 13- Jun 14 Oct 11- Jun 14 1 605156 / 0634.0002 Jagriti Mahila Maha Sangh (JMMS) Capacity Strengthening of JMMS Network 16-Jan￾2012 30-Jun￾2016 159,258 18,590.05 29,487.99 24,601.66 72,680 46% 2 605157 / 0634.0003 ChhahariMahilaSamuha Positive prevention activities for PLHIV in Chitwan district 1-Oct￾2011 30-Jun￾2016 118,760 15,689.79 22,631.72 20,597.95 58,919 50% 3 605158 / 0634.0004 Child and Women Empowerment Society (CWES) HIV and STI Prevention and FP promotion for FSWs and their clients in Kaski district 1-Oct￾2011 30-Jun￾2016 194,907 29,654.77 35,276.55 31,286.01 96,217 49% 4 605159 / 0634.0005 Dang Plus, Dang Positive prevention activities for PLHIV in Dang district 1-Oct￾2011 30-Jun￾2016 119,171 17,520.01 22,485.49 19,130.23 59,136 50% 5 605160 / 0634.0006 Federation of Sexual and Gender Minorities Nepal Capacity strengthening of MSM/MSW network in Nepal 1-Oct￾2011 30-Jun￾2016 129,564 17,017.97 22,503.55 25,167.13 64,689 50% 6 605161 / 0634.0007 Lumbini Plus Positive prevention activities for PLHIV in Nawalparasi district 1-Oct￾2011 30-Jun￾2016 127,909 17,561.35 23,585.49 23,082.42 64,229 50% 7 605162 / 0634.0008 Sakriya Plus Nepal Positive prevention activities for PLHIV in Kavrepalanchowk district 1-Oct￾2011 30-Jun￾2016 135,245 19,931.86 25,437.56 24,583.76 69,953 52% 8 605163 / 0634.0009 Syangja Support Group (SSG) Positive prevention activities for PLHIV in Syangja district 1-Oct￾2011 30-Jun￾2016 117,680 19,457.22 22,516.51 19,157.29 61,131 52% 9 605164 / 0634.0010 Trisuli Plus Positive prevention activities for PLHIV in Nuwakot district 1-Oct￾2011 30-Jun￾2016 126,289 18,865.19 23,370.58 22,288.87 64,525 51% 10 605165 / 0634.0011 SnehaSamaj Essential package of care services for women living with HIV in Lalitpur district 1-Oct￾2011 30-Jun￾2016 92,204 16,031.20 17,951.68 14,653.53 48,636 53% 11 605166 / 0634.0012 STEP-Nepal HIV and STI prevention and FP promotion for FSWs and their clients in Kathmandu district 1-Oct￾2011 30-Jun￾2016 442,557 62,939.67 76,062.77 74,779.72 213,782 48% 12 605167 / 0634.0013 Community Development Forum Expanded community and home￾based care services for PLHIV in Doti district 1-Oct￾2011 30-Jun￾2016 152,114 23,526.60 29,522.45 23,322.15 76,371 50% 13 605168 / 0634.0014 Dhaulagiri Positive Group (DPG) Positive prevention activities for PLHIV in Baglung district 1-Oct￾2011 30-Jun￾2016 107,021 15,141.20 20,037.52 19,324.08 54,503 51% 14 605169 / 0634.0015 Drishti Nepal Capacity strengthening of former female IDUs network in Nepal 1-Oct￾2011 30-Jun￾2016 78,377 11,539.46 14,839.75 13,944.17 40,323 51% S.N FCO/ID# Implementing Partners Project Title Period of Performance LOP Budget Expenditure Total Expendi ture (%) Yr 1 Yr 2 Yr 3 Total Begin End Oct 11- Jul 12 Aug 12- Jul 13 Aug 13- Jun 14 Oct 11- Jun 14 15 605170 / 0634.0016 Namuna Integrated Development Counsil Integrated FP, HIV prevention, care and treatment services for FSWs, clients of FSWs and PLHIV in Rupandehi and Kapilvastu districts 1-Oct￾2011 30-Jun￾2016 579,802 68,328.89 97,181.25 96,824.42 262,335 45% 16 605171 / 0634.0017 National Federation of Women Living with HIV and AIDS (NFWLHA) Capacity strengthening of network of women living with HIV in Nepal 1-Oct￾2011 30-Jun￾2016 136,143 25,112.33 26,865.11 21,988.30 73,966 54% 17 605172 / 0634.0018 Social Awareness Center Expanded community and home￾based care services for PLHIV in Surkhet district 1-Oct￾2011 30-Jun￾2016 120,863 17,824.06 23,291.54 18,668.39 59,784 49% 18 605173 / 0634.0019 Sparsha Nepal Integrated HIV treatment, care and support services for PLHIV in Lalitpur district 1-Oct￾2011 30-Jun￾2016 165,049 26,191.95 30,401.24 27,501.02 84,094 51% 19 605174 / 0634.0020 Community Action Center (CAC-Nepal) Integrated FP, HIV prevention, care, treatment services for FSWs, clients of FSWs and PLHIV in Bhaktapur district 1-Oct￾2011 30-Jun￾2016 439,097 63,183.39 71,957.58 76,393.91 211,535 48% 20 605175 / 0634.0021 Institute of Community Health HIV and STI prevention and FP promotion for FSWs and their clients in Banke, Bardiya and Dang districts 1-Oct￾2011 31- May￾2014 190,848 59,470.17 66,392.05 59,433.62 185,296 97% 21 605176 / 0634.0022 JHPIEGO JHPIEGO: Saath-Saath Project 1-Oct￾2011 31-Oct￾2013 694,378 - 488,983.8 5 205,393.84 694,378 100% 22 605177 / 0634.0023 NauloGhumti Nepal (NGN) Integrated treatment, care and support and FP services for FSWs, clients of FSWs and PLHIV in Kaski district 1-Oct￾2011 30-Jun￾2016 658,199 80,377.34 128,615.8 5 107,337.39 316,331 48% 23 605178 / 0634.0024 Nepal National Social Welfare Association Integrated treatment, care, support and FP services for FSWs, clients of FSWs and PLHIV in Kailali and Kanchanpurdistricts 1-Oct￾2011 30-Jun￾2016 407,176 65,867.97 72,706.76 68,471.83 207,047 51% 24 605179 / 0634.0025 Rural Development Foundation (RDF) HIV and STI prevention and FP promotion for FSWs and their 1-Oct￾2011 30-Jun￾2016 302,701 45,514.82 53,173.97 50,287.51 148,976 49% S.N FCO/ID# Implementing Partners Project Title Period of Performance LOP Budget Expenditure Total Expendi ture (%) Yr 1 Yr 2 Yr 3 Total Begin End Oct 11- Jul 12 Aug 12- Jul 13 Aug 13- Jun 14 Oct 11- Jun 14 clients in Dhanusha, Mahottari and Sarlahi Districts 25 605180 / 0634.0026 Asha KiranPratisthan, Kailali Expanded community and home￾based care services for PLHIV in Kailali district 1-Oct￾2011 30-Jun￾2016 251,618 36,083.34 51,733.36 40,609.95 128,427 51% 26 605181 / 0634.0027 Dharan Positive Group Expanded community and home￾based care services for PLHIV in Sunsari district 1-Oct￾2011 30-Jun￾2016 107,435 18,104.24 19,031.70 17,823.72 54,960 51% 27 605182 / 0634.0028 Gangotri Rural Development Forum, Achham Expanded community and home￾based care services for PLHIV in Achham district 1-Oct￾2011 30-Jun￾2016 254,594 36,897.31 48,363.04 44,321.26 129,582 51% 28 605183 / 0634.0029 General Welfare Pratisthan (GWP) Integrated FP, HIV prevention, care, treatment services for FSWs, clients of FSWs and PLHIV in Makwanpur, Bara and Rautahat districts 1-Oct￾2011 30-Jun￾2016 755,292 83,840.23 113,529.9 0 179,083.59 376,454 50% 29 605184 / 0634.0030 Junkiri, Banke Expanded community and home￾based care services for PLHIV in Banke district 1-Oct￾2011 30-Jun￾2016 123,342 18,481.13 25,602.66 18,092.40 62,176 50% 30 605185 / 0634.0031 NariChetnaSamaj (NCS) HIV and STI prevention and FP promotion for FSWs and their clients in Lalitpur district 1-Oct￾2011 30-Jun￾2016 217,775 29,390.21 35,060.20 34,102.97 98,553 45% 31 605186 / 0634.0032 STI/AIDS Couseling and Training Services (SACTS) Integrated treatment, care, support and FP services for FSWs, clients of FSWs and PLHIV in Kathmandu and Lalitpur districts 1-Oct￾2011 30-Jun￾2016 454,478 61,602.37 71,464.65 78,202.77 211,270 46% 32 605187 / 0634.0033 Thagil Social Development Association HIV and STI Prevention and FP promotion for FSWs and their clients in Kailali and Kanchanpur districts 1-Oct￾2011 30-Jun￾2016 259,078 40,306.13 41,552.29 43,561.82 125,420 48% 33 605188 / 0634.0034 Student Awareness Forum (BIJAM) Integrated FP, HIV prevention, care and treatment services for FSWs, clients of FSWs and PLHIV in Parsa district 1-Oct￾2011 30-Jun￾2016 337,416 56,030.49 63,014.20 53,456.68 172,501 51% 34 605189 / National Association of Capacity strengthening of national 1-Oct- 30-Jun- 128,725 48% S.N FCO/ID# Implementing Partners Project Title Period of Performance LOP Budget Expenditure Total Expendi ture (%) Yr 1 Yr 2 Yr 3 Total Begin End Oct 11- Jul 12 Aug 12- Jul 13 Aug 13- Jun 14 Oct 11- Jun 14 0634.0035 PLHA in Nepal PLHIV network in Nepal 2011 2016 266,131 40,125.71 48,473.28 40,126.38 35 605190 / 0634.0036 Nepal STD & AIDS Research Center Integrated treatment, care, support and FP services for FSWs, clients of FSWs and PLHIV in Banke, Bardiya and Dang districts 1-Oct￾2011 30-Jun￾2016 618,959 71,969.44 78,382.92 64,561.45 214,914 35% 36 605191 / 0634.0037 Recovering Nepal Capacity strengthening of recovering drug users' network in Nepal 1-Oct￾2011 30-Jun￾2016 111,287 18,842.31 18,766.61 18,295.87 55,905 50% 37 605192 / 0634.0038 Sahara Nepal HIV and STI prevention and FP promotion for FSWs and their clients in Jhapa, Morang and Sunsari Districts 1-Oct￾2011 30-Jun￾2016 413,873 64,631.81 76,007.40 69,581.08 210,220 51% 38 605193 / 0634.0039 Association of Medical Doctors in Asia AMDA Saath-Saath Project 1-Oct￾2011 30-Jun￾2016 1,417,46 3 216,631.7 0 270,423.5 8 255,138.06 742,193 52% 39 605194 / 0634.0040 SAHAVAGI Integrated FP, HIV prevention, care and treatment services for FSWs, clients of FSWs, migrants and PLHIV in Chitwan and Nawalparasi districts 1-Oct￾2011 30-Jun￾2016 708,593 77,484.15 98,443.36 134,117.02 310,045 44% 40 605197 / 0634.0043 New ERA Ltd Baseline FP Survey including rapid assessment of HIV, STIs and FP Situation among Migrant Couple in Bara 1-Jul￾2012 15-Dec￾2012 63,871 34,879.47 28,991.61 - 63,871 100% 41 605196 / 0634.0042 IndreniSamaj Kendra (ISK) Integrated FP, HIV prevention, care and treatment services for migrants and spouse of migrants in Palpa district 1-Apr￾2012 30-Jun￾2016 369,705 30,194.97 70,608.15 66,215.47 167,019 45% 42 605197 / 0634.0043 Conscious Media Forum Radio proram for HIV prevention and FP promotion among migrants and their spouses in Palpa, Kapilvastu, Nawalparasi and Bara Districts 1-Apr￾2012 30-Sep￾2014 79,099 9,937.97 31,608.58 27,243.67 68,790 87% 43 605198 / 0634.0044 Chandra Jyoti Integrated Rural Development Integrated FP, HIV Prevention, care, treatment services for FSWs 1-Apr￾2012 30-Jun￾2016 232,985 22,999.17 46,948.91 37,598.52 107,547 46% S.N FCO/ID# Implementing Partners Project Title Period of Performance LOP Budget Expenditure Total Expendi ture (%) Yr 1 Yr 2 Yr 3 Total Begin End Oct 11- Jul 12 Aug 12- Jul 13 Aug 13- Jun 14 Oct 11- Jun 14 Society (CIRDS), Dhading and lcients of FSWs in Dhading district 44 605199 / 0634.0045 South Asia Institute for Policy Analysis and Leadership Vulnerable Youth Mapping and Behavioral Study in Kathmandu, Kaski and Sunsari Districts of Nepal 1-Aug￾2013 31-Dec￾2013 16,108 - - 16,107.94 16,108 100% 45 605200 / 0634.0046 Right Direction Nepal Sexual Network of Female Sex Workers (FSWs) in Kathmandu Valley, Jhapa and Kailali Districts of Nepal 1-Aug￾2013 31-Dec￾2013 11,595 - - 11,595.43 11,595 100% Total 12,866,0 09 1,723,769 2,683,285 2,338,055 6,745,110 52% Source: SSP, June 2014 ANNEX 3: ACHIEVEMENT AGAINST SSP M&E PLAN INDICATOR TARGETS Achievement against SSP M&E Plan Indicator Targets Note: Y1 = FY12 (10 months); Y2 = FY13; Y3 = FY14 ND = No data; NA = Not applicable Outcome and Indicators Level Base line (Year) Target Achievement Achievement % Means of Verificatio n Remarks and Justification of Progress Y1 Y2 Y3 Y1 Y2 Y3 Y1 Y2 Y3 HIV prevalence among FSWs Impact Ktm:1.7%; Pkr. 1.2 (FY11) <2% <2% <2% 1% ND ND NA NA NA IBBS Achievement from IBBS Terai (2012) CPR among migrant couples Outcom e 22.5% (2010) 25% 27.50% 30% 24% ND ND NA NA NA Baseline Achievement from baseline 2012 Outcome 1: Decreased HIV prevalence among selected KAPs Output 1.1: Reduced HIV and STI risk among FSWs in selected districts Percent of FSWs reporting the use of a condom with their most recent client* Outcom e PKR 78.8% (FY11 80% 82% 84% ND ND ND NA NA NA IBBS Percent of FSWs who say they consistently use a condom when they have sex with clients* Outcom e PKR 61.4% (FY11) ND ND ND NA NA NA NA NA NA IBBS Number of FSWs receiving STI treatment by age Output 6,778 (FY11) Routine program data Targets included under output 4.2 Number of Clients of FSW receiving STI treatment by age Output 939 (FY11) Routine program data Targets included under output 4.2 Number of FSWs who have accessed HIV testing services and received their results in the last 12 months Output 6,680 (FY11) Routine program data Targets included under output 4.2 Number of Clients of FSW who have accessed HIV testing services and received their results in the last 12 months Output 10,396 (FY11) Routine program data Targets included under output 4.2 Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and Number of FSWs reached with individual or small group interventions that are based on evidence and/or meet the minimum standards required Output 25,263 (FY11) 22,000 26,000 28,000 22,055 25,873 26,513 100% 100% 95% Routine program data FY14 achievement is for 11 months only Number of Clients of FSW reached with individual or small group interventions that are based on evidence and/or meet the minimum standards required* Output 58,236 (FY11) 50,000 55,000 58,000 49,319 69,631 69,031 99% 127% 119% Routine program data Target for clients was set by multiplying a factor of 2.3 but achievement was 2.5 times higher. Number of condoms distributed to FSWs (in million) Output 3,574,524 (FY11) 2.97 3.74 4.03 3.04 4.28 4.62 102% 114% 115% Routine program data The achievements includes condoms distributed to both FSWs and their Clients Number of individuals trained to promote HIV/AIDS prevention through other behavior change beyond abstinence and/or being faithful disaggregated by sex Output 664 (FY11) 1,000 1600 1,000 1,286 839 3,800 129% 52% 380% Training reports Overachievement in FY14 was due to FCHV training planned in FY 13 were carried out and completed in this year. Output 1.2: Reduced HIV and STI risk among migrants in selected districts Percentage of migrants reporting the use of a condom with most recent partner (FSW)* Outcom e MFW 50% (FY06) 60% 70% 75% 80% ND ND NA NA NA Baseline Achievement from baseline 2012 Percentage of migrants who say they consistently use a condom when they have sex with FSWs* Outcom e MFW 50% (FY06) 60% 70% 75% 73% ND ND NA NA NA Baseline Achievement from baseline 2012 Number of male migrants and their spouses receiving STI treatment by age Output 724 (FY11) ND ND ND NA NA NA NA NA NA Targets included under output 4.2 Number of male migrants and their spouses who have accessed HIV testing services and received their results in the last 12 months Output 4,708 (FY11) ND ND ND NA NA NA NA NA NA Targets included under output 4.2 Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and Number of male migrants and their spouses reached with individual or small group interventions that are based on evidence and/or meet the minimum standards required* Output 56,738 (FY11) 17,000 34,000 40,000 4,980 39,022 53,031 29% 115% 133% Routine program data Due to expansion of migrant program focus in program districts, achievement of overage of migrants and their spouse is increasing each year. Number of condoms distributed to migrants and their spouses (in million) Output 574,196 (FY11) 0.38 0.81 0.96 0.06 0.25 0.36 16% 31% 38% Routine program data Number of condoms distributed among migrants and wives is increasing but slow progress was due to low demand among women whose husband is away. Output 1.3: Increased knowledge about HIV transmission and service delivery sites among selected KAPs Percentage of FSWs who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission* Outcom e Terai: 26.7% (FY09) 35% 40% 45% 18.5% ND ND NA NA NA IBBS Achievement from IBBS Terai (2012) Percentage of male migrants who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission* Outcom e MFW: 15.8% (FY08) 20% 25% 35% 17% ND ND NA NA NA Baseline Achievement from baseline 2012 Percentage of wives of migrants who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission* Outcom e 12.5% (FY10) 20% 25% 35% 13% ND ND NA NA NA Baseline Achievement from baseline 2012 Outcome 2: Increased use of family planning services among KAPs Output 2.1: Strengthened integration of FP counseling and information services within HIV services Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and Percent of FSWs using any FP modern method* Outcom e Terai: 62.5% (FY09) 70% 80% 85% 60% ND ND NA NA NA IBBS Achievement from IBBS Terai (2012) Percent of migrants wives using modern FP methods* Outcom e ND ND ND NA NA NA NA NA NA This was included under CPR indicator study. Number of individual provided any FP method, by method Output NA 1,000 3,000 4,000 0 462 1,876 0% 15% 47% Routine program data Program initiated only from the end of FY13. Percent of USG-assisted service delivery points(SDPs) that experience a stock out at any time during the defined reporting period of any contraceptive method that the SDP is expected to provide Output NA 0 0 0 ND 0 0 NA 100% 100% LMIS Reports Number of people that have been trained in FP/RH with USG funds Output 3 (FY11) 300 50 300 308 352 4,537 103% 704% 1512% Training reports Overachievement in FY14 was due to FCHV training conducted in 4 district carried from FY13 Couple Years Protection in USG supported program Output 29,000 (FY11) 25,000 33,000 38,000 25,982 40,031 ND 104% 121% NA Routine Program Data Achievement for FY13 was calculated using the conversion factor provided by USAID. Output 2.2: Integrated FP and HIV messages into community outreach Number of HIV-related service delivery points (SDPs) with FP/HIV integrated services* Output 2 (FY11) 44 49 50 2 50 56 5% 102% 112% Routine program data Service sites were expanded as per program needs Number of additional USG￾assisted community health workers (CHWs) providing family planning (FP) information and/or services during the year Output 5 (FY11) 39 39 47 5 47 ND 13% 121% NA Routine program data It is the count of trained paramedical staff and counselors at EIHS sites. FY14 result is not ready yet. Output 2.3: New program approaches developed to reach migrant families with FP and HIV information and counseling, and access to services Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and Number of community outreach activities with integrated HIV/FP messages by type of activity.* Output NA 200 2,000 2,000 205 2,480 ND 103% 124% NA Routine program data Radio Episodes, RLGs, PGDs were counted. FY14 results is not ready yet. Outcome 3: Increased GON capacity to plan, commission and use strategic information Output 3.1: Improved capacity at GON to plan, commission and supervise HIV/AIDS essential data collection using appropriate research Functional plan in place to commission IBBSs* Output NA 1 1 1 1 1 1 NA NA NA Meeting updates Surveillance Strategies and Plan are functional Output 3.2: Improved capacity within the NCASC to conduct Second Generation HIV surveillance in accordance with national plans Number of SGS training curriculum in place Output NA 1 1 1 0 1 1 NA NA NA Meeting updates IBBS curriculum is in place Output 3.3: Improved capacity in NCASC and DACCs to analyze and use strategic information on HIV/AIDS for planning and management purposes Meetings held to promote HIV database integration with HMIS Output NA Yes Yes Yes Yes Yes Yes NA NA NA Meeting updates HIV reporting system is integrated into HMIS Number of government staff trained in strategic information (includes M&E, surveillance, and HMIS)* Output NA 210 325 150 139 382 219 66% 118% 146% Training Report In FY12, database related training not carried out. In FY14, more training Data Analysis, GIS training conducted in coordination with NCASC Output 3.4: Periodically updated national research agenda with involvement from all stakeholders including NGOs Rigorous ethics review process in place at NHRC applied to both government and non￾government research proposals* Outcom e NA Yes Yes Yes Yes Yes Yes NA NA NA Meeting updates All studies have gone through standard review process in time. Outcome 4: Increased quality and use of HIV services Output 4.1: Improved quality of clinical management of HIV positive people institutionalized Number of Health Care Workers completed an in-service training program* Output 230 (FY11) 80 110 120 374 239 167 468% 217% 139% Training report Y1 = achievement is high due to caregivers training by CHBC partners, and CHBC basic and refresher training for NGOs and Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and PMTCT training in Sunsari and Kailali Number of health workers correctly using the clinical protocols for HIV/AIDS management * Output NA 39 39 39 39 47 ND 100% 121% NA QA/QI checklist Technical review reports says all clinical staff adhering to the standards Number of individuals trained in HIV-related stigma and discrimination reduction* Output 10,678 (FY11) 7,000 7,500 8,000 8,128 10,923 9,196 116% 146% 115% Training reports Over 40 NGOs report these training from 33 districts. Marginal overachievement is due to more people have attended trainings Number of individuals trained in medical injection safety Output 36 (FY11) 90 300 50 41 215 41 46% 72% 82% Training report Training conducted as per plan, due to having less number of participants, achievement is low in this training Number of SDPs with safe Medical Waste Management system in place Output 40 (FY11) 46 46 52 48 52 57 104% 113% 110% Service site mapping Due to expansion of new HTC/STI sites, it is more than 100% achievement Output 4.2: Improved systems for sustaining NGO community care and support services Percentage of NGOs leveraging government and other donor financial support Outcom e NA NA NA NA NA NA NA NA NA NA Program reports Targets was set for end of project year. So it is not measured now. Number of individuals trained in HIV-related institutional capacity development* Output 359 (FY11) 1,000 1,100 800 731 1,389 898 73% 126% 112% Training report After TOCAT, more trainings were planned in Y2 of the project. Number of individuals trained in counseling and testing according to national and international standards Output 35 (FY11) 40 20 40 48 45 46 120% 225% 115% Training report Two training on HTC carried out each year. Number of outlets providing STI management and treatment Output 39 (FY11) 44 44 50 46 50 55 105% 114% 110% Service Site Due to expansion of sites, the achievement Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and services Mapping is more than 100% Number of people who receive STI diagnosis and treatment services Output 9,106 (FY11) 7,000 8,000 8,500 7,224 9,421 10,332 103% 118% 122% Routine program data Number of outlets providing VCT services* Output 40 45 45 51 47 51 56 104% 113% 110% Service Site Mapping Due to expansion of sites, the achievement is more than 100% Number of people who received counseling and testing and results* Output 24,983 (FY11) 17,000 22,000 24,000 16,310 22,704 24,950 96% 103% 104% Routine program data Number of outlets for palliative care* Output 41 (FY11) 46 46 52 48 52 57 104% 113% 110% Service Site Mapping Number of HIV+ adults and children receiving a minimum of one clinical service* Output 5,856 (FY11) 5,500 6,000 6,500 5,531 6,390 6,770 101% 107% 104% Routine program data Number of People Living with HIV/AIDS (PLHIV) reached with a minimum package of Prevention with PLHIV (PwP) interventions Output 1,190 (FY11) 3,000 3,300 3,600 3,036 3,544 3,681 101% 107% 102% Routine program data Output 4.3: Improved uptake and quality of Community-Based Prevention of Mother-to-Child Transmission (CB-PMTCT) in selected districts Percentage of CB-PMTCT scale up districts receiving CHBC team support among those districts overlapping with Saath￾Saath Project CHBC services districts Output NA 100% 100% 100% 100% 100% 100% NA NA NA National reports Outcome 5: Strengthened coordination among all HIV/AIDS partners Output 5.1: Improved process for joint planning and harmonization of HIV/AIDS strategic planning and review NAP and AWP plans citing SI resources as an evidence base Outcom e NA 1 NA 1 0 NA 0 NA NA NA National reports Output 5.2:Improved systems within the GON established for tracking and monitoring HIV/AIDS activities within all GON ministries Existence of national level working group on HIV-related Outcom e NA Yes Yes Yes ND ND ND NA NA NA Meeting records No HSEB Functional Outcome and Indicators Level Base line Target Achievement Achievement % Means of Remarks and data activities Output 4.3: Improved uptake and quality of Community￾Based Prevention of Mother￾to-Child Transmission (CB￾PMTCT) in selected districts Achham, Kailali and Sunsari Outcome 5: Strengthened coordination among all HIV/AIDS partners Output 5.1: Improved process for joint planning and harmonization of HIV/AIDS strategic planning and review National Strategy, National M&E and Surveillance Guidelines Output 5.2:Improved systems within the GON established for tracking and monitoring HIV/AIDS activities within all GON ministries No HSEB Functional Source: SSP, June 2014 ANNEX 4: LIST OF DISTRICTS FOR FIELD VISIT AND MINI￾SURVEY QUESTIONNAIRE List of districts for field visit and mini-survey questionnaire Component wise districts of Saath-Saath Project Saath-Saath Project-- NationalProgra m OtherProgramforCollaboration Region District Code DistrictName FSW and clients (prevention including FP promotion and EIHS) Micro planning SMS JMMS CBOsdi stricts Safer & Healthy Workplac e Activity for FSWs SSP Migrants (Preventio n and EIHS) FP counseling and provide contracepti ves thru EIHS CHBC PP ART Sites PMTCT Sites H4L GGMS districts Suaahara districts Education & IG projectdist ricts Anti￾traffickingpr ojectdistrict s Eastern 04 Jhapa X X X X X 1 4 05 Morang X X X X X 1 1 06 Sunsari X X X X X X X X 1 2 Central 17 Dhanusa X X X X 1 4 18 Mahottari X X X 19 Sarlahi X X X X 24 Kavrepalancho k X X 1 1 X X 25 Lalitpur X X X X 26 Bhaktapur X X X X 1 1 27 Kathmandu X X X X X X X 6 7 X 28 Nuwakot X X 1 4 X X 30 Dhading X X X X 31 Makwanpur X X X X 1 1 X 32 Rautahat X X X 1 1 33 Bara X X X X X 1 1 34 Parsa X X X X X X 1 1 35 Chitawan X X X X X X X 1 3 Wesste rn 38 Tanahu X X X X 1 2 X 39 Syangja X X 1 1 X X Component wise districts of Saath-Saath Project Saath-Saath Project-- NationalProgra m OtherProgramforCollaboration Region District Code DistrictName FSW and clients (prevention including FP promotion and EIHS) Micro planning SMS JMMS CBOsdi stricts Safer & Healthy Workplac e Activity for FSWs SSP Migrants (Preventio n and EIHS) FP counseling and provide contracepti ves thru EIHS CHBC PP ART Sites PMTCT Sites H4L GGMS districts Suaahara districts Education & IG projectdist ricts Anti￾traffickingpr ojectdistrict s 40 Kaski X X X X X X X 1 4 45 Baglung X X 1 3 X X 47 Palpa X X 1 1 X 48 Nawalparasi X X X X X X X 2 X 49 Rupandehi X X X X X 1 1 X 50 Kapilbastu X X X X X 1 1 X Mid-west 56 Dang X X X X X X 1 3 X X 57 Banke X X X X X X X 1 1 X X X 58 Bardiya X X X X 1 1 X X 59 Surkhet X X 1 5 X X X Far West 69 Achham X X 2 1 X X 70 Doti X X 1 3 X X 71 Kailali X X X X X X 2 2 72 Kanchanpur X X X X X 1 1 X Total districts 25 25 9 19 3 4 26 20 13 35 63 5 10 4 4 5 Note: Blue: 6 Field visitdistricts Yellow: 10 Surveydistricts Orange: 5 more surveydi stricts Updated : June 27, 2014 ANNEX 5: SUMMARY OF IAs – TOCAT PRINCIPAL CAPACITY DOMAINS PHASE II Summary of IAs- TOCAT principal capacity domains Phase- II 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 Number, Mix and Capacity of Technical Staff Technical Quality Standards Technical Supervision Internal Training and Mentoring Cl i ent Communi cations Community Involvement Service Delivery Organizati on Quality Assuran ce Referral System for Continuum of Prevention, Care and Support Services Technical Program Planning and M&E Organizati onal Structure and Systems Managem ent Practices Operation al Planning Structure: Roles and Responsib i l i ti es Structure: Delegation of Authority and Deci si on￾Making Staffing and Human Resource Managemen t Regi onal Presence / Geograph ical Capacity Partnering and Networking Adequacy of Physical Infrastructure Effective External Rel ati ons Sustainability Foundation Financial Planning and Budgeting Cash and Banking Accounting and Record Keeping Procurement Quality Assurance for Critical Equipment and Supplies Data Col l ecti on System Data Use and Culture of Information Stakeholder Communicatio ns and Reporting S.N. IA 1 GWP 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 RDF 3 4 4 4 4 4 4 3 4 4 2 4 4 4 4 4 4 4 3 3 3 4 4 4 4 3 4 4 4 3 CIRDS 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 2 4 4 4 4 RN 3 4 3 3 3 3 3 3 2 3 3 4 4 3 3 4 4 4 3 4 3 4 4 4 3 3 4 3 4 5 JMMS 3 4 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 3 3 4 4 4 4 4 4 4 4 4 6 Dristi Nepal 3 3 2 3 4 3 4 2 3 3 4 2 3 3 3 3 4 3 3 2 4 4 4 4 3 4 4 4 4 7 NFWLHA 3 2 3 3 4 4 3 4 3 3 4 3 3 4 3 4 4 3 3 2 3 3 4 4 2 2 4 4 3 8 SACTS 4 4 4 4 4 4 4 4 4 3 4 4 3 4 4 3 4 4 4 3 3 3 4 4 3 3 4 4 4 9 TP 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 3 4 4 4 4 4 4 4 4 10 CWES 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 3 3 4 4 3 3 4 4 4 4 4 4 4 4 11 CMS 3 4 4 3 4 3 4 3 4 4 4 3 4 4 4 3 3 4 3 3 3 4 4 4 4 3 4 3 4 12 ISK 3 4 4 4 4 4 4 3 4 4 4 4 4 4 4 4 4 4 3 4 4 4 4 4 4 1 4 4 4 13 N'SARC 3 4 4 4 4 3 4 3 4 4 3 3 2 4 4 4 4 4 4 4 3 3 4 4 4 3 3 3 3 14 SAC 3 3 4 3 4 3 4 4 4 4 4 3 4 4 4 3 4 4 4 3 3 4 4 4 4 4 3 3 3 15 TSDA 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 3 4 3 4 4 4 4 4 4 4 4 4 16 GaRDeF 4 4 4 4 4 4 4 4 4 3 4 4 4 4 4 4 4 4 4 4 3 4 4 4 4 3 4 4 4 17 CMF 3 3 3 4 4 4 4 4 3 4 3 3 4 3 2 4 4 3 3 3 4 4 4 4 3 3 3 3 Principal Capacity Domains Core Functional Areas 1. A Institutional and Programmatic: Technical Capacity 1 B. Institutional and Programmatic: Organizational Capacity 2. Financial Management Systems pp y Management 4. Monitoring for Management ANNEX 6: QUANTITATIVE ANALYIS OF FSW FGD RESPONSES TO QUESTIONS # 9 AND # 10 QUANTITATIVE ANALYIS OF FSW RESPONSES TO QUESTIONS # 9 AND # 10 AT THE END OF THE FSW FOCUS GROUP DISCUSSONS. Methods SPSS software is used for analysis of the data. Code number given to each rating scales (Poor=1, Good=2, Very Good=3 By using descriptive statistics tool of data analysis, frequencies are derived along with respective percentages. # 9. How would you rate/assess the quality of the services that you have received in the past two years? Information Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 0 0.0 0.0 0.0 Very Good 13 100.0 100.0 100.0 Total 13 100.0 100.0 Counseling Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 1 7.7 7.7 7.7 Very Good 12 92.3 92.3 100.0 Total 13 100.0 100.0 Testing Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 1 7.7 7.7 7.7 Very Good 12 92.3 92.3 100.0 Total 13 100.0 100.0 Treatment Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 1 7.7 7.7 7.7 Very Good 12 92.3 92.3 100.0 Total 13 100.0 100.0 # 10. How would you rate the following aspects of the services at the clinic/DIC you go to? Short waiting time Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 1 7.7 7.7 7.7 Very Good 12 92.3 92.3 100.0 Total 13 100.0 100.0 Friendly and warm communications by staff? Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 0 0.0 0.0 0.0 Very Good 13 100.0 100.0 100.0 Total 13 100.0 100.0 Respectful interaction with you by staff? Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 0 0.0 0.0 0.0 Very Good 13 100.0 100.0 100.0 Total 13 100.0 100.0 Adequate medicine and treatment supplies? Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 0 0.0 0.0 0.0 Very Good 13 100.0 100.0 100.0 Total 13 100.0 100.0 Follow up after treatment? Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 2 15.4 15.4 15.4 Very Good 11 84.6 84.6 100.0 Total 13 100.0 100.0 Other aspects? Frequency Percent Valid Percent Cumulative Percent Poor 0 0.0 0.0 0.0 Good 0 0.0 0.0 0.0 Very Good 4 30.8 30.8 30.8 N/A 9 69.2 69.2 100.0 Total 13 100.0 100.0 ANNEX 7: EVALUATION RESPONSES FROM DPHOs AND IAs Evaluation Areas DPHO IA 1. Model effectiveness How does the current SSP model in the four areas of capacity building provide the optimum in service delivery? How will this model be effective in providing optimum service delivery under Nepal’s new “test and treat” model? HIV Services • SSP has been successful in providing good quality STI testing and treatment, FP service, general health testing, HIV testing to beneficiaries • In high-risk areas like Chitwan, SSP has provided support to identify almost 1000 cases HIV positive • Identified PLHIVs are referred for CHBC services • SSP provides linkage for care &support of PLHIVs • Although target population in certain remote areas of Chitwan, and Bara District have not yet been reached, SSP has been a pioneer project in Chitwan District • HIV testing is offered after counseling and confidentiality/privacy of clients are maintained • In general, data shows that there has been considerable progress in reduction and prevention of transmission of STI/HIV among target populations • Through targeted intervention, there has been behavior change of FSWs and their clients, mainly in awareness, prevention, and reduction of high-risk behaviors. • SSP has been able to make significant progress in reaching target populations as is evident in target vs. achievement (utilizing the baseline of national estimate) • Detected HIV cases for referral to EPC and leveraging for livelihood programs. • Data is provided to GON about how many reached and treatment, more quantitative than qualitative Test, Treat and Retain • Only a few of the DPHO and DACC coordinators had heard about the new GON plans to implement “test, treat, and retain” model using a community-based approach. Hence, most of them did not have concrete plans. • However, they were enthusiastic about expanding the HIV Services • Target Group/Beneficiaries: FSWs &clients, migrant workers & their spouses, IDUs, PLHIVs • Services Provided at the Clinic - Counseling - FP services - HIV testing/counseling, [VCT] - STI/STD counseling, testing and treatment - PMTCT - Syphilis testing • Comprehensive Testing Package - EIHC - FP service, counseling - EPC package (HIV positive) - General health checkup - Linkage to ART clinics -Technical support by FHI & DPHO: PMTCT, VCT - Referrals to other clinics for TB testing - Refer for CD4 testing DIC Services • To get information about HIV and STI related services • To get more information about FP services (contraceptives: Depo￾Provera injections, implants, condoms) • Games (on correct ways to use condoms; risks and transmission of STI, HIV) • Edutainment [audio/visual] • Counseling/information • Referrals for medical checkups and treatment • Recreational activities SSP Support • FOR SOME DICs - Rent for the office Evaluation Areas DPHO IA services currently provided by the DPHO and further their reach into communities. • If the GON implements the new approach and provides guidelines, then DPHO and DACC were certain that they could work with SSP in providing the services. • They felt that their coordination with SSP would help to break the existing socio-cultural barriers within Terai and other non-receptive communities and increase the HIV program reach despite prevalent stigma and discrimination. • DPHO and DACC members emphasized the major and crucial roles of peer educators and community mobilizers to reach out to more target populations, especially in the closed communities and remote areas. • A good approach for the new model would be to make comprehensive testing available through mobile health camps. - Comprehensive Testing would be more effective than Targeted approach due to stigma and discrimination prevalent among communities. Coordination with GON • At the central level, SSP has supported data analysis, DQA, M&E • At the district level, SSP has been fully involved in service sites, VDCs • SSP has also helped to develop M&E guidelines at district level • SSP has provided support to develop a national integrated database • Support in data collection, database management, reporting and DQA • SSP has provided support to DPHO and DDC: - Technical support and financial - Salary for management/staff - Budgets for programs - Technical Support - Data management - SSP provides materials • 2 main Components of SSP in PLHIV care and treatment - EPC - CHBC (Community home-based care) Training • Field Staff training [OECM Training] - Training for Community Mobilizer (from target group) - Outreach Educator - Peer Educator (from FSW target group) - Field Supervisor - DIC Operator training - Condom Negotiation Training • Training on M&E • Financial • NGO development • Provided Training to other IAs of SSP: illustrates role as Capacity Building Organization (CBO) Administration - Documentation - Feedback Survey - Exit Interview - Quality Assurance - Scope of Work Procurement & Logistics - Quality Assurance - Cost-effectiveness Evaluation Areas DPHO IA - Reporting & planning - Sustainability analysis & action plan -Sustainability Assessment Workshop, FP/HIV Training Workshop • Coordination with SSP at the district level through National Strategy Plan • Involvement in trainings, conferences, review meetings, strategic plan, sustainable development workshop through IAs, regional review meetings [presentation HIV/data compilation] • The IAs supported by SSP submit and report to the GON: - DPHO: bi-monthly - NCASC: monthly - DDC: annually • SSP and GON correspond through [monthly feedback, quarterly review meeting] • DACC is one system they can work with in the district; coordinated effort among the partners • SSP has helped to strengthen DACC • ORE-support FCHV • SSP provides support for outreach and field visits Improvements/Suggestions • SSP has provided good support but there is still room for improvement • SSP could coordinate and provide more support to DPHO in technical, financial areas • Should be an increase in collaboration, coordination and co-operation between SSP and DPHO - Especially if the HIV program is to be expanded at the peripheral level • SSP can help the GON to expand service coverage/ accessibility - Monitoring for Management - Data Generation (improve the flow-system) - Document and Present Data - Some Supervision & Monitoring by the Govt. - DACC Coordinator - Stakeholder Meeting [held Quarterly] - Medicines, condoms (NCASC, DPHO) - Good collaboration with Govt. IMPROVEMENTS/SUGGESTIONS • Adult Literacy Program • For complete behavior change- need to offer complete package with treatment [offer alternatives and options such as links to IGP • Beneficiaries expressed their interest to have more linkages to IGPs in the forms of: - Skill- based trainings - Seed money to start their own business (tea shops, beauty parlor, small restaurant) • Main Hindrances - Illiteracy - Lack of seed money to start own business - Lack of necessary skills Test, Treat and retain • Plan to Test in Cluster Areas could be tricky, challenging to reach high-risk group/ target populations - Clients could be identified by volunteers - Health camp could be organized by DPHO in Health Posts and supported by local NGOs and CDOs • Possible Impacts - Reduction of stigma and discrimination [by mobilizing prominent political representatives, community members, volunteers] - Support of community mobilizers, outreach educators crucial Evaluation Areas DPHO IA • GON and SSP could work together on a Sustainability plan for the HIV program in the country in order to make the plan more feasible, reach more targeted areas, and maintain quality of the services • SSP could support the establishment of ART sub￾centers • Service ownership should be GON but SSP could provide technical support at periphery health services (hospital, health post) • SSP could provide trainings to low-level workers such as health post workers, social workers, educators, and others working in the field of HIV/STI prevention and reduction • Need for in-depth investigation and research on the accuracy of data provided by IAs • Need for research on the effectiveness of the SSP model in reduction of STIs transmission and prevalence • Need for GON and SSP to work together towards forming common policies SSP Drawbacks • Current model may be too rigid if GON implements the new “test, treat, and retain” model utilizing community￾based approach • SSP current documentation and reporting is labor￾intensive • Treatment plan needs to be more flexible - Improved accessibility to services in remote areas - Identification of untreated HIV positive cases - Reduction and prevention of transmission of STI/STD and HIV 2. Service delivery effectiveness High quality service delivery is the goal of SSP. How does SSP and how do Reporting • Reporting tool provided by NCASC in HIMS format [SSP must adhere to the format and report to the DPHO] Guidelines • SSP and DPHO use different guidelines – those of SSP Criteria/Protocols/Guidelines National Guidelines • NCASC guidelines • Provided by SSP on the following: - Prevention - OE - Condom Negotiation Skill Evaluation Areas DPHO IA sub- grantees assess the quality of the six services? are more detailed than those of the MOHP. The SOWs and protocols are the same and both use GON SOWs. • From the government side with support from SSP helped to develop - STI testing guidelines - SOPs - VCT guidelines - Lab testing guidelines [same model is also utilized in the government hospitals] - Education Guidelines • From the government side with support from SSP - STI testing guidelines - SOPs - VCT guidelines - Lab testing guidelines [same model is also utilized in the Govt. Hospitals] OTHERS • NCASC chart • WHO clinical staging of HIV & AIDS • For adult &adolescents Reporting • Outreach and peer group educators report to MIS • Guidelines are provided • Monthly target reporting • Field staff report to CM • CM report to outreach educator • Trend for reporting since inception - Cross-verified by MIS officer - Then Project-Coordinator - Then Program Officer at FHI - PO at FHI provide feedback - Feedback is reported to Field Staff, CM, Outreach Educators Guidance - Through trainings - Guide individually - Guidelines for training (micro-planning) - Meetings once a month - Follow-up - Report staff turnover Evaluation Areas DPHO IA 3. Capacity Building SSP efforts were aimed at strengthening the capacity of some 43 sub grantees. To what extent has the capacity in the four key areas been strengthened? SSP key tools for capacity development have been the TOCAT tool and the TOCAT corrective action plans. To what extent have these been effective? • Some DPHO staff had no knowledge of the TOCAT model used by the SSP • No TOCAT training provided to DPHO SSP provides support to: • MoHP- support • DPHO- technical support • NCASC- support • During Regional Review Meeting, IAs present data, best practice, lesson learned. DPHOs would like data collected to be shared with them. • SSP provides DACC coordinator training M&E help to GON • Data validation workshop (done by DPHO) helped to identify consistency, data quality and finalized district profiles • Monitoring system different from SSP • Need for uniformed reporting System (policy level dialogue needed) TO IAs • Capacity building - Training for Board Members - Good governance - USAID RR training - Gender inclusion Recommendations • M&E recommendation- since all services are related, M&E trainings for all components needed. • There is a need for more reporting by SSP and IAs to DPHO. Capacity Building of IAs Technical - TOCAT training - Leadership Development Training - Proposal Writing - Project Management Training - Communication Strategy training - Regular training for capacity building [Eg. MIS officer is given GIS Training] Board Members • TOCAT for Board Members - NGO Management - Good Governance - Waste Management Senior Level Staff • Best practices/lessons learned • Report writing- 5 days • Proposal writing • Leadership • Procurement • Monitoring for management • Technical • Financial policy developed • Institutional (strengthened BOD, PR relations) • Annual strategic plan SSP Support for Trainings • OCM and FP training • Condom negotiation skills • Guidelines • DIC operator training Evaluation Areas DPHO IA • Need to share info/data with NGOs at forums regularly • Need for monthly meetings with DPHO. • Prevention services of IAs are good but it is missing component for livelihood support program. If SSP could have a livelihood support component in its program, it would help to make the clients self-reliant and also strengthen the overall sustainability of the project. Moreover, it would greatly help to integrate PLHIVs and FSWs into the general population. • In the Far-West region, Livelihood Support Program is running well [cattle-herding, goat-keeping]. Therefore, there is a need for funding for these programs. • There are still 30% unidentified HIV positive cases in the area so SSP needs to build the capacity of its partner IAs to reach those unidentified target population. • There is a need for more expansion of services in many areas, 24-29 VDCs in Bara District still need to be reached. Limited access to health clinics and testing has greatly affected the scope of work and target reached. • SSP could increase access and reach through mobile clinics. Then, based on the findings of the mobile clinics about the concentration of target populations, the satellite clinics could be moved to the areas with more concentration of target groups. • SSP can help in effectively reducing and ultimately eliminating the socio-cultural barriers in the Terai Communities. Stigma and discrimination is still prevalent despite efforts so there is a need for more outreach. • Mobile camps are a great way to reach communities in remote areas. In our experience, these camps need to be publicized as providing General health services • Refresher courses SSP Technical Support • Guidelines • SOPs-executive procedure All the staff are trained in: • CHBC SOP • EPC SOP • FP SOP • PMCCT SOP Others • Economic empowerment • Leadership training • Reduce sexual exploitation training • Voters’ education TOCAT Assessment • Found gaps in work • Strategic planning • Communication strategy • Strengthened policies of the organization • Action Plan was formed which resulted in sound policies, documentation, policy documentation • After TOCAT assessment, IAs utilized various tools • Learned how to strengthen the capacity of the organization • Increased Co-ordination between Board Members and staff • SSP provided training on NGO governance Highlights • M&E • Strategic planning, human resources Evaluation Areas DPHO IA rather than specific for HIV Testing, or STI/STD camps. General testing should be provided to these areas and then high-risk groups could be screened out through risk assessments. These high-risk groups could be further referred for HIV counseling and testing. • There could be even more coordinated efforts between SSP and DPHO in mobilizing volunteers, especially from the target populations such as FSWs. This collaboration could help in building stronger networks of NGOs. FCHVs, CMs, outreach workers, and other volunteers. • Need for SSP coordinated efforts with GON, collaborative planning, strengthening and sustaining the DACC- providing access to livelihood programs in a coordinated manner, as well as links to other organizations. • PLHIVS need to be supported to be more organized to be able to present their needs? • CHBC Services are provided by Bijam in Bara District and beneficiaries (IDUs) are happy as they receive syringes, and other services. However, there is a need for such CHBC services for FSWs, MSMs. • SWAT analysis done by M&E and then the policy was formed • Regular reviews • Management meeting every 2 weeks Financial • Made TORs/more receipts for purchases (stationeries) • Procurement • Manual system/accounting system tally system using computers • Increased efficiency Budget Plan • Reviewed plan • Administration • Strategic • M&E training • Data quality assurance • Monthly assessment • Quarterly assessment Comments • Formed the Action Plan after TOCAT assessment. However, some questions not relevant to NGO culture in Nepal • TOCAT assessment was time-consuming • Knowledge provided by training was applied to other fields/ and others • Helped to minimize errors • Refresher courses • Condom negotiation skills (eg: rotten apple) • Regular monitoring by SSP - Check mistakes and give feedbacks - Give recommendations • Due to trainings, staff have become technically strong in their analysis Evaluation Areas DPHO IA 4. Best Practices • Counseling in general hospitals are mostly poor. However, counseling in VCT, STI clinics is very good. • Efficient system for case finding and tracking people • Good networks-IAs have helped to identified 612 FSWs • Capacity assessment and development are good • SOPs and protocols conform with those guidelines and protocols provided by the GON. Best Practices According to IA Staff • Homogeneous and good coordination • SSP documentation is good • SSP conducts a lot of trainings so even the illiterate staff have built confidence and knowledge (mostly peer educators) • Good quality of service • Planned group discussion • Trafficking in person (TIP) referral • Linkages to Income Generation Program - Leveraging/referrals for IGP - Help in reducing partners - Although there is no certainty and guarantee about leaving sex work - Some FSWs faced discrimination at IG work so some returned back to sex work • Support in micro-planning • In the clinical side, client satisfaction survey is done along with quality assurance checklist provided by SSP on a fortnightly basis According to Beneficiaries • Services are free of cost • Confidentiality and privacy are maintained • Community mobilizers are selected from the target group (FSW) and trained • Peer group educators are also recruited from target group (FSWs) • Some beneficiaries are chosen for the Positive Speaker Bureau • Seed money for small business and to earn a livelihood (cattle rearing, beauty parlor, teashop, attend coaching classes, farming) • Leveraging for trainings in IG activities (beauty parlor, henna, sewing, embroidery) • Skill-based trainings have been provided to daughters of FSWs so that they do not become sex workers themselves Evaluation Areas DPHO IA • During post-test counseling, written consent of the client is needed for partner disclosure and only then the spouse or relatives can be informed about the client’s status • Provide educational scholarship to children of FSWs • Allowance is given for uniform (group 12-25 years old), stationary for children of FSW ANNEX 8: TOCAT ROLLOUT PROCESS AND STEPS ANNEX 9: SAATH SAATH PROJECT DATA FLOWCHART ANNEX 10: SAATH SAATH PROJECT DATA MANAGEMENT INFORMATION SYSTEM FRAMEWORK ANNEX 11: MAP OF SSP COVERAGE ANNEX 12: DISTRICT-WIDE TARGETS VS. ACHIEVEMENT ON KEY INDICATORS Number of individualsreachedthroughcommunityoutreachthatpromotes HIV prevention Indicators Bara Bhaktapur Chitwan Jhapa Lalitpur Morang Parsa T A T A T A T A T A T A T A Number of individualsreachedthroughcommunityoutreach 861 846 4,279 4,006 1,279 1,394 1,207 1,317 3,859 3,547 698 754 861 1,057 Number of individuals who received test results 282 283 2,251 2,817 598 602 561 595 648 799 419 381 452 468 Number of individuals examined for sexually transmitted infections (STI) 296 287 2,261 2,818 626 621 570 602 769 861 437 390 457 470 Number of individuals diagnosed and treated for STI 283 282 2,107 2,791 619 602 567 596 758 847 431 384 447 447 Number of FSWs diagnosed HIV positive NA 2 NA 13 NA 2 NA 12 NA 4 NA 2 NA 7 Note: T = Target for Oct 2011- Jul 2014 A= Achievement for Oct 2011- Jun 2014 NA = Not Applicable ANNEX 13: IMPLEMENTING AGENCY SURVEY DATA MINI-SURVEY QUESTIONNAIRE FOR IAS Number of IA staff participated in the mini survey Component Percentage Male 67.9 Female 32.1 Total 100.0 Out of the total 28 IA’s staffs that participated in the mini survey, 69% of them were male and 32.1% were female. As the survey shows more male staff representation, it is obvious that males dominate the staff compositions of districts. Relevance and effectiveness of Model as perceived by IAs The kinds of services provided to FSW supported by SSP Services Percentage (of total) Information for FSWs 73.3% Counseling for FSWs 93.3% Testing for FSWs 86.7% Treatment, STI and Pre-ART for FSWs 86.7% Referral-SSP supported clinics for FSWs 60.0% Others 33.3% The above table shows the kinds of services provided to FSWs by IAs through technical and financial support of SSP. 73.3 % of the IAs provide Information service through DICs, 93.3% provide Counseling services to FSWs and 86.7% provide HIV/STI Testing as well as Treatment and Pre-ART for FSWs. 60% of the IAS provide Referrals to SSP supported clinics for FSWs. 33.3% of the IAs provide Other services such as care services for PLHIV, referral to TIP related organizations, coordination with JMMS (National Network of FSWs) and local CBOs, referrals for livelihood support, Family Planning services, referrals to income generating activities as well as referrals to other service centers as per the needs of key population. Rating of the quality of Service Delivery support provided to IAs through SSP Service Areas Quality of Services Poor (%) Somewhat Poor (%) Moderate (%) Good (%) Very Good (%) a. Information b. Counseling Service Areas Quality of Services Poor (%) Somewhat Poor (%) Moderate (%) Good (%) Very Good (%) c. Testing d. Treatment, STI & pre-ART e. Referral- SSP supported c f. Other (please specify): CH Family Planning services The table above shows the ratings of the respondents on the quality of services namely, Information, Counseling, Testing, Treatment, STI & pre-ART, Referral to SSP supported clinics and Other activities. None of the respondents rated any of the service quality as poor with most of the respondents rating the quality of services as “Very Good.” Among the given service areas, both Counseling services and Treatment, STI and pre-ART, were rated as “Very Good” by 42.9% of respondents. Effectiveness of SSP Model Ef Does SSP follow government pr FSWs with your IA? Has SSP told your IA about the Strategy of the government? Has SSP explained what chang may be needed in order for you Treat and Retain Strategy? If yes, do you think SSP can pro government to implement this n According to the above table, more than half (53.5 %) of the respondents stated that SSP follows Government protocols in supporting clinical services to FSWs. With regards to Government approach of Test, Treat and Retain (TTR), only 21.43% of IAs are aware of TTR and were told by SSP. Still, more than 75% of IAs are unaware of TTR and about 29% of the IAs have not been explained by SSP about the changes needed in services and approaches in order to support the new Community Test, Treat and Retain Strategy. Effectiveness and quality of service delivery Q_2, 1 To what extent do you think quality service (prevention and treatment) are being provided to FSWs in your IA? Quality of se Perc Poor Moderately Very Effectively Not Applicable No Response Total 50% of the respondents mentioned that quality service (prevention and treatment) are being provided to FSWs “very effectively” and 28.6 % of the respondents did not provide any response to the question. 2.2 What tools or processes does SSP use to assess the quality of your IA’s services? Tools to assess quality Percentage5 MOHP/DPHO protocols and guidelines? 87.5% SSP protocols and guidelines 100.0% SSP clinical reviews 75.0% Data quality assessment 100.0% Program reports 100.0% Other (Best practices and lesson learned, Field/site visit semi-annual review meeting, supportive supervision and m Regarding the tools used by SSP to assess the quality of services in IA’s, 100% of the respondents mentioned three tools namely, SSP protocols and guidelines, Data Quality Assessment and Program Reports. 87.5% of respondents mentioned MOHP/DPHO protocols and guidelines and 75% of respondents mentioned SSP Clinical Reviews. Only 16.7% of respondents mentioned that Other tools (Best practices and lesson learned, Field/site visit (CHBC client), Quarterly review meeting, semi annual review meeting, supportive supervision and monitoring) are used by SSP to assess the quality of IA’s services. 2.3 FSWs may think that there are other aspects important in service delivery. In your opinion, how important are the following aspects to FSWs? Please check in the appropriate boxes. As Importance S 1 Short waiting times 0 7.1 53.6 39.3 2 Friendly and warm communicatio 0 3.6 60.7 35.7 3 Respectful interaction by staff 0 0 64.3 35.7 4 Clinical care provided according t 0 0 64.3 35.7 5 Adequate medications and treatm 0 0 64.3 35.7 6 Follow-up after treatment 0 0 64.3 35.7 According to respondents, four aspects in service delivery are, in their opinion, most important to FSWs : Respectful interactions by staff, Clinical care provided according to protocols, Adequate medications and treatment supplies and Follow-up after treatment. 64.3% of the respondents rated each of the above-mentioned aspects as “Very Important.” 53.6% of respondents mentioned Short waiting times as “Very Important” for FSWs. Capacity Building Effectiveness Q_3.1 Have you heard of TOCAT? 5 % of multiple responses Percentage Yes No Total The above table shows that about 97% of the respondents of IAs have heard about TOCAT. 3.1.1 If yes, in your opinion how effective has TOCAT been in strengthening your IA’s operations? Place a check mark in the appropriate column. IAs’ operations In the follow-up question for the respondents who have heard about TOCAT, about 61% of them mentioned that TOCAT has been “Effective” in strengthening IA’s operations and nearly 29% of them mentioned that TOCAT has been “Very Effective.” 3.2 Which of the following factors have helped the TOCAT’s to be effective? Please check all that apply. Factors Percentage6 Leadership Support Widespread use by the board and staff with in the IA. Staff and boards well trained in the instruments’ use Staff and boards can see improvements due to TOCAT use Other (Revised policy, procedures, strategies and guidelines; Technical, financial and 100% of the respondents mentioned that Leadership Support as the main factor and nearly 93% of the respondents mentioned Staff and boards can see improvements due to TOCAT use as the other factor that has helped the TOCAT to be effective. Only 21.4 % of the respondents mentioned Other factors (Revised policy, procedures, strategies and guidelines; Technical, financial and other training to staffs). Q_3.3 In which of the following Technical/clinical service areas has the TOCAT provided effective support? Please check all that apply. Service Areas Percentage7 Technical quality standards Client communications Community Involvement Checklist of service delivery organizations Quality assurance Referral Other {Client’s satisfaction, helped to make strategy and policy in different sector (i.e 6 % of multiple responses 7 % of multiple responses The above table shows that 100% of the respondents mentioned Quality Assurance and 92.6% mentioned Checklist of service delivery organizations as the areas that TOCAT has provided effective support. Similarly, according to 81.5% of the respondents, TOCAT has provided effective support to both Technical quality standards and Community Involvement areas. The lowest rating was given to Other service areas {Client’s satisfaction, helped to make strategy and policy in different sector (i.e. gender policy)} as only 25.9% of the respondents expressed TOCAT as being effective in these areas. 3.4 How effective have the TOCAT Action Plans been in strengthening your IA? Place a check mark in the appropriate column. Effectiveness of TOCAT Action Plans Some Your IA While rating the Effectiveness of TOCAT Action Plans, 60.7% of respondents mentioned it as “Effective” and 32.1% mentioned it as “Very Effective” in strengthening IAs. None of the respondents gave negative responses in this regard. 3.5 Which of the following SSP capacity development areas have been effective in supporting the highest quality services for FSWs in your IA? Please check all that apply. Capacity Development Areas Percentage8 Technical capacity Organizational capacity Financial Management Systems Procurement and Supply Management Monitoring for Management Other (To review the policy, to search fun activities) Don’t know According to respondents, all the SSP Capacity Development Areas have been effective and applicable in supporting highest quality services for FSWs.100% of the respondents mentioned Technical Capacity, 93.8% mentioned both Organizational capacity and Monitoring for Management, 75% mentioned Financial Management Systems and 68.8% mentioned Procurement and Supply Management as being effective in supporting the highest quality services for FSWs in their IAs. 3.6 Which of the following positive changes in your IA’s performance in the past two years do you think are due to SSP support during those years? Positive Changes Percentage9 Higher quality HIV/AIDS and family planning services Better tracking of financial resources 8 % of multiple responses 9 % of multiple responses Improved use and administration of facilities, materials, transport, etc. Improved planning, oversight and monitoring of HIV/AIDS and family planning serv Other (Best coordination and collaboration with district level stakeholders, Capacity system, to explore new project) In above table, 100% of respondents expressed that providing Higher quality HIV/AIDS and family planning services have been the positive changes in their IA’s performance in the past two years due to SSP support. 92.9% of respondents also mentioned Improved planning, oversight and monitoring of HIV/AIDS and family planning services as positive changes. 89.3% mentioned Better tracking of financial resources and 67.9% mentioned Improved use and administration of facilities, materials, transport, etc. 3.7 Which of the following negative changes are due to lack of SSP support in the past two years? Negative Changes Percentage10 Poor tracking of financial resources 16.7% Decreases in the administration and use of facilities, materials, transport, etc. 50.0% Poorer or incomplete planning, oversight and monitoring of the selected HIV/AIDS and family planning services 33.3% Others (No salary in the basis of survival, mostly in the case of field staff) 16.7% 50% of the respondents stated that there has been Decreases in the administration and use of facilities, materials, transport, etc. due to lack of SSP support in the past two years. 33.3% of respondents stated Poorer or incomplete planning, oversight and monitoring of the selected HIV/AIDS and family planning services due to lack of SSP support. 3.8 SSP sometimes works with district government offices in surveillance and M&E. Has SSP worked with the district government offices in your district on surveillance and M&E in the last two years? Percentage Yes No Not Applicable No Responses Total With regard to whether SSP has coordinated with district government offices in their districts in surveillance and M&E in the last two years, 78.6 % of the respondents stated “Yes” and only 14.3% of respondents stated “No.” 3.9 What tools and processes has SSP used to strengthen the capacity of your district’s government agencies in surveillance and M&E? Please check all that apply. Tools/Process Percentage11 Data Quality Assessment (DQA) 10 % of multiple responses 11 % of multiple responses Health Management Information System (HMIS) Conduction of surveys in the district Others Don't know The table shows that all the above- mentioned tools/processes are applicable and used by SSP to strengthen the capacity of district’s government agencies in surveillance and M&E. 88.9% of respondents mentioned Data Quality Assessment (DQA), 70.4% mentioned Health Management Information System (HMIS) and 59.3% of respondents mentioned Conduction of surveys in the district as the main tools/process used by SSP, Best Practices 4.3 How actively does your IA share your best practices with others? Place a check mark in the column. Sharing Best Practices With SSP With other IAs With DPHO With DACC The above table shows that 53.6% of IAs share their best practices with SSP, 39.3 % of the IAs share their best practices with DACC, and 35.7% of IAs share their best practices with DPHO “Very Actively.” However, only 25% of IAs share their best practices “Very Actively” with other IAs. ANNEX 14: DISTRICT OFFICIALS SURVEY General Information of Respondents Gender of Respondents Gender Frequency Percent Male 22 88.0 Female 3 12.0 Total 25 100.0 Position or organization Position Frequency Percent Chief DPHO 8 32.0 HIV Focal Person 7 28.0 DACC Coordinator 10 40.0 Total 25 100.0 District District Frequency Percent Sunsari 3 12.0 Sarlahi 2 8.0 Kavrepalanchok 1 4.0 Palpa 3 12.0 Dhading 3 12.0 Tanahu 1 4.0 Kaski 3 12.0 Dang 3 12.0 Kailali 1 4.0 Surkhet 3 12.0 Mahottari 2 8.0 Total 25 100.0 1.1 What HIV and AIDS services for Female Sex Workers (FSWs) are implemented by NGO projects supported by SSP in your district? Services Frequency Percentage12 Information for FSWs 18 100.0% Counseling for FSWs 18 100.0% Testing for FSWs 18 100.0% Treatment, STI and Pre-ART for FSWs 16 88.9% Referral-SSP supported clinics for FSWs 14 77.8% Other (CHBC, FP services, Referral to related programs e.g. 4 22.2% 12 % of multiple responses CABA, CHBC program) Not Applicable 4 No Response 3 According to district officials, Information, Counseling, and Testing services are the major services implemented by NGO projects supported by SSP in their districts. Excluding the number of Not Applicable responses along with No Response answers (28%), rest (72%) of the respondents said that aforementioned services supported by SSP are available in their districts. 88.9% of total valid respondents mentioned that Treatment of STI and pre-ART for FSWs are available in their district. Similarly, 77.8% of respondents stated that Referral services are available for FSWs. Only 22.2% mentioned that CHBC and FP services or referrals to CHBC or CABA services as well as other services supported by SSP are available. 1.2 How do you rate the quality of service delivery of the following five services in that/those SSP supported NGO? Please check in the appropriate boxes. Service Areas Quality of Services Poor (%) Somewhat Poor (%) Moderate (%) Good (%) Very Good (%) Not Applicable (%) No Response (%) g. Information 4 0 0 64 20 0 12 h. Counseling 4 0 0 64 16 4 12 i. Testing 4 0 48 32 4 12 j. Treatment, STI & pre-ART 0 0 0 72 8 4 16 k. Referral- SSP supported clinics 0 0 4 68 0 0 28 l. Other (CHBC, Positive Prevention, Family Planning, Coordination with partners, Coordination to DACC) 0 4 0 28 4 0 64 64% of district officials said Information services provided by local NGO projects supported by SSP is good, 20% very good, and 4% poor. 64% also said that Counseling service is good whereas 20% said it is very good and only 4% said it is poor. 48% of respondents said the quality of Testing service is moderate, 32% said it is good whereas only 4% said it is poor. 72% of respondents said Treatment of STI and Pre￾ART service is good and only 8% said it is very good. 68% respondents rated Referrals to SSP supported clinics as good and 4% respondents rated it as moderate. Other services such as CHBC, Positive Prevention, Family Planning, Coordination with partners, Coordination with DACC are rated good by 28% of respondents, very good by 4% and somewhat poor by other 4% of respondents. 1.3 Does SSP follow government protocols in supporting clinical services to FSWs in your district? Yes (%) No (%) Don't Know (%) Not Applicable (%) No Response (%) Total Percentage (%) Does SSP follow government protocols in supporting clinical services to FSWs in your district? 76 0 0 8 16 100 Majority of district officials (76%) said that SSP follows Government protocols in supporting clinical services to FSWs in their districts. Rest of the respondents (8%) said it is not an applicable question. 16% of respondents did not provide responses to the question. 1.4 Have you heard about the Community Test, Treat and Retain strategy of the Government? Yes (%) No (%) Don't Know (%) Not Applicable (%) No Response (%) Total Percentage (%) Have you heard about the Community Test, Treat and Retain Strategy of the government? 32 16 36 0 16 100 32% of respondents said they have heard about the Community Test, Treat and Retain strategy of the Government. 16% said they have not heard of the strategy. 36% of respondents said they do not know whereas 16% did not respond to the question. 1.4.1 If yes, do you think SSP can provide support in implementing this new strategy? Yes (%) No (%) Don't Know (%) Not Applicable (%) No Response (%) Total Percentage (%) If yes, do you think SSP can provide support in implementing this new strategy? 32 0 0 56 12 100 In a follow up question, 32% said SSP could provide support in implementing the Community Test, Treat and Retain strategy. 1.5 How could SSP support the Government’s Test, Treat and Retain strategy? Frequency Percentage13 Funding to NGOs 9 60.0% Providing Trainings 12 80.0% Monitoring progress 11 73.3% Supporting development of protocols and guidelines 9 60.0% Other (Supporting for recording and reporting, Supporting government is providing service through government institution, assisting government process) 2 13.3% According to 89% of respondents, SSP could support the Government’s Test, Treat and Retain strategy by providing trainings. 73.3% respondents stated monitoring progress, 60% respondents stated providing funding to NGOs and 60% respondents stated supporting development of protocols and guidelines. Only 13% said SSP could provide support for recording and reporting and/or support to Government by assisting them to provide services through their institutions. Effectiveness and quality of services delivery 2.1 To what extent do you think quality services to FSWs are being supported by SSP in your district? Frequency Percentage Poor 0 0 Moderately 7 28.0 Very Effectively 11 44.0 Not Applicable 3 12.0 No Response 4 16.0 Total 28 100.0 13 % of multiple responses 44% respondents said that the services provided to FSWs with the support of SSP are very effectively provided and 28% said the services were moderately provided. 2.2 What criteria do you use to assess the quality of services? Tools to assess quality Frequency Percentage14 MOHP/DPHO protocols and guidelines? 15 62.5% SSP protocols and guidelines 10 41.7% SSP clinical reviews 12 50.0% Data Quality Assessment 21 87.5% Program reports 24 100.0% Other (Coordination and collaboration with stakeholders) 1 4.2% Mostly, district officials use Program reports to assess the quality of services. All valid respondents said they use Program Reports to assess the quality whereas 87.5% said they do Data Quality Assessment, 62.5% said they use MOHP/DPHO protocols and guidelines whereas 50% said they use SSP clinical reviews and 41.7% said they use SSP protocols and guidelines. Capacity building effectiveness 3.1 Have you heard of TOCAT? Frequency Percentage Yes 6 24.0 No 19 76.0 Total 25 100.0 76% of respondents said they have not heard of TOCAT whereas 24% said they have heard of TOCAT. 14 % of multiple responses 3.1.1 If yes, in your opinion, how effective has TOCAT been in strengthening the operations of NGOs supported by SSP? Please check in the appropriate box. Servic e Areas Quality of Services Po or (% ) Some what Effecti ve (%) Mode rate (%) Effec tive (%) Very Effec tive (%) Not Applic able (%) No Resp onse (%) NGOs ’ operat ions 0 0 0 24 0 76 0 According to the 24% of respondents who have heard of TOCAT, they stated that it is effective in strengthening the operations of NGOs. 3.2 Which of the following factors have helped the TOCAT to be effective? Please check all that apply. Factors Frequency Percentage15 Leadership Support 5 83.3% Widespread use by the board and staff within the NGO. 6 100.0% Staff and boards well trained in the instrument’s use 4 66.7% Staff and boards can see improvements due to TOCAT use 4 66.7% Others 0 0% 100% of respondents said that widespread use by the board and staff within the NGO has helped the TOCAT to be effective. 83.3% stated leadership support, 66.7% stated staff and boards well trained in the instrument’s use and 66.7% stated staff and boards can see improvements due to TOCAT use, as the factors that have helped the TOCAT to be effective. 3.3 Have you seen the NGOs’ TOCAT Action Plans? 15 % of multiple responses Frequency Percentage (%) Yes 4 16.0 No 2 8.0 Not Applicable 19 76.0 No Response 0 0 Total 25 100 16% of respondents said they have seen TOCAT Action plans whereas 8% said they have not seen TOCAT Action plans. 3.3.1 If yes, how effective have they been in strengthening SSP supported NGOs in your district? Please check in the appropriate box. Effectiveness of TOCAT Action Plans Poor (%) Somewhat Effective (%) Moderate (%) Effective (%) Very Effective (%) Not Applicable (%) No Response (%) SSP supported NGOs 0 0 0 16 0 84 0 In the follow-up question, the same 16% respondents also think that TOCAT action plans are effective in strengthening SSP supported NGOs. 3.4 As part of its project, SSP is to work with Government offices in surveillance and M&E. Has SSP worked with the DPHO, DACC or other Government offices in your district on surveillance and M&E? Frequency Percentage (%) Yes 6 24.0 No 0 0 Not Applicable 19 76.0 No Response 0 0 Total 25 100 24% of respondents said SSP worked with the DPHO, DACC or other Government offices in their districts on surveillance and M&E. 3.5 What tools and processes has SSP used to strengthen the capacity of district level Government agencies in surveillance and M&E? Please check all that apply. Tools/Processes Frequency Percentage16 Data Quality Assessment (DQA) 5 83.3% Health Management Information System (HMIS) 5 83.3% Conduction of surveys in the district 4 66.7% Other (Monitoring and Evaluation) 2 33.3% Total 6 100% 83.3% of respondents mentioned that Data Quality Assessment (DQA) is used by SSP to strengthen the capacity of district level government agencies in surveillance and M&E. 83.3% of respondents also said that Health Management Information System (HMIS) is used for strengthening the GON capacity. 66.7% mentioned the conduction of surveys in the districts and 33.3% mentioned that other (Monitoring and Evaluation) tools help to strengthen the capacity of district level government agencies in surveillance and M&E. 3.6 Which of the following positive changes in your district’s NGO’s performance in the past two years do you think are due to SSP support? Please check all that apply: Positive Changes Frequency Percentage17 Higher quality HIV/AIDS and family planning services 5 83.3% Better tracking of financial resources 3 50.0% Improved use and administration of facilities, materials, transport, etc. 5 83.3% Improved planning, oversight and monitoring of HIV/AIDS and family planning services 6 100.0% Other (Monitoring and Evaluation) 2 33.3% 16 % of multiple responses 17 % of multiple responses Total 6 100% Improved planning, oversight and monitoring of HIV/AIDS and family planning services are the most common positive changes noticed by district officials due to SSP support. Similarly, 83.3% respondents stated higher quality HIV/AIDS and family planning services and 83.3% respondents stated improved use and administration of facilities, materials, transport, etc. as the other positive changes. Better tracking of financial resources is also the positive change noticed by 50% of district officials. A small number (33.3%) of respondents said monitoring and evaluation are also the positive change they have seen due to SSP support. 3.7 Which of the following negative changes in SSP funded NGOs are due to lack of SSP support in the past two years? Please check all that apply. Negative Changes Frequency Percentage18 Decline in the quality of HIV/AIDS & FP services 0 0% Poor tracking of financial resources 2 100.0% Decreases in the administration and use of facilities, materials, transport, etc. 0 0% Poorer or incomplete planning, oversight and monitoring of the selected HIV/AIDS & Family Planning services 0 0% Others 0 0% Total 2 100% Poor tracking of financial resources is the only negative change seen due to lack of SSP support in NGOs. Best Practices 18 % of multiple responses 4.1 What are SSP’s most effective approaches/set ups that would help NGOs improve their performance in your district? Please check all that apply. Frequency Percentage19 Community mobilization and outreach to FSWs? 21 95.5% Counseling and testing at NGO centers? 16 72.7% Support to FSW networks? 14 63.6% Monitoring and reporting? 20 90.9% Other (Monitoring and Evaluation, Positive Prevention and CHBC services, Satellite clinics for T&C & STI where and when necessary, increased the area of CHBC to rural areas) 4 18.2% No Response 3 12% According to 95.5% of respondents, Community mobilization and outreach to FSWs is SSP’s most effective approach that has helped NGOs to improve their performance. Monitoring and reporting are most effective approaches according to 90.9% of respondents. 72.7% of respondents stated Counseling and Testing at NGO centers as the most effective approaches of SSP. Only 18.2% mentioned Monitoring and Evaluation, Positive Prevention and CHBC services, Satellite clinics for T&C & STI where and when necessary, increased the area of CHBC to rural areas as the other most effective approaches of SSP. 19 % of multiple responses 4.3 How actively do SSP supported NGOs share their best practices with others: SSP, DPHO, DACC, and other NGOs? Please check in the appropriate boxes. Sharing Best Practices Poorl y (%) Somewh at Actively (%) Modera te (%) Active ly (%) Very Active ly (%) No Respon se (%) With SSP 0 0 24 28 0 48 With DPH O 0 0 8 60 32 0 With DAC C 0 0 8 60 32 0 With Othe r NGO s 0 0 16 76 8 0 Overall, it is seen that sharing best practices with DPHO, DACC and other NGOs are actively done by SSP supported NGOs at the district level. 60% of respondents said that SSP supported NGOs are actively sharing best practices with DPHO and DACC, whereas 76% said that SSP supported NGOs are actively sharing with other NGOs. 32% of respondents stated that SSP supported NGOs are very actively sharing with DPHO and DACC. 28% of respondents mentioned that NGOs are sharing actively with SSP about their best practices whereas 24% stated that NGOs are sharing moderately about their best practices with SSP. ANNEX 15: LISTENING SESSION NOTES Listening Session: Kathmandu Valley, August 26, 2014 On August 26, 2014, the Kathmandu Listening Session was held by the Team with the presence of 28 participants from IAs in Bhaktapur, Lalitpur and Kathmandu districts as well as GON representatives (DPHOs, DACC, DDCs) from these districts and members of civil society. The list of the participants is in Annex… Policy and SSP Model • IAs discussed the need to, and the way in which, to recommend addressing the issue of MDG 6a (HIV and AIDS) beyond 2016 • Roles of IAs, Government and SSP: Some of the IAs mentioned that the current model should be continued for some years even after the completion of the SSP because the government will not be fully prepared and capable of taking all responsibilities of providing services related to Community Test Treat and Retain. One IA mentioned that it has already started to work with the approach similar to CTTR approach. • Coordination among IAs, the GON, and SSP requires urgent attention and NCASC needs to take the lead role to make this happen. • IAs stated that they are aware that, in order to fully implement the CTTR approach, NCASC’s current HIV guidelines will need to be revised to include CTTR, and to provide the guidance for the implementation of CTTR. • Participants discussed that the DACC’s role has increased particularly related to HIV coordination of services, but it is funded by donors and it is outside of the formal GON structure. • Bottom up approach of the GON is praiseworthy but its implementation has a great challenge. Currently, in the SSP IAs are implementing this approach through “Micro￾Planning”, a similar approach to bottom up planning. In the government side, lowest level FCHVs and health workers will also be required to do “Micro-Planning”. However, they are not able to do this exercise because most of them are not qualified. They require rigorous training and SSP can help train them and build their capacity. • The problem with the government is that it itself is unstable and has given less priority for FP, person in trafficking, HIV and AIDS. • Participants, particularly from government said that group counseling can be done, target groups can be involved in testing, and everyone can test after one whole day training according to GON guidelines. • One of the DACC coordinators asked whether SSP model would be able to sustain capacity of IAs at the district level. Based on this issue, he suggested the evaluation report could recommend sustaining the capacity development initiatives of the SSP. He further mentioned that training can be provided through DACC. If VACC and other government line agencies are empowered, services for CTTR can expand. • One DPHO participant suggested that if some recommendations could be given in the evaluation report, it would be better and district level capacity gap will not be experienced. • In the case of IDUs they find specific groups with coordination with other specialized organizations. • IAs in Kathmandu district stated that it is an influential district in all aspects, so if the Team leaves Kathmandu district data out (because it was the team’s district for testing the data collection tools), some findings and conclusions may not truly reflect the real situation and/or current status of FSWs. • Outputs under Outcome 5 (Strengthening coordination among all HIV/AIDS partners) could not make much progress because the evaluation is focusing other districts but leaving Kathmandu out. In Kathmandu there has been some progress under Outcome 5 but not with tangible output(s). • JMMS expressed their dissatisfactions with not being informed/interviewed to fully involve FSWs. ( they are the national association of FSWS) • One of the IAs expressed that there are two major systems strengthening - health system strengthening and community strengthening. NGOs should work on community strengthening and EDPs should work on government strengthening for service delivery. • In the discussion these two key questions were asked by the presenters. o Do you think the HIV vulnerable FSW can do “in-reach” into the four Overlapping high risk communities to which they belong? o Can they – the FSWs and especially the overlapping high risk groups-- lead this with the IAs supporting them? • Some of the IAs said that they have already started this in-reach CTTR related services for two years, especially working with high risk IDU communities. They said it would be easier if the new strategy started as soon as possible. • Testing goes and covers both groups: general population and key population. Service Delivery • All IAs who participated are following government’s HIV protocols and SSP’s guidelines in each service delivery area • Regarding the overlapping risk for the FSWs and their vulnerability, it was stated that the present services were inadequate, this vulnerable group needs more work in future planning and providing services. • The EIHS service providing centers mentioned about the unavailability of the treatment for co infection- TB and Hepatitis C, However treatment for Hepatitis C started by EIHS/CCC (SPARSH) , but still guidelines have not been developed • Static Clinic, Lab Technician from the district (Chitwan) mentioned that testing 9-11 Static clinic coverage, but could cover more 25-30 clients per day. While in Bhaktapur and in Kathmandu, the lab service is covering 25-30 clients a day. • CAC Bhaktapur has formed groups of FSWs for IGP program through IA (DIC). Most of the IAs are supporting school fees, books and supplies to support the children of PLHIV in Chitwan. • Participants also raised that if FP integration with HIV should be discontinued it needs a strong justification before it’s phased out. This program will have some negative impact if it is suddenly dropped from their program part. In fact, outreach activities such as condom negotiation skills and condom distribution are helping FSWs a lot. • The performances of IAs have not been fully reflected in the presentation. The Team discussed that this was because the focus of the evaluation was the performance of SSP. • The Program Manager of SPRASH was concerned that service data were not included in the presentation. He is aware that this was because the pre-test of the instruments was carried out in Kathmandu district. In addition, SPRASH mentioned that CHBC teams reached 4,681 people out of which 1,003 were people being followed up seems to be considerably low number. This may imply loss of PLHIV due to death. • The IAs discussed the difficulty of getting birth certificates for their children: marriage registration of single women and FSWs was also difficult. Many people are facing these problems. Usually when FSWs are left by their husbands they will face difficulty in marriage registration and will face problems in acquiring citizenship certificates. CAC Nepal is lobbying for the issue. Response to FSW Focus Group Information Most of the participants seemed to be in agreement with FGD findings, however they commented mostly on the following: • Need to look at the relevance of SSP Model with regard to empowerment of the FSWS. - In order to make the program more sustainable and empower the FSWs, Income Generation Programs should be a key component. However, it is missing or still inadequate in some of the IAs. - There are good trainings and information sessions with FSWs about condom negotiation and safer sex and the IAs say it has been effective in FSW behavior change. However, IGP should be addressed to provide opportunities for willing FSWs to shift their profession • One IA discussed their intervention to link FSWs to opportunities for Micro-credit or provision/s to save money through IA (DIC) by forming small groups • SSP should provide support to PLHIV children and FSW children for education. In FSWs case, it is more difficult to admit their children in school (personal conversation with JMMS). Kathmandu, Bhaktapur and Lalitpur participants opined their views – currently funding comes from UNICEF, Pool Fund and other sources • FSWs need linkages for legal counseling for property rights, citizenship issues/ lobbying for Single mothers’ rights- issue of birth certificate for children, Marriage registration. • IAs mentioned about the importance of continuing FP/HIV Integration, for beneficiaries. There has been behavior change in the beneficiaries due to the integration as seen by the increased use of FP services by MARPs. • IAs stated that, in conducting the study without considering the male clients of FSWs, are we not missing a major section? It will not be a complete analysis without that. • Jagriti Mahila Mahasangh- The organization of FSWs expressed their dissatisfaction that they were not included in our interviews but the Team clarified that we were aware of the organization: we had tried to contact them but were not able to meet them, due to time constraints. However, they were invited to the Kathmandu Listening Session in order for the team to get their input. The Team is also including their representatives in the Listening sessions in Jhapa and Chitwan in order to get additional information and feedback from their organization. Capacity Building – TOCAT The discussion of IAs in response to the TOCAT presentation focused on two major areas: 1) TOCAT instrument and process of use of the instrument 2) Comprehensive Action Plans Comments and responses were made by several different IAs. The DPHO, DACCs and other GON participants had not heard of, and had not received any TOCAT training, therefore they did not comment on this and were not very interested in IAs’ discussions on TOCAT. TOCAT instrument and process: • Good for Capacity Building & sustainability assessment • Supports IA transparency, good governance, review of policies and documentation • Resources and new staff are needed to do the assessment and carry out the activities in the Action Plan • IAs demand for M&E training is also essential and valid: training in M&E currently only provided to SSP supported staff and is needed for all staff if the IA is to be self-sustainable (in case the SSP project is phased out) • TOCAT is helping to develop the whole organization, is not focusing on individual members, although, sometimes training does not include all staff members • Several different types of training programs were as part of “Roll Out” to increase capacity • M&E training, done after the TOCAT initial assessment, actually helped IA to monitor the organization’s capacity • There are 242 indicators-- Many are not applicable in Nepalese NGO culture --- So maybe SSP can tailor it to fit the NGO culture. Even the IA’s categories might be different so IAs should also have their own categories (customize) • Earlier in the process of working with the TOCAT instrument and process, it was exciting, but when we went to more go more in deep, we found the instrument complicated • We have met many goals that are in the TOCAT instrument but still few things are yet to be met • Once our organization has used the tool and worked to develop and strengthen our/IA’s systems, then it is easier to follow • At the beginning, it was tough to do the TOCAT instrument and process but in large staff organizations, TOCAT helps because it puts a system in place but at first it is difficult to implement • Some IAs took 18 months by taking time to do the initial one year TOCAT assessment • We need resources to do this work and have meetings to make this happen. Comprehensive Action Plans • A few NGOs have submitted TOCAT Comprehensive Annual Action plans and it took them 18 months: the reason that it may have taken that length of time is that IAs may not have prioritized those activities in the Action Plan or may have been in need of some technical support from SSP • Action Plans were done as well as Strategic Plans even though a second TOCAT assessment scoring has not been done – at least for us present here • As per the findings presented in the Power Point, some IAs have made Comprehensive Action Plans and have extended their plan if not finished in the first 12 months • Measurements/scoring still needed in IA Action Plans: now the actions are measured/ marked as “completed”, or “in progress”, etc. Recommendations offered by IA participants include: • Since TOCAT scoring is not yet done (for the second time), it would be advisable to revise indicators, reduce it to the extent to which will be applicable to IAs scenario. [Reduced indicators should be more applicable to IAs] • IAs recommended that SSP reduce and revise tools to make compatible in Nepal • SSP & IAs indicators different so they should be revised • SSP should review the initial level result now and also the requests or recommendations from IAs given to SSP, such as trainings to develop the overall capacity of the organization rather than just SSP associated staffs to make the organization sustainable. • Few trainings have been rolled out like GIS but certain trainings like M&E and others should also be rolled out according to the capacity of the IA. Coordination between IAs and GON Counterparts • There is coordination at a moderate level; however, IAs said that coordination is inadequate. • Some IAs questioned on what basis are the prerequisites for coordination defined and asked clarification on the basis for concluding as ‘no coordination’ with government agencies. • According to IAs, they have been following government guidelines and protocols. • They said conducting activities is not possible without coordination with government stakeholders and beneficiaries. • IAs have made objections to the statement that relates to the SSP Principle C) institutionalization of coordination and collaboration, stating that the coordination and collaboration are not “nil” between the IAs and government counterparts. They stated that, in Kathmandu valley, the scenario is different than rest of the districts. • However, with regard to community based test, treat and retain approach, IAs said they have not received any guidelines and concept papers from government although one IA said it is applying this approach in its own programs. • Government officials stated that DACC platforms are essential for performing coordination activities at the district level for HIV activities. The DACC committee is permanent and they are already formed in fifty districts. GON representatives stated that village level coordination committees (VACCs) have also been formed. • They continued with the statement that, even if a particular DACC coordinator changes, the DACC committee will be there. DACC should be continued and have more responsibilities because it provides the essential link between the GON and IAs. GON Capacity Building Most of the comments made about government capacity building were made by GON participants— DACC coordinators, DPHOs, and HIV Focal Persons. They stated that if there were more capacity building at the district level and below, it would be possible to try Test Treat and Retain. Comments from Participants: • The DACCs are here and will always be here (as part of the structure of government at the district level), even if there is a change in their internal structure, so investing in the DACC is worthwhile. Therefore, if the DACCs get more training, they can help with the roll out of TTR. • The government is not stable; they do not prioritize family planning or the trafficking of persons. • If family planning is dropped by SSP, the government would not be able to take up the responsibility for it. • Building GON Technical Capacity recognizes that - Health Personnel at the lower levels i.e. FCHV, Health Post Workers, are not well trained - Capacity building should be more oriented to the lower level - The GON still needs SSP support for facilitation of capacity building • It takes time to build the capacity of district level government staff and local mechanisms. • It takes time to empower the district level government staff and district level mechanisms. • Capacity building should allow information to flow from the bottom up and from the top down. Listening Session: Bharatpur, Chitwan, September 1, 2014 COORDINATION BETWEEN GON AND SSP/ IAs Chief of DPHO • Need more coordination between GON and IAs • There should be strengthening of the whole system, both GON and IAs. Currently, NGOs are working parallel to the Government. However, if the Government highly prioritizes the TTR approach, NGOs should work under the Government network. NGOs should work in supplementing GON’s work, where the GON does not have their wings or networks such as with FSWs, IDUs. Only when the IAs work to build networks in areas where the GON does not have any networks, there will be real strengthening of the coordination between GON and NGOs. SSP should support to strengthen this coordination. • Moreover, SSP needs to support GON Networks that are already present such as T&T area, ART Centers DPHO from Bara • We need to reflect on the current SSP model and the changes as well as adaptations that it needs in case new GON approach is implemented such as Community TTR. • He agrees that we should use epidemiology to focus our activities in HIV • He states that there is a lot of turnover in the SSP Regional Coordinator positions – Multiple coordinators in the year he has been in office in his district. He has spoken with the deputy director of SSP of this concern. The Regional Coordinators do not seem to have a vision of where the program has to go but rather “operate like carthorses with blinders” – checking to see that reports are getting out. The IAs are working but need technical TA and guidance. DDC Officer • SSP is works at central level and has more coordination with NCASC. At the district level, only IAs are coordinating. It has to be bottom up coordination, which they do not have. IAs Views • SSP strengthens DACC and supports IAs but there is a need to provide more support at the District level • Government stakeholders should take ownership of service delivery part and IAs should adopt awareness part. • SSP provides much more financial and technical support at Central level such as NCASC and MOH but SSP has not supported at district level • SSP has completed TOCAT at IAs organizational level, but government level staffs do not even know about TOCAT • SSP wanted a decentralized system and as a result SSP operates at regional level. However, Regional level officials do not coordinate with district level stakeholders, so IAs face problem to coordinate and collaborate with GON and other stakeholders. • In any district, before SSP supported IAs and DHO plan and implement programs together, IAs should look after the Prevention part and gradually, the GON could look after the EIHS part. For that, GON staff should be given S&D Reduction training as well as Capacity building trainings. That way, even if the current SSP project is phased out, GON will have built their capacity and taken ownership of the program as well as the beneficiaries would still get the services they require. This would make the program more sustainable. • For sustainability, treatment part should be the responsibility of the GON. IAs are ready to make this move but do not think the GON currently has the capacity to take over the responsibility. SSP could provide support to the GON to build its capacity. • SSP should consider the bottom to top approach rather than top to bottom approach and should emphasize more S&D reduction activities at the District level. • Recording and reporting for the GON and SSP funded IAs should be uniform. If the current GON reporting format does not fulfill the requirements of SSP, then NCASC should be consulted about the additional requirements. • Finances provided to the IAs should be a bit flexible in order to implement the project in the district in close coordination with district stakeholders like DDC, DHO, and DACC. • DACC/VACC/MACC should also be involved with IAs during Annual District Meetings. SSP MODEL IAs Views • Need adaptations • Need to address vulnerable groups from the target population as well • It might be challenging to reach beneficiaries at the community level with the current SSP model • There are challenges in reaching and providing services to the target population of FSWs. Furthermore, stigma and discrimination creates more challenges. Since FSWs prefer to come to DIC for services and do not want to go to Government hospitals for the same services, it is also further increasing their stigmatization. Once the project is phased out, they have to go to the Government hospital for the same services so we need to consider this aspect as well. Chief of DPHO • Special programs such as DIC services and treatment of beneficiaries at Clinics has resulted in further isolation • There is a need for more S&D Reduction Trainings • SSP Supports the Central level (S&D Reduction) but less at the District level • Need Community Sensitization in order to reach unidentified groups • Program specialized for HIV has increased Stigma & Discrimination • Has socially detached the beneficiaries rather than attach them • In a way, we have created stigma and discrimination, because in the early days of the occurrence of STI/HIV, we raised a big hue and cry over HIV, and gave a very negative message to the people. Govt. and NGOs, sort of, terrorized the masses with these negative messages and this led to stigma in the society. Now, a lot of awareness raising programs are done by GON and NGOs, but HIV positives still face strong stigmatization • IAs should send beneficiaries through referrals to Government Hospital - Government Hospitals offer general services - SSP runs satellite clinics for services but FSWs/beneficiaries also go to GOVT. hospitals for Antenatal, FP Services, Immunization, and Medical Abortion) - Govt. Hospitals also maintain confidentiality and privacy - Specialized services for beneficiaries are not sustainable, as ultimately the project will be phased out - IAs should focus on conducting Risk Assessments and providing referrals to beneficiaries to Govt. hospitals for services. - IAs should work for Prevention measures and GON should take ownership of Clinical services - If this is done, stigma and discrimination will be gradually reduced DDC Officer • Stigma and discrimination will continue until the society becomes HIV friendly, so it is important to make the society HIV friendly. Programs should not be launched in isolated manner, but should be community friendly. • CCC and CHBC will also work under the ownership of the Government. • SSP is working rigorously in sensitization it still needs to do more. MISC. POINTS IAs Views • 4 Overlapping categories - Tracking through Micro-planning is done but have only been recording since 6-7 months • More Referrals are needed to find unidentified groups • Need activities tailored to fit categories • Vulnerable in FSWs already tracking but no specific guidelines have been provided for support of these groups • There are no technical guidelines by SSP for “overlapping groups” • CHBC/EPC services would be even more effective if PLHIV Volunteers could do “inreach” in the communities • Currently, only a few staff members from CHBC and EPC are PLHIVs. Paid Peer Volunteers recruited from the target groups would benefit CHBC and EPC services and make them more effective. It would further help to reduce Stigma and Discrimination. • If the IA had the financial support to buy medicines for EPC services, then more people would come for EPC services. • There is a high turnover of OE/CM because the salary is low. Considering the fact that the OE/CM are the pillars of the program, there should be increase in their salaries. • The workload of Reporting/Recording by OE/CM should be reduced so that they could work more in the field. • The main problem occurring in most of the IAs from month to month is Staff turnover. This is because of high load to IAs staff and low payment. If SSP could provide financial support after researching market price value, staff retention would increase at least for 1-2 years or more. • Due to the Program format, the workload of employees is high and labor intensive so SSP needs to consider workload and pay • Salary benefits of IA staff should be as of Govt. level to motivate the staffs towards the tasks. • There are no entertainment events between all staffs for encouragement • Considering financial part, IAs need approvals from SSP for even small amounts of money and it leads to a lot of difficulties. • SSP needs to coordinate and involve IAs during program planning phase. • Program and management are all done by SSP and they only call IAs for signing the agreement. All IAs want participatory planning and budgeting. • In the issue of Program planning, most IAs also spoke out that they want participatory planning. Due to the low salary, it is difficult to retain staff- they never get invited in their retreat program and also in their IAs planning and budgeting. • Planning and Program should be done in close coordination with key staffs and Board Members of the IAs by SSP. • SSP also needs to provide more financial support to IAs • Financial Management of IAs should also be evaluated - IAs face problem of retention of staffs and turnover of staffs - SSP should support in sharing costs and Overhead costs • IAs do not have sharing cost, organization cost and even over head and are facing a lot of difficulties. • Another problem is that NCASC provides the medicines but the IAs need to buy medicines when there is less availability, but IAs do not have overhead costs • If IAs are given Overhead costs to build the capacity of IAs for M&E program then the project would be even more effective. • It would be better if SSP could share costs with IAs for different programs such as Annual Review Meeting, Monthly Meeting, Tri-monthly Meeting, Monthly Board and Management Meeting, Annual Meeting. • Even if the project is phased out, the IAs should have the capacity to work directly with USAID. For this, SSP’s main focus for the next 2 years should be to provide technical support so that IA can provide direct support and work with USAID. • Essential gadgets and instruments should be upgraded or brought new. For example, most IAs have been using the same computers for 6-8 years, and they are highly unreliable and leads to difficulties in data entry, preservation, and reporting to SSP. Thus, new computers should be provided which would also help in overall efficiency. • In Nepal, Load shedding is a huge problem. Offices do not have backup in case of load shedding. So, there is no electricity during office hours, it comes around 4 pm, so no work can be done during office hours. Therefore, it is essential to provide backup for electricity with invertors/generators. IA Views Regarding IGP • In providing IGP program, bank’s priority and FSWs’ priorities has always been different. Therefore, there is a challenge to provide IGP program to FSWs through Co-operatives. TOCAT • As per the request of all IAs in Chitwan , SSP called for pre-assessment TOCAT training , regarding re-scoring they do not whether SSP conducted or not they do not know. What is the overall recommendation of this MTR? Asked by one of the IAs Our answer: how to reach, enrich and empower FSWs. ANNEX 15: BIBLIOGRAPHY BIBLIOGRAPHY 1. ASHA Project: National Centre for AIDS and STD Control. (2011). Integrated Biological and Behavioral Surveillance (IBBS) Survey among Female Sex Workers in Pokhara Valley, Nepal. Kathmandu: USAID/Nepal; ASHA Project and Government of Nepal. 2. Family Health International (FHI). (2007). Standard Operating Procedures for FHI supported STI Clinics in Nepal. Kathmandu: Family Health International (FHI). 3. Family Health International: Saath Saath Project Cooperative Agreement No. AID 367-A-11-0000 x 4. Family Health International and UNICEF Nepal. (2010). Manual of Standard Operating Procedures (SOPs) for the Community-Based Prevention of Mother-to-Child Transmission of HIV (CB-PMTCT) Program and Research Study. Kathmandu: Family Health International and UNICEF Nepal. 5. National Centre for AIDS and STD Center. (2011). National Guidelines on Community and Home-Based Care and Standard Operating Procedure. Kathmandu: Government of Nepal. 6. National Centre for AIDS and STD Control. (2008). Clinical Management of HIV Standard Operating Procedures. Kathmandu: Government of Nepal. 7. National Centre for AIDS and STD Control. (2008). The National Guidelines for the Management of HIV and AIDS in Children in Nepal. Kathmandu: Government of Nepal. 8. National Centre for AIDS and STD Control. (2009). National Guidelines on Case Management of Sexually Transmitted Infections. Kathmandu: Government of Nepal. 9. National Centre for AIDS and STD Control. (2010). National Medical Standard for Reproductive Health (Volume I: Contraceptive Services) Fourth Edition. Kathmandu: Government of Nepal Ministry of Health and Population Family Planning Division. 10. National Centre for AIDS and STD Control. (2011). HIV Monitoring and Evaluation System Strengthening (MESS) Workship Report 2011. Kathmandu: Government of Nepal. 11. National Centre for AIDS and STD Control. (2011). National Guidelines for HIV Diagnosis and Laboratory Monitoring of Antiretroviral Therapy. Kathmandu: Government of Nepal. 12. National Centre for AIDS and STD Control. (2011). National Guidelines on Prevention of Mother-to-Child Transmission of HIV in Nepal. Kathmandu: Government of Nepal. 13. National Centre for AIDS and STD Control. (2011). National HIV/AIDS Srategy 2011- 2016. Kathmandu: Government of Nepal. 14. National Centre for AIDS and STD Control. (2012). Integrated Biological and Behavioral Surveillance (IBBS) Survey among Female Sex Workers in 22 Terai Highway Districts of Nepal. Kathmandu: Government of Nepal. 15. National Centre for AIDS and STD Control. (2012). National Antiretroviral Therapy Guidelines. Kathmandu: Government of Nepal. 16. National Centre for AIDS and STD Control. (2012). National Estimates of HIV Infections in Nepal. Kathmandu: Government of Nepal. 17. National Centre for AIDS and STD Control. (2012). National Guidelines for Early Infant Daignosis. Kathmandu: Government of Nepal. 18. National Centre for AIDS and STD Control. (2012). National Guidelines on Monitorign and Evaluation of HIV Response in Nepal, 2012. Kathmandu: Government of Nepal. 19. National Centre for AIDS and STD Control. (2013). National HIV Research Agenda in Nepal 2013. Kathmandu: Government of Nepal. 20. National Centre for AIDS and STD Control. (2013). The Nepal HIV Investment Plan 2014-2016. Kathmandu: Government of Nepal. 21. National Centre for AIDS and STD Control. (2014). National Guidelines on Case Management of Sexually Transmitted Infections. Kathmandu: Government of Nepal. 22. National Centre for AIDS and STD Control. (2014). Review of The National Response to HIV in Nepal. Kathmandu: Government of Nepal. 23. National Centre for AIDS and STD Control. (2014). Technical Report on The Epidemiological Analysis of HIV in Nepal and The Way Forward. Kathmandu: Government of Nepal. 24. Saath Saath Project. (2013). Saath-Saath Project Monitoring and Evaluation Plan (October 2011- September 2016). Kathmandu: Saath Saath Project. 25. Saath Saath Project. (2014). Saath-Saath Project Recording and Reporting Guidelines for Expanded Integrated Health Services. Kathmandu: FHI 360 Nepal/Saath Saath Project. 26. Saath-Saath Project. (2012). Guidelines for The Technical and Organizational Capacity Assessment Tool (TOCAT). Kathmandu: Saath-Saath Project. 27. Saath-Saath Project. (2012). Saath-Saath Project Annual Report (October 2011- July 2012). Kathmandu: Saath-Saath Project. 28. Saath-Saath Project. (2012). Standard Operating Procedures for Family Planning Services. Kathmandu: Saath-Saath Project. 29. Saath-Saath Project. (2013). Baseline Family Planning Survey Including Rapid Assessment of HIV, Sexually Trasmitted Infections and FP Situation among Migrant Couples in Bara, Kapilbastu, Nawalparasi and Palpa District (Volume I: A Qualitative Report). Kathmandu: Saath-Saath Project. 30. Saath-Saath Project. (2013). Baseline Family Planning Survey Including Rapid Assessment of HIV, Sexually Trasmitted Infections and FP Situation among Migrant Couples in Bara, Kapilbastu, Nawalparasi and Palpa District (Volume II: Report on Qualitative Survey among Male Labor Migrants. Kathmandu: Saath-Saath Project. 31. Saath-Saath Project. (2013). Baseline Family Planning Survey Including Rapid Assessment of HIV, Sexually Trasmitted Infections and FP Situation among Migrant Couples in Bara, Kapilbastu, Nawalparasi and Palpa District (Volume III: A Qualitative Report of Wives of Male Labor Migrants). Kathmandu: Saath-Saath Project. 32. Saath-Saath Project. (2013). Essential Package of Care Operational Guidelines. Kathmandu: Saath-Saath Project. 33. Saath-Saath Project. (2013). Migrant District Profile: Bara, Kapilbastu, Nawalparasi and Palpa. Kathmandu: Saath-Saath Project. 34. Saath-Saath Project. (2013). Saath-Saath Project Annual Report (August 2012- July 2013). Kathmandu: Saath Saath Project. 35. Saath-Saath Project. (2013). Sustainability Analysis Workship at District Level Guidelines. Kathmandu: Saath-Saath Project. 36. Saath-Saath Project. (2014). Data Quality Assessment (DQA) First Semi-Annual, FY 2014 (Aug 2013– Jan 2014). Kathmandu: Saath-Saath Project. 37. Saath-Saath Project. (2014). Recording and Reporting Guidelines For Prevention Activities. Kathmandu: FHI 360 Nepal/ Saath Saath Project. 38. Saath-Saath Project. (2014). Saath-Saath Project Third Semi-Annual Report (August 2013- January 2014). Kathmandu: Saath-Saath Project. 39. Saath-Saath Project. (n.d.). Rolling Out the TOCAT: Experiences from Phase II. Kathmandu: Saath-Saath Project. 40. United States Department of Agriculture. (2011). Monitoring and Evaluation Policy. United States Department of Agriculture.