[Month Year] This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Social Impact, Inc. Performance Evaluation of the Strengthening Private Sector Services Project in Zimbabwe PERFORMANCE EVALUATION OF THE STRENGTHENING PRIVATE SECTOR SERVICES PROJECT IN ZIMBABWE January 2015 Performance Evaluation: Task Order No. AID-613-M-14-00006 SPSS Project: Cooperative Agreement No. 674-A-00-10-00081-00 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Kudzai Makoni, Jean–Camille Kollmorgen, Edmore Munongo, and Zibusiso Jokomo on behalf of Social Impact, Inc. Contact: Social Impact, Inc. 2300 Clarendon Blvd., Suite 1000 Arlington, VA 22201 Jean-Camille Kollmorgen: jkollmorgen@socialimpact.com DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS ACRONYMS ................................................................................................................... i EXECUTIVE SUMMARY ............................................................................................. iii EVALUATION PURPOSE & EVALUATION QUESTIONS ...................................... 1 PROJECT BACKGROUND........................................................................................... 2 EVALUATION METHODS & LIMITATIONS............................................................. 3 FINDINGS, CONCLUSIONS & RECOMMENDATIONS........................................... 8 Evaluation Question 1: What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS?............................................................................ 8 Evaluation Question 2: How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals?.............................................................................................. 25 Evaluation Question 3: How effective and efficient was the administration of sub-awards to local and private sector organizations in meeting the project goals? ............................................. 28 Conclusions and Recommendations .................................................................................. 33 Annex 1: Evaluation Scope of Work Annex 2: Evaluation Geographic Scope Annex 3: Data Collection Instruments Annex 4: Desk Review and Secondary Sources Annex 5: Sites Visited Annex 6: Respondent Matrix Annex 7: APR Data Annex 8: Site Outputs Annex 9: Data Capture Annex 10: Disclosure of Conflict of Interest i ACRONYMS AIDS Acquired Immunodeficiency Syndrome ART Antiretroviral Therapy ARV Antiretroviral (drugs) CDC Centers for Disease Control and Prevention CESSHAR Centre for Sexual Health and HIV AIDS Research Zimbabwe CHAI Clinton Health Access Initiative CSW Commercial Sex Worker DEC Development Experience Clearinghouse DFID Department for International Development DHS Demographic and Health Survey DQA Data Quality Assurance FGD Focus Group Discussion FACT Family AIDS Caring Trust FP Family Planning GBV Gender Based Violence GoZ Government of Zimbabwe HIV Human Immunodeficiency Virus HTC HIV Testing and Counseling HQ Headquarters IPC Interpersonal Communication ISP Integrated Support Programme KII Key Informant Interviews MARPs Most At-Risk Populations MMC Mass Media Communications MOHCC Ministry of Health and Child Care NAC National AIDS Council NEMC North Eastern Medical Centre NGO Non–Governmental Organization PEPFAR President's Emergency Plan for AIDS Relief PITC Provider-Initiated HIV Testing and Counseling PLHIV People Living with HIV PMTCT Prevention of Mother to Child Transmission PSI/Z Population Services International/Zimbabwe QA Quality Assurance SBCC Social Behavior Change Communications SI Social Impact SOW Scope of Work SPSS Strengthening Private Sector Services SRH Sexual and Reproductive Health SSI Semi–Structured Interviews STI Sexually Transmitted Infection TB Tuberculosis USAID United States Agency for International Development USG United States Government VIAC Visual Inspection with Acetic Acid and Cervicography VMMC Voluntary Medical Male Circumcision VTC Voluntary Testing and Counselling ii ZACF Zimbabwe AIDS Caring Foundation ZNASP Zimbabwe National HIV and AIDS Strategic Plan iii EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The purpose of the end-of-project performance evaluation of the Strengthening Private Sector Services (SPSS) Project in Zimbabwe is (1) to identify and understand the drivers of the project performance in order to guide similar interventions in the future, (2) to advise the Zimbabwean MOHCC and other actors regarding how to best strengthen complementary private sector health services within the MOHCC strategic framework, and (3) to share project performance information and lessons learned with key stakeholders in Zimbabwe and the US. The evaluation addressed three key questions: 1. What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS? 2. How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals? 3. How effective and efficient was the administration of sub-awards to local and private sector organizations in meeting the project goals? PROJECT BACKGROUND Zimbabwe has made the fight against HIV a national priority and achieved a severe decline in HIV prevalence from 25% in 1997 to 15% in 2011, although the national HIV/AIDS burden remains high with an estimated 1.4 million adults and children living with HIV. The United States Agency for International Development (USAID) funded Zimbabwe Strengthening Private Sector Services (SPSS) project, implemented by Population Services International/Zimbabwe (PSI/Z) and local partner organizations began in 2010 as a follow-on to the USAID-funded Zimbabwe HIV and AIDS Partnership project. The SPSS project aims to: 1) expand and improve private sector based health services; 2) improve the availability and range of affordable health products or supplies; and 3) promote positive health behaviors. In order to attain these objectives, the project uses a social franchising model buttressed by social marketing and communications to provide standardized services for HIV prevention and post-test care through New Start and New Life centers, to sell Protector Plus male condoms and care female condoms, and to provide voluntary medical male circumcisions (VMMC) in partnership with the MOHCC. The project has striven to provide a continuum of care through the integration of Family Planning (FP), Tuberculosis (TB), Gender-Based Violence (GBV), and Sexual and Reproductive Health (SRH) services. EVALUATION DESIGN, METHODS, STRENGTHS AND LIMITATIONS The team implemented a 3-part evaluation design consisting of a desk review, analysis of quantitative data, and collection and analysis of qualitative data. The desk review included program design and performance documents, national level reports, and research studies. For the quantitative analysis, the team examined project performance data for the President’s Emergency Plan on AIDS Relief (PEPFAR) essential indicators on which PSI reports annually to USAID/Zimbabwe and the MOHCC, as well as project outputs for each New Start and New Life center. The evaluation team gathered primary qualitative data from stakeholders in order to identify and understand project success and challenges, as well as provide suggestions for improving the project. The team employed three different qualitative iv research methods depending on the type of respondent. The team conducted key informant interviews (KIIs), semi-structured interviews (SSIs) with program staff, and focus group discussions (FGDs) with beneficiaries. The evaluation team visited a total of 49 sites in 16 districts across all 10 provinces in order to collect qualitative data. Sites were selected using purposive sampling based on several sampling criteria: geographic representation, type of service-delivery point, type of delivery model, implementing partner, and variety of services provided. Key informants were purposively sampled based on their relevance to the project. The evaluation team used a combination of purposive and convenience sampling for SSIs with staff members. Most FGD participants were recruited via convenience sampling, although a few select FGDs were purposively sampled with the assistance of the implementing partners to ensure participant availability. The evaluation exhibited the following strengths due to its robust design: (1) ensuring project representation by conducting an extensive number of site visits across a wide range of geographic areas and service-delivery points, as well as interviews with multiple stakeholders; (2) enabling the cross￾referencing and validation of findings through the triangulation of quantitative and qualitative data, and primary and secondary data sources; (3) exploring gender dynamics via gender-segregated focus groups for youths aged 18-24. The evaluation also encountered the following limitations and took appropriate steps to mitigate them: (1) the evaluation used annually reported national level data from national reports in order to assess SPSS contributions to national efforts, due to an inability to receive the preferred national level data from the MOHCC on a monthly basis; (2) the evaluation did not look at project financial data by sub-awardee site in order to determine site-level efficiencies, but instead conducted an in-depth interview with PSI’s Director of Finance and Administration; (3) the evaluation’s use of purposive and convenience sampling may have introduced selection bias, but this risk is lessened by the large sample size for sites and respondents; (4) political rallies and a concurrent USAID evaluation inhibited site visits in a few instances, to which the team responded by rescheduling site visits. FINDINGS AND RECOMMENDATIONS Evaluation Question 1: What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS? Finding 1.1 SPSS is in alignment with and has contributed to all three major priorities of the Zimbabwe National AIDS Strategic Plan (ZNASP II) 2011–2015, which are to reduce new HIV infections, to reduce mortality and morbidity related to HIV and AIDS through increasing access to treatment and care for people living with HIV (PLHIV), and to reduce stigma and discrimination against PLHIV. The project employed HIV testing and counseling (HTC), voluntary medical male circumcision (VMMC), provision of Antiretroviral Therapy (ART), condom distribution and social behavior change communication (SBCC) for HIV prevention. HIV care and treatment, TB screening and referrals, blood pressure testing and condom distribution served mortality and morbidity reduction purposes, while the project’s psychosocial support services and confidentiality protocols contributed to neutralizing the impacts of stigma and discrimination. Finding 1.2 Overall, the project has reached the majority of its performance targets over the evaluated period, created demand for its products and services, been a thought leader for VMMC and other interventions, influenced national policies, contributed research, and strengthened public health systems. SPSS increased HTC uptake figures through increasing the number of outreach teams providing services to underserved communities, intensifying and diversifying demand creation campaigns, and spearheading v VMMC, which accounted for the increase in the number of men testing for HIV and the number of men who knew their HIV status. As a result, 30% of national HTC outputs and 95% of national VMMC outputs were attained through the project’s efforts between 2011 and 2013. The project integrated HTC with post-test support services including ART. The project has further distributed 110,504,400 male condoms and 5,748,540 female condoms from January 2010 to December 2013, making Zimbabwe a country with one of the best female condom distribution records in the Southern Africa region. In qualitative interviews stakeholders credited the project with filling capacity gaps in the public sector including staffing challenges, resource constraints, limited geographical coverage and operational scope. The project’s achievement in HTC and VMMC, coupled with research that it has conducted over the years, have generated useful lessons that are now being applied at scale across the health sector. National policy changes such as the approval of nurses to do VMMC surgical procedures were done with the project’s active participation. Finding 1.3 Enabling factors for project performance include: integration of other services, mobilization and outreach, evidence-based market segmentation and targeted promotions, multi-stakeholder engagement, good coordination with MOHCC, leveraging of resources from other donors to fill gaps, use of electronic data capture for HTC, favorable national policies, project participation at strategic platforms, and organizational flexibility. Finding 1.4 Inhibiting factors for project performance include: funding and resource constraints, poor roads and long distances to reach communities, client hesitancy to openly purchase condoms, inhibiting policies (related to MOE, PEPFAR, and religious institutions), stigma and discrimination against PLHIV, misconceptions about VMMC, and operating challenges within the MOHCC (i.e. availability of government healthcare workers, low client satisfaction with public sector services, and limited cooperation with SPSS for referral tracking). Finding 1.5 The project has met different needs for men and women, different age groups, and key populations. More women than men access services as New Start and New Life, although overall the male-female ratio for HTC is close to 50-50 due to referrals from VMMC services. Outreach to workplaces and other isolated communities with large cadres of men, the support of community leaders and the use of male-fora or networks, and referrals from VMMC clients have helped increase the number of men reached by New Start, New Life, and VMMC services. The project strategically targets young males aged 13-29 for VMMC with school campaign, but has been more successful in reaching boys aged 13-19. The project’s use of voluntary testing and counselling (VTC) has more successfully reached people <30 years old. Although not in the SPSS project mandate, the project has gone above its initial purview and attempted to serve the needs of children and adolescents, although staff perceive a training gap for serving young clients. SPSS has worked to address the needs of both Most At-Risk Populations (MARPs) as defined by PEPFAR, as well as other key populations such as workplaces and other isolated communities that are not included within the PEPFAR definition. In particular, SPSS has either directly provided or partnered with organizations to provide services to Commercial Sex Workers (CSWs). Evaluation Question 2: How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals? Finding 2.1 A number of services have successfully and with positive results been integrated into HIV services, albeit at varying degrees at different sites. Furthermore, services can be accessed at fixed sites, during outreach, and at referral sites external to SPSS. TB services were found to be highly integrated, as all New Start and New Life centers implement TB screening, some centers offer TB microscopy, and the tracking accuracy of TB referrals emerged to be vi among the highest, at 63% in 2014. FP counselling and services were found to be highly successful as these services were directly offered at site. Integrated STI identification and treatment is limited to only four New Start sites, although quantitative figures thus suggest that demand for this service is high. GBV services, while not SPSS-funded, have been integrated into HIV services albeit at a smaller scale. Finding 2.2 While HTC is often the entry point for accessing integrated services, the reverse is also true, as the integration of services has increased uptake for HTC and other existing HIV services. For example, as clients come for HTC they also received FP counselling and service. As part of FP counselling, clients are routinely urged to use dual protection, and condom distribution at all visited sites was integrated with information on HTC, FP, VMMC, VIAC and GBV. Finding 2.3 Factors enabling service integration include: the development of “one-stop-shops” whereby all services are offered at one location, proximity of referral sites to referring ones or the client’s home, low end–user costs of accessing the referral site, supportive mass media and IPC campaigns, service provider’s partnership with other organizations, and life–saving attributes of the integrated service. Finding 2.4 Factors inhibiting service integration include: clients’ perceived poor service delivery quality or inefficiency at referral sites run by MOHCC, referrals to health facilities or service delivery sites that are distant from clients’ homes or the referring sites, training gaps among project staff on the integrated services, high fees at non-SPSS sites for referred services, negative messages about some services that clients are not culturally familiar (i.e. VMMC among certain ethnic groups, female condoms), and lack of cooperation with referral sites for referral tracking. Evaluation Question 3: How effective and efficient was the administration of sub-awards to local and private sector organizations in meeting the project goals? Finding 3.1 The evaluation revealed that sub–awards meaningfully ensured the project’s attainment of its objectives and enabled it to do so within a short time period after inception. It further showed no significant differences between sites that PSI directly managed and those managed by sub–awardees. Major determinants of positive sub-awardee effectiveness include: sub–awardee technical and contextual expertise, ability to leverage complementary resources from concurrent programs, adequate capacity to manage funds and implement SPSS activities, provision of support from PSI enabled sub-awardees to meet standards and expectations for effectiveness. Major limiting factors influencing sub-awardee effectiveness include: human and material resource constraints linked to overall budget limitations (i.e. limited vehicles, CD4 cell count machines, and trained staff), and delays in the sub-granting and centralized procurement process which may lead to implementation delays. Finding 3.2 The administration of sub–awards is efficient due to the strategic use of sub-awardees based on geographic and demographic characteristics of the target population. Examples of efficient strategies for service delivery include: use of mobile outreach, co-location of New Start and New Life fixed sites and managed by the same entity, limited doubling of site managers as counsellors, and leveraging resources from other programs. Finding 3.3 Local NGOs have the potential to support the expansion of the SPSS project’s scope and scale of activities in future, as long as they meet several qualifying criteria, such as previous experience in the field of HIV/AIDs, established relationships with other local stakeholders, developed financial monitoring systems, and other funding sources besides SPSS. For most current implementing partners, continued supervision, support, and capacity building from a stronger organization remains necessary. vii Recommendations In light of the findings for Evaluation Question I, the evaluation recommends the following:  Given the effectiveness of the project’s targeted approach to social marketing, funding should be made adequate so as to allow for the targeting of additional segmented population groups (based on Findings 1.3, 1.4).  Given the effectiveness of community mobilizers and mass media campaigns to create awareness and demand for services, funding should be made adequate for these labor and resource intensive strategies (based on Findings 1.3, 1.4).  Advocacy efforts with the MOE should be increased so that they better support VMMC and HTC efforts in schools with eligible age populations (based on Finding 1.4).  Due to the project’s reliance on outreach services, funders should be sensitive to the project’s needs for adequate and reliable vehicles and, if funding allows, make sure the project is resourced accordingly (based on Findings 1.3, 1.4).  Continue and expand strategies that have proven effective for increasing male involvement and access to services at New Start and New Life sites, such as mobilization through community leaders and men’s fora (based on Finding 1.5).  The project should continue and, if possible, expand its outreach service as this strategy has proven effective for reaching marginalized and inaccessible populations (based on Finding 1.5). In light of the findings for Evaluation Question 2, the evaluation recommends the following:  For a given site, the project should strive to offer the same integrated services on outreach as they do at fixed sites (based on Findings 2.3, 2.4).  If resources allow, the project should train all site counsellors to be able to perform all the integrated services offered at a given site (based on Finding 2.4).  Although challenges associated with the MOHCC are beyond the project’s control, the project can consider furthering its communication and advocacy efforts with the Ministry at the local, provincial, and national levels. This may entail sharing evaluation findings related to client barriers for accessing and/or using public sector facilities, and continuing to explore ways to better cooperate regarding client referral tracking (based on Finding 2.4). In light of the findings for Evaluation Question 3, the evaluation recommends the following:  USAID should continue to consider the use of sub-awardees for contexts where the ratio of fixed costs to client flow is cost-effective, and where an organization capable of handling a sub-award already exists (based on Findings 3.1, 3.2, 3.3).  PSI should continue to provide prompt and adequate support to sub-awardees to ensure that services are delivered in a timely manner and according to quality standards (based on Finding 3.1).  PSI should make sure that centralized procurement processes and any sub-granting procedures occur as rapidly as possible so as to avoid implementation delays down the line. PSI may also consider building in time during its routine supervision visits to briefly mentor sub-awardees on managing their financial risk-mitigation strategies to prevent delays at the sub-awardee level (based on Finding 3.1).  Donors should consider an influx of funding to fill the resource gap (e.g. vehicles, CD4 machines, training and staffing needs) which severely limits sub-awardees’ potential to reach and serve clients (based on Findings 3.1, 1.4).  If SPSS expands in scope and scale, it is recommended that additional New Start and New Life sites be co-located to be more efficient (based on Findings 3.1, 3.2). 1 EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of the end-of-project performance evaluation is (1) to identify and understand the drivers of the project performance in order to guide similar interventions in the future, (2) to advise the Zimbabwean MOHCC and other actors regarding how to best strengthen complementary private sector health services within the MOHCC strategic framework, and (3) to share project performance information and lessons learned with implementing partners, GoZ, as well as USAID/Zimbabwe and USAID/Washington. EVALUATION QUESTIONS The evaluation addressed the following three questions and associated sub-questions: 1. What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS? a. What, if any, “niche” populations has SPSS served? Has SPSS served different needs for men and women, different age groups, etc.? 2. How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals? a. To what extent were the integration of FP, TB, GBV and SRH within HIV services achieved? b. What were the major factors influencing the achievement or non-achievement of the integration effort? 3. How effective and efficient was the administration of sub-awards to local and private sector organizations in meeting the project goals? a. To what extent did the administration of sub awards to local NGOs and private sector organizations ensure that desired objectives were achieved? b. Were the administration of sub awards to local NGOs and private sector implemented in the most efficient (cost, time, etc.) way compared to alternatives (vis-à-vis key health and social objectives of the project)? c. What is the potential to work with local NGOs in supporting an expanded scope and scale of current activities? 2 PROJECT BACKGROUND Context of HIV/AIDS and the SPSS program in Zimbabwe Zimbabwe has made considerable progress in combating HIV and AIDS, experiencing a decline in HIV prevalence from 25% in 1997 to 15% in 2011.1 However the national HIV/AIDS burden remains high with an estimated 1.4 million adults and children living with HIV,2 making it the leading cause of death among adults and accounting for 27% of the mortality rate for mothers and infants. The latest thrust however is in looking at the HIV incidence since the prevalence has also been affected by the longer survival due to access to ARVs. Among young adults aged 15-24, the prevalence of HIV among females is 1.5 times more than for males according to the 2010-11 ZDHS. This is largely because 24% of young women age 15–19 have already begun childbearing: 19% are mothers, and an additional 5% are pregnant with their first child. The Government of Zimbabwe (GoZ) through the National AIDS Council (NAC) and the Ministry of Health and Child Care (MOHCC) has made the fight against HIV a national priority and has outlined its priorities in the Zimbabwe National HIV and AIDS Strategic Plan (ZNASP II) 2011- 2015. The GoZ and its partners have made great strides in promoting awareness and education about HIV/AIDS and in expanding USAID supported prevention and treatment schemes such as behaviour change communication (BCC), voluntary medical male circumcision (VMMC), HIV testing and counselling (HTC), antiretroviral therapy (ART), condom use and prevention of mother to child transmission (PMTCT) which has now been cascaded to elimination of mother to child transmission (eMTCT). 3 Despite these commitments, national efforts have been hampered by limited resources. The USAID funded Zimbabwe Strengthening Private Sector Services (SPSS) project, implemented by Population Services International/Zimbabwe (PSI/Z) and local partner organizations began in 2010 as a follow-on to the USAID funded Zimbabwe HIV and AIDS Partnership project. The project uses private sector approaches to: 1) expand and improve private sector based health services; 2) improve the availability and range of affordable health products or supplies; and 3) promote positive health behaviors. The project uses a social franchising model to provide standardized services for HIV prevention and post-test care at New Start and New Life franchises throughout the country. Services include HIV testing and counseling, CD4 cell counts, and HIV care and support such as psychosocial counseling and ART. In addition to New Start and New Life centers, the project socially markets the Protector Plus male condom and the care female condom, and partners with the MOHCC to provide voluntary medical male circumcisions (VMMC) at stand-alone and MOHCC–embedded sites. The project employs targeted mass media communications (MMC) and interpersonal communications (IPC) strategies informed by market-based research to create awareness of and demand for products and service delivery activities. Furthermore, the project integrates an array of other counseling, services, and products for Family Planning (FP), Tuberculosis (TB), Gender-Based Violence (GBV), and Sexual and Reproductive Health (SRH), as these areas impact HIV dynamics. The SPSS project’s design incorporates joint funding with other donors (e.g., Global Fund, DFID, Gates Foundation, etc.), so any delay or unavailability of funding from one of the other joint donors can potentially compromise the project’s ability to reach its combined targets. 1ZIMSTAT (2012). Zimbabwe Demographic and Health Survey 2010-11 2UNAIDS (2013). Zimbabwe HIV and AIDS Estimates. Retrieved from: http:/www.unaids.org/en/regionscountries/countries/zimbabwe 3NAC, MOHCC (2014). Global AIDS Response Country Progress Report Zimbabwe 2014 3 EVALUATION METHODS & LIMITATIONS Evaluation Team The Social Impact (SI) evaluation was led by senior-level local Team Leader, Kudzai Makoni. Zibusiso Jokomo and Edmore Munongo, both senior-level evaluators from Zimbabwe, as well as Jean-Camille Kollmorgen from SI-Headquarters (HQ) in the United States, comprised the rest of the team. SI’s Senior Director for Performance Evaluation, James Fremming, and SI Program Assistant, Julia Benjamin, provided backstopping and quality assurance support. Desk Review The evaluation team conducted a desk review of secondary data sources including program design and performance documents (i.e. SPSS Cooperative Agreement, Quarterly and Annual Project Reports), national level reports, and both internal and external research studies and assessments. These data sources were used to gather background information pertinent to the HIV/AIDS situation and prevention efforts in Zimbabwe, understand SPSS project design and changes over time, inform data collection strategy and tools, and extract quantitative data for analysis. Quantitative Data Collection The evaluation team examined project performance data for PEPFAR essential indicators on which PSI reports annually to USAID/Zimbabwe and the MOHCC (Table 1 and Annex 7). The data was collected from Annual Performance Reports (APRs) and cross-checked against SPSS Annual Reports for 2011, 2013, and 2014.4 In 2014 some PEPFAR indicators on which the project reports were revised. In order to examine performance over time, the 2014 data for revised indicators were matched to previous indicators where applicable. Qualitative narratives from the APRs and Annual Reports were used to help clarify and substantiate quantitative figures. The evaluation team compared the figures for project applicable PEPFAR indicators against national-level data collected from national reports in order to assess SPSS’s percentage contribution to national results over the same reporting period. Because PEPFAR indicators do not capture condom sales and activity, the evaluation team looked at condom distribution figures for the project as compared to the public sector. These figures were received from PSI (See Annex 8 for site outputs). The evaluation additionally looked at site-level performance through an analysis of project outputs for each New Start and New Life center. Site-level outputs were also received from PSI. Table 1: Annual Performance Report Indicators for SPSS Indicator Year Reported On FY 2011 FY 2012 FY 2013 FY 2014 C1.1D: Number of eligible adults and children provided with a minimum of one care service X X X C2.1D: Number of HIV-positive adults and children receiving a minimum of one clinical service X X X X 4 An Annual Report was not required and thus not submitted for 2012. 4 P5.1D: Number of males circumcised as part of the minimum package of MC for HIV prevention services; VMMC_CIRC_DSD: Number of males circumcised as part of the voluntary medical male circumcision (VMMC) for HIV prevention program within the reporting period 5 X X X X P7.1D: Number of People Living with HIV/AIDS (PLHIV) reached with a minimum package of 'Prevention with PLHIV (PLHIV) interventions X X X P8.1D: Number of the targeted population reached with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet the minimum standards required; GPY_PREV_DS: Number of the target population who completed a standardized HIV prevention intervention including the minimum components during the reporting period (DSD)6 X X X X P8.3D: Number of MARP reached with individual and/or small group level HIV preventive interventions that are based on evidence and/or meet the minimum standards required; KP_PREV_DSD: Number of key populations reached with individual and/or small group level HIV preventive interventions that are based on evidence and/or meet the minimum standards required (DSD)7 X X X X P11.1D/HTC_TST_DSD: Number of individuals who received Testing and Counseling (T&C) services for HIV and received their test results X X X X T1.1D: Number of adults and children with advanced HIV infection newly enrolled on ART; TX_NEW_DSD: Number of adults and children newly enrolled on ART8 X X Qualitative Data Collection The evaluation team gathered primary qualitative data from interviews with stakeholders in order to identify project successes and challenges, understand the drivers of project performance, and gather suggestions for improving the project. Qualitative data was collected through key informant interviews (KIIs), semi-structured interviews (SSIs) with program staff, and focus group discussions (FGDs) with program beneficiaries. In total, the evaluation team conducted 98 interviews with 301 respondents (Table II and Annex 6). The evaluation team acquired participants’ informed consent before each interview. Table II: Number and Type of Qualitative Interviews Conducted Interview Type # of Interviews KIIs IP 11 Donors 4 GoZ 9 SSIs – Program staff 60 FGDs – Beneficiaries 14 TOTAL 98 5 In 2014 indicator P5.1D was revised to VMMC_CIRC_DSD 6 In 2014 indicator P8.1D was revised to GPY_PREV_DSD. 7 In 2014 indicator P8.3D was revised to KP_PREV_DSD. 8 In 2014 indicator T1.1D was revised to TX_NEW_DSD. 5 Key Informant Interviews with Donors, GoZ, and Implementing Partners Separate data collection instruments were developed for each key informant type in order to focus the discussion based on their pertinent expertise (Annex 3a-3c). In general, interviews with key informants explored strategic matters of the project, the project’s relevance to changing contexts, and project effectiveness and efficiency (e.g., sub–granting). Semi-Structured Interviews with Program Staff Managerial staff (site managers), administrative staff (receptionists, drivers), clinical staff (nurse counselors, general counselors, VMMC doctors and nurses and theater assistants) at SPSS project sites, as well as condom outlet owners and community mobilizers, were interviewed either individually or in small groups using an SSI instrument (Annex 3d). The interviews generated answers primarily related to evaluation questions 1 and 2, exploring staff experiences with the delivery of the program in practice (i.e. availability of supplies, client demand and utilization of services, results of outreach to MARPs, and identification of enabling and inhibiting factors for project performance). Focus Group Discussions with Program Beneficiaries The evaluation team conducted FGDs with beneficiaries guided by an FGD instrument in order to gather beneficiaries’ perspectives regarding the provision and uptake of SPSS services for HIV/AIDS, TB, FP, GBV, and SRH (Annex 3e). Beneficiaries were grouped into four categories: 1) people living with HIV (PLHIV), 2) youths aged 18-24, 3) commercial sex workers (CSWs), and 4) other SPSS clients present during site visits. The evaluators acquired participants’ informed consent before proceeding with the FGDs. Recognizing that health behaviors and outcomes are inexorably linked to gender norms in Zimbabwe, and that the HIV prevalence rates for youths aged 15-24 is almost twice as high for women than for men, focus groups with youths were sex-segregated in order to more candidly explore gender dynamics, as well as the uptake and perceptions of gender-specific SPSS interventions. Sampling and Respondent Selection Site Visit Selection The evaluation team visited a total of 49 sites in 16 districts across all 10 provinces in order to collect qualitative data (Table III and Annex 2). Sites were selected using purposive sampling based on several sampling criteria: geographic representation, type of service-delivery point (New Start, New Life, VMMC, condom outlet), type of delivery model (fixed, outreach), implementing partner (PSI or local entity), and variety of services provided (See Annex 5 for the full list of sites visited). The evaluation team selected when to visit an outreach site, although the outreach location itself was dictated by the predetermined outreach schedule of the anchoring fixed site. Table III: Geographic Areas Visited Province Districts/Sites Visited Harare Harare urban (PSI HQ, USAID, CESHHAR, FACT, NAC, City Health, MOHCC, DFID, ISP, ZACF, DAPP) Mashonaland East Chitungwiza (NS fixed) Manicaland Mutare (MOHCC, City Health) Chipinge (NS and NL fixed and outreach, VMMC outreach) Buhera (NS and NL fixed and outreach, VMMC outreach) Mashonaland West Makonde (NS and NL fixed sites) Mashonaland Central Bindura (MOHCC, NS fixed, NL fixed, NS outreach, condom distribution) Shamva (NL outreach, VMMC, outreach) Guruve (NS outreach) 6 Masvingo Masvingo (MOHCC, NL fixed, NS fixed, VMMC fixed, condom outlets) Chiredzi (NS fixed, NL fixed) Bulawayo Bulawayo urban (MOHCC, City Health, NAC, NS fixed, MAC, VMMC outreach) Matabeleland South Insiza (NS outreach) Plumtree (NS fixed) Matabeleland North Nkayi (NS outreach) Midlands Gweru urban (MOHCC, City Health, NS fixed, NL outreach) Respondent Selection for Qualitative Interviews Key Informant Interviews: Key informants were purposively sampled based on their relevance to the project and availability to be interviewed. Participants represented a variety of implementing partner organizations, donor organizations, and GoZ structures, including: NAC, MOHCC, City Health Departments, USAID, DFID, Integrated Support Programme (ISP), Clinton Health Access Initiative (CHAI), Zimbabwe AIDS Caring Foundation (ZACF), the Centre for Sexual Health and HIV AIDS Research Zimbabwe (CESHHAR), PSI/Zimbabwe, Batanai, Family AIDS Caring Trust (FACT), Murambinda Mission Hospital, Colin Saunders Hospital, DAPP/Home Humana People to People, Matabeleland AIDS Council (MAC), and North Eastern Medical Centre (NEMC). Contact information and initial introductions were facilitated by either USAID or PSI, whereas the evaluation team coordinated the interviews. Semi-Structured Interviews: The finalized site selection list was shared with the sites in order to ensure staff availability for interviews and to coordinate logistics for visiting outreach sites. The evaluation team used a combination of purposive and convenience sampling which balanced the need to interview various types of staff members with staff availability given work demands. Focus-Group Discussions: FGD participants were recruited through communications with the SPSS site ahead of the scheduled arrival. Because of the logistical need to make the most efficient use of the evaluation team’s time, most FGD participants were recruited via convenience sampling of clients present at a site on a given day. Although the majority of FGDs occurred without advanced notice, few FGDs—such as those sex workers and some FGDs with PLHIV—were facilitated by the implementing partners to ensure participant availability. Strengths and Limitations of the Methodology Strengths • The evaluation team visited a variety of project site types across a wide range of geographic areas, and conducted an extensive number of interviews with stakeholders at multiple levels of engagement with the project. This diversity allowed the evaluation to capture a wide range of opinions representing all aspects of the SPSS project. • The evaluation team triangulated quantitative and qualitative data, as well as primary and secondary data sources, thus allowing it to cross-check findings and to identify and explain nuances that could have been otherwise missed. • The evaluation team conducted sex-segregated focus groups for youths aged 18-24, in order to encourage the freer participation of young women whom the evaluation presumed would not be as forthcoming in a non-segregated FGD. 7 Limitations • The evaluation team tried but was unable to acquire national level data on a monthly basis in order to more accurately compare SPSS contributions to national achievements. Instead, the evaluation was able to use annually reported national level data from national reports. Although the SPSS reporting cycle (September-October) and the national reporting cycles (January-December) do not exactly coincide, the three-month difference was not deemed significant enough to impeded direct comparison. • The evaluation did not look at project financial data by sub-awardee site in order to determine site￾level efficiencies as part of Evaluation Question 3. This was due to limited time and resources for conducting a cost-effectiveness analysis using quantitative financial data. Instead, the evaluation team conducted an in-depth interview with PSI’s Director of Finance and Administration in order to understand sub-awardee effectiveness and efficiency. • Because random sampling was not used, there exists the potential of self-selection bias on the part of participants for focus groups in which convenience sampling was used, selection bias for focus groups facilitated by implementing partners, and desirability bias on behalf of respondents. However, the large sample size for both sites and respondents should substantially mitigate the risk of these biases. • Political rallies and a concurrent USAID evaluation hampered site visits in a few instances. The evaluation team was able to mitigate this disturbance by rescheduling site visits or doing additional interviews at more accessible sites in the affected areas. 8 FINDINGS, CONCLUSIONS & RECOMMENDATIONS FINDINGS Evaluation Question 1: What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS? The evaluation assessed the SPSS project’s contribution to Zimbabwe’s HIV prevention efforts at two levels: (1) the project’s alignment to the national HIV and AIDS response strategy as guided by the second Zimbabwe National HIV and AIDS Strategic Plan (ZNASP II) 2011–2015, and (2) the project’s proportional contribution to the national outputs and outcomes. The evaluation analyzed each project component individually, attempting, wherever possible, to identify contextual factors behind differences between sites in different geographical locations and/or sites offering different blends of service packages. Alignment and Contribution of SPSS to the National Strategy The goal of ZNASP II is to improve the health of the people of Zimbabwe through: a. reduced HIV prevalence among young adults, and; b. reduced mortality and morbidity among People Living with HIV (PLHIV). SPSS aims to increase the availability of social sector services and related products through the private sector. This makes its goal a strategy, or the means of achieving the national strategic plan. Table IV summarizes how SPSS is aligned with all priorities of ZNASP II. Table IV: SPSS Alignment with ZNASP II ZNASP II priority SPSS strategies responding to priority SPSS Responsiveness To reduce new infections in children and adults through vertical HIV transmission by scaling up VMMC, promoting condom use for the HIV– positive and HIV–negative, HTC and behavior change communication HIV Testing and Counseling (HTC), VMMC Scale Up, male and female condom social marketing and distribution. Entirely responsive To reduce mortality and morbidity related to HIV and AIDS by increasing access to treatment for adults and children who test HIV–positive and supporting these to enjoy a good quality life (balanced diet, viral load suppression, adherence to treatment, regular reviews) Provision of ART at four treatment centers, as well as treatment adherence, nutritional counseling, and facilitation of support groups at all New Life centers. Entirely responsive To reduce stigma and discrimination against PLHIV – Zimbabwe is pursuing a zero tolerance to stigma and discrimination against PLHIV through social and behavior change communication as well as advocacy targeted at some national policies and laws. Counseling sessions are highly confidential, New Life promotes disclosure of one’s HIV status, VMMC is done on HIV–positive and negative clients Entirely responsive 9 An interview with a senior NAC official further revealed that SPSS contributes very actively to the sub– strategies of ZNASP II, including the National Male Circumcision Strategy (through VMMC), National Testing and Counseling Strategy (HTC), Treatment Strategy and PMTCT Strategy (New Life), and Combination Prevention Strategy (service integration). The project’s alignment to ZNASP II priorities has enabled it to contribute meaningfully to the national HIV prevention targets. Project contribution to national targets The project’s ability to meet its own targets meaningfully determines the level of its contribution to national targets. The evaluation therefore analyzed the project’s performance against its own targets before determining its contribution to national targets, using available MOHCC statistics. i. HIV Testing and Counseling (HTC) through New Start The Zimbabwe Global AIDS Response Progress Report of 2014 recognizes that “HTC services are available to all citizens that need them, including key populations,” and it counts PSI among “the [four] key implementing partners of the [national HTC] program” (others are OPHID, ZAPP and WHO”). The SPSS project has either met or exceeded its target each year for indicator P11.1D/HTC_TST_DSD: “Number of individuals who received Testing and Counseling (T&C) services for HIV and received their test results,” which reflects both new and repeat clients (Figure 1.1). Figure 1.2 shows a steady rise in the uptake of HTC from October 2010 to 2014.9 After reaching 98% of the target in Year 1 and 2 where uptake was gathering momentum, the project managed to surpass its target in Year 3 (110%) and Year 4 (119%) as well as overall (106%) owing to its refinement of strategies along the way, especially for creating demand. In effect, 30% of national HTC clients were attained through the project’s efforts between 2011 and 2013. The achievement of the target in 2013 is notable considering that the project closed the New Start site in Kadoma due to issues with the implementing partner. Throughout the four years, the majority of clients (ranging from 58–67% of the 9 Whereas the data source for Figure 1.1 is project APRs, the data source for Figure 1.2 is the sum of the target and output figures for all New Start sites which were received directly from PSI. See Annex 8 for more information. Figure 1.2 Figure 1.1 Figure 1.2 10 yearly output over four years) accessed HTC through mobile outreach services. The 2014 SPSS Annual Report attributes the high increase in clients accessing HTC that year to the integration of services and cross–referrals from the growing VMMC program. Cross–referrals from VMMC have significantly increased numbers of male HTC clients in line with the national call, beginning in 1999, to boost male participation in HIV prevention. ii. New Life Qualitatively the evaluation showed meaningful progress in positive living among PLHIV as a result of New Life, which contributes towards the national plans to reduce “annual death rates from 71,299 in 2010 to 51,808 by 2015.” Indeed, some PLHIV confirmed through their life stories that before the introduction of New Life they defaulted on ART because of either denial or fear of stigma and, as noted by the 2014 Global AIDS Response Progress Report, some PLHIV admitted that they continued to transmit HIV to their sexual partners. All New Life clients engaged in this evaluation felt a difference between the times when they collected their medication at the clinic without supportive counseling, and now with those services available through New Life. The intensive counseling that New Life teams introduced complements drug dispensing services that public health facilities are limitedly able to deliver. Part of ZNASP II reads: “The strategic plan will accelerate the provision of ART to close the gap (between current achievements and the target). It is projected that ART coverage will be increased from 31.5% in children and 59% in adults in 2010 to 85% in both populations by 2015. Zimbabwe aims at reducing annual death rates from 71,299 in 2010 to 51,808 by 2015.” It further states that “Sustained provision of ART will not only help reduce death rates but also contribute to HIV prevention efforts.” With the Zimbabwe Global AIDS Response Progress Report 2014 observing that “levels of condom use among PLHIV are low despite high levels of sexual activity,” the SPSS Project has played an integral part in addressing national concerns for PLHIV through its New Life brand. SPSS-funded ART services is a pilot program primarily delivered through clinics at New Life sites and select New Start sites as well, and is mainly oriented to key populations rather than the general population. Nonetheless, its existence and the achievements thus far have contributed to the national strategic plan to increase “ART coverage … from 31.5% in children and 59% in adults in 2010 to 85% in both populations by 2015.” Figure 1.3 shows that while the project did not provide comprehensive HIV treatment and care services for its first two years due to delayed funding, it almost reached its target for 2013 and surpassed its 2014 target for indicator T1.1, “Number of adults and children with advanced HIV infection newly enrolled on ART.”10 This indicator reflects the number of new clients 10 In 2014 this PEPFAR indicator was modified to TX_NEW_DSD, “Number of adults and children newly enrolled on ART” Figure 1.3 11 receiving ART through project–funded clinics in Harare, Bulawayo, and Mutare. The drastic increase within one year for both targets and outputs may be linked to the expansion of mobile teams from one to three, and the provision of ART services to CSWs at six outreach sites at the “Sisters with a Voice” CESHHAR clinics. Sex workers are among the most at risk populations (MARP) – a specially targeted group in the ZNASP. iii. VMMC Since inception (October 2010), the project has expanded the coverage of VMMC services to 41 districts and two urban areas, reaching 290,746 men from 2010-2014. This number was achieved through three dedicated VMMC sites and 70 mobile teams. The figure alone constitutes 24% of the national VMMC target for 2015 and 95% of national VMMC outputs, which is a major contribution. VMMC service delivery was supported by a strong quality assurance system that accounted for a low adverse events rate of <0.5% against a national target of <1%.This achievement is very important considering that qualitative interviews suggest that the absence or low rate of adverse events positively influences VMMC demand and uptake. Respondents noted that a single major adverse event can erase all the gains in reputation that the initiative may have attained. The SPSS project’s combination of aggressive demand creation efforts in schools, traditional circumcision camps and other strategic platforms, coupled with strict quality assurance measures to prevent adverse events, has enabled an increase in the numbers of men undergoing VMMC between 2011 and 2014. According to Figure 1.4, the project only achieved half its targets in its first two years for indicator P5.1D/VMMC_CIRC_DSD: “Number of males circumcised as part of the minimum package of MC for HIV prevention services.” 11 The corresponding annual reports linked this to startup–related challenges (e.g., delayed funding). Furthermore, USAID informs that even though PSI reached a higher number of clients than reported in 2014, changes to the PEPFAR guidance in 2014 limited the age at which VMMC clients could be counted and reported under SPSS. Despite these challenges, the project managed to exceed its target in 2013 (101.4%) and largely meet its 2014 target (96.6%). 11 In 2014 this PEPFAR indicator was changed to VMMC_CIRC_DSD, “Number of males circumcised as part of the voluntary medical male circumcision (VMMC) for HIV prevention program within the reporting period (DSD)” Figure 1.4 12 iv. Condom social marketing From 2010-2014 in Zimbabwe, 365,295,616 male condoms and 19,816,595 female condoms have been distributed between PSI and the public sector. PSI’s Protector Plus and care brands claimed a significant share of these volumes (approximately 30%) as shown in Table V. 12 Table V: Condom Consumption for SPSS and Public Sector Calendar year (Jan– Dec) Condom consumption Protector Plus Public sector Total male condoms distributed Protector Plus % of total sales Care Public sector Total female condoms distributed Care % of total sales 2010 24,662,180 44,806,120 69,468,300 36% 1,417,540 3,270,693 4,688,233 30% 2011 29,839,890 61,711,140 91,551,030 33% 1,294,680 4,028,078 5,322,758 24% 2012 27,925,290 67,437,496 95,362,786 29% 1,409,600 3,566,056 4,975,656 28% 2013 28,077,040 80,836,460 108,913,500 26% 1,626,720 3,203,228 4,829,948 34% Total 110,504,400 254,791,216 365,295,616 30% 5,748,540 14,068,055 19,816,595 29% The volumes of national condom distribution through the public sector, which is free of charge, have generally increased over the years from 2010 to 2014, whereas the pattern for Protector Plus and care condoms has been fluctuant because of dynamics in the market related to macroeconomic challenges beyond the project’s control. The number of socially marketed Protector Plus condoms sold increased between 2010 and 2011, before decreasing in 2012 and increasing again by 2013. Sales of the care female condom have been increasing with each year beginning in 2011, thanks to the project’s strategy of channeling this condom through such targeted markets as sex workers and hair stylists. Combined, the Protector Plus and care condoms have successfully constituted 30% and 29% of the total annual condom sales respectively for combined public sector and SPSS socially marketed condoms. Qualitative interviews suggest that the lower uptake of female condoms may be due to cultural barriers. For example, interviews in pharmacies and service stations showed that people could not freely buy male condoms in the presence of other customers probably because “of a perception that the product is associated with loose sexual behavior and used among unfaithful people cheating [on] their spouses” (Pharmacist). As a result, with the exception of sex workers, women generally do not buy male condoms from retail outlets, and the SPSS project strategically decided to sell female condoms through IPC channels such as hair salons and CSW networks. v. Social and behavior change communication According to Zimbabwe’s Global AIDS Response Country Progress Report 2014, social behavior change communication “interventions were intensified in the community, workplace and in schools, targeting most at risk and key populations.” Acknowledging “SBCC as a high impact programme in Zimbabwe, and as one of the key interventions (behind) the decline of HIV prevalence,” the same progress report identifies two implementing partners of the SPSS project, FACT Mutare and Midlands AIDS Support Organisation (MASO), among the key partners leading the implementation of SBCC. SPSS has been very innovative in using a multi–media approach to spread social and behavior change messages and to create demand for its services, whereby service uptake constitutes behavior change. Figure 1.5 shows the performance of the project’s SBCC efforts against targets for Indicator P8.1: “Number of the targeted population reached with individual and/or small group level HIV prevention interventions that are based on evidence and/or meet the minimum standards required.” This indicator reflects the number of 12 The figures do not include private sector brands. 13 individuals who participated in small group level HIV prevention interventions on VMMC and/or male and female condoms driven by IPC agents. The project exceeded its targets in 2011 and 2014, but fell short in 2012 and 2013. According to the project APR, the 2012 shortfall resulted from budgetary constraints after some anticipated Global Fund complementary funding was not availed. In addition, the project made a programmatic decision to integrate messages on concurrent sexual partnerships into small group discussions on VMMC. Due to budgetary constraints, stand-alone activities for concurrent sexual partnerships were stopped in response to research data suggesting lack of epidemiological impact. Instead, more emphasis was placed on demand creation for bio medical interventions. In 2013 the target shortfall followed the project’s scaling down of IPC activities because of national elections that were held that year. In general, therefore, the project has mostly met or exceeded its targets over the evaluated period. The project has also contributed between a quarter and more than a third of national targets in all components discussed above. The cross cutting contributions of the project to the national response to HIV and AIDS include the following: i) Raising HTC uptake and boosting numbers of early treatment seekers –HTC is widely viewed as “a crucial first step in the cascade of HIV treatment and an entry point to other prevention and care interventions including male circumcision, prevention of mother–to–child HIV transmission (PMTCT), and treatment of opportunistic infections (OI).”13 The SPSS Project served about 30% of all Zimbabweans who have been tested for HIV and know their status, thereby multiplying the number of people enrolling for antiretroviral therapy (ART) and OI. This is because the project carefully streamlined its fixed sites, reducing them from 19 in 2011 to 16 currently operating, and steadily increased its outreach sites over the years. Indeed 67% of HTC clients are served during outreaches. The project further intensified its demand creation activities and spearheaded VMMC, which encourages clients to be tested for HIV first, thereby boosting numbers of men who know their HIV status. 13 MOHCC (2014). Global AIDS Response Country Progress Report Figure 1.5 14 ii) Leading the way in VMMC service delivery – “Male circumcision is one of the key components of the National Combination Prevention Strategy.” 14 “The government (of Zimbabwe) has set a target to circumcise 1.2 million men by 2015 and Zimbabwe has had one of the largest increases in VMMC of any country, with the number of operations done increasing from 2,801 in 2009 to 36,742 in 2011.” 15 PSI, through the SPSS project, pioneered this work and has contributed 95% of the circumcisions done to date in Zimbabwe. It has spearheaded the training of doctors, nurses and counselors in surgical and, more recently, the Prepex methods of circumcision. Between 2012 and 2014, the project trained 511 healthcare workers in VMMC service delivery. On the other hand SPSS has generated demand for VMMC even for clients who are seeking the services elsewhere. iii) Filling gaps in public health sector services – SPSS has served as a private sector alternative for health services that were either absent or delivered with difficulty through the public health system. For example, all interviewed clinical staff at public health facilities felt they could not afford time for counseling patients as intensively as SPSS staff because of their heavier workload and compromised numbers. With the advent of New Life outreach services, which target clinics on days when PLHIV replenish their drug supplies, rural health facilities have managed to attract increased numbers of ART and OI clients, according to FGDs with PLHIV: “This is because clients receive adequate group and individual counseling from New Life teams, allowing us to concentrate on dispensing drugs,” a nurse at a rural clinic said. SPSS has also alleviated the government’s challenges related to resources—SPSS outreaches have enabled MOHCC, which otherwise is limited by its small vehicle fleet, to widen the geographical coverage of some of its health services. Reflecting this, a MOHCC official said, “if a private partner is providing the same service as the public service provider, then what makes it relevant is its ability to reach areas that the public service provider cannot access.” Accordingly, in all government and donor officials’ interviews the complementary presence of SPSS was viewed as having added value to referrals. iv) Generating new knowledge through pioneering key interventions and research – The project’s pioneering work in HTC, VMMC and promotion of the female condom has generated highly useful lessons that can be productively applied in other settings. PSI-initiated research studies have generated new knowledge on trends in demand, acceptability of specific services, and cost effectiveness of given strategies. Through a partnership with CESHHAR, the project is generating new research on the concept of “treatment as prevention” for CSWs. v) Adding value to ART, opportunistic infections (OI) treatment and HIV prevention – PLHIV confirmed during all FGDs that ART, OI treatment and support group membership alone were insufficient to promote adherence to treatment or decent nutrition. “Without the kind of education, psychosocial and nutritional counseling that we now receive from New Life, we made many mistakes that negatively affected our own and other people’s health,” one discussant said. Common examples of errors that PLHIV made before the introduction of New Life services include defaulting on ART once their health conditions improved, unprotected sex among PLHIV couples, giving up on ever having children again, and disregarding the prescribed times of taking drugs. In fear of being stigmatized and/or discriminated against, PLHIV reportedly kept their HIV status secret even to the extent of having unprotected sex with uninfected partners. This situation has changed, as PLHIV reported they now adhere to condoms more consistently than before their interactions with New Life, and they encourage other community members to take HIV tests in order to make informed life–enhancing post-test decisions. Some couples are receiving professional advice on bearing HIV–negative children. 14 MOHCC (2014). Global AIDS Response Country Progress Report 15 UNGASS (2012). Zimbabwe Global AIDS Response Progress Report 15 vi) Multiplying the sectors responding to HIV and AIDS –“Multi–sector response” is one of the key terms of ZNASP II, and strongly encouraged, although limited resources within the public sector made it difficult to accomplish. The efforts of SPSS to widen community access to such services as HTC, VMMC and condoms have culminated in mining companies and commercial farms, traditional leaders, community–based organizations (CBO), and schools all actively participating in the national response to HIV and AIDS. VMMC services were offered at prisons, while leaders of traditionally circumcising ethnic groups, such as Shangaani in Manicaland and Chewa in Mashonaland Central, were engaged to allow for the safe surgical circumcision of their young men. In August 2014, some 2,150 Shangaani men were surgically and safely circumcised at their circumcising camp through SPSS. Existing CBOs and community networks of such volunteers as village health workers (VHW) partook in mobilizing communities and schools for VMMC outreaches, whereas female condoms permeated the market through networks of sex workers and hair salons. Male condoms have been effectively marketed through wholesale and retail outlets, pharmacies and service stations, thereby involving business people in the national response. Tertiary students’ networks were engaged to facilitate the marketing of condoms among youths at colleges and universities where casual sexual activity is rampant. This multi– stakeholder buy–in has been and will remain an important foundation for the sustainability of future health programs in Zimbabwe. vii) Demystifying HIV and AIDS through open discussions of the epidemic – Through the project’s New Life brand, the open discussion of HIV and AIDS was encouraged, with PLHIV being urged to publicly disclose their status to fellow community members, their spouses and/or children. Qualitative interviews suggest that the disclosure of one’s HIV status has enabled some PLHIV to become effective leaders in HIV prevention in that they mobilize community members to take HIV tests. “People who are afraid of facing a possible positive HIV test result come to us secretly to ask questions, especially about how we have made it, and we encourage them to be tested,” one support group member echoed the general view of PLHIV. Indeed half of the participants of this evaluation’s FGDs comprised people who said they had been encouraged to take HIV tests by their peers already on ART or OI treatment. All PLHIV who informed the evaluation had disclosed their HIV status to their children, communities or spouses, with two couples having started dating during support group meetings. FGDs with PLHIV noted that while stigma and discrimination against PLHIV remain a challenge, involvement with SPSS supported activities has improved the resilience of some PLHIV to withstand stigma. HIV–related services were previously associated only with hospitals and clinics, being viewed as very complex operations, but according to this evaluation’s FGDs, “today we get tested for HIV, get a CD4 count test and collect results of both under a tree very close to our village because of New Start” (FGD participant at a New Start outreach). viii) Stabilizing supply of essential health commodities – The evaluation revealed that SPSS brought consistency to the supply of key equipment, medication, and related supplies. In line with the general view among MOHCC professionals, one doctor said, “referrals are now more meaningful because there is always a place where clients will find the help they need because of this project’s intervention.” At all condom outlets that the evaluation team visited it was shown that stocks of the product were never out of supply and that the project has always improved them to increase their appeal to clients of various tastes, notably with the scenting and flavoring of the male condom. Hospital staff conducting male circumcisions with SPSS support similarly cited “efficient supplies of relevant kits” as one of their most key enabling factors. The same was the case with family planning products. Before the project the National Pharmaceutical Stores (NATPHARM) could not meet demand from health facilities for medical supplies and equipment on time or at all. ix) Capacity building – The SPSS project helped build the technical capacity of the entire health sector of Zimbabwe through trainings of staff and resource persons, research, and partnerships with 16 local organizations or franchise holders. Staff at service delivery sites received training in diverse medical and clinical operations such as VMMC surgical operations, use of the Prepex device, HIV care and treatment, referrals management, and counseling. Sex workers and hair stylists were trained on using female condoms and demonstrating to others how to wear them. Trainings are good for the sustenance of activities related to the project in the future because a considerable proportion of the trainees serve in mainstream health facilities and communities where they can use their new skills outside the context of the SPSS project. SPSS–commissioned research generated new knowledge that has benefited the entire health sector of Zimbabwe (e.g., the 50 pilot surgical circumcisions in 2010 and the Prepex device were tested as part of the project but are implemented at scale across the health sector). Opportunities exist for the project to similarly pilot the self–testing model of HTC prior to scale up. SBCC messages currently being delivered are evidence–based, and demand for HIV–related health services has increased across the health sector. The evaluation’s FGDs confirmed this with HTC and VMMC clients, about 80% of them reporting as follows: “it was easy for IPC agents to convince us to be circumcised (or HIV tested) because we had heard about the service on radio or television.” The idea of combining MMC with IPC was research–informed. Research further inspired the packaging of MMC, including the use of celebrities to market services, as well as the decision to tone down on relating benefits of behavior change to the ‘scary’ HIV and AIDS threat and replacing that with associations of “smartness,” “Pinda muSmart/Ngena kuSmart- Be Smart” tag. Lastly, the partnership of PSI with national and local NGOs managing New Start and New Life franchises, coupled with quarterly supervisory visits, financial, technical and material support, cemented the standards of quality that these service brands have established over the years under PSI. The upholding of these standards outside a partnership with PSI in the future will potentially benefit and add value to Zimbabwe’s health sector response to HIV and AIDS. x) Creating demand for health services –SPSS significantly generated demand for HTC, VMMC, condoms, post–test HIV services and other integrated services management, etc.) through its IPC efforts, community mobilizations prior to outreach, and MMC. Such demand is enjoyed beyond sites that PSI and its partners manage, and going for HTC, VMMC, buying condoms and so on reflect positive behavior change that is required for the effective containment of HIV and AIDS. xi) Widening scope, coverage and reach of services – With the project’s experience has come innovations that have progressively widened the scope, coverage and reach of essential services. Outreaches, for instance, have taken HIV services to commercial farms, mines, prisons, schools and hard–to–reach populations where transport flow is erratic. 16 The decision to integrate services saw the project responding to challenges outside the clinical and medical domains, including such socioeconomic aspects of the epidemic as gender–based violence (GBV) and medical conditions that are unrelated to sexual behavior (e.g., blood pressure testing). xii) Contributing to positive policy and strategy shifts – The participation of PSI and its SPSS implementing partners in technical working groups and district–level structures such as District AIDS Action Committees (DAAC) for lobbying and advocacy has seen the national health system introducing policies to smoothen the delivery of essential services. One of the most celebrated moves of this kind is the approval of task shifting, which has allowed nurses to administer surgical male circumcision, as noted in the 2014 Annual Report: “PSI’s continued lobbying for the broadening of the scope of practice for nurses to include the male circumcision surgery paid dividends as it was approved by the Permanent 16 This evaluation’s definition of ‘hard–to–reach’ encompasses geographical areas that are distant from health facilities, economically poor individuals who cannot afford health services, and people who because of their literacy levels or living conditions are scantly informed or lack information about HIV/AIDS. 17 Secretary for Health in January 2014 and it was included in the scope of work of nurses but limited only to the nurses working in the VMMC program.” Enabling factors for project success i) Service integration – The availability of multiple HIV–related services under one roof at most sites increased convenience to customers and made the services more attractive than when standalone services were provided. The low cost of accessing the integrated services made them more popular. The evaluation revealed that client congestion has always been higher at sites offering a wider range of integrated services than others. Respondents in 27 out of 74 interviews (36%) noted that service integration has led to clients accessing a wider range of SPSS services. A good example is the contrast between New Africa House New Start site, which offers a comprehensive package of services and has been surpassing its targets, and Harare City Center VMMC Clinic, which specializes in VMMC and a few other complementary services reported of struggles to meet its targets. ii) Strong reliance on IPC and mobile outreach – According to the 2011 SPSS Annual Performance Monitoring Report, “IPC–supported channels (e.g., hair salons, organizations, faith–based organizations, support groups and sex worker groups) continued to be the major volume drivers (for female condoms) by contributing 89% to total sales.” Mobile outreach enabled the program to reach underserved communities such as mining areas, commercial farms, and other workplaces and hard–to–reach communities where services are badly needed but were inaccessible. iii) Evidence–based market segmentation and targeted promotions –SPSS–supported research informed the segmentation of the market for its services, accounting for such decisions as the need to channel female condoms via sex workers and hair stylists, and the need to associate VMMC with ‘smartness’ and celebrities. This enabled the project to penetrate hard–to–reach areas, resistant populations (e.g., non–circumcising ethnic groups such as the Ndau) and to demystify the HIV epidemic by taking it out of the medical domain and to the community. iv) Multi–stakeholder engagement – Demand and uptake of the HIV services have been increasing, due to the project’s engagement of major stakeholders at various levels of society – traditional leaders (promotion of VMMC), business operators (condom marketing), miners and farmers (HTC, VMMC, family planning, condoms), prison authorities (HTC, VMMC, condoms), celebrities (VMMC), CBOs and volunteer networks (demand creation in general). This factor is tellingly behind the increased uptake of HIV services among men who traditionally have been associated with a poor health seeking record. v) Strong coordination between MOHCC and PSI (and partners) – The project and MOHCC integrated their resources and strengths most notably in the delivery of VMMC and New Life outreaches. The project’s support facilitated the outreach that widened the coverage of health services while MOHCC facilities that were used as sites for delivering the project’s services availed captive audiences. On the other hand the project’s demand creation efforts have promoted services delivered at both the project sites and MOHCC health facilities. vi) Smooth transition from demand creation to service provision – One of the reasons why service uptake has been generally high is that IPC agents create demand for services that are within accessible range in communities courtesy of mobile outreach teams. This prevents clients from taking an “I will think about it” response, as clients who express an interest in the services are offered the service at their convenience (VMMC clients are transported from their communities to the circumcising site and back). vii) Leveraging of resources from different sources – The evaluation noted that PSI and many implementing partners had other funding partners apart from USAID. Additional funds enabled PSI and local implementing partners to positively respond to clients’ request for additional services. For example, Global Fund, DFID, Bill and Melinda Gates Foundation and other funders were found to support complementary services such as the packaging of condoms (DFID), demand creation (DFID, 18 Bill and Melinda Gates, and Global Fund), and cervical cancer screening (DFID). These complementary initiatives facilitated service integration, which emerged as a key pull factor to HIV service delivery sites. While transportation shortage is a serious constraint across all program components and areas, PSI and implementing partners promoted the sharing of the few available vehicles among departments to make outreaches a success. viii) Favorable national policies – The MOHCC’s approval of the policy allowing nurses to carry out surgical male circumcision operations coupled with the subsequent training of 82 nurses thereafter have increased the efficiency of VMMC delivery in public health facilities where doctors’ availability can be erratic due to their need to divide attention between VMMC and other medical duties. ix) Project participation at strategic platforms – The MOHCC policy position to authorize nurses to perform surgical male circumcision procedures resulted from persistent waves of lobbying and advocacy that the project spearheaded through platforms ranging from the grassroots level types such as DAACs to national ones like technical working groups. x) Results–driven sub–granting – PSI sub–granted project resources to partners that it had been working with for several years before receiving SPSS funding. These partners, despite having proven their capacity to properly manage projects and resources, underwent prior capacity assessment and entered contracts that require grants to be disbursed upon the submission of key deliverables and either evidence of meeting the agreed targets or reasonable justification for failing to do so. This has sustained high performance standards across the project. xi) Organizational flexibility – The evaluation noted that the project’s ability to meet its targets, sometimes amid challenging circumstances, is a result of the implementers’ flexibility to deal with emerging shocks. One good example is the project’s maintenance of high VMMC uptake figures in spite of a change in PEPFAR policy effected after receiving WHO guidelines discouraging the use of the forceps guided method for the circumcision of boys aged <14 years. In this case, PSI turned to DFID funding to support the continuance of circumcisions for <14 year old boys. Inhibiting factors and challenges The notable inhibiting factors and challenges that adversely affected project performance can be classified into five groups: resource–related, national socioeconomic and political, physical/geographical, and behavioral or cultural factors (Table VI). For example, although the project has widened awareness among clients of the range of services that are available through SPSS, respondents in 50 out of 74 interviews (68%) commented that access to services at the static site or referral sites remains a challenge due to the long distances in rural areas, poor quality of roads, and the erratic flow of public transport.17 From the project implementation side, vehicle breakdowns or lack of vehicles were cited in 45 out of 74 interviews (61%) as preventing teams from going on outreach and thus inhibiting project performance. 17 Implementation issues were predominantly explored in the 74 total SSIs with program staff and FGDs with beneficiaries, and not in the additional 24 KIIs with donors and government officials which focused on strategic issues. 19 Table VI: Factors Inhibiting Project Performance Category Challenge Effects on Project Performance Internal Factors18 Resource related Transport challenges The project’s vehicle fleet has been overtaken by travel requirements given the project’s growing emphasis on outreach, which has proven to be the most effective strategy of widening access to the project’s services. Rough roads in some outreach sites have accounted for the further wearing down of available vehicles. However, USAID notes that the project is in the process of up￾dating and adding to its vehicle fleet. Financial constraints Limited or delayed funding has contributed to the project’s inability to meet its targets or to postpone planned activities. “Due to restricted funds, no activities have been undertaken to integrate OI treatment and ART service delivery with New Life services this year.” 19 “No other mass media activities were implemented due to budgetary constraints.” 20 Delayed VMMC funding in 2011 and 2012 was responsible for target shortfalls in that area. Physical/ geographical Long distances to communities While observing that the gravest need for the project’s services, notably HTC, VMMC, condoms, STI treatment and family planning is in the hard–to– reach areas, distances to these communities are so long that few visits are possible per defined period (e.g., quarter). Following up referrals done to distant service providers usually yields very poor results. Poor roads Roads to some areas (e.g., Mutare or Chipinge to Murambinda, Concession to Guruve, Insiza roads, etc.) are sufficiently rough to cause fatigue that may compromise staff performance in the field and wearing down of vehicles. Cultural and behavioral Stigma and discrimination against PLHIV The evaluation revealed that stigma and discrimination against PLHIV is still common, and that the people who can better withstand it are those who undergo repeated sessions of sensitization and counseling. There remain some people who have not and do not desire to be tested for HIV in fear of being stigmatized and discriminated against. Misconceptions about VMMC The evaluation revealed that as much as VMMC uptake figures continue to soar among the target population (males aged 13–29 years), the bulk of circumcised men are in the 13–19 years age group because older men in the target group (20–29 years) seemingly viewed MC as being irrelevant to HIV prevention (given the need to still wear a condom) and risky. Misconceptions and negative attitudes towards condom use Although high volumes of condoms are being sold through SPSS, respondents in 31 out of 74 interviews (42%) spoke about barriers which continue to inhibit consistent condom use, including the limited ability of married women to negotiate condom use, associations of condoms with sexual immorality, the spontaneity of sexual intercourse among youths and debutants without a condom on-hand, the perception of safety in long term sexual relationships, and other factors. 18 Internal factors are those which the project and its stakeholders may control or reasonably respond to. 19 PSI (2011) Zimbabwe Strengthening Private Sector Services (SPSS) Project, Annual Performance Monitoring Report, 1 October 2010 – 30 September 2011. 20 Ibid 20 Category Challenge Effects on Project Performance External Factors21 Resource related Irregular availability of government employees The availability of key government healthcare workers, especially doctors, at public health facilities for VMMC operations is sometimes compromised because healthcare workers have to divide their attention between their routine duties and VMMC work. The SPSS project is limited in its ability to address this challenge, given that the challenge involves public sector employees who are beyond the control of the project. Policy related Church policies Some church denominations that run health facilities such as the Roman Catholic–managed Murambinda Mission Hospital (MMH) do not approve of family planning methods that the project promotes. So while potential clients for more integrated services are readily available at these sites, opportunities to offer a full range of family planning services are missed. USG restrictions Current USAID restrictions do not allow the project to channel funding through government structures. While this has its own merits that this evaluation will not question, it is one of the issues about which MOHCC officials have always contended with SPSS teams whenever stakeholder buy–in has been discussed. Ministry of Education Sports Arts and Culture stance on VMMC Although most demand for VMMC currently comes from school-age males, decision makers in the Ministry of Education, Sports, Arts and Culture (MOESAC) offers limited support to the program (e.g., this ministry’s insistence on having MCs done during school holidays increases costs of outreaches and reduces VMMC uptake). National, socioeconomic, and political National elections; political rallies, etc. In 2013 the project scaled down its IPC work, as political campaign rallies and subsequent national elections that year were not compatible with any form of group work or gatherings in communities. A major school campaign was canceled in 2012 to allow for national census errands to proceed without disruption. Business viability challenges Sales declined following the closing down of some retail or wholesale outlets through which the project channeled its condoms. “Two of the country’s major wholesalers (Jaggers and Redstar) closed all 55 of their branches as they failed to recapitalize their business under the multi–currency system introduced in 2009. PSI therefore made efforts to recruit new upcoming wholesalers into the network of Protector Plus stocking outlets. However, these new wholesalers have limited reach given their relatively smaller size and limited geographical coverage.”22 Poverty among beneficiaries PLHIV and healthcare workers reported that participation in support groups was declining as members prioritized activities meant to address their poverty. Condom dealers reported that because of poverty, condoms competed with other priorities for the few incomes in people’s pockets and basic necessities such as food would always prevail. MOHCC operating challenges As much as the partnership with MOHCC is highly commendable, limited cooperation for referral tracking on the part of government health facilities to which SPSS clients are referred makes it difficult for SPSS to prove referral success and ensure a continuum of care. Some respondents also suggested that clients have lower satisfaction with the quality of public sector services than the quality of service at SPSS sites due to privacy issues and staff shortages at government health facilities. 21 External factors are those that lie beyond the project’s control. 22 PSI (2011). Zimbabwe Strengthening Private Sector Services (SPSS) Project, Annual Performance Monitoring Report, 1 October 2010 – 30 September 2011. 21 Sub-question 1a: What, if any, “niche” populations has SPSS served? Has SPSS served different needs for men and women, different age groups, etc.? Gender The project has provided services that are equally relevant to both women and men. These include HTC, post-test support and care services, HIV treatment through ART, TB screening, as well as sexual and reproductive health services like STI awareness and condom distribution. In addition, the project has recognized and sought to fulfill the different sex-specific needs of its clientele. For example, family planning counselling and linkage to a range of FP methods is routinely offered to female New Start and New Life clients. VMMC is provided for males, and different communication strategies are utilized to create demand for care and Protector Plus condoms respectively to females and males. GBV counselling, although for both men and women, is usually targeted towards women, and was sometimes referred to by respondents in interviews as “prevention of violence against women and girls services.” Even for these sex-specific services, the project has attempted to engage the opposite sex as a strategy for ensuring that a person of the targeted sex has the necessary social support to follow-through with the service. For example, VMMC messaging has emphasized the procedure’s contributions to cervical cancer prevention, thus indirectly meeting a female-specific need. Family planning counselling and referrals for clients who come as couples have the added benefit of meeting men’s needs for family planning, even if the service itself is undertaken by the female partner. Although the project has served different needs for men and women, respondents in five out of 62 interviews that touched on the issue of “niche” populations mentioned that more services are offered for women than men,23 which in some instances has impacted client-perception of the New Start and New Life aspect of the project. As perceived by one respondent, “Our BCC partners in NAC have said that people are starting to associate New Start as a for-women organization,” although it is unknown how widely this association is held among the general population. The perception may contribute to the finding in 13 out of 62 interviews that it is harder to attract men than women to New Start and New Life centers, and in six out of 62 interviews that it is hard to attract couples, although couples have historically represented only a small portion of clients. To attract couples, the project has experimented with offering promotions for couples to some success, but this method is subject to sustainability concerns. Respondents in only three out of 62 interviews commented that there is an approximate 50- 50 split in the number of men and women who access services at their New Start and New Life centers: “Overall there is a higher number of women who come than men” (respondent at a New Life center). The qualitative evidence that more women visit New Start and New Life centers is validated by quantitative data for APR indicator C2.1D, “Number of HIV-positive adults and children receiving a minimum of one clinical service,” which includes the male-female breakdown of New Life clients (Figure 1.6). 23 Respondents did not necessarily comment on niche populations in all 98 interviews, which is why the denominator differs for evaluation sub-question 1a. Figure 1.6 22 Seven out of 62 interviews noted that outreach to workplaces and other isolated communities with large cadres of men have helped increase the number of men reached by New Start, New Life, and VMMC services. It was stated that “more men are coming when we go to workplaces, but in general the number of men is still low.” For sites that have successfully attracted men, the support of community leaders and the use of male-fora or networks were cited as successful strategies for encouraging men’s use of SPSS services, including couples testing. Other suggestions for increasing male involvement at New Start and New Life included offering either more male-specific (i.e. prostate cancer counselling) or general services (i.e. blood pressure checks such as at Bambanani New Start). Figure 1.7 Despite the challenges attracting men to the social franchises, quantitative data supports the finding that for HTC the project overall has succeeded in achieving an approximate 50-50 split in the number of males and females clients, with slightly more men receiving this service from 2012-2014 (Figure 1.7). Qualitative responses suggest that HTC provided at VMMC delivery points make up for a deficit in male testers at New Start sites: “VMMC has increased the access of men to HTC. Previously men constituted the smaller proportion of people taking HIV and AIDS tests, but now because of VMMC the numbers of men testing for HIV and AIDS is increasing.” Age Groups The project’s strategy of segmenting and targeting a population based on prior research means that it focuses its efforts on populations and/or age groups on which it expects to have the largest impact. For example, young men aged 20-30 have been the primary target audience for Protector Plus condoms, and advertising and packaging changes were implemented to appeal to this group—a finding established in the cooperative agreement and validated in four out of 62 qualitative interviews: “New packaging has also been introduced to target younger guys...there’s more style and swag within the packaging.” Similarly, the project strategically targets young males aged 13-29 for VMMC through school campaigns and other MMC efforts as outlined in its cooperative agreement and noted by respondents in 16 out of 62 interviews. In 2014 when changes in PEPFAR policies limited the use of the forceps Figure 1.8 23 guided method to boys 14+ years, the project responded to meet the needs of those <14 by adopting the dorsal slit method instead. However, of this age group the project has been more successful in reaching, and thus meeting, the needs of boys aged 13-19. This finding was confirmed qualitatively in eight out of 62 interviews, and also quantitatively when examining age-disaggregated data for APR indicator P5.1D/VMMC_CIRC_DSD, “Number of males circumcised as part of the minimum package of MC for HIV prevention services.” 24 Thirteen through 19 year olds comprised 54% of males circumcised in 2012, 69% of males circumcised in 2013, and 75% of males circumcised in 2014 (Figure 1.8). For HTC, qualitative interviews (three out of 62 interviews) suggest that the project’s use of voluntary testing and counselling (VTC) has more successfully reached people <30 years old. In contrast, the public sector’s implementation of provider-initiated HTC (PITC) is better at reaching individuals aged 30 years old and above. The evaluation was unable to examine this suggested finding further using quantitative analysis due to limited age-disaggregation in APRs.25 Although not in the SPSS project mandate, 14 out of 62 interviews noted examples in which the project has gone above its initial purview and attempted to serve the needs of children and adolescents through a family-oriented approach. Respondents shared that the project has allowed HTC for individuals <15 years old who receive parental permission, provided OI and ART services to children and adolescents, formed separate support groups for HIV+ youths, partnered with other organizations providing children and adolescent services, and has served children and adolescents who have survived GBV. Six out of 62 respondents, including both SPSS program staff and MOH staff, note that there is a gap for pediatric HIV/AIDS services in Zimbabwe in general, and recommend that health providers need better training on counseling and service-provision for children and adolescents versus adults if they are indeed serving young age groups. Key Populations SPSS has worked to address the needs of both Most At-Risk Populations (MARPs) as defined by PEPFAR,26 as well as other key populations such as workplaces and other isolated communities that are not included within the PEPFAR definition. The project exceeded its targets in 2011 and 2014 for indicator P8.3D, “Number of MARP reached with individual and/or small group level HIV preventive interventions that are based on evidence and/or meet 24 For 2011, age-disaggregated data was only provided for ages 1-14 and 15+, and so was excluded from Figure 1.8. 25 Age-disaggregation for those tested and counselled by the project is limited to <15 and 15+ years old. 26 The PEPFAR definition of MARPs includes commercial sex workers (CSWs), intravenous drug users (IDUs), men who have sex with men (MSMs) and “Others” (military and other uniformed services, incarcerated persons, mobile populations, clients of sex workers, and non-injecting drug users). SPSS only impacts and reports numbers for CSWs and “Others” (incarcerated persons, mobile populations, and clients of sex workers). Figure 1.9 24 the minimum standards required.” 27 However, it fell short of its target for this indicator in 2012 and 2013 (Figure 1.9). APR narratives cite the project’s intensive efforts to target high-risk areas, map new areas of sex work activity, and recruit new CSWs as factors enabling good project performance. Targets were set based on APR trends, which may explain the increase in targets from 2011-2012 and the decrease from 2012- 2013 based on the previous year’s performance. Due to the revision of PEPFAR indicators in 2014, the target and output for that year only include CSWs and not “Other” MARPs. Disaggregated data reveals that both the targets and outputs for CSWs reached have increased over the four years, with only a slight reduction in 2012. Despite the projects mixed record on achieve overall totals for MARPs reached, the project has consistently exceeded its targets for reaching CSWs (See Annex7b). Qualitative interviews substantiate the quantitative finding that SPSS has worked to address the specific needs of MARPs and other key populations. Respondents in 19 out of 62 interviews spoke about how SPSS has either directly provided or partnered with organizations like CESHHAR to provide services to CSWs, and respondents in five out of 62 interviews mentioned how IPC for care female condoms have specifically targeted CSWs. A few interviewees spoke about how the care condom packaging was redesigned to be more discreet based on the needs of CSWs. As much as the project has done to target CSWs, this group only makes up <15% of the MARPs reached by the project (Figure 1.10). The majority of MARPs reached fall under the “Other” category. In addition to the PEPFAR defined MARPs, PSI explains in APR narratives that it includes people living with disabilities and those living in enclosed settlements (i.e. commercial farms, mines) in the counts for “Other” MARPs reached. Respondents in 22 out of 62 interviews made note of how the project has been able to serve those in workplaces, prisons, refugee camps and other resettlements, as well as mobile traders and truck drivers through outreach: “We provide services to workplaces, especially the mining and agricultural sectors. Twenty percent of clients come from these workplaces.” Respondents in eight out of 62 interviews, as well as project Annual Reports, mention that the project has made efforts to provide condoms in high-risk “hot zones” for HIV/AIDS transmission. 28 27 In 2014 this PEPFAR indicator was modified to KP_PREV_DSD, “Number of key populations reached with individual and/or small group level HIV preventive interventions that are based on evidence and/or meet the minimum standards required (DSD)” 28 “Hot zones” include locations such as mining areas, border towns, growth points, resettlement areas, commercial farms and encampments. Figure 1.10 “PSI has worked with us to reach CSWs and mobile truck drivers. We are seeing a lot of improvement and achievements with these key populations.” –MOHCC respondent 25 Evaluation Question 2: How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals? Sub-question2a: To what extent were the integration of FP, TB, GBV and SRH within HIV services achieved? The findings that address the first evaluation question above show the integration of HIV services through the SPSS project as one of the enabling factors behind the project’s attainment and/or surpassing of its output targets. Under SPSS a number of services have successfully and with positive results been integrated into HIV services, albeit at varying degrees at different sites. Not all sites offer the full gamut of integrated services. Rather, services are integrated based on the presence and proximity of other service providers in the area, according to PSI. Furthermore, services can be accessed at fixed sites, during outreaches, and at referral sites external to SPSS. Integrating services within HTC The evaluation’s social network analysis manipulations showed HTC to be the service with the highest rating of centrality,29 being directly related or connected to the widest range of other HIV services such as ART, STI treatment, FP, condom promotion, TB treatment and others. It is seen as the gateway for clients to receive all other services. In addition, it is routinely offered to clients who patronize SPSS sites for other services including FP, treatment for illnesses suspected to be HIV–related (PITC), VMMC, and medical reviews of survivors of GBV. SPSS Project annual reports throughout the evaluated period show that “all HTC clients were counseled on the need to access HIV prevention and care services appropriate for them from New Start sites, including family planning counseling and services; TB screening and diagnosis and treatment; CD4 cell testing and referral to public sector ART/OI clinics and post-test support services; VMMC counseling and referral.” Tuberculosis (TB) Screening TB integration with HIV gained momentum right from the inception phases of the project, with intensified MMC and IPC campaigns being launched in 2011 to increase knowledge of TB/HIV co– infection and reaching 177,354 people in 50 districts in 2011 and 332,774 in 2014. Following the training of laboratory scientists from all New Start sites in TB smear microscopy (13 lab scientists trained in 2011) and procurement of essential infrastructure, which allowed for TB smear microscopy to be offered at two New Start sites and on–site TB laboratory diagnosis to be offered at four New Start sites, the project has reached growing numbers of TB clients. All New Start and New Life sites screen clients for TB, referring those who screened positive to the four sites that have laboratory diagnostic equipment installed. To date 347,865 clients have been screened for TB between 2013 and 2014. Of these, 10,660 TB suspects have been identified and referred for further tests. Referral tracking accuracy of TB emerged to be among the highest, at 63% in 2014. This, according to interview respondents, is because of the threat to life that TB poses and the massive educational campaigns that have been done on TB and HIV. The evaluation found TB to be better integrated with New Life’s post–test support services than New Start or other SPSS services because of the peer counseling work of PLHIV through their support groups. Life stories of PLHIV revealed that TB clients sometimes get to take HTC services long after enrolling for TB treatment because “support groups advise them that it is not enough to be on TB treatment without knowing their HIV status.” 29 Centrality in social network analysis language refers to a ‘node’ or character (individual, service, place, etc.) that relates with the highest number of others, which themselves have no direct relationship. 26 Family Planning (FP) and Sexual and Reproductive Health (SRH) According to the ZDHS (2010-11) knowledge of contraception is nearly universal in Zimbabwe with 98% of women and 99% of men reportedly knowing a contraceptive method. As SPSS clients come for HTC they also received FP counselling and services, although FP services are only provided through DFID funding. As part of FP counselling, clients are routinely urged to use dual protection. Condom distribution at all visited sites was integrated with information on HTC, FP, VMMC, VIAC and GBV. Condom information is highly accessible, as condoms are displayed at reception areas and toilets of New Start and New Life facilities. Qualitative interviews with condom distributors suggest that condom distributors refer clients to other HIV services. In all the districts, the condom distributors at hair salons stated that it was easier to discuss VIAC with clients as an entry point for introducing the subject of female condoms to them. Sex workers, who have been targeted as end-users of the female condom, reported through FGDs that their participation in the project’s condom promotions linked them with the integrated services offered at New Start and New Life sites. To ensure efficient service provision, refresher training for counsellors in FP and SRH integrated services are offered each year. In general, interview respondents expressed that referrals for FP services were highly successful as these services were directly offered at site. For example, of the 1,464 New Life clients referred for FP services in 2011, 50% were success whilst of the 1,158 clients referred for the same service in 2013, 45% received the services. It was however noted that FP referrals were more difficult at Catholic Mission Hospitals which did not offer FP services due to the policies of these religious institutions. In 2013, SPSS introduced STI identification and treatment at two New Start sites Harare and Bulawayo, treating a total of 235 STI clients that year. The project expanded this service to two more New Start sites in 2014 and treated a total of 1,874 clients, a 150% achievement of the target. Quantitative figures thus suggest that demand for this service is high, although it has not yet been integrated into many SPSS sites. Very few qualitative interviews mentioned this service. Gender Based Violence (GBV) GBV services, funded through DFID, have been integrated into HIV services, albeit at a smaller scale. The modalities included counseling all the females who came for HTC on GBV. Of the 60 program staff from the New Start and the New Life sites who were interviewed, 36 confirmed that GBV had been integrated. FGDs with New Life clients also validated this finding. The project assisted clients who were survivors of GBV in writing affidavits. For SPSS sites at which GBV was not offered, SPSS sites could refer clients to nearby organizations offering GBV services. However, program staff reported that there was a lot of GBV work happening in the communities and the inability to make follow ups was negatively affecting its integration. Due to cultural norms in the community, some women were not coming forward to report GBV and some clients even withdrew cases after they had been reported to the police. Some clients who had to travel long distances faced transport challenges and were lost to follow up. The New Start and New Life outreach were also unable to follow up with the clients as they were unable to visit certain sites during a certain period. There was still a need to strengthen this aspect as GBV exposes clients to HIV infection. The majority of the program staff also felt that there was need for capacity building in GBV among staff as few staff are currently trained in GBV counselling. Referrals As a result of the strong pull of HTC services, 92,749 HTC clients were referred to other departments or service providers in 2011, and 25% of the referrals that the project tracked were successful. This is a quite low rate because it includes a mixture of services that are highly valued enough to demand urgency and other services which clients may not prioritize. FP is one service that has had effective referrals. The 27 SPSS annual reports show that between 2011 and 2013 a total of 8,336 clients have been referred for FP from New Start (69%) and New Life (31%), with an average referral tracking accuracy of 50%, which is much higher than for all other service combinations. This was credited to women’s positive health seeking behavior and perceived role in intimate relationships as “custodians of birth control” (FGD participant). Referrals to services with readily perceived life-saving value such as OI/ART registered high rates of referral tracking accuracy. SPSS annual reports show that 33,433 post–test support clients were referred for ART/OI treatment, SRH services, social welfare and NGO support group services from 2013 to 2014, and 27,367 of them (66%) were successfully tracked. Sub-question 2b: What were the major factors influencing the achievement or non￾achievement of the integration effort? Table VII provides a summary of enabling and inhibiting factors influencing the integration effort: Table VII: Enabling and Inhibiting Factors Affecting Integration of Services Internal Factors Enabling factors Availability of a range of critical services under one roof (one–stop shop) – clients get to access follow up services immediately because it is convenient. Where all services are SPSS–managed clients are guaranteed to receive them at a minimum or no cost. Proximity of referral sites to referring ones or the client’s home – convenience attracts clients to referral sites which do not require them to spend their incomes on transport. This is especially true for rural areas where access to cash is limited, and there are competing priorities to be considered (e.g., food) Low end–user costs of accessing the referral site – absence of user fees or low cost of services at both SPSS sites or non-SPSS sites are attractive to clients, especially for services that they do not perceive as urgent (e.g., VMMC, condoms, etc). Supportive mass media and IPC campaigns – clients reported that they gained awareness of the integrated services through mass media campaign messages that IPC agents reinforced. Service provider’s partnership with other organizations – implementing partners who have strategic synergies with organizations that offer complementary services are viewed as conduits that connect clients to services of other organizations. Life–saving attributes of the service – clients honor referrals to sites offering TB management, ART/OI, cervical cancer treatment and services for other life–threatening health conditions. Inhibiting factors For New Start and New Life sites, training gaps among project staff meant that not all counsellors were equipped to provide all integrated services offered at that site. Referrals to health facilities or service delivery sites that are distant from clients’ homes or the referring sites can be totally disregarded because of the transport costs that they attract, or clients delay their scheduled visits in order to raise the money. By the time they visit the service provider they may forget their referral slips home. Referral sites are often uncooperative as well and do not keep the referral slips for tracking purposes. For services that communities perceive as being alien to or incompatible with their cultural values (e.g. VMMC among some ethnic groups like the Ndau, use of female condoms, etc.), respondents expressed that some community members such as parents discouraged others from utilizing the services. The absence of organizational partners that address special needs of clients (e.g., psychosocial/nutritional support of children affected by or living with HIV, livelihood ventures targeting PLHIV, etc) were associated with the reduction in sizes of PLHIV support group memberships. The inability of clients to perceive the life–saving value of specific services results in them taking referrals for granted. External Factors Challenges affecting service delivery quality or efficiency at the referral site – once clients referred to other sites discover that quality of service at referral site is poorer than at the referring site they reportedly ignore the service, especially if they perceive a minimal death threat. High user fees can repel clients, especially for services perceived to be non–threatening to life. Clients reported that sometimes they give up on pursuing life–saving services (e.g., viral load testing) if the fees are inhibitive (US$50). Respondents solely associated inhibitive fees with referral sites that are not managed by PSI or its partners, and are hence beyond the project’s direct control. 28 Evaluation Question 3: How effective and efficient was the administration of sub￾awards to local and private sector organizations in meeting the project goals? Sub-question 3a: To what extent did the administration of sub awards to local NGOs and private sector organizations ensure that desired objectives were achieved? The support of USAID enabled PSI to scale up HTC, VMMC, post–test support services, condom social marketing/distribution and SBCC under SPSS. This scale up occurred at two levels: increased geographical coverage (the project had to reach new areas that were yet to receive its services) and increased scope of services (additional services had to be integrated into the primary ones already offered). The engagement of sub–awardees to manage New Start and New Life franchises was strategically utilized as a way to economize resources and maximize on the expertise of local implementing partners already active in targeted areas: “It would be costly and time consuming to set up PSI offices in all the provinces where SPSS is implemented,” a PSI official reported. At the time of this evaluation, HTC and other integrated services are being offered at 16 sites in all 10 provinces. Twelve of the sites are managed by sub–awardees, including private institutions (e.g. Saunders Hospital in Triangle), mission health facilities (e.g. Murambinda Mission Hospital), municipal health facilities and NGOs (e.g. Harare City Health, FACT, Hope Humana People to People, Batanai, etc.). For New Life, all nine sites are currently managed by sub-awardees, following PSI’s strategic decision in 2013 to close three directly-managed sites. The evaluation revealed that sub–awards meaningfully contributed to the project’s attainment of its objectives and enabled it to do so within a short time period after inception. It further showed no significant differences between sites that PSI directly managed and those managed by sub–awardees. In general, both PSI and partner–managed New Start sites reached their targets, whereas New Life sites have had a more difficult time reaching their targets. Figure 1.11 illustrates the performance of all New Start sites across the evaluated period. “Because New Life and New Start are housed under one roof it means that clients are able to receive all integrated services in one site. PLHIV can come here for CD4 testing, then psychosocial counseling and if you are talking about positive living then the FP factor is introduced. Clients may not end up having all services in one day because of time, but they can get booked for another day” –Program staff “Referral tracking is a problem at site and outreach. Clinics are supposed to collect referral tear offs, and although you might be able to find the name in the registers, the actual tear off slip is missing. We talk to all clinics. Even here at [the hospital] the nurses hardly collect the tears off. Maybe people are not used to collecting these things. It’s easier to put it in their records. They don’t think about the other person who has referred to them” –Program staff 29 Sub–awardees operating in outlying towns and rural communities where access to health services is a challenge managed to reach 688,143 clients with HTC services, which is 44% of the total number of clients HIV tested across the partnership. PSI reached 56% of the clients through directly managed sites situated in the major cities of Harare, Bulawayo, Masvingo and populous towns such as Chitungwiza. In qualitative interviews, sub–awardees provided evidence that their efforts have increased awareness of HIV/AIDS and services, reduced HIV/AIDS related stigma, promoted healthy behaviors and normalized of HTC. In addition, interview respondents expressed that local partners have managed to adhere to both the quality standards of the New Start and New Life social franchising model, as well as quality standards set by the MOHCC. For New Life, the project reached 86% of the targets on average over the four years, but the evaluation revealed that this was more a result of challenges that the project experienced than the attributes of the partners. Figure 1.12 below shows the pattern. Different partners were allocated varying targets subject to the population differentials in their areas of operation. For example, Triangle New Life’s cumulative target for the four years is 24,420 new clients compared to Chinhoyi New Life’s target of 62,880 new clients. Chiredzi and Murambinda New Life Centers—co-located at general hospitals—managed to surpass their targets by margins as wide as 122% and 129% respectively because their clients are a captive audience – patients who visit the health Figure 1.12 Figure 1.11 30 facilities for treatment – and the New Start and New Life centers are co-located, enabling the easy referral of HIV+ clients to New Life. Qualitative data suggests that the uptake of HIV testing is high at the hospitals because PITC is provided through the hospitals in addition to New Start offering VTC. The New Life and New Start teams at Murambinda Mission Hospital reported that HIV prevalence among clients testing voluntarily via VCT is generally lower (between 18% and 22%) than when they test through PITC (around 40%). This is because PITC happens when health workers suspect that a client’s illness is HIV–related, “and most times their suspicions turn out to be accurate” (health worker). This may explain why higher numbers were served at hospital–based New Life sites than NGO–based sites, which had to rely heavily on outreach to access new clients amid their transport challenges. The evaluation identified the following elements which caused sub–granting to be effective: • Sub–awardees have expertise related to HIV/AIDS in their geographic domain. Sub-awardees have a good track record stretching over several years (some as far back as 2000 or before) of working in the same geographical area, and addressing problems associated with HIV and AIDS. This means that they understand local people’s needs and the best ways of working with different groups. They have established relationships with traditional leaders, resource persons (e.g., volunteers, etc) and had synergies with local community–based organisations (CBO), thus enabling them to secure the buy–in of local stakeholders. Specific examples cited include the partnership of Padare Men’s Forum on Gender with Hope Humana People to People in raising the uptake of HTC among men in Mashonaland Central, and the partnership of most New Life sites with Zimbabwe National Network for People Living with HIV, which added weight to the project’s promotion of support groups. Batanai was able to link some of its affiliate support groups to partner CBOs that specialize in supporting income–generating projects. The entire project’s partnership with CESHHAR increased its reach to sex workers and found them sufficiently informed to accept the projects. • Many sub-awardees were able to leverage complementary resources from concurrent programs and existing infrastructure (vehicles, office space, staff, etc). This allowed them to create demand for SPSS services and to reduce the impact of challenges such as vehicle shortages of vehicles. For example, FACT-managed sites were able to spread BCC messages through a concurrent program funded by another donor, and focus SPSS resources on service delivery. • Sub-awardees exhibited a vetted capacity to manage funds and implement SPSS activities. They had developed systems for project administration management, and monitoring and evaluation. PSI’s financial oversight, use of results-based disbursements (resources are only dispersed on the basis of the sub-awardee having met set targets), and GPS tracking of vehicles helped monitor and bolster the management capacity of local implementing partners. • The frequent and adequate provision of support from PSI enabled sub-awardees to meet standards and expectations for effectiveness. Quality services were maintained in part due to PSI’s supervisory role, including regular site visits from PSI throughout the implementation period, consistent quarterly supportive and mentorship visits to implementing partners to discuss with them their experiences of approaches that were effective, challenges faced, and to explore better ways of improving service delivery. All partner staff received training in skills relevant to their components of the project (e.g., rapid HIV testing, counseling, etc.). PSI shared with all partners the Standard Operating Procedures Manual, including the version that was revised in 2014 in line with new World Health Organization (WHO) guidelines. It invested heavily sharing information among its partners to maintain high service delivery standards: “There is no difference in the quality of services whether you are served at a New Start Center in Harare, in Chiredzi or wherever,” a HTC client reported in reference to the uniformity of service quality standards across the project’s partnership. The timely provision of essential commodities from PSI (HIV testing kits, drugs, and related commodities) enabled sub-awardees to implement activities without delay. One outstanding 31 resource that PSI centrally procured was the electronic data capturing equipment, which is used in the New Start sites and was well received among implementing partners. The evaluation furthermore identified the following factors which at times limited sub-awardees’ effectiveness: • Despite well thought out strategies for meeting targets, human and material resource constraints linked to overall budget limitations have limited partners’ ability to realize their full potential. Respondents cited that their achievements could have been even greater if they had a sufficient number of reliable vehicles, CD4 cell count machines, and relevantly trained staff to meet demand, especially on outreach. In particular, respondents mentioned the short supply of CD4 cell count machines in 20 out of 74 (27%) interviews. • Although the 6-month sub-granting process and centralized process of major procurements, operate well as a financial/risk reduction mechanism, delays in these processes at times has led in turn to implementation delays for the sub-awardees which can affect project effectiveness. Although some partners felt that this results–focused funds disbursement mechanism, especially PSI’s release of funds at quarterly intervals, creates periodic anxieties among affected partners regarding the future of the support, the system has undoubtedly kept partners on their toes and striving to keep their standards high. “We have a perfect financial management record with USAID,” a PSI representative said while justifying “our stringent financial policy with partners.” Sub-question 3b: Were the administration of sub–awards to local NGOs and private sector implemented in the most efficient (cost, time, etc.) way compared to alternatives (vis-à-vis key health and social objectives of the project)? In general, both PSI and local IPs felt that the administration of sub–awards was a more efficient system than if PSI attempted to manage the entire program on its own. This is due to the strategic use of sub￾awardees based on geographic and demographic characteristics of the target population. Because overhead and salaries are more costly for directly-managed sites, this model only works for densely populated areas which will draw high numbers to a static site, thus off-setting overhead costs. By contrast, sub–awarding makes sense if awardees operate in areas where population density is low, client flow to the static sites lower and reliance on outreach is higher. The evaluation found that sub–awardee efficiency is virtually assured because they are only given limited responsibility for project funds, whereas PSI maintains central management of larger procurement operations. Accordingly, sub–awardees were left to handle simple expenditures such as salaries, field allowances and minor commodity procurements – a system that also reduced the amount of effort that partners would need to invest in acquitting expended funds before receiving subsequent disbursements of the grants. Budgets are set-up so that all sub-awardees are expected to spend 100% of their monthly disbursement before receiving the next payment. However, partners who managed grants from other partners such as Centers for Disease Control and Prevention (CDC), Global Fund, and other major donors felt that they had developed sufficient capacities to manage huge grants and complex projects like SPSS. Mission Hospitals like Murambinda preferred to continue receiving as sub–awardees of PSI because of their limited additional sources of funding while private institutions like Saunders Hospital in Triangle valued the partnership with PSI for exposing them to new dimensions of delivering health services. All partners concurred with the MOHCC officials that having a central coordination and management of the project, which is done by PSI, is an efficient mechanism that eases government’s oversight of the project. 32 Qualitative interviews explored how sub-awardees have developed different strategies to efficiently use limited resources, with some being more efficient than others: • The use of mobile outreach by all SPSS partners has proven highly efficient, allowing sub-awardees to reach more clients with few material and human resources. • New Start and New Life franchises co-located at a single site, as well as those managed by the same entity, showed improved efficiency by better enabling resource sharing and planning between the fixed sites. However, respondents suggested that resource sharing between New Life and New Start does not lead to the same level of efficiency and effectiveness on outreach, as the target outreach sites differ between the two franchises. Resource sharing, especially with regards to vehicles and equipment, often results in one franchise having to compromise in its objectives which can impact target attainment. • In addition to enabling effectiveness, the ability of sub-awardees to leverage other resources from other programs and donors allows sub-awardees to more efficiently use SPSS funds by focusing this money on direct service-provision. • Site managers are expected to dedicate a certain amount of time to counselling in addition to their administrative duties, according to PSI. This is efficient if done sparingly, but qualitative interviews suggest that if this happens too often then it becomes less efficient as administrative duties end up being neglected. • A few sub-awardees have made use of public transport to conduct outreach services due to lack of an available vehicle. This strategy has not proved to be efficient as it takes up a lot of time with little output. However, sites recognize this inefficiency and have resorted to this strategy only because they have had no other option of reaching an outreach site. Sub-question 3c. What is the potential to work with local NGOs in supporting an expanded scope and scale of current activities? On account of the findings on the effectiveness and efficiency of sub–awardees discussed above, this evaluation’s regards local NGOs as strategic partners to support the expansion of the SPSS project’s scope and scale of activities in future. However, based on lessons that the evaluation illuminated through its analysis of (a) the sub–awardees’ defining features and (b) PSI’s support thereto, NGOs will be best placed to support the project’s expansion in the following circumstances: • If they share a relationship with the principal fund recipient prior to the project’s inception – the project showed that working with existing partners eliminated the need for PSI to take its partners through series of orientations and room for error was minimized. • If the implementing partners are already implementing their own responses to HIV and AIDS, which enables them to bring expertise to the partnership. The evaluation further revealed that partners with extensive HIV and AIDS management experience enriched the project with their separate collaborating partners who offered strategic services that added value to the project’s service integration efforts. • If they have complementary funding sources, especially for services that can be integrated with New Start or New Life services- notably income–generating project support, GBV services, and so forth. Partners with complementary funding were able to leverage resources to cover for shortcomings that were created by the shortage or poor working conditions of equipment, especially vehicles. FACT is among the partners that profitably used resources from other funding partners to cover for transport challenges in its New Life division, reaching 95% of its New Life targets in Chinhoyi, 58% in Chipinge, 122% in Chiredzi and 92% on average. 33 • If they have operated in given local communities over long periods of time with a good record of consistently producing desired results and relating with stakeholders. SPSS was able to hit the ground running because of its reliance on partners that were firmly established and operating in the same localities for long periods. Their relations with local leaders, volunteers, and fellow organizations facilitated the swiftness with which the project teams settled and started delivering positive results. • If partners handle limited administrative and logistics management burden while concentrating on project implementation. Although some implementing partners felt that they could manage large grants of the magnitude of SPSS, all key informants agreed that it would take massive adjustments to the structures of partners and be time consuming for them to create appropriate financial management systems. Continued oversight, supervision, and mentoring from a larger organization are still required to ensure quality service delivery and defer much of the administrative burden. CONCLUSIONS AND RECOMMENDATIONS The project made several significant contributions to Zimbabwe’s response to HIV and AIDS including creating demand and increasing service uptake, adding input to policy, raising awareness, and generating evidence for planning and programming. In this vein, the project has met the different needs of women and men, age groups, and key populations which they have specifically targeted. The project was largely effective in integrating services, although the level of integration varies by site, with some sites not yet offering a full package of integrated services. In general, the use of sub–awardees has been an effective and efficient management strategy for locations with low population density and greater reliance on outreach, whereas having directly managing sites is more efficient for densely populated areas. The local implementing partners showed capacity to support the expansion of the scope and scale of current activities, albeit with continued oversight and mentoring. The project in its entirety is relevant and has always flexibly responded to demand and changes in its external environment. It should continue doing the same while addressing challenges identified by this evaluation. Table VIII provides recommendations for improving the project moving forward based on the findings from the evaluation. Table VIII: Findings and Recommendations Findings Recommendations Evaluation Question 1: What has been the overall contribution of the SPSS project in Zimbabwe’s efforts to prevent HIV/AIDS? Finding 1.1 SPSS is in alignment with and has contributed to Zimbabwe’s National Strategy to prevent HIV/AIDs. • Given the effectiveness of the project’s targeted approach to social marketing, funding should be made adequate so as to allow for the targeting of additional segmented population groups (based on Findings 1.3, 1.4). • Given the effectiveness of community mobilizers and mass media campaigns to create awareness and demand for services, funding should be made adequate for these labor and resource intensive strategies (based on Findings 1.3, 1.4). Finding 1.2 Overall, the project has reached the majority of its performance targets over the evaluated period, created demand for its products and services, been a thought leader for VMMC and other interventions, influenced national policies, contributed research, and strengthened public health systems. Finding 1.3 Enabling factors for project performance include: integration of other services, mobilization and outreach, evidence-based market segmentation and targeted promotions, multi-stakeholder engagement, good coordination with MOHCC, leveraging of resources from other donors to fill gaps, use of electronic data capture for HTC, favorable 34 national policies, project participation at strategic platforms, and organizational flexibility. • Advocacy efforts with the MOE should be increased so that they better support VMMC and HTC efforts in schools with eligible age populations (based on Finding 1.4). • Due to the project’s reliance on outreach services, funders should be sensitive to the project’s needs for adequate and reliable vehicles and, if funding allows, make sure the project is resourced accordingly (based on Findings 1.3, 1.4). • Continue and expand strategies that have proven effective for increasing male involvement and access to services at New Start and New Life sites, such as mobilization through community leaders and men’s fora (based on Finding 1.5). • The project should continue and, if possible, expand its outreach service as this strategy has proven effective for reaching marginalized and inaccessible populations (based on Finding 1.5). Finding 1.4 Inhibiting factors for project performance include: funding and resource constraints, poor roads and long distances to reach communities, client hesitancy to openly purchase condoms, inhibiting policies (related to MOE, PEPFAR, and religious institutions), stigma and discrimination against PLHIV, misconceptions about VMMC, and operating challenges within the MOHCC (i.e. availability of government healthcare workers, low client satisfaction with public sector services, and limited cooperation with SPSS for referral tracking). Finding 1.5 The project has met different needs for men and women, different age groups, and key populations. More women than men access services as New Start and New Life, although overall the male-female ratio for HTC is close to 50- 50 due to referrals from VMMC services. The project strategically targets young males aged 13-29 for VMMC with school campaign, but has been more successful in reaching boys aged 13-19. The project’s use of voluntary testing and counselling (VTC) has more successfully reached people <30 years old. Although not in the SPSS project mandate, the project has gone above its initial purview and attempted to serve the needs of children and adolescents, although staff perceive a training gap for serving young clients. SPSS has worked to address the needs of CSWs, and used an outreach model to provide services to other key populations such as mobile persons, workplaces and other isolated communities. Evaluation Question 2: How effective is the integration of FP, TB, GBV and SRH within the HIV services, including referrals? Finding 2.1 A number of services have successfully and with positive results been integrated into HIV services, albeit at varying degrees at different sites. • For a given site, the project should strive to offer the same integrated services on outreach as they do at fixed sites (based on Findings 2.3, 2.4). • If resources allow, the project should train all site counsellors to be able to perform all the integrated services offered at a given site (based on Finding 2.4). • Although challenges associated with the MOHCC are beyond the project’s control, the project can consider furthering its communication and advocacy efforts with the Ministry at the local, provincial, and national levels. This may entail sharing evaluation findings related to client barriers for accessing and/or using public sector facilities, and continuing to explore ways to Finding 2.2 While HTC is often the entry point for accessing integrated services, the reverse is also true, as the integration of services has increased uptake for HTC and other existing HIV services. Finding 2.3 Factors enabling service integration include: the development of “one-stop-shops” whereby all services are offered at one location, proximity of referral sites to referring ones or the client’s home, low end–user costs of accessing the referral site, supportive mass media and IPC campaigns, service provider’s partnership with other organizations, and life–saving attributes of the integrated service. Finding 2.4 Factors inhibiting service integration include: clients’ perceived poor service delivery quality or inefficiency at referral sites run by MOHCC, referrals to health facilities or service delivery sites that are distant from clients’ homes or the referring sites, training gaps among project staff on the integrated services, high fees at non-SPSS sites for referred 35 services, negative messages about some services that clients are not culturally familiar (i.e. VMMC among certain ethnic groups, female condoms), and lack of cooperation with referral sites for referral tracking. better cooperate regarding client referral tracking (based on Finding 2.4). Evaluation Question 3: How effective and efficient was the administration of sub-awards to local and private sector organizations in meeting the project goals? Finding 3.1 The evaluation revealed that sub–awards meaningfully ensured the project’s attainment of its objectives and enabled it to do so within a short time period after inception. It further showed no significant differences between sites that PSI directly managed and those managed by sub– awardees. Major determinants of positive sub-awardee effectiveness include: sub–awardee technical and contextual expertise, ability to leverage complementary resources from concurrent programs, adequate capacity to manage funds and implement SPSS activities, provision of support from PSI enabled sub-awardees to meet standards and expectations for effectiveness. Major limiting factors influencing sub-awardee effectiveness include: human and material resource constraints linked to overall budget limitations (i.e. limited vehicles, CD4 cell count machines, and trained staff), and delays in the sub￾granting and centralized procurement process which may lead to implementation delays. • USAID should continue to consider the use of sub-awardees for contexts where the ratio of fixed costs to client flow is cost￾effective, and where an organization capable of handling a sub-award already exists (based on Findings 3.1, 3.2, 3.3). • PSI should continue to provide prompt and adequate support to sub-awardees to ensure that services are delivered in a timely manner and according to quality standards (based on Finding 3.1). • PSI should make sure that centralized procurement processes and any sub￾granting procedures occur as rapidly as possible so as to avoid implementation delays down the line. PSI may also consider building in time during its routine supervision visits to briefly mentor sub￾awardees on managing their financial risk￾mitigation strategies to prevent delays at the sub-awardee level (based on Finding 3.1). • Donors should consider an influx of funding to fill the resource gap (e.g. vehicles, CD4 machines, training and staffing needs) which severely limits sub-awardees’ potential to reach and serve clients (based on Findings 3.1, 1.4). • If SPSS expands in scope and scale, it is recommended that additional New Start and New Life sites be co-located to be more efficient (based on Findings 3.1, 3.2). Finding 3.2 The administration of sub–awards is efficient due to the strategic use of sub-awardees based on geographic and demographic characteristics of the target population. Examples of efficient strategies for service delivery include: use of mobile outreach, co-location of New Start and New Life fixed sites and managed by the same entity, limited doubling of site managers as counsellors, and leveraging resources from other programs. Finding 3.3 Local NGOs have the potential to support the expansion of the SPSS project’s scope and scale of activities in future, as long as they meet several qualifying criteria, such as previous experience in the field of HIV/AIDs, established relationships with other local stakeholders, developed financial monitoring systems, and other funding sources besides SPSS. For most current implementing partners, continued supervision, support, and capacity building from a stronger organization remains necessary.