1 August 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by International Business & Technical Consultants, Inc. (IBTCI). USAID/Zambia: Partnership for Integrated Social Marketing (PRISM) Program Mid-term Evaluation ACKNOWLEDGEMENTS This mid-term evaluation of the PRISM program was made possible with assistance from various stakeholders at national, regional, and community levels. Many institutions and stakeholders provided valuable time and opinions during the assessment. IBTCI and the evaluation team would like to thank the following institutions and individuals for their support and information provided: USAID/ Zambia, Office of Population and Health; Government of the Republic of Zambia; SFH head office and their offices in the provinces; PRISM project partners; and Other donors and implementing agencies. Special gratitude is also extended to respondents and participants who provided invaluable information without which this evaluation would not have been possible. i USAID/ZAMBIA: PARTNERSHIP FOR INTEGRATED SOCIAL MARKETING (PRISM) Authors: Jennifer Peters, Milka Juma, Dr. Moses Simuyemba, Dr. Phoebe Bwembya, Dr. Peggy Chibuye July 17, 2013 AID-RAN-I-00-09-00016/AID-611-TO-13-00003 Cover Photo Credit: Dr. Moses Simuyemba Caption: Women and children from Chiparamba ward in Chipata District of Eastern Province at an under-five clinic. DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii TABLE OF CONTENTS ACRONYMS .................................................................................................................................................................. iv EXECUTIVE SUMMARY.......................................................................................................................................... 1 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS ..................................................... 6 1.1 PROJECT BACKGROUND ................................................................................................................................ 6 1.2 EVALUATION PURPOSE .................................................................................................................................... 6 1.3 EVALUATION QUESTIONS .............................................................................................................................. 7 2. PROJECT BACKGROUND............................................................................................................................... 8 2.1 SITUATIONAL ANALYSIS: HEALTH AND WELFARE SITUATION IN ZAMBIA ............................. 8 2.2 MINISTRY OF HEALTH PRIORITIES AND PROGRAMS ........................................................................ 10 2.3 SUMMARY OF USAID HEALTH PROGRAMS ............................................................................................ 10 3. EVALUATION METHODS AND LIMITATIONS............................................................................... 11 4. FINDINGS................................................................................................................................................................. 13 4.1 EVALUATION THEME 1: PROGRESS TOWARDS intended RESULTS ............................................. 13 4.2 EVALUATION THEME 2: EFFECTIVENESS OF INTERVENTIONS ...................................................... 34 4.3 EVALUATION THEME 3: SUSTAINABILITY ............................................................................................... 37 5. CONCLUSIONS.................................................................................................................................................... 39 6. RECOMMENDATIONS ..................................................................................................................................... 40 6.1 SAFE WATER AND CHILD HEALTH ........................................................................................................... 40 6.2 MALARIA PREVENTION.................................................................................................................................. 41 6.3 FAMILY PLANNING & REPRODUCTIVE HEALTH .................................................................................. 41 ANNEX A: SCOPE OF WORK FROM THE TASK ORDER...................................................................... 46 ANNEX B: TOOLS/SURVEY INSTRUMENTS, KII GUIDE ........................................................................ 59 ANNEX C: DATA COLLECTION TABLE: NUMBER OF HOUSEHOLD INTERVIEWS DONE BY DISTRICT, PROVINCE ......................................................................................................................................... 72 ANNEX D: LIST OF INTERVIEWEES ............................................................................................................. 73 ANNEX E: LIST OF DOCUMENTS REVIEWED .......................................................................................... 77 ANNEX F: SUSTAINABILITY OF ITNS, CLORIN AND MALE CONDOMS........................................ 79 ANNEX G: EVALUATION TEAM BIOS .......................................................................................................... 83 iii Table of Figures Figure 1 – Age of Respondents ...................................................................................................................................13 Figure 2 - Method of water purification ..................................................................................................................... 18 Figure 3 - Mosquito net availability in household ..................................................................................................... 21 Figure 4 - Method of modern contraception used .................................................................................................. 24 Figure 5 - Knowledge of partner HIV status ............................................................................................................. 31 Figure 6 - HIV Counseling & Testing .......................................................................................................................... 31 Figure 7 - Knowledge of benefits of male circumcision .......................................................................................... 32 iv ACRONYMS ANC Antenatal Care CBD Community Based Distribution CBV Community Based Volunteer CSO Civil Society Organizations CYPs Couple Years of Protection DALYs Disability Adjusted Life Years DAPP Development Aid from People to People DfID UK Department for International Development DHS Demographic and Health Survey DHMT District Health Management Teams FGDs Focus Group Discussions GFATM The Global Fund to fight HIV, TB and Malaria GRZ Government of the Republic of Zambia ITN Insecticide-Treated Mosquito Net IUD Intrauterine Device KII Key Informant Interview LARC Long-Acting Reversible Contraception LLIN Long-Lasting Insecticidal Treated Net MARP Most At-Risk Populations M&E Monitoring and Evaluation MCDMCH Ministry of Community Development Maternal and Child Health MIS Malaria Indicator Survey MOH Ministry of Health MSL Medical Stores Limited NGO Non-governmental Organization NMCC National Malaria Control Centre NMSP National Malaria Strategic Plan PLHA People Living with HIV/AIDS PMI United States Government’s President’s Malaria Initiative PRISM Partnership for Integrated Social Marketing PSI Population Services International v SFH Society for Family Health TBA Traditional Birth Attendant TRaC Tracking Results Continuously UNICEF United Nations Children’s Fund USAID United States of America International Development ZHECT Zambia Health Education Communication Trust ZISSP Zambia Integrated Services Strengthening Program 1 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The objectives of USAID/Zambia: Partnership for Integrated Social Marketing (PRISM) Program mid￾term evaluation were three-fold: a) to evaluate progress towards intended results made by PRISM towards intended results and describe the context and other possible factors also contributing to the observed changes; b) to determine the effectiveness of the activities, techniques, and processes deployed by PRISM to achieve the intended results within the five-year performance period and recommend modifications to project activities or priorities to address implementation issues, apply lessons learned, or capitalize on new opportunities; and c) to determine the sustainability of gains made through PRISM beyond the life of the program and recommend techniques and/or processes that would further ensure sustainability. A thorough evaluation of the PRISM program was conducted to assess progress towards results; appropriateness and effectiveness; PRISM responsiveness to Government of the Republic of Zambia (GRZ) strategies and protocols; and sustainability. PROJECT BACKGROUND The Partnership for Integrated Social Marketing (PRISM) Program is a five-year USAID-funded project running 2009-2014 with a total budget of $65 million. To date, $43,406,861 has been obligated into the award. The program conducts social marketing programs for health services and products in HIV prevention, family planning, malaria, and maternal and child health and is implemented by Society for Family Health (SFH), the local affiliate office of Population Services International (PSI). The PRISM’s local partners include the Zambia Health Education Communication Trust (ZHECT), Mwami Adventist Hospital, Luapula Foundation, Solwezi Youth Alive, and Development Aid from People to People (DAPP). International partners include Care International, the Population Council, Booz Allen Hamilton, Overseas Strategic Consulting, Jhpiego, and IntraHealth. PRISM employs social marketing techniques to distribute male and female condoms for HIV prevention, family planning methods (oral and injectable contraceptives and implants), insecticide treated bed nets to prevent malaria and a point-of-use home water treatment solution to disinfect drinking water. The PRISM program also provides stand-alone testing and counseling for HIV and male circumcision services to prevent HIV infection among HIV-negative males. EVALUATION DESIGN, METHODS AND LIMITATIONS The evaluation employed a mix of quantitative and qualitative methods including a document review, as well as primary data collection through 92 key informant interviews (KIIs), 38 focus group discussions (FGDs) and 240 household respondents in selected communities in two districts across six provinces. Quantitative data was analyzed using Epi-info while qualitative data was analyzed using N-Vivo. Where possible and with insightful results, data was segregated by gender, age, region, and urban versus rural across all PRISM products and services. The evaluation took place between April and July 2013 and the evaluation team comprised five consultants, three research assistants, and one logistician (See Annex G for evaluation team bios). The evaluation had some limitations including: a small sample size for the quantitative household survey; a higher proportion of female respondents than male in household surveys; inaccessibility of certain key informants; and purposive selection of project sites, limiting the ability to compare project intervention and non-intervention areas or compare pre- and post￾intervention results due to lack of baseline data. 2 FINDINGS AND CONCLUSIONS Theme 1: Progress towards Intended Results Provision of Clorin for safe water: With over 3 million unit sales annually, SFH/Zambia’s Clorin program funded by PRISM is one of PSI’s most successful safe water programs globally. Awareness among household survey respondents is high (97%) primarily as a result of activities through health facilities and community-based volunteers (CBVs). Availability is lower (83%) where CBVs and smaller retailers struggle to maintain stocks of Clorin. Use is impressively high with 75% of households surveyed found with Clorin. The product is also distributed free during high-risk periods through the Ministry of Health (MOH) and to people living with HIV/AIDS (PLWHAs) through implementing partners providing HIV care and support services. Provision of Insecticide treated nets (ITNs) for malaria prevention: To date, SFH has assisted the National Malaria Control Centre (NMCC) program to deliver over 2 million ITNs free to pregnant women through District Health Offices (DHOs) and health facilities. Society for Family Health also assists the MOH with mass distribution campaigns funded by other donors. Both awareness and use of ITNs were found to be high (93% and 71% respectively), with rural use (77.8%) higher than urban (47.5%). Results of primary data collection match that found in the Malaria Indicator Survey 2012. However, perceived availability – at DHOs, health facilities and amongst communities - is low; though net coverage is high. Understanding of the antenatal care (ANC)/ITN program in the broader context of ITN distribution needs improvement. SFH also needs to focus more attention on systems strengthening for ANC ITN distribution. Family planning services: PRISM activities include community-based distribution of short-term family planning methods combined with technical assistance to strengthen service delivery of long-acting reversible contraception (LARC) through MOH facilities. Awareness of all family planning methods is high, ranging from 83% for intrauterine devices (IUD) to 99% for Maximum male condoms. Myths and misconceptions surrounding family planning methods, and particularly LARC, hamper demand. Availability of family planning services is generally high at 93%, primarily due to the use of free family planning services available through all MOH facilities. Free family planning through the MOH hinders sales of PRISM’s family planning products and, in contrast, allowed the project to shift focus to increased provision of LARC, primarily in training more than 400 providers within the MOH. Use of family planning was found to be high at 58.26% of which only 15% are using a long-term method. More work is needed in this area. Misoprostol provision: In the end of 2012, SFH was awarded funds for optional task 5 Misoprostol for prevention of postpartum hemorrhage. Project pilot activities implemented in 2009 were successful, and the project began to roll out activities to ten districts. However, due to USAID restrictions on the purchase of misoprostol, activities are currently halted. An alternative source of misoprostol is urgently needed. Male condoms promotion: Of all PRISM interventions, awareness of Maximum male condoms is highest at 99%. Maximum condoms were also found to be widely available in retail, through CBVs and at hotspot locations such as bars and nightclubs. Use of male condoms is unfortunately low still: only 18.5% of respondents used a condom at last sex. Condom use amongst trusted partners is reported to be lowest by couples and adults in FGDs though reportedly somewhat higher amongst high-risk groups such as sex workers and truckers. 3 Female condoms promotion: Of all interventions, female condoms are least well known (87%), least available (according to the majority of FGD respondents, both beneficiaries and CBV workers), and least used; a lowly 1.6% of all household respondents reported using the female condom. Little is known about the typical consumer for female condoms; activities mostly target sex workers, and whether or not this acts as a constraint to use amongst non sex workers, and particularly young women, is unclear. Counseling and testing services for HIV: Under PRISM, ten new New Start sites have been established (eight of which are functional with the other two just beginning services). Society for Family Health also undertook a national behaviour change communication (BCC) campaign to encourage couples to come for testing together and share results with each other. Through ANCs targeting women and routine counselling and testing for men who come for male circumcision, counseling and testing rates have increased significantly in Zambia in recent years. Knowledge of counseling and testing is universal, and availability of services also reported amongst all project target groups as high. Amongst household respondents, 71% had been tested for HIV and knew their results. A further 58% knew their partner’s status. Male Circumcision services: Amongst implementing partners, SFH is the leading provider of male circumcision accounting for over 40% of the total national number of male circumcisions provided in the first half of 2013. In the last year, SFH has undertaken considerable efforts to implement mobile counseling and testing and male circumcision services. Mobile male circumcision services now account for 75% of circumcisions performed. This has increased both reach and access but comes at a higher cost than static provision of the services. While awareness about male circumcision services is high, misconceptions and fears about the procedure, healing time, and short- and long-term effects, hamper demand and use of services. Uptake of male circumcision is on the rise, though it is still higher in urban than rural areas. Theme 2: Effectiveness A number of best practices and effective strategies and activities have been implemented through PRISM. PRISM activities are implemented in both rural and urban areas, though the scope, reach, coverage, and uptake of most of these interventions is significantly higher in urban than in rural areas. Nearly all of PRISM’s health service interventions have been urban focused until recently, when SFH began offering mobile services and campaign-styled events to increase access to rural and underserved populations. These include mobile services and community campaigns to extend reach of and access to male circumcision and counseling and testing services to rural and remote/underserved communities; piloting the provision of postpartum IUD insertion and using lessons learned to assist the MOH in scaling up provision of this service; and assistance with “universal access” to counseling and testing services in market places and other public locations where high volumes of people can be reached. Networks of CBVs targeting rural areas with little or no access to commercial markets and located too far from health facilities are developed to increase access to family planning and other preventive health products. Outreach and BCC activities are targeted towards both men and women; for example, encouraging men to support and encourage their partners to use family planning, and targeting women with messages about the benefits of male circumcision for both men and women; and encouraging couples to come for counselling and testing together resulting in increased awareness of partner status and open dialogue about safer sex. To reach high-risk populations, the program is targeting hotspots with male and female condoms to improve condom use in high-risk behaviors. To improve sustainability of community-based distribution and sales activities, trainings are provided to CBV staff in small business skills and encouraging linkages between wholesale and retail; retail and CBVs; 4 CBVs and DHOs; and health facilities to facilitate access to the resupply of social marketing products when needed (without reliance on a project partner and vehicle for resupply). PRISM program activities are well targeted and appropriate for both men and women. However, more work is needed to insure coverage of rural populations and youth. Theme 3: Sustainability Sustainability of socially marketed products is varied, but a number of PRISM activities were identified as increasing the likelihood of sustainability over the longer term. These include strengthening linkages within the commercial sector and between distributers and retailers, the MOH and CBVs; sustained demand-creation activities and innovative means of incentivizing CBVs; and incremental increases in pricing consistent with willingness-to-pay surveys. Likelihood of sustainability of PRISM assistance for MOH services is by and large deemed to be high. Ministry of Health systems strengthening, training, and other technical support is all deemed in- and outside the Ministry as sustainable. Key constraints include MOH financing and inadequate human resources. Of note, handover of the ANC ITN program to the MOH is not likely to be achieved, nor is graduation of Clorin to a full priced commercial product. Critically, the evaluation team does not feel either of these activities is recommended. The MOH and central medical stores are already overburdened with logistics and management of MOH supplies. Handover of Clorin is likely to result in losses in reach and use to target populations; considerable market and consumer research is recommended before advancing with this activity. CONCLUSIONS The PRISM program has achieved or is likely to achieve 21 of its 32 total indicators. Three are unlikely to be achieved, while for four it is too early to evaluate, and another four are not applicable. Some indicators require revision or reconsideration, notably a family planning indicator which measures total couple years of protection (CYP) would be a better indicator of success than the current indicator for sales of oral contraceptives. Similarly, PRISM indicators for handover of the ANC program to the MOH and Clorin to the commercial sector should be reconsidered. The PRISM program has made significant gains in awareness, access, and use of the majority of products and services. More work is needed to dispel myths and misconceptions surrounding family planning and male circumcision as well as to reach youth. Access in rural areas remains a constraint across interventions. Best practices of the project are many and should be documented and shared with the MOH and other implementing agencies. Project interventions in support of MOH service delivery are largely deemed to be sustainable; the primary constraints are in financing and human resources. Gains in sustainability of socially marketed products were noted also. The primary constraint is the ability to pay amongst project beneficiaries, most notably, rural poor. RECOMMENDATIONS Details of recommendations are provided in section six of the report. Here, a summary of major recommendations are provided. 5 Safe water with Clorin provision: In addition to the planned market research for Clorin, SFH and USAID need to carefully examine many factors related to cost-effectiveness, health impact, and reach before deciding whether handover of this product to a commercial entity is prudent. The program needs to increase BCC activities including use of available TV spots to promote Clorin and increase use of distributors to reduce transport costs. The program also needs to consider removing subsidies from sales to donors and other partners and to improve linkages between rural communities and retail markets where CBV networks do not exist. ITNs for malaria prevention: It is recommended that Indicator 1.4 “Transfer of ANC ITN program to MOH” be revised to include systems strengthening indicators at all levels of MOH to better implement, monitor, track, and report on the ANC ITN program. Additionally there is a need to; develop a strategic overview and communications plan for MOH, DHOs and health facilities to improve awareness and understand of ANC versus mass distribution nets; and to undertake a tracking (TRaC) survey to understand net ownership, coverage, and use at a household level. Family Planning services: There is need to revise SFH indicator “Market 25.6 million cycles of oral contraceptives” to one that measures couple years of protection (CYP) across all family planning products; develop a strategy and activities to improve access to youth friendly family planning services in consultation with MOH taking into consideration the national policy guidelines; lessons learned from operational research on male circumcision and voluntary counseling and testing (VCT) mobile and outreach services should be duplicated and implemented for family planning, particularly LARC, where feasible and appropriate; pilot the training and franchising of private sector and/or retired midwives in rural areas to increase access to quality family planning provision; increase systems strengthening within the MOH for family planning; increase BCC activities to improve uptake of LARC and dispel myths and misconceptions; and advocate for communications and education programs targeting young women with family planning services. Provision of Misoprostol: There is a need to secure supply chain for commodities, short- and long-term and develop an improved, low-cost Safe Delivery Kit with Misoprostol to pilot through SFH franchise of midwives. Female condoms promotion: There is a need to undertake a consumer profiling survey and possibly a cost-effectiveness study to determine how best to target female condoms in future. Counseling and Testing services for HIV: The indicator, “Train 2,300 individuals in HIV counseling and testing,” needs to be clarified. Additionally, scale-up of comprehensive service delivery for counseling and testing and CD4 counts should be done; and linkages to care and treatment for HIV should be strengthened. Male circumcision services: Recommendations include the following: increasing funds and focus on training of trainers and supervisors for male circumcision provision; centralizing procurement of kits through JSI/DELIVER; operational research and best practices on mobile service provision; strengthening coordination efforts with partners and communities for mobile and campaign service delivery; include Central province in PRISM program male circumcision activities; introduction of new non-surgical male circumcision kits; and increase of demand-creation activities targeting older men and myths and misconceptions. 6 1.EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1 PROJECT BACKGROUND The Partnership for Integrated Social Marketing (PRISM) Program is a five-year USAID-funded project running 2009-2014 with a total budget of $65 million. To date, $43,406,861 has been obligated into the award. PRISM conducts social marketing programs for health services and products in HIV prevention, family planning, malaria, and maternal and child health and is implemented by Society for Family Health (SFH), the local affiliate office of Population Services International (PSI). PRISM’s local partners include the Zambia Health Education Communication Trust (ZHECT), Mwami Adventist Hospital, Luapula Foundation, Solwezi Youth Alive, and Development Aid from People to People (DAPP). International partners include Care International, the Population Council, Booz Allen Hamilton, Overseas Strategic Consulting, Jhpiego, and IntraHealth. PRISM employs social marketing techniques to distribute male and female condoms for HIV prevention, family planning methods (oral and injectable contraceptives and implants), insecticide-treated bed nets to prevent malaria, and a point-of-use home water treatment solution to disinfect drinking water. PRISM also provides stand-alone testing and counseling for HIV and male circumcision services to prevent HIV infection among HIV-negative males (see Annex A for evaluation scope of work). 1.2 EVALUATION PURPOSE The objectives of this mid-term evaluation of USAID/PRISM program were three-fold:  Progress towards Intended Results: To evaluate the progress made by PRISM towards intended results and change, and describe the context and other possible factors (such as health programs by other donors or actors, campaigns by a variety of actors, etc.) that may be contributing to the observed changes.  Effectiveness: To determine whether the activities, techniques, and processes agreed upon between USAID and PRISM and deployed by PRISM are appropriate to achieve the intended results and/or change within the five-year performance period, and to recommend modifications to program activities or priorities, as necessary, to address implementation issues, apply lessons learned, or capitalize on new opportunities.  Sustainability: To determine whether gains made through PRISM are sustainable beyond the life of PRISM and, if not, recommend techniques and/or processes that would ensure sustainability of the gains made. Audience The aim of this mid-term evaluation was to provide USAID/Zambia, the Government of the Republic of Zambia, and other in-country stakeholders with objective information on what has been achieved to date, what is working and why, as well as what is not working, with recommendations for short-term program modifications and recommendations for longer-term strategic planning and future programming. 7 1.3 EVALUATION QUESTIONS Progress towards intended results: Are the social marketing activities, techniques, and processes employed by PRISM contributing to the following intended results? 1. Increased awareness of socially marketed health services and products? 2. Increased availability of socially marketed health services and products? 3. Increased use of socially marketed health services and products? Effectiveness of interventions: 1. Are the social marketing activities, techniques, and processes agreed upon between USAID/Zambia and PRISM, and deployed by the latter, effective in achieving the intended results? a) Gender-Are the social marketing activities, techniques, and processes deployed by PRISM addressing the unique needs of men and women? b) Target population-Are the social marketing activities, techniques and processes deployed by PRISM equitably addressing the needs of rural and urban populations? c) Are the protocols and best practices identified by the program being implemented according to the work plan? d) Is PRISM applying a level of effort that can ensure achievement of the intended results? 2. Does the Government of the Republic of Zambia (GRZ) consider PRISM to be accountable to the country’s strategic objectives? Are there activities that the GRZ would rather see implemented differently? Sustainability: Is PRISM incorporating host country systems for Zambia to continue to implement some or all of the currently socially marketed services and products after the conclusion of the program? 1. To what extent is PRISM establishing systems that will make the currently socially marketed products and services continue to be socially marketed in Zambia by a commercial entity after the PRISM Task Order ends? Will the program be able to increase the number of individuals/clients willing to pay for currently socially marketed services and products at unsubsidized rates by 2014? 2. Is PRISM generating evidence and lessons learned which could help other implementing partners and/or the MOH? 3. Is PRISM providing useful skills to Zambians (judged by locally engaged staff) to improve their ability to socially market health services and products? 8 2. PROJECT BACKGROUND 2.1 SITUATIONAL ANALYSIS: HEALTH AND WELFARE SITUATION IN ZAMBIA Although Zambia has experienced more than a decade of economic growth and now boasts an average income of $1,460, making it a lower-middle-income country, this growth has not benefited an estimated two-thirds of the population living in absolute poverty (defined as living on less than $US1 per day). Of the estimated 13 million1 people in Zambia, two-thirds are rural￾dwelling and nearly three-quarters fall below the national poverty line. Rural areas are largely undeveloped and access is difficult as roads are either very poor or non-existent.2 Inequity rates are high and growing, with poverty rates highest among remote and rural populations. The poorest of the poor suffer disproportionately from preventable diseases and are less likely to have access to health services compared to their urban counterparts.3 Zambia’s challenge is to promote broad-based, pro-poor economic growth, create employment and develop its human capital. Zambia is committed to achieving all the health-related MDGs by the year 2015, and as such, the Ministry of Health has identified priority health intervention areas in the National Health Strategic Plan (NHSP) 2011-2015 which include child health, integrated reproductive health, HIV/AIDS and sexually transmitted infections (STIs), and malaria. Safe water and Child Health Contaminated water is a leading cause of diarrheal disease and child mortality in Zambia, where only 64% of the population has access to safe drinking water sources. Among children under five, 21% have had diarrhea in the past two weeks, regardless of water source or location. Mortality among children under five is particularly high, as attempts to rehydrate children, usually with more contaminated water, often fail. Rural children are disproportionately at risk as they lack both access to safe water and to health services and treatment for diarrheal diseases, including cholera and typhoid4. Despite significant efforts to combat malaria, it remains the leading cause of morbidity and mortality in Zambia, with more than 4 million clinical and laboratory confirmed cases of malaria with more than 4,500 malaria deaths. (Zambia National Health Management Information System in 2011). The 2010 Malaria Indicator Survey indicates 70% of Zambian households have at least one mosquito net, and 64% of households have at least one ITN, representing an increase from 2006. Fifty-five percent of all Zambian children under age five slept under a mosquito net the night before the survey, an increase from 2008 despite challenges in increasing overall net availability. Ninety percent of children under age five who slept under a net the night before the survey slept under a treated net. (MOH, 2010 Malaria Indicator survey). The most up-to-date information on nationwide coverage of malaria prevention and control measures in Zambia shows progress in recent years. More than 64% of households own at least one insecticide-treated net (ITN), and 50% of children under five years of age had slept under an ITN the previous night. Almost 73% of households owned at least one ITN. In 2011, the NMCP 1 Central Statistics Office, 2010. Census of Population and Housing Report. 2 Ministry of Finance and National Planning, Fifth National Development Plan 2006-2010. June 2006. 3Central Statistical Office, Living Condition Monitoring Survey Report 2006 & 2010. 4 Zambia Child Survival Report, 2010. 9 distributed over 4 million nets through mass distribution campaigns and routine distribution at antenatal clinics (ANC) (MOH, 2010 Malaria Indicator survey). Family Planning and Reproductive Health Zambia has the world’s third highest fertility rate; the 2007 ZDHS found the total fertility rate (TFR) to be 6.2 (4.3 in urban areas and 7.5 in rural areas). Only 25% of married women currently use a modern method of family planning and approximately 40% of all births occur within three years of the preceding birth; Zambian women bear their first child around age 19. These fertility trends have generated a youth bulge in Zambia and teenage pregnancy rates are also alarmingly high. More than 20% of all young women aged 15–19 have at least one child. Worse, the proportion of teenage pregnancies increased from 6.6% in 2002 to 8.3% in 2007. Worryingly, a recent survey undertaken by the Abt Associate’s “Scaling Up Family Planning” project indicates that the majority of young women prefer seeking abortions to deal with unwanted pregnancy over seeking family planning services. There is an urgent need to target young men and women with youth-friendly services providing both information and access to quality family planning services. The Zambian government has committed to raising the nation's contraceptive prevalence rate (CPR) from 33% to 59% by 2020 but faces many constraints in doing so. 5 Amongst them are a severe shortage of skilled family planning providers, particularly in rural areas; low demand for LARC, particularly amongst younger women; cultural and traditional norms that encourage young women to begin childbearing early and without proper spacing of births; and widespread myths and misconceptions that hamper demand. Maternal mortality remains high in Zambia; in 2007, the maternal mortality ratio (MMR) was 591 per 100,000 live births6, well above the MDG target (162). Post-partum hemorrhage (PPH) is the leading cause (34%) of maternal mortality in Zambia. Inter-related factors contributing to Zambia’s maternal mortality include high rates of unattended births, geographic and transportation barriers to accessing health facilities, and shortages of trained health personnel as well as essential drugs and medical supplies. More than half of all women deliver at home; in rural areas, 71% childbirths take place in the home compared to 21% in urban areas. Only 27% of all rural births are attended by a skilled provider. Globally and in Zambia, a number of studies have concluded/proven that a three-pill dose of misoprostol, given to a woman immediately after she delivers, significantly reduces the incidence of PPH, resulting in lower rates of maternal mortality. HIV/AIDS and STIs Zambia has a generalized HIV epidemic estimated at 14.3% among adults aged 15–49 years.5 Adult HIV prevalence remains higher amongst women (16.1%) than men (12.3%) and higher in urban areas (19.7%) than rural areas (10.3%). The estimated number of annual new infections in the adult population aged 15-49 years in 2007 and 2008 was 79,755 and 80,442, respectively. Although HIV incidence has begun to stabilize in recent years, the absolute number of people living with HIV/AIDS and needing care and treatment continues to grow due to new HIV infections and the increased life expectancy amongst people living with HIV/AIDS (PLHAs) receiving ART treatment. 5 http://www.mariestopes.org/news/zambia-commits-raising-contraceptive-prevalence-rate-59-2020 6 Central Statistical Office (CSO), Ministry of Health (MOH), Tropical Diseases Research Centre (TDRC), University of Zambia, and Macro International Inc., 2009. Zambia Demographic and Health Survey 2007. Calverton, Maryland, USA: CSO and Macro International Inc. 10 In Zambia, high prevalence rates are fueled by early initiation of sex, unprotected sex with non-regular partners, concurrent sexual partnerships, low incidence of condom use among high risk groups and trusted partners, sexual violence against women, and poverty that forces women and girls to sell sex for food, good grades, small gifts, or money. The most at risk individual in Zambia, however, is the sero-negative partner in a discordant couple. Annually there is an 11.8% sero-conversion of negative partners. It is estimated that 21% of couples are discordant in Lusaka. Other high-risk groups include mobile populations such as migrant workers, sex workers, long distance truck drivers, minibus drivers, refugees, prisoners, uniformed personnel (such as the military and police), and fishermen. 2.2 MINISTRY OF HEALTH PRIORITIES AND PROGRAMS Zambia is committed to achieving all the health-related MDGs by the year 2015. However, challenges at household, community, health services delivery, and health sector policy and strategic management levels constrain Zambia’s ability to achieve its national health sector goals and the MDGs. Use of essential health services and products such as testing and counseling for HIV, family planning, male circumcision, and male and female condoms remains low, especially in rural areas due to low demand (poor health seeking behaviors) and barriers such as impassable roads, limited numbers of well-staffed health facilities, high population densities in peri-urban areas, gender inequities, traditional beliefs and negative cultural practices, and low household incomes. The Ministry of Health (MOH) is supposed to have 30,000 health care workers on its register. However, to date the MOH only has 15,000. As a result, Zambia experiences weak technical guidance, program management, and supervision in several health areas. Further, lack of equipment, infrastructure, and medical supplies, and a weak supply chain system for drugs and medical supplies compound the situation. The burden of health sector financing is largely shouldered by the MOH because the policy environment has not provided adequate incentives to attract sufficient private sector investment in health. For example, employers are not compelled by legislation to contribute to medical insurance schemes of their employees. Recruitment and placement of government workers is centralized and undertaken by the Public Service Commission in Lusaka. This has led to inequitable distribution of health care workers because the central level does not always understand the unique needs of provinces and districts. 2.3 SUMMARY OF USAID HEALTH PROGRAMS USAID/Zambia’s health sector response aligns with the NHSP and National AIDS Strategic Framework (NASF) through a strategic approach focused on three areas: Health Service Delivery; Health Systems and Accountability; and Community Health Practices. Currently, USAID/Zambia supports five key health programs, namely Zambia Integrated Systems Strengthening Project (ZISSP); Zambia Prevention, Care, and Treatment Partnership (ZPCT) II; Communication Support for Health (CSH); JSI/DELIVER; and the Partnership for Integrated Social Marketing (PRISM) Program. Under the U.S. President's Emergency Plan for AIDS Relief (PEPFAR), USAID/Zambia supports comprehensive activities under the Ministry of Health and National AIDS Council national plans. USAID partners work directly with the Ministry of Health to expand quality services for prevention of mother-to-child HIV transmission, HIV counseling and testing, anti-retroviral therapy, and other treatment and care. USAID also supports HIV counseling and testing and 11 male circumcision services as well as socially marketed male and female condoms to high-risk groups. At the national level, USAID invests in the supply chain system for HIV/AIDS-related commodities. USAID procures anti-retroviral drugs, HIV test kits, and HIV-related lab equipment and supplies for the public sector. USAID supports the National Malaria Control Program to reach vulnerable pregnant women and children under age five with prevention and treatment services. USAID helps expand availability and use of insecticide-treated mosquito nets (ITNs) and targeted indoor residual spraying. Activities also address the dangers of malaria in pregnancy by reaching pregnant women with preventive treatment and strengthening diagnosis and treatment of malaria￾especially for children. USAID procures LLINs, insecticides for indoor residual spraying, diagnostic equipment, and other commodities for the national malaria program. USAID supports other priority activities to reduce child mortality under Zambia's National Health Strategic Plan via technical assistance and training that strengthen antenatal, post-abortion, and emergency obstetric care; address childhood immunization and micronutrient needs; expand an integrated approach to managing childhood illnesses; and improve access to safe water. USAID also invests in family planning and other reproductive health services to mitigate the economic and environmental impact of population growth and help improve maternal and child health. As family planning and reproductive health services are not uniformly available and are not often well linked to HIV/AIDS interventions, USAID/Zambia helps achieve Zambian family planning and reproductive health goals by providing technical assistance and training to expand access to family planning services in the public sector and through social marketing and support of communication for behavior change. Partnership for Integrated Social Marketing activities are positioned to contribute to USAID/Zambia’s Development Objective of improving the health status of Zambians, and through this goal, contributing to Zambia’s achievement of health-related Millennium Development Goals (MDGs). PRISM has a presence in all the ten provinces of Zambia. 3. EVALUATION METHODS AND LIMITATIONS The evaluation employed a mix of quantitative and qualitative methods including a document review, as well as primary data collection through key informant interviews (KIIs), focus group discussions (FGDs), and quantitative household surveys. The evaluation was conducted April￾August 2013. The evaluation team comprised five consultants, three research assistants, one logistician, and one public health expert serving as team leader. Evaluation activities began in April with document review and development of methodology plans (see Annex E: List of documents reviewed). The evaluation team convened in Lusaka in May to finalize detailed methodology plans and data collection tools. Sites for data collection were selected purposively. Six provinces—Lusaka, Luapula, Eastern, Northern, North-Western and Southern—and twelve districts were chosen because of the comparatively high density of project interventions in each. Within each province, one urban and one rural district and four communities per district were chosen (see Annex C for a 12 summary of site data collection activities by region). Respondents for key informant interviews (KII), mini-surveys, and focus group discussions (FGD) were also selected by virtue of their roles with the PRISM program (see Annex B for data collection tools). First round data collection activities were conducted in three provinces—Eastern, Luapula, and Northern—after which the team reconvened to prepare their midterm briefing with USAID mid-June. Second round data collection commenced in three more provinces—Southern, North Western, and Lusaka—in late June. The team spent two weeks in July analyzing and summarizing findings and submitted the final evaluation report to USAID/Zambia on August 8, 2013. A total of 92 KIIs were conducted at national, provincial, and district levels and included representatives of the MOH and Ministry of Community Development Maternal Child Health (MCDMCH), USAID, PRISM partner staff, and retail and service providers. Thirty-eight FGDs were held with community-based volunteers and project beneficiaries (including women and men of reproductive age, youth, most at-risk peoples (MARPs), and mothers of young children). Mini-surveys were conducted at 240 households in rural communities where CBVs were working and in peri-urban areas. (See Annex D for a summary of stakeholders interviewed as well as FGDs and mini-surveys conducted.) The mini-survey took place in the six selected provinces as outlined above. Purposive sampling was used in selecting these provinces, which were Lusaka, Luapula, Eastern, Northern, North￾Western and Southern Provinces, as the areas where PRISM was active across a wide range of interventions. In each of these provinces, two districts were selected. The communities in the 12 districts were further clustered into rural and urban before selection to ensure that there was a fair combination of rural and urban communities. The communities targeted were then randomly selected. The mini-survey was therefore conducted in 12 urban communities and 12 rural communities. The mini-survey targeted 240 community members in all the 24 selected communities. The number interviewed in each district depended on the proportion of the district population to the total population of the 24 districts (using population figures in the Zambia Census of Population and Housing 2010). In each of the selected communities, the interviewer randomly selected a house from the first five houses to start the interview. From the selected households the interviewer systematically selected every fifth house in urban areas and every other house in rural areas. The interviewer interviewed one person per household and continued until the required sample for the district was covered. Of the 240 mini-survey respondents, 87% were female (n= 209) and 13% were male (n=31); and 70% were from urban communities (n = 168) and 30% from rural communities (n =72). The majority of respondents had either primary school or secondary school level of education (43% and 50% respectively), with only 5% having never been to school and 2% having had tertiary education. Seventy percent (n=168) were married, 12% (n= 29) had never been married, 11% (n = 27) were widowed, and the remainder were either separated (4%) or divorced (3%). The majority of respondents were between 20 and 44 years of age (80%). Figure 1 below gives a detailed breakdown by age. 13 Figure 1 - Age of respondents Quantitative data was analyzed using Epi-info while qualitative data was analyzed using N-Vivo. Where possible and with insightful results, data was segregated by gender, age, region, and urban versus rural across all PRISM products and services. The evaluation had some limitations. The sample selection for the mini-survey included considerably more women than men as more women were available in their households at the times the survey was conducted. Thus gender representation was not even. A few key informants could not be interviewed due to problems with availability. Finally, purposive sampling prioritizing areas with a full range of PRISM program activities being implemented did not allow for comparison in non-PRISM program sites. Thus, it is not possible to compare increases in awareness, availability, and use of PRISM products and services with non-PRISM program sites. There was no baseline information on awareness, knowledge, and use of PRISM products, and therefore pre- and post-intervention data can also not be compared to understand the effect of the intervention on awareness, knowledge, and use. 4. FINDINGS 4.1. EVALUATION THEME 1: PROGRESS TOWARDS INTENDED RESULTS Table 1 presents a summary of PRISM’s progress towards results to date under each task and activity. Overall, SFH/PRISM has achieved 21 of a total 32 targets. 0% 5% 10% 15% 20% 15-19 20-24 25-29 30-34 35-39 40-44 45-49 Above 50 Don't Know 7% 20% 18% 17% 13% 12% 5% 6% 2% % o f r e s p o n d e n t s Age Range Respondent Age Source: Mini-Survey 14 Table 1: Task 1: Increase the supply and diversity of health products and services to distribute and deliver through the private sector. 1.1 Increase supply and diversity of child health products and services. Indicator with target Progress to Date ( % achievement against target) Whether Achieved, Likely to be Achieved or Unachieved? Market Clorin to disinfect 10 billion liters of water 10,566,529 Clorin distributed; 7 billion litres of water (70%) Likely Market 5.53 million diarrheal kits (ORS & Zinc) to reduce severity of diarrhea in under-fives N/A N/A 1.2 Increase supply and diversity of integrated RH products and services. Market 25.6 million cycles of oral contraceptives 7,216,110 cycles distributed (28%) Unlikely Increase demand for RH/FP through 30,526 counseling visits Over 172,000 conducted (563%) Achieved Train 200 people in FP/RH 1909 trained (955%) Achieved Establish 300 RH/FP service delivery points 97 outlets, 1,432 mobile outlets (510%) Achieved Train 60 people in research for reproductive health 18 trained in research (30%) Behind/Indicator needs clarification 1.3 Increase supply and diversity of products and services to prevent and manage HIV infection and STIs. Market 91.7 million male condoms, 10% to at risk groups 78,351,771 Maximum Classic condoms distributed (85%) Likely Market 1.83 million female condoms, 40% to at-risk groups 1,183,836 Care female condoms distributed (65%) Likely 2,553 male and 1,064 female condom outlets; of which 1,021 and 852 are hotspots respectively 1,786 condom outlets & more than 350 new hotspot outlets opened Likely Establish 10 New Start HIV testing and counseling outlets 10 established: YWCA, Cairo, Ndola, Kitwe, Mansa, Chipata, Solwezi, Livingstone, Mongu, and Kasama (100%) Achieved Provide HIV counseling and testing services to 1,026,000 clients, of which at least 10% are couples 618,149 (60%) clients received CT services (380,846 males, 237,303 females). 127,202 (124%) as couples Likely Train 2,300 individuals in HIV 2,720 (118%) training in Needs clarification 15 counseling and testing HIV prevention. Data is still being collated for HIV testing and counseling specifically Increase number of Horizon graduates by 22% 22.829 clients graduated (34% increase) Achieved Distribute 400,000 Clorin bottles to at least 55,000 PLHA through basic care services 1,109,316 Clorin bottles distributed to PLWHAs to date. 400,000 planned in Year 4 Achieved Circumcise 30,000 men, at least 75% are 13-29 years old 78,508 through April 2013 (262%) Achieved Establish 3 MC outlets annually which offer 3 or more integrated HIV/RH services (CT, STIs, TB screening, RH/FP) 8 established (53%): YMCA, Chachacha, Mpenzeni, Obote, Chipata, Solwezi, Mongu, Kasama (likely to be achieved) Likely Reach 595,328 individuals with non-A&B HIV community outreach 850,124 individuals reached (143%) Achieved Train 2,700 individuals to promote HIV preventive behaviors 2,720 trained (101%) Achieved 1.4 Increase supply and diversity of products and services to prevent malaria. Distribute at least 2 million ITNs 2,020,676 ITNS distributed (101%) Achieved Develop and implement strategy for transfer of the ANC ITN program to the MOH to handle independently No strategy developed yet Unlikely Maintain supplies of ITNs to all 72 DHMTs nationwide Done in all 72 districts (100%) Achieved Train 2,000 community outreach workers to promote correct and consistent ITN use 2,486 volunteers trained (124%) Achieved Task 2: Increase awareness & demand for products and services to prevent childhood illness, pregnancies, HIV and STIs, and malaria. Train 2,000 community outreach workers to promote use of ITNs and other child health products 2,486 volunteers trained (124%), 770 planned in Year 4 Achieved Reach at least 593,328 individuals through community promotion of non-A&B HIV/AIDS preventive messages 850,124 individuals reached (143%) Achieved 16 Reach at least 186,040 people to increase awareness and demand for family planning methods 340,865 individuals reached (183%) Achieved Task 3: Develop a commercial sector entity to produce and market one socially marketed product sustainably. Develop a commercial/private sector entity to produce and market Clorin in a sustainable, self￾sufficient manner Willingness to pay surveys underway. Market survey scheduled in 2013. Unlikely OPTIONAL Task 5: Develop a commercial sector entity to produce and market one socially marketed product sustainably. 5.1 Market 150,000 doses of Misoprostol in priority districts annually Jan-Apr, 2013: 32,000. (21%) Too early to evaluate 5.2 Increase Misoprostol sales by 5% annually. All Misoprostol is given free N/A 5.3 Increase awareness and demand for Misoprostol by reaching 186,040 people SFH just starting to collect data now Too early to evaluate OPTIONAL Task 6: Increase awareness, demand for and use of Zinc with ORT. 6.1 Market 5.53 million diarrheal kits (DTKs) containing ORS and Zinc to reduce length and severity of acute diarrhea in under-fives. N/A N/A 6.2 Increase demand for diarrheal treatment kits (DKTs) containing ORS and Zinc. N/A N/A A. Safe Water Products Clorin 1. Summary of Key Activities PRISM partners sell slightly more than 3,000,000 bottles of Clorin annually, 75% of total sales through commercial sector wholesale and retail outlets and 25% through Care CBVs and other partner agency volunteer networks. Community-based volunteers also undertake community￾based promotion of safe water and Clorin. In times of diarrheal disease outbreaks and during rainy season, the MOH and UNICEF buy or sometimes receive free stocks of Clorin from SFH with which to target known high-risk communities. Additionally, SFH has supplied millions of Clorin bottles through PLHA programs as part of various implementing partners’ care and support programs. SFH/Zambia’s Clorin program is the most successful of all of PSI’s safe water programs globally. Sales could still increase with increased budget to procure Clorin. Until recently, SFH has sold 17 Clorin at a subsidized price to the retail trade and partners. However, as a result of the willingness-to-pay survey conducted by Population Council in 2013, SFH concluded that it could increase the price of Clorin significantly. The product is now sold to the trade at 85% of cost of goods (COGs).7 SFH plans to increase the price of Clorin again shortly and then by at least 5% annually. If incremental increases in pricing do not significantly hamper access and use amongst target groups, SFH will soon be able to match demand for Clorin as there will be no programmatic cost for production. With the price increase and concurrence from USAID to shift additional funds for Clorin production, SFH anticipates selling more than 3.4 million units in the course of Year 4. 2. Key Findings and Analysis Awareness and Demand for Clorin Household survey results found extremely high awareness of Clorin at 97% (233/240) consistent with KII and FGD results with respondents commonly reporting awareness of the product and knowledge of benefits of use—to ensure their family drinking water is safe. This awareness was through health facilities (77%, 185/240) and Community Volunteers (25%, 60/240), followed by radio (6.3%, 15/240) and television (4.2%, 10/240). However, the qualitative data also found knowledge gaps about Clorin: The perception that Clorin is toxic and may cause health problems and complications; The misconception that the product is “medicinal” and to be used only during high risk periods and cholera or typhoid outbreaks; Confusion about correct dosing procedures (for example, some use an entire bottle to treat their wells). Some respondents reported a problem with using the right dose of Clorin in their drinking water, which creates more barriers to use as the water has an unpleasant smell and/or taste; and Amongst household respondents who were not using Clorin, 15% said they were put off by the taste of treated water, an indication that more education and outreach activities to improve understanding of correct dosing procedures for Clorin are needed. Of household respondents, 74.7% indicated willingness to pay more for Clorin. More insight into willingness to pay needs to be explored considering that 35% of household respondents who were not using Clorin indicated price and lack of money as their primary constraint to purchase and use. The survey did not gage how much more target consumers might pay, so it is difficult to assess how price-sensitive Clorin truly is. Availability and Access to Clorin Results of household surveys found that 83% (198/240) of respondents knew where they could purchase Clorin. Beneficiaries in rural FGDs also indicated that where CBVs were not selling Clorin in theimunities, they had been given no information on where else they could buy it. Better links to retail trade and health facilities willing to stock Clorin would help to overcome this barrier. Clorin was found to be widely available through commercial retailers such as market stalls, pharmacies, and drug stores in urban and peri-urban areas, and to a lesser extent, through 7 COGs for Clorin include the total cost of manufacturing of the bottle, cap, solution, and label. Boxing, warehousing, and transport are excluded from the total cost of goods sold calculations presented here. 18 community volunteers in rural areas. From time to time, DHOs as well as UNICEF purchase Clorin to distribute free to communities during high risk period and disease outbreaks (Mwense District, Luapula Province; Kasempa District, North-Western Province; and Mungwi District, Northern Province). Clorin is also sometimes distributed free by Neighbourhood Health Committees and Safe Motherhood Action Groups members, as well as through treatment and care programs working with PLWAs. While CBVs have all been trained to sell Clorin alongside other socially marketed products, community based sales of Clorin are lower than for other products, resulting in reduced numbers of volunteers willing to stock the product and ultimately less access in rural areas. The size, bulk and cost of Clorin coupled with long distances to transport the product present barriers to sales of Clorin for Care’s CBVs activities. Use of Clorin Use of Clorin is impressively high; nearly three-quarters of all households surveyed treat their water and amongst these respondents, 89% (158/177 that purified their drinking water) use Clorin, over boiling or other methods (Figure 2 below). At the time of the survey, however, only 42% (n=75) of households surveyed that purified their drinking water had Clorin available in their households. Amongst respondents not using Clorin, 15% (n=9) didn’t like the taste and smell of treated water, 35% (n=21) gave affordability and price as the barrier to use, and 37% (n=22) chose to use other methods, such as boiling. Figure 2 - Method of water purification Qualitative results found significant seasonal variation in purchase and use of Clorin, with higher sales and consistent use in the rainy season when the risk of waterborne diseases is greater, as well as during outbreaks of typhoid and cholera. In Chipata District, there was an outbreak of typhoid at the time of the evaluation and all respondents interviewed indicated that demand for Clorin had increased so significantly that the DHO was unable to meet this demand and need in high-risk communities and had already appealed for help from SFH to provide them with additional Clorin. Qualitative data also found mixed responses regarding price and willingness to pay for Clorin; while some respondents felt SFH should reduce the price others noted that when perceived risk was high, demand increased and individuals were more willing to pay for this product. While the 7.9% 88.2% 3.9% Method of Purifying Water Source: Mini-Survey Boiling Clorin Other 19 vast majority of respondents felt that use of Clorin was positively correlated with prevention of waterborne diseases, SFH and partners might consider including messages that address perceived risk and consistent use outside of rainy season and disease outbreaks. Clorin Process Indicator Progress The program was tasked with marketing Clorin to disinfect 10 billion liters of water and distributed 10,566,529 bottles Clorin adequate to disinfect 7 billion liters of water suggesting the program is likely to meet its target. The second indicator to market 5.53 million diarrheal kits (ORS & Zinc) to reduce severity of diarrhea in under-fives was not implemented. B. Malaria Prevention ITNs 1. Summary of Key Activities ITN Distribution to Districts SFH partners with the National Malaria Control Center (NMCC) in a nationwide project to distribute Mama Safenite LLINs in an effort to prevent malaria. As a national implementer of the Malaria in Pregnancy program, SFH has rolled out the program to all nine provinces. The program is managed throughout District Health Management Team centers and then through antenatal clinics at district and ward levels, making free nets easily accessible to pregnant women and children under five. SFH has delivered over two million ITNs to DHOs to date through the PRISM program; more than 370,000 were delivered to 59 districts out of 72 in Year 1; over 990,000 ITNs to 72 districts in Year 2; and more than 730,000 nets to 65 districts in Year 3. Outside the PRISM program, SFH has also assisted the NMCC with warehousing and logistics of millions of other ITNs from other donor for example DfID, PMI over the life of the project. At the start of the PRISM program, SFH worked with the DHOs to assist them with management of the ANC ITN system and delivery of nets to facilities. SFH provided the DHOs with its own internal ITN tracker system and guided them through implementation and use. DHOs were intended to use this system to record, monitor, and report on receipt of nets from SFH and the subsequent transfer of these nets to health facilities and project beneficiaries. Since then, SFH has increasingly been asked to assist the NMCC with warehousing, logistics, delivery, and distribution of ITNs, both within the ANC program and in other campaign events. However, they have not provided any further technical support to either the NMCC or MOH nationally or to DHOs in management of ITN distribution programs nor in systems strengthening for forecasting, planning, logistics, tracking, or reporting on these. 2. Key Findings and Analysis Awareness and Demand for ITNs Household surveys found that 93% (224/240) of respondents knew about ITNs. Of the total respondents, 83% (n=200) heard of them through health facilities and 13% (n= 31) through community volunteers, followed by radio at 4.6% (n=11) and television at 4.2% (n=10). Thousands of CBVs have been trained by PRISM partner CARE International to provide education in their communities about malaria and ITN use, especially for high risk groups. However, FGDs with PRISM volunteers indicated that this was not a priority for them or a 20 major focus of their work. In part, this is because they are only sporadically engaged in community ITN distribution efforts. In equal part, CBVs are not paid for their time; they earn money only from product sales, so there is little incentive for them to spend time addressing products they cannot also sell. FGDs with beneficiaries (mothers of children under five) found high knowledge of ITNs in general, though gaps in ITN information exist, including the belief that ITNs itch and irritate skin (if not aired before use or come in direct contact with skin), as well as who are the high-risk groups who should be given priority for net use in the home. Availability and Access to ITNs Availability of and access to ITNs through ANC clinics varied considerably between the DHOs and health facilities visited; in the majority of sites visited, no stocks of ITNs were found on hand and a considerable number of DHO and health workers were unable to confirm when last they had received ANC nets. Of note, DHOs in Luapula and Southern Province indicated they had not received ANC nets since 2011. Also of note, a number of DHOs, such as in Mansa and Mwense in Luapula Province and Mazabuka in Southern Province, reported no ANC nets received from SHF since 2011. This finding is troubling, as the evaluation team confirmed via JSI/Deliver (who procured and developed the 2012 district net distribution plans jointly with the NMCC), SFH MIS data confirming total numbers of nets delivered to the DHOs and from the PMI Audit Report 2012 that thousands of nets had in fact reached each DHO. (An in-depth analysis of weaknesses and challenges within the MOH supply chain systems for ANC nets is included in Annex F.) While FGD respondents knew that pregnant women were supposed to receive a free ITN through ANCs at the time of birth, most respondents across the majority of sites visited reported frequent stock outs and insufficient supplies of ITNs through ANCs. In 2012, a mass campaign was carried out in which roughly 4 million nets were distributed door to door through community leaders and volunteers as well as through health facilities. Households received one or more ITNs according to family size and need. Provision of free ITNs to pregnant women is intended to fill gaps in net coverage in the one to two years between mass campaigns; however, findings from this evaluation indicated that the systems needed to insure correct implementation and distribution of ANC nets are lacking, and neither health staff nor beneficiaries understand the objectives and differences between ANC and community campaign net programs. Challenges and constraints to the current ITN program include the following: lack of defined procedures and/or adherence to guidelines for free targeted distribution (allowing loopholes and leakages whereby preference is given to family and friends of those distributing nets as well as stock-outs of ITNs for intended beneficiaries); inequities in reaching and coverage to poor and deep rural communities; and inadequate supplies of ITNs delivered to health facilities to cover ANC needs. Use of ITNs Quantitative results of mini-surveys found that 71% (171/240) of all households owned at least one ITN (Figure 3 below). Sixty-five (65) per cent of urban households sampled had an ITN compared to 84.7% of rural households. Over half (57.8%) of all respondents with an ITN had slept under an ITN the night preceding the survey (47.5% of urban respondents slept under an ITN compared to 77.8% of rural respondents); 44.6% of respondents who had children under five years of age reported that the child had slept under an ITN the previous night (64.2% of rural respondents and 35% of urban respondents). Clearly ITNs are more available and more utilised in rural areas at household level. 21 Figure 3 - Mosquito net availability in household The majority of project beneficiaries in FGDs also reported owning ITNs, though responses also showed that ITN use was inconsistent amongst some respondents who said the ITNs in their home were not being used by anyone. Use of ITNs amongst intended target beneficiaries (pregnant women and children under five) also varied considerably between respondents in FGDs; in some communities, young mothers indicated they did not use ITNs in their homes as they had not had access to a free one in the last two to three years, and the price to buy one in the market was prohibitive. Zambia’s National Malaria Indicator Survey from 2012 includes encouraging data and findings regarding ITN coverage and use in rural areas and amongst intended beneficiaries: in aggregate, 59.9% of children under five slept under an ITN the night before the survey; ITN use was highest amongst young children in Eastern (79.9%), Luapula (77.6%), and Northern Provinces (65%). Rural women were more likely to sleep under ITNs (60.2%) than women in urban areas (47.6%). In addition, the percentage of pregnant women who slept under an ITN the night before was higher (74.8%) in the lowest wealth quintile than in the highest quintile (45.1%). These finding indicate that despite the many constraints and challenges identified during this evaluation, ITNs are nonetheless reaching rural areas and being used by target groups. “72.1% of households in Zambia had a mosquito net… and 56.9% had more than one ITN. The average number of ITNs per household was 2.4. Ownership of nets (all categories) was typically higher in rural than urban areas suggesting the prioritization of rural areas through mass distribution has been largely successful. For example, average number of ITNs per household was 2.01 for urban compared to 2.56 in rural areas. In Luapula Province, 93.5% of households reported having at least one mosquito net, and 90.0% of households reported owning at least one ITN. This is the highest percentage of mosquito net and ITN ownership reported among the provinces.” (Malaria Indicator Survey 2012, page 22) 28.9% 71.1% ITN in Household? Source: Mini-Survey No Yes 22 ITN Process Indicator Progress There are four ITN indicators: three already achieved, and one is unlikely to be achieved (See Table 1). The program has distributed over 2 million ITNs above the target of 2 million; maintained supplies of ITNs to all 72 DHMTs nationwide as targeted; and trained 2,486 volunteers outreach workers versus the target of 2,000 to promote correct and consistent ITN use. However the target to develop and implement strategy for transfer of the ANC ITN program to the MOH to handle independently is unlikely to be achieved as no strategy has been developed. C. Family Planning and Reproductive Health Family planning products 1. Summary of Key Activities The PRISM portfolio of family planning products which are sold at subsidized prices through community based distributors in rural communities includes SafePlan oral contraceptives, injectable contraception, IUDs, and Jadelle implants. To date, nearly 2,000 CBV workers have been trained and are working through PRISM partners to reach women of reproductive age with family planning methods in deep rural areas not reached by the commercial market and too far from health facilities to benefit from free MOH distribution. SafePlan is a well-known brand across Zambia, though the decision to adopt a free family planning policy through all MOH facilities, coupled with a substantial increase in availability of free family through all health facilities in recent years has had a marked effect on PRISM sales of oral contraceptives and other family planning products. Despite this, CBV activities fill a vital gap in family planning coverage, reaching women who would otherwise go without consistent access to family planning options. While PRISM’s CBV program is relatively small and reaches only selected communities in certain districts and provinces, both primary and secondary data confirm the importance of activities such as this to increase the reach and scope of health interventions for rural populations. Society for Family Health - PRISM and partners also work with private and public sector service providers to improve the availability of contraceptive products, including long-acting reversible contraception (LARC). In addition to retail and community based sales of short-term FP methods, SFH works in collaboration with the MOH to improve access to and use of long-term family planning methods. Society for Family Health has a cadre of trained service providers who are positioned within MOH facilities permanently or on rotation/as needed in response to MOH human resource gaps and resulting overburdened urban health facilities as well as under￾supported rural MOH clinics. Society for Family Health also provides training and equipment for LARC to 103 MOH and private health facilities providers to date. In Year 2, SFH piloted immediate insertion of post-partum IUDs in collaboration with the University Teaching Hospital (UTH) and MOH, which proved a success and provided many lessons learned and best practices from which MOH, SFH, and partners are now scaling up and rolling out the program nationally. SFH has also expanded access to family planning services through contracting trained midwives to work Saturdays when MOH facilities are normally closed. Constraints to the program include myths and misconceptions about all family planning methods, but particularly for LARC; critical gaps in reaching youth and youth friendly services, leading to early teen pregnancies and preference for abortion over family planning; and perceived lack of partner support amongst women. CBD programs work well for provision of 23 products, but as livelihoods depend on sales commissions, they are a less reliable source for referrals for LARC; large number of providers not yet trained to provide LARC combined with exorbitant government per diems has restricted access to skilled family planning service and care. The MOH has not yet laid out a systematic plan for the training of family planning providers in areas where they are needed most. Family planning service provision is inconsistent as a result of low staffing levels and high staff turnover. Constraints which have been key to increased sales of oral contraceptives and SafePlan include the following: 1) stock out due to USAID shift in oral contraceptive brand/type; 2) Warehouse fires; 3) MOH flooding of market with free family planning; and 4) an inability to sell off FP off￾script, limiting commercial sales. 2. Key Findings and Analysis Awareness and Demand for Family Planning Household survey results found very high knowledge and awareness of all family planning methods: Awareness was highest for Maximum condoms (99%, 237/240)) followed by SafePlan and Jadelle (95% and 96% respectively, with knowledge lowest for IUDs (83%, 198/240)). Target beneficiaries (women of reproductive age) in FGDs also confirmed high levels of knowledge about the range of family planning methods available. The main channel through which messages on family planning had been heard was health facilities (over two-thirds of respondents) followed by Community volunteers (about one-fifth of respondents). Nearly all FGDs’ results confirmed that Depo-Provera remains the most popular method amongst women of all age, particularly amongst young women (15-24 years). Despite high levels of awareness, constraints and barriers exist; amongst them, myths and misconceptions continue to hamper demand for almost all family planning methods. Misconceptions include the belief that SafePlan causes cancer and infertility; that long term methods cause sterility and also cancer; and that IUDs are believed to travel within the body and sometimes cause heart problems. Some women also fear that Jadelle could also be dislodged and lead to health problems. In addition to myths and beliefs about family planning methods, the free provision of some methods such as oral contraception in health centers alongside socially marketed sales of SafePlan has led to confusion and misgivings amongst women in some communities. Focus groups with young women and community volunteers indicate that youth friendly family planning services are lacking and young women lack the information and services they need to prevent unwanted pregnancies. Focus group discussions also found that while the majority of male participants indicated their full support for whatever family planning method their partner chose to use, the majority of women felt they needed their partner’s approval and consent before choosing a method. Targeting men to better encourage family planning use amongst partners and programs designed to reach adolescent women (15-19 years) are needed. Availability and Access to Family Planning The successful implementation of free family planning services throughout the Ministry of Health’s network of health facilities is evident: 93% of household respondents confirmed that family planning methods were available for free at their nearest health center. A further 39% said that they could also get family planning free in communities. While the MOH should be applauded for its achievements in scaling up access to modern family planning methods countrywide, it has also negatively impacted the PRISM programs success with regards to sales of SafePlan. While SFH recognizes that this is a success for Zambia and Zambian women, they 24 also recognize the need to revise program focus from the strategies included in the original program design. Where sales of SafePlan have lagged behind indicators, SFH has focused efforts on increasing access to LARC (in response to a contract modification from USAID adding LARC to their portfolio in January 2011). Family planning methods and services were reported to be more widely available in urban than rural areas amongst beneficiaries in FGDs. The provision of LARC in particular is not easily accessible in rural and remote health centers or mobile clinics. Where PRISM does not have CBVs distributing SafePlan, women are left without options and are at higher risk of unwanted pregnancy as a result. Oral contraceptives and Depo-Provera continue to be the most popular methods; however, whether this is simply a function of availability and familiarity/positive experience or actual choice is difficult to assess. SFH has assisted the MOH with the training of nearly 400 providers in provision of LARC to assist in increasing access to these services. Whereas LARC was available in less than 50 facilities nationwide in 2009, in 2011 this number had increased to more than 350. Despite increased numbers of facilities now providing LARC services, uptake for these methods is slow. Use of Family Planning According to the Ministry of Community Development Maternal and Child Health (MCDMCH), new family planning acceptors increased from 45,500 in 2010 to 51,300 in 2011. Utilization rates for family planning amongst households surveyed was found to be fairly high: 54.6% (131/240) of the respondents currently use a modern method, compared to a national CPR of 40.8 (DHS, 2007). Figure 4 - Method of modern contraception used Of note, the above figure demonstrates that only 15% of all women using modern contraception chose a long term method; female condoms are the least popular method; and worryingly, barely more than 5% of respondents are using dual methods to protect against HIV too. The survey also found that 67.1% of rural respondents had discussed FP with their partner compared to 52.3% of urban respondents. Over 61% (60.7%) of males and 56.1% of females had discussed FP with their partners. 0.0% 10.0% 20.0% 30.0% 29.5% 5.4% 1.6% 4.7% 10.9% 17.8% 0.8% 23.3% % o f R e s p o n d e n t s Method of Modern Contraception Method of Contraception Source: Mini-Survey 25 Family Process Indicator Progress: The program has five family planning indicators/targets (Table 1). Three are already surpassed: conducted 172,000 counseling visits to increase demand for RH/FP compared to the target of 30,526; established 97 outlets, 1,432 mobile outlets versus the target of 300 RH/FP service delivery points; and the target to train 200 people in FP/RH compared to1909 trained. The program target is to market 25.6 million cycles of oral contraceptives with 7,216,110 cycles distributed suggesting the target is unlikely to be achieved. Misoprostol 1. Summary of Key Activities At inception of Task 5, SFH had no USAID funding with which to purchase Misoprostol8 hence opted to invest $50,000 of PSI unrestricted funds to procure the first lot of stocks needed to begin activities9. Simultaneously, SFH assisted the MOH with registration of Misoprostol for postpartum hemorrhage; developed training and demand-creation activities and materials for community-based distribution of Misoprostol; and piloted Misoprostol distribution in ten districts with high home deliveries: Chongwe, Luangwa, Kafue, Mumbwa, Senanga, Mongu, Kazungula, Lufwanyama, Mwense, and Chiengi. A total of 591 providers and community volunteers were trained in and around 382 facilities with messages regarding Misoprostol use. Constraints and challenges to distribution of Misoprostol include commodity security for current activities and future scale up; short shelf-life for Misoprostol; long distances / hard to reach rural areas; and high training costs due to revised GRZ daily subsistence allowance. While they await a means of procuring more Misoprostol, SFH is currently preparing to scale-up activities to include four Saving Mothers Giving Life districts; Kalomo, Nyimba, Lundazi, and Mansa, and to continue supportive supervision at facilities to keep providers engaged. Society for Family Health is also planning to engage MOH/MCDMCH in discussing ways to reduce the training costs. SMAGS will receive training for future community involvement, and SFH will advocate for government ownership of community distribution of Misoprostol with the aim of improving commodity security in the future. Between March and December 2009, SFH distributed 234,140 tablets to 205 health facilities located in the ten designated districts. During the same time period, SFH trained 556 facility￾based health workers and 59 community members in PPH prevention. Monitoring data also reveals that 54% of the 205 facilities involved in the SFH PPH pilot program did not have oxytocin on-hand when the Medical Detailer first visited them during the months of March￾October 2009. No adverse events or complications associated with Misoprostol were reported between March and December 2009. Monitoring data suggests that women returned unused Misoprostol to facilities in cases where they delivered at home and forgot or were unable to take the tablets (e.g., if they delivered while traveling and did not have the tablets with them) or delivered at a facility. SFH’s PPH prevention pilot project involved distributing Misoprostol supplies to health facilities and training facility-based health workers to give every antenatal client three pills of Misoprostol, along with instructions on how and when to take them, at first contact. In Lufwanyama district 8 USAID funds are restricted from purchasing Misoprostol because of its alternative use as an abortifacent. 9 MOH promised 200,000 doses, but these have yet to come through to the program. Further, the program is currently at a halt. DfID has expressed interest, and this should be explored. 26 only, the MOH approved SFH to train TBAs who had received significant prior training through another public health operations research project. Society for Family Health trained these TBAs to help facility-based health workers distribute Misoprostol to pregnant women and promote correct use following home deliveries. Women reported reasonably high levels of knowledge regarding Misoprostol for PPH prevention. Interviewees were able to describe benefits, side effects, and risks associated with Misoprostol. They reported receiving this information from facility-based health providers or TBAs. Their knowledge about the drug was based on information they received during antenatal counseling and/or training from SFH. HIV and STIs Prevention Male condoms 1. Summary of Key Activities PRISM works through a range of private, public, and non-governmental channels to improve availability of and access to HIV prevention products and services including male condoms. To improve commercial availability of Maximum condoms amongst peri-urban retailers and outlying markets, SFH sales teams focused on “uplifting” activities to strengthen linkages between these smaller retail outlets and the existing network of urban based wholesalers stocking Maximum condoms. In Year 3, SFH began phasing out these intensive uplifting support activities, while closely monitoring commercial sales to insure there was no drop in volumes sales as a consequence. While it appears that uplifting activities were successful in strengthening sustained linkages between wholesalers and retailers, without a mapping survey it is not possible to confirm how many retail outlets currently stock Maximum condoms. A mapping survey to assess market reach for all socially marketed products would assist the project to evaluate the coverage, reach and penetration of its commercial distribution and sales activities. The program is also currently conducting a willingness-to-pay survey for condoms, to determine if the retail price for Maximum condoms can be raised and how this affects access and use amongst target groups. Results will inform future pricing for condoms as well as marketing and BCC messages and activities. PRISM also sells Maximum condoms through its partnerships with private sector companies such as Barclays Bank, Bank of Zambia, and the three mobile providers in Zambia. These companies in turn distribute Maximum condoms free of charge to employees. PRISM’s Community Based Volunteers (CBVs) have been trained to educate, promote and sell Maximum male condoms (alongside other health products and services) in selected communities and districts across Zambia. The CBVs carry out community outreach activities, including door￾to-door outreach and mass campaigns in markets, health facilities, and at traditional and other local events. 2. Key Findings and Analysis Awareness and Demand for Male Condoms Mid-term household surveys indicate universal awareness of male condoms at 99%, a finding that was corroborated with FGD respondents who were all aware of Maximum male condoms. Both key informants and FGD respondents reported condoms to be in high demand. Focus group participants were generally aware that the condom is the best product for its dual purpose of protecting against sexually transmitted diseases and preventing pregnancy with the exception of 27 some sex workers in Livingston who were not unaware. Youth reported having observed that people are relying more on condoms for fear of contracting diseases. Myths and misconceptions about condoms still persist: for example the belief that the lubricant on the condom makes people sick, and that it is not 100% safe. Availability and Access to Male Condoms Maximum condoms were reported to be widely available through PRISM CBVs, health facilities, retailers, pharmacies and drug stores, and other CSOs. Sales of male condoms were said to have increased due to education efforts. Use of Male Condoms Male condoms were found to be well received and widely used in the communities visited. Sales of male condoms were reported to have increased due to education efforts. Some couples reported not using condoms because they trusted each other; some women said that their partners refuse saying “you are my wife” and some associated condom use with promiscuity. Volunteers from a peri-urban area reported increased number of people accessing condoms but doubted if they were used correctly or consistently as incidences of HIV infections remained high (quote from Chongwe volunteer FGD). These factors help to explain the low level of condom use at last sex (18.5%) as well as multiple sexual partnerships reported by 15.5% of respondents in the household surveys. Youth in FGDs reported using condoms primarily for infection prevention purposes. In one FGD, four out of eight male youth respondents reported using condoms; three claimed to be using condoms consistently; and one admitted he used them irregularly. Two young men indicated they were not currently sexually active, and the remaining two were not using any condoms at all (neither male nor female). Male Condom Process Indicators The program target is to market 91.7 million male condoms, 10% to at risk groups of which 78,351,771 Maximum Classic condoms have been distributed. Additionally PRISM was to set up 2,553 male condom outlets. The program has established 1,786 male and female condom outlets and more than 350 new hotspot outlets opened. These results indicate the targets are likely to be achieved within the program period. Female Condoms 1. Summary of Key Activities The Care female condom (FC) is distributed through several channels, including pharmacies, drug stores, health facilities, and VCT centers as well as in partnership with non-governmental organizations. PRISM activities target high-risk groups, such as sex workers, miners, and truckers, through hair salons, barbershops, bars, brothels, and nightclubs. Activities are focused in areas with high concentrations of commercial sex work and/or migratory and transitory populations (such as truckers and miners). In Lusaka Province, PRISM partner ZHECT targets most at-risk populations (MARPs) in four districts of Lusaka province with male and female condoms alongside HIV prevention messages. Consistent with global findings for female condoms, sales and uptake of Care female condoms are low, leaving both retailers and CBVs reluctant to invest in stocks of this product. Plagued by negative consumer perceptions, low advocacy at a national level and limited knowledge about the characteristics of women who buy and/or use Care female condoms, the program faces 28 significant challenges with this intervention. In Year 4, SFH is working more closely with non￾traditional outlets (such as NGOs, bars, motels, and hair salons) to promote Care to their clients. SFH is also planning a consumer insight and profiling study to better understand its target market and to hone future BCC messages and channels used to reach them. With lessons learned from PSI Zimbabwe’s recent investments in female condom promotion and distribution, SFH plans to intensify BCC activities in the remainder of Year 4 and Year 5. 2. Key Findings and Analysis Awareness and Demand for Female Condoms Household surveys indicated an 87% (208/240) awareness of the female condom. Some of them knew Care was sold by CBVs; others knew it was distributed in health facilities. A few respondents associated female condoms with sex workers. There is inadequate promotion of female condoms by CBVs and health centres, and most women report preference for male condoms over female. In some cases Care is seen as positioned for sex workers; in others it is unclear to whom the female condom is targeted. Availability and Access to Female Condoms Care female condoms are not widely available. Retailers, pharmacies, and drug stores shun the female condom as sales are very low and expiration is common. In some cases, Care female condoms were purchased not for use during sex but instead as jewelry to be worn as a bracelet. The pricing of the products was seen to be too expensive for the youth to afford. Care female condoms are reported to be the least purchased and used of the products sold by CBVs. As a result, Care expires a lot and CBVs are forced to give them free or dispose of them. Female condom use is perceived by some to be for sex workers. Use of Female Condoms Self-efficacy/confidence of use is low. Female condom acceptance and use remains very low with all volunteers, retailers, pharmacies, and condom outlets interviewed indicating difficulties selling it and ending up with expired products. In most focus groups discussions, the general perception is that women prefer to use male Maximum condoms rather than Care female. Women themselves prefer male partners to use male condoms. A number of men also prefer to use male condoms than their partners using female condoms. Two women in two different FGDs shared that they even pack Maximum condoms for their spouses when they go on trips. Community Based Volunteers end up discarding them due to lack of sales. Reasons given for non-use of female condoms include difficulty to insert, time wasting, and making noise during sex, and that men don’t like being guided during sex by their partners. The price is also reported to be high for some women and youth to afford. Other factors reported by women include the misconception that it may remain inside them or that Care female condoms are only bought by women who are HIV-positive and don’t want be re-infected with another strain of HIV. These factors help to explain the low level of use at 1.6% in the household survey. It is unclear who is targeted by the female condom and if sales or free distribution result in use, or used for other purposes/thrown away. It was said that women don’t like the Care female condom due to its size, noise, and unattractiveness. 29 Female Condom Process Indicator The PRISM program was to market 1.83 million female condoms, 40% to at risk groups. Over one million have been distributed (1,183,836 Care female condoms). The program has also set up 1,064 female condom outlets that are part of the 1,786 male and female condom outlets and opened more than 350 new hotspot outlets. Counseling and Testing for HIV 1. Summary of Key Activities To increase access to Counseling and testing SFH/PRISM has established ten New-Start sites as targeted by the program (Table 1). Of the ten sites PRISM has established to date, five are managed by PRISM partners and five by SFH. Mobile outreach counseling and testing services are conducted from these sites, often in tandem with mobile provision of male circumcision. Routine HIV testing for male circumcision has had a significant impact on the volume of counseling and testing clients nationally. In addition, it has helped increase the numbers of couples who test together and/or know their partner’s status. With the new MOH policy to provide “universal access” to counseling and testing, SFH and partners have assisted the MOH in handling the significant rise in demand and surplus of patients that would otherwise have overwhelmed MOH capacity. Society for Family Health also assists the MOH with training and logistical support at non-PRISM sites. A total of 251 people were trained including public facility providers, male circumcision and counseling and testing counselors, and SFH staff. Society for Family Health has received multiple requests from the Provincial Health Office (PHO) in Central Province for assistance with counseling and testing service delivery, training, and logistical support. However, PRISM cannot operate in Central with supplies and stocks from Lusaka Province or the services provided would have to be reported and “counted” in Lusaka; so SFH would need both USAID approval and an agreement in place with the Central Province PHO before it can respond to this. Society for Family Health and partners have undertaken a variety of BCC and community mobilization activities to improve uptake of counseling and testing services. CBVs undertake community mobilization in anticipation of campaign and mobile service activities scheduled in their areas. SFH undertook an intensive mass media and interpersonal campaign to encourage couples to go for counseling and testing together. The campaign helped to increase the numbers of couples testing and sharing their status with each other, a critical step in prevention of HIV transmission between discordant couples, particularly as low and inconsistent use of condoms amongst trusted partners is a key driver of HIV transmission in Zambia. 2. Key Findings and Analysis Awareness and Demand for HIV Counseling and Testing Awareness of counseling and testing is universal: 99% (237) of households surveyed were aware of counseling and testing services. Awareness of counseling and testing is attributed to activities under PRISM (specifically for the New-Start counseling and testing clinics), and government and other civil society organizations (CSO’s) efforts in educating people about and making referrals for counseling and testing. Men, women, and youth are aware of the importance of knowing their status so that they can either protect their negative status or get treatment since availability of ART is now more widespread, too. Women in FGDs indicate they believe it is 30 important to know their status so that measures can be taken to prevent transmission to their new-borns. Married couples in FGDs attached great importance to voluntary counseling and testing as a tool for protecting the family from the adverse effects of HIV. One married woman in a FGD said “we need to know our status to keep the family together.” However, FGDs held with both female and male youth indicated that they lacked the information and social support in particular to seek counseling and testing services. While some respondents indicate fear of knowing their status and stigma around this as a barrier still, most respondents indicated that due to outreach and education activities on the importance of counseling and testing, most people in their communities felt positively about the need for the services. Availability and Access to HIV Counseling and Testing Counseling and testing is widely available at PRISM static sites including New-Start clinics and in public health facilities and other civil society organizations (CSOs). Of the 12 districts evaluated, six had New-Start Centres, but two of these, in Kasama and Mungwi, had only begun operating in mid-2013. Mobile counseling and testing is also provided through New-Start centres and public health facilities to reach rural communities. However, limited human resources and transport for mobile counseling and testing pose challenges to reaching rural areas without access to static sites. The adequacy of HIV information and services for youth varies from one place to another. For instance in Mungwi, Malole community, people think the youth are too young to know the details, while in other districts such as Mansa, Solwezi, Kasempa, Livingstone, and Kazungula, access was adequate. Youth in Solewzi praised the strong Youth Alive activity there. The evaluation team observes that programs such as Youth Alive deserve more attention and focus, and a best practice. Lessons learned review from this should be undertaken, shared, and duplicated where possible. Lessons from Youth Alive might also be applicable in reaching young people with family planning information and services. Use of HIV Counseling and Testing The SFH-PRISM program target is to provide HIV counseling and testing services to 1,026,000 clients, of which at least 10% are couples. The project has reached 618,149 (60%) clients with counseling and testing services (380,846 males, 237,303 females) of which 127,202 (124%) are couples. The findings indicate the target has been achieved. Household surveys found that 71% of respondents had tested for HIV and received their results; 58% of respondents also had knowledge of their partner’s HIV status (Figure 5 and 6 below). These findings are corroborated with FGD results. The majority of target groups interviewed (couples, pregnant women, mothers, sex workers, truckers, and men targeted for counseling and testing) knew their HIV status. However the majority of youth did not know their HIV status. It was mentioned that for some the fear of knowing HIV status and stigma drove some people not to test; while other people do not care about knowing their status. However, according to respondents, due to sensitization, the number of such people is on the decline and stigma is also decreasing. Increased numbers of people, youth and adults, are testing for HIV, and those who test positive are referred for care and treatment. 31 42.0% 58.0% Know HIV Status of Partner? Source: Mini-Survey No Yes Figure 5 – Knowledge of partner HIV status Male Circumcision 1. Summary of Key Activities Since 2007, SFH has offered male circumcision services through static stand-alone sites, MOH health facilities, campaign, and mobile services, which are organized and run out of hospitals and rural health centers. Alongside refurbishment of sites and establishment of integrated HIV/RH service sites, PRISM partners have trained hundreds of MOH and partner staff in male circumcision provision, outreach counseling, and follow-up care. In Year 3, SFH is providing training for 16 MOH and partner staff to assist with scaling-up male circumcision clinical training capacity, in-service training opportunities, and on-site supervision. A total of 288 people have been trained, of which 132 are MOH and SFH male circumcision providers (MC skills) and 156 are Counselors. SFH has developed a “hub” approach to the provision of male circumcision services. In this approach, male circumcision services are available from a central static site and this provincial hub also coordinates all outreach and mobile services, coordinates technical support and trainings with the MOH and partners, and provides logistical support when requested. The hub approach has proven a cost-effective means of increasing access to male circumcision services and also provides monitoring, supervising, reporting, and ultimately lessons learned and best practices for scaling up male circumcision service provision and improving coverage and reach in rural areas. SFH has taken the lead in provision of mobile services, and the majority of their male circumcision clients (roughly 75%) are reached through mobile and not static sites. The MOH has recently begun a quarterly male circumcision campaign approach to the procedure, which has resulted in triple the number of male circumcisions performed compared to non-campaign months. SFH and partners have also undertaken mass and community-based BCC campaigns to increase awareness about male circumcision. The recent involvement of community leaders in promotion 29.4% 70.6% Tested & Received Results in Past 12 Months? Source: Mini-Survey No Yes Figure 6 - HIV Counseling & Testing 32 of male circumcision has helped encourage men in their communities to seek male circumcision services, particularly where circumcision is not traditionally practiced or culturally accepted. Despite huge successes in the provision of technical support, service delivery and scale-up of male circumcision services PRISM partners have provided to the MOH, Ministry facilities and all male circumcision partners are under enormous pressure to meet ambitious annual targets. This sometimes comes at a cost with poor follow-up to training, low levels of supervision, stock out of supplies, or demand-creation activities that service providers are unable to meet. 2. Key Findings and Analysis Awareness and Demand for Male Circumcision Male circumcision awareness and demand are increasing due to demand-creation efforts by PRISM as well as government and a great number of other implementing partners and civil society organizations (CSOs). While awareness that male circumcision services exist is high and as these are being heavily promoted through a multitude of organizations and channels, results of the household surveys show that benefits of the procedure are still not well understood. While 65% of respondents knew male circumcision reduced the risk of contracting STIs, only 34% knew it also reduced the risk of HIV transmission. Only 9% of respondents knew that male circumcision reduced the risk of cervical cancer. Figure 7 below illustrates these findings. Involvement of community and traditional leaders proved useful in the last round of male circumcision campaigns, and PRISM will seek increased involvement from this influential group for future campaigns. Partner support has also proven important: women who are involved in promotion of male circumcision and support their partners in seeking these services have helped to increase demand. Figure 7 - Knowledge of benefits of male circumcision 0 50 100 65 9 34 15 12 11 % o f r e s p o n d e n t s Benefit of MC Awareness of Benefits of Male Circumcision Source: Mini-Survey 33 Misunderstanding, fears, and misconceptions continue to hamper demand for male circumcision services:  Fear of pain and healing time;  Fear of long term repercussions, including infertility, sterility, loss of feeling, erectile dysfunction, failure to perform sexually;  Fear of being perceived to be HIV-positive if male circumcision services are accessed at comprehensive site with ART and other support and care services;  Traditional and cultural norms and barriers in Northern Province and some regions of Eastern Province;  Fear of embarrassment if performed by a female service provider; and  Fear due to stories of complications during or after male circumcision. (The few MC procedures in which patients have had poor outcomes or contracted an illness unrelated but perceived to be linked to male circumcision have led to increased fear and misperceptions in some communities.) Availability and Access to Male Circumcision Male circumcision services are done through MOH hospitals and health centers, through PRISM partner static and mobile sites, and through other donor-funded implementing partners. Male circumcision services are more available in urban than in rural areas, though the initiation of mobile male circumcision provisions is seen to be increasing rural access. PRISM offers male circumcision static services and collaborates with MOH on mobile services. However, some community members indicated that the mobile unit does not always turn up when they say they are coming to conduct male circumcision, indicating that even when demand is there the service may not be readily available. PRISM has trained 132 SFH and MOH providers to conduct male circumcision in health facilities, and SFH staff also assisted with procedures at MOH sites when there is an overflow of clients. This has contributed to availability of male circumcision. Some health facilities only provide the procedure on certain days of the week. Static male circumcision sites under PRISM provide the services daily, but lack of kits reportedly hampered the constant availability of services. Loss of trained providers was also mentioned as a constraint to access and availability. Use of Male Circumcision In all provinces visited male circumcision demand and utilization rates are increasing for men of all ages, but particularly amongst youth and children. Parents want their male children to get circumcised, especially during the school holidays and that makes demand for male circumcision very high during those periods. According to respondents, male circumcision has been well accepted among the young more than the older age groups with older men saying they felt it was too late to get circumcised, and that they would have little benefit from it and/or take too long to heal. Male circumcision acceptance in communities where it is not a traditional practice is a barrier to uptake of the services, for example the Ngoni people of Eastern province. Routine HIV testing before circumcision also discourages some men from male circumcision. Workers may avoid circumcision because they would be incapacitated and not be able to make a living during the healing period, especially among the self-employed, truckers, and rural farming communities. The project target was to circumcise 30,000 men, at least 75% aged 13-29 years old. By April 2013, 78,508 men had undergone the procedure indicating the target has already been surpassed. 34 4.2 EVALUATION THEME 2: EFFECTIVENESS OF INTERVENTIONS Effectiveness and best practices of PRISM social marketing techniques and activities to achieve intended results As the findings presented in the above section demonstrate, PRISM has been effective in contributing to increased awareness, access, and use of both socially marketed products as well as for health services through stand alone and MOH health facilities. A number of strategies and activities have proven highly effective and are worthy of noting here:  Mobile services and community campaigns to extend reach of and access to male circumcision and counseling and testing services to rural and remote/underserved communities;  Piloting the provision of postpartum IUD (PPIUD) insertion and using lessons learned to assist the MOH in scaling up provision of this service;  Assistance with “universal access” to counseling and testing services in market places and other public locations where high volumes of people can be reached;  Networks of CBVs targeting rural areas with little or no access to commercial markets and located too far from health facilities to access FP and other preventive health products;  Targeting hotspots with male and female condoms to improve condom use in high-risk sex;  Encouraging couples to come for counseling and testing together, resulting in increased awareness of partner status and open dialogue about safer sex;  Outreach and BCC activities targeting both men and women; for example, encouraging men to support and encourage their partners to use family planning, and targeting women with messages about the benefits of male circumcision for both men and women;  Provision of training in small business skills to CBV to improve sustainability of community based distribution and sales activities; and  Encouraging linkages between wholesale and retail; retail and CBVs; CBVs and DHOs and health facilities to facilitate access to resupply of SM products when needed (and without reliance on a project partner and vehicle for resupply). SFH and its partners should be commended for their considerable achievements and innovative activities, increasing availability and reach of services, particularly in rural areas with mobile service provision of counseling and testing of HIV and male circumcision services. While the evaluation team did not do a cost-effectiveness study of static clinic versus mobile service provision, however, donors and the MOH should compare the cost benefits of mobile vs. static services as they expand these interventions. It’s important to highlight that CBV programs were found to be vital in filling gaps where communities have little or no access to either health services or commercial markets. However, decades of experience across the globe have proven that CBV activities are not scalable, are extremely financially and human resource intensive, and rarely prove sustainable. Programs strategically designed to reduce volunteer fees and in-kind payments and which provide life skills training in addition to training in health intervention areas, are more likely to be sustained in 35 future. The interventions being implemented through CARE International have strived to do just this, and a closer analysis of best practices and lessons learned summary from these would be fruitful for future programs. Gender The evaluation team received a wealth of evidence through qualitative interviews and document reviews indicating that the PRISM program integrates an appropriate mix of activities to ensure interventions reach both women and men appropriately. Of note, findings from most of the FGDs found that the use of both male and female CBV workers helped ensure that community members could choose to speak to someone of the same sex when desired, but also that dialogue between men and women about sexual and reproductive health issues was increasingly seen to be okay and acceptable by both sexes and amongst men and women of all ages. FGDs also found that women feel more empowered to talk openly with their partners about sexual and reproductive health issues as well as to make reproductive health choices independently of their partners. Recognition of partner support in seeking services such as family planning and male circumcision was noted. Finally, a handful of key informants from SFH highlighted the fact that SFH is now female headed and a significant number of directors and program managers are also female. Reaching Target Populations PRISM activities are implemented in both rural and urban areas, though the scope, reach, coverage, and uptake of most of these interventions is significantly higher in urban than in rural areas. Nearly all of PRISM’s health service interventions have been urban focused until recently, when SFH began offering mobile services and campaign-styled events to increase access to rural and underserved populations. CBV networks are managed by PRISM partners CARE International and ZHECT. CARE targets rural and remote communities with health education information and sales of socially marketed products. ZHECT targets most at risk populations (MARPs) in high-risk zones of Lusaka with information about a wide range of HIV preventive services and products. While both volunteer networks target high-risk populations, ZHECT has struggled with product sales, as PRISM products are already widely available in the commercial sector as well as in many hotspots frequented by target MARPs, so there is less incentive to buy from a volunteer. Conversely, CARE has found that using sales commissions as the sole incentive for CBVs has been successful in two ways: one, CBVs who are not motivated or effective in this role drop out (because their sales commissions are too low to motivate them to continue) while dynamic and dedicated volunteers remain with the program, with average monthly earnings ranging from 200 Kwacha (USD 37) and as much as 460 Kwacha (USD 85). In deep rural communities where many families do not earn that much money in three months, this provides a considerable incentive to CBVs to continue their work and to undertake more dedicated and thorough demand-creation activities than might be the case for volunteers who are paid a monthly stipend. Conversely, ZHECT volunteers are more likely to expend as much time educating target populations on free services such as male circumcision and LARC, whereas CBVs who sell products have little incentive to focus on this. However, there remain a vast number of rural communities across Zambia still in need of health intervention and prevention products and services not being reach by PRISM or other implementing agency programs. As SFH plans to increase product pricing by 5% annually, it might be useful to pilot new and innovative means of connecting CBVs and community leaders 36 from rural populations with alternative and less program-intensive means of accessing health products. Feasibility studies could be undertaken to answer the following questions: Where DHOs are willing to stock and sell PRISM products, is there a means of building in a small margin for the individual in charge of managing social marketed product sales at the DHO or facility? Could the project identify rural farmers who frequently travel to local markets to sell their wares be incentivized to travel back with PRISM products to be sold in their communities? A recent PSI innovation in Mozambique has created a mobile phone voucher system where small retailers in outlying markets are recruited as focal points for product promotions and e-voucher sales and mobile campaigns sent to target communities via SMS text message. For example, a recent flood in northern Mozambique left many communities stranded without Clorin and with a cholera epidemic sweeping through their areas. With impassible roads and bridges, UNICEF could not get its trucks there. At the touch of a button, PSI sent e-vouchers to individuals in the affected communities which connected them to their nearest participating retailer. The response was immediate and effective. Replication of similar cost-effective means of increasing demand for and access to health products in rural areas of Zambia could prove successful. PRISM products and services are increasingly accessible and available down to community level. In Solwezi district for example, the DHO stated that: “We have reached/covered about 70-75% of the population. Most sites have been reached, done follow up programmes, and all populations are covered irrespective of where people are.” Still, the need to expand access and availability of PRISM interventions to more deep rural areas, and in close coordination with MOH and local partners, was noted by a great number of respondents, and across all interventions. As one key informant said: “those ten kilometers or more from an outlet (to a rural community) are harder to achieve than the ‘low lying fruit’.” While some PRISM program activities are targeted to youth, gaps exist. In particular, there is an urgent need for youth friendly family planning services, centers, and providers. A key informant in charge of LARC services at one of Lusaka’s most overburdened health centers indicated that while she does see a lot of young women (aged 15-17), many of these come to her because they already have one child and do not want to risk a second pregnancy. Alongside scaling up provision of youth-friendly family planning and reproductive health services, many young people in FGDs indicated a need for more peer educators (aged 15-19). Using young women of this age who already have children, who are currently using family planning (and specifically LARC), and who can speak from their own experience about the difficulties and challenges of teenage pregnancy could be even more effective. FGDs with youth also indicated that they need more in-depth and detailed information on sexual and reproductive health matters but felt they were often perceived to be too young for this and were too often dismissed without the information needed to make informed choices. Of note, a number of younger participants felt particularly uncomfortable with using counseling and testing services for a number of reasons: fear of being seen at the service site; fear of non-confidentiality or results being shared with their parents, peers, or teachers as they are still young; and fear of being judged as promiscuous for seeking such services. Appropriate level of effort to ensure achievement of results As the progress against indicators table in Table 1 demonstrates, the PRISM program has or is likely to achieve 21 of its total 32 indicators. As such, it is clear that PRISM has applied the correct focus and level of effort to achieve all of these, and in most cases, has gone beyond simple achievement of contract indicators alone, responding also to the needs of the country as well as numerous requests from the MOH for activities beyond the scope of the program. However, it is also clear that the project is stretched immensely thin on the ground; staffing 37 "SFH is one of the key partners to the ministry."- MOH personnel levels, particularly at provincial and district level are low given the considerable number of interventions being implemented and the increasing demands on the project to assist with mobile and campaign activities in particular. Both for SFH and its PRISM partners (notably CARE and ZHECT) there is often only one provincial coordinator, and sometimes this individual covers more than one province. Transport is an issue that came up in every province and district surveyed; transport for commodities, training, outreach and mobile services, monitoring, and supervision were all examples of where the project simply did not have the resources at hand to comfortably achieve what it set out to do. As PRISM is a contract, SFH noted that it has struggled to keep up with inflation and other factors limiting its ability to retain and support good qualified staff. For example, SFH staff were only recently awarded increases in salaries, and these increases still lag significantly behind salaries offered from other implementing agencies and private sector companies. Constrained resources have hindered SFH and partners from doing all that they might were the circumstances different and they in a position to be more flexible to respond to the needs and objectives of both the project and the MOH. Accountability and adherence to GRZ strategic objectives and protocols Key informants interviewed from the MOH at national, provincial, and district levels were nearly unanimous in their praise of PRISM and its efforts, achievements, and responsiveness to MOH strategies, activities, and requests. First, they confirmed that all project activities are not only within the strategic plans of the government, but that the assistance from PRISM partners in implementation of these is often central to their successes. Society for Family Health and relevant partners sit on the technical working groups of all of the interventions in which it is involved, including male circumcision, counseling and testing, sexually transmitted, and ART. Central level government MOH directors all provided numerous examples of how PRISM has been responsive to and supportive of their departments and objectives. While there are too many to list here, some examples include assistance in development of the voluntary male circumcision training package for counseling, and piloting and best practices developed for postpartum IUD insertion. Of note, at national level, key informants were very clear about the fact that SFH and partners were supportive, but that ownership of all programmatic issues remains with the Ministry, which is the key to sustainability of these interventions and supportive activities. 4.3 EVALUATION THEME 3: SUSTAINABILITY Sustainability of PRISM service delivery support to MOH Sustainability of PRISM supported MOH services—including male circumcision, counseling and testing, and family planning (notably LARC)—were all deemed to be likely at varying levels of the MOH and amongst key informants interviewed. Sustainability of services is most hindered by two factors: MOH financing available to maintain and take to scale these interventions, especially given the enormous pressure they are under to do so (particularly in the case of male circumcision). This is exacerbated due to the paucity of trained, skilled health care providers. Without the human resources to adequately provide the services intended, PRISM, USAID, the MOH, and the country at large will ultimately suffer as a result. Scaling up a program such as male circumcision (and this lesson has been learned time and again with VCT, PMTCT, and ARVs to name but a few) requires substantial scale up of manpower, both those skilled to provide quality services and those skilled to train, supervise and monitor the roll out of a program as ambitious as this one. 38 While PRISM should be commended for its considerable role in provision of male circumcision services, the donor community, partners involved, and MOH would be wise to reconsider slowing down initially and building the systems and capacity required to ultimately reach the same or better results in a few years’ time. However, in general, PRISM was praised for its focus and assistance on building systems and capacity – from strategies, to protocols, to follow up and support – for all the services it is engaged in with the MOH. In the short term, the placement of trained staff to assist in service delivery is seen as a good thing. Over the longer term, for counseling and testing and family planning at least, the MOH will require continued assistance until it can address the human resource gaps within its own network of facilities. Sustainability of commercial and community-based sales of socially marketed products Sustainability of PRISM’s portfolio of socially marketed products has been enhanced in a number of ways. SFH has implemented intensive retail trade support activities such as uplifting to increase retail availability through smaller markets and outlying retail kiosks and to improve sustained linkages between them and their nearest urban wholesalers. This activity seems to have paid off; as SFH scaled back uplifting activities to allow retailers to restock unassisted, they have not noted any drop in commercial sales. However, with a mapping survey it is hard to know the extent of product availability, penetration, and reach in peri-urban and rural markets nationally (SFH plans to undertake a MAP survey in 2014). Linking DHOs and health facilities to the retail trade have also proven beneficial: in some instances, the DHO or facilities have bought products and resold these to CBVs, assisting in linking retail to this important cadre of volunteers. Finally, SFH has begun to increase its pricing (primarily for condoms and Clorin) in response to a willingness-to-pay survey conducted by PRISM partner Population Council in 2012. Research is ongoing but SFH plans to continue to increase prices by at least 5% annually to reduce donor subsidies on product costs and increase willingness to pay over time. Variations amongst products exist and are worthy of noting here. The free availability of condoms and family planning (notably oral contraception), coupled with the inability of selling oral contraceptives over the counter and without prescription (due to MOH policy), hinders the program’s ability to increase prices. Targeted sales of these products to rural poor communities of Zambia mean that increases in price, particularly to one that is full cost-recovery, never mind one that covers the costs of transport and logistics, staff support, BCC and research activities, and other direct costs, would be prohibitive to these target populations and counter-intuitive to the objectives of USAID and the program. However, primary data collection noted a number of areas in which sustained and sustainable improvements in demand, access, and use have been achieved. Amongst them training of CBVs in small business skills has improved their ability to manage stocks, sales, and revenue of PRISM products; income generation as the sole incentive to CBVs assists in motivating volunteers without further dependence on donor funds; linkages with the MOH and other local community-based organizations and groups (such as church groups, traditional leaders, and SMAGs) increases the likelihood of sustained demand over time; and ownership and support for PRISM socially marketing activities by the MOH which has also proven important. A comprehensive analysis of sustainability factors related to Clorin in the commercial sector and the ANC ITN program in the MOH can be found in Annex F. 39 5. CONCLUSIONS The PRISM program has achieved or is likely to achieve 21 of its 32 total indicators. Three are unlikely to be achieved, while for four it is too early to evaluate, and the other four are not applicable. Some indicators require revision or reconsideration notably: an FP indicator which measures total CYPs would be a better indicator of success than the current indicator for sales of oral contraceptives. Similarly, PRISM indicators for handover of the ANC program to the MOH and Clorin to the commercial sector should be reconsidered. Awareness of the portfolio of PRISM products and services is high (between 83% and 99%). More work is needed to target myths and misconceptions surrounding family planning and male circumcision, as well as to target youth with needed information about sexual and reproductive health. Access and availability of products and services varies: in rural areas, access is limited except where CBV networks are in place and where mobile provision of services has been made available. Access to male circumcision and counseling and testing services have increased considerably with the provision of mobile and campaign-style services, and this intervention warrants a review, so that lessons learned and best practices can be used for provision of LARC as well as by the MOH and partners to scale up such services. Use of PRISM products and services is on the increase across the majority of interventions, notably, male circumcisions, counseling and testing for HIV, ITNs, and Clorin. Female condoms are the least used of all of PRISM’s interventions and uptake of LARC lags behind other forms of modern contraception. A number of best practices and effective strategies and activities have been implemented through PRISM including the following: mobile services and community campaigns to extend the reach of and access to male circumcision and counseling and testing services to rural and remote/underserved communities; piloting the provision of postpartum IUD insertion and using lessons learned to assist the MOH in scaling up provision of this service; assistance with “universal access” to counseling and testing services in market places and other public locations where high volumes of people can be reached; networks of CBVs targeting rural areas with little or no access to commercial markets and located too far from health facilities to access FP and other preventive health products; targeting hotspots with male and female condoms to improve condom use in high-risk sex; encouraging couples to come for counseling and testing together, resulting in increased awareness of partner status and open dialogue about safer sex; outreach and BCC activities targeting both men and women—for example, encouraging men to support and encourage their partners to use family planning and targeting women with messages about the benefits of male circumcision for both men and women; provision of training in small business skills to CBV to improve sustainability of community-based distribution and sales activities; and encouraging linkages between wholesale and retail, retail and CBVs, CBVs and DHOs and health facilities to facilitate access to resupply of socially marketed products when needed (and without reliance on a project partner and vehicle for resupply). PRISM program activities appear well targeted to both men and women; however more work is needed to insure coverage of rural populations and youth. The likelihood of sustainability of PRISM assistance for MOH services is by and large deemed to be high. MOH systems strengthening, training and other technical support is deemed sustainable both inside the MOH and out. Key constraints include MOH financing and human resources. Sustainability of socially marketed products is varied, but a number of PRISM activities were identified as increasing the likelihood of sustainability over the longer term. These include strengthening linkages within the commercial sector and between retail, the MOH, and CBVs; 40 sustained demand-creation activities and innovative means of incentivizing CBVs; and incremental increases in pricing consistent with willingness-to-pay surveys. Of note, handover of the ANC ITN program to the MOH is not likely to be achieved, nor is graduation of Clorin to a full-priced commercial product. Critically, the evaluation team does not feel either of these activities is recommended. The MOH and central medical stores are already overburdened with logistics and management of MOH supplies. Handover of Clorin is likely to result in losses in reach and use to target populations; considerable market and consumer research is required before any advancement with this activity is recommended. 6. RECOMMENDATIONS 6.1 SAFE WATER AND CHILD HEALTH Clorin: 1. Conduct the planned market research study for Clorin: In addition to the planned market research for Clorin, SFH and USAID need to carefully examine many factors related to cost-effectiveness, health impact and reach before deciding whether handover of this product to a commercial entity is prudent. The market survey planned should be coupled with a MAP survey and consumer profiling survey to assess a number of critical issues in order to accurately assess the effect of graduating Clorin to the commercial sector and at an unsubsidized price on reach and use by rural poor and other key target groups. Questions to address include the following:  Price and Cost-effectiveness: At what price would Clorin sell if handed over to Pharmanova (or another private company)? What would a fully commercial product cost to the consumer, and how does this compare with the subsidy currently applied on the product alone? How much investment would still be required from USAID to reach those unable to pay this? Would DHOs still purchase Clorin and how would this effect their response to emergencies and disease outbreaks?  Consumer profiling: Who is currently buying Clorin and what is their socio-economic status, income level, ability to pay more? Where do they buy Clorin and would they continue to buy at a full commercial price? Is Clorin reaching intended beneficiaries at the current subsidized price, and what percentage of Clorin sales and subsidy is lost, or currently goes to consumers able to pay more?  Retail trade and mapping survey: Where is Clorin available commercially and would small retails and outlying markets still carry this product (e.g., invest in stocks) at the estimated full price? 2. Increase BCC activities, and use TV spots developed, to promote Clorin. SFH has a number of TV spots developed, which were pretested with consumers, and the response was positive. These should be aired prior to this year’s rainy season. Increased BCC activities are also likely to help maintain and increase demand as SFH continues to increase product price towards one that is unsubsidized. 3. Increase use of distributors to reduce transport costs. SFH should continue to expand its use of distributors to improve efficiencies and reduce subsidies needed on product sales and distribution. SFH has begun to work with distributors who come and collect Clorin from their warehouse and move product to wholesale warehouses nationwide. 41 4. Consider removing subsidy from sales to donors and other partners. SFH currently sells Clorin to UNICEF and other partners at a subsidized price. Unless USAID is comfortable subsidizing product sales to other donors, subsidies should be reserved for program activities targeting high-risk groups, and other donors should be required to pay the full cost of the product and subsidize activities from their own budgets. 5. Improve linkages between rural communities and retail markets where CBV networks do not exist. SFH should explore opportunities to improve availability and access to Clorin amongst rural communities outside CBV networks. Promotion of rural and outlying retailers, use of e-vouchers and mobile systems (such as the Mozambique “Movercado” program, sales through DHOs and health facilities, etc., should be explored and piloted to see how availability and access to Clorin can be improved and sustained in the absence of CBVs. 6.2 MALARIA PREVENTION ITNs 1. Revise Indicator 1.4 “Transfer of ANC ITN program to MOH” to include Systems strengthening indicators at all levels of MOH to better implement, monitor, track, and report on ANC ITN program. While handover of all aspects of the ANC ITN program is neither desirable nor feasible at the present time, considerably more could be done to strengthen MOH capacity and systems for this program. SFH should be tasked with systems strengthening activities similar to those it has successfully achieved in other interventions such as male circumcision and counseling and testing, starting from DHO level down, where systems for logistics and supplies of nets to health facilities, as well as distribution, tracking, monitoring, and reporting to end beneficiaries are lacking or not being implemented. 2. Develop strategic overview and communications plan for MOH, DHOs and health facilities to improve awareness and understanding of ANC versus mass distribution nets. Alongside systems development for tracking of nets, the NMCC needs a clear document summarizing the objectives of those nets distributed every two to three years in mass distribution campaigns aimed to increase general household coverage and use and those distributed in ANCs to target high-risk beneficiaries only. This document should improve understanding of the entire net program and also help ease issues of implementation at all levels. 3. Undertake a “Tracking Results Continuously” (TRaC) survey to understand net ownership, coverage and use at household level. 6.3 FAMILY PLANNING & REPRODUCTIVE HEALTH Family Planning: 1. Revise SFH indicator “Market 25.6 million cycles of oral contraceptives” to one that measures CYPs across all family planning products. As the landscape for family planning has shifted significantly and all health facilities now provide family planning, including oral contraceptives free of charge, it is unrealistic to expect PRISM partners to succeed with its original oral contraceptives target. Similarly, the program currently lacks any indicator for 42 LARC; a CYP indicator would be a better measure of health outputs and impact across all family planning interventions. 2. Develop a strategy and activities to improve access to youth friendly family planning services in consultation with MOH in line with national policy guidelines. Alongside the increase in scale, scope, and reach of family planning and LARC services nationwide, there is an urgent need to reach younger women, aged 15-19, and critically, under 15, with family planning information and accessible services. Strategies to develop peer educators who can provide information on family planning as well as products (condoms, pills) are needed. Training to providers to sensitize them on the needs of youth also needs improvement. Lessons learned from previous MOH experience with youth friendly corners, Ministry of Education activities in schools, and global best practices for reaching youth should be built upon. Advocacy for youth sex education programs at school – and targeting girls and boys starting ages ten and up – is also urgently needed. 3. Document lessons learned by conducting operational research: Evidences on effectiveness and cost benefits should be generated on mobile and outreach services providing male circumcision, VCT, family planning, particularly LARC, where feasible and appropriate to guide development of future implementation strategies. A careful analysis of gaps in coverage and mapping of where CBV activities are most needed is essential for future efforts. Strengthening linkages between CBVs and MOH health workers, again to insure sustainability and supportive community structures are in place in lieu of intensive supervision and support requirements from implementing agencies, is also recommended. 4. Pilot the training and franchising of private sector and/or retired midwives in rural areas to increase access to quality family planning provision. To increase access to quality reproductive health and family planning services, SFH should pilot a program to provide training to retired midwives and those still working outside the public health system. These midwives could then receive a seal or brand of some kind that is sanctioned by the MOH and allows women in their communities to know where they can receive reproductive health/family planning services (including safe delivery kits with Misoprostol included) outside of the MOH, both at hours when the MOH health facilities are not open and in areas without access to MOH facilities. 5. Increase systems strengthening within the MOH for family planning. PRISM and other implementing agencies should work with the MOH to improve targeting of family planning support and interventions, particularly through high volume facilities. Lessons learned from male circumcision and counseling and testing should be applied in targeting areas where demand is greatest, as well as to strategically identify and target areas without access to family planning through mobile service delivery. While a significant amount of training in family planning and provision of LARC has taken place, there is a lack of supervision and support to newly trained midwives and providers to insure they can confidently and adequately provide quality LARC services. Where needed refresher courses or in-service supervision and support would help increase quality of care provided at rural health centers. 6. Increase BCC activities to improve uptake of LARC and dispel myths and misconceptions. SFH plans to undertake a TRaC survey in 2014, which will help the MOH, PRISM, and other partners working in FP to better understand who current FP acceptors are, and what the primary barriers to uptake are. This information is vital and 43 should be used to ramp up BCC activities at all levels, with special emphasis on younger women. 7. Advocate for communications and education programs targeting young women with FP services. Policymakers and national level decision makers need to be better educated and supportive of the health and economic benefits of long-acting contraception and supportive policies for young women. Misoprostol: 1. Secure supply chain for commodities, short- and long-term. SFH and USAID need urgently to explore other options for procurement of Misoprostol. DfID expressed interest to the evaluation team in assisting in the short-term, provided a longer term, sustainable plan was in place. SFH should develop a proposal for short-term assistance by DfID (and possibly others, such as the UNFPA) to pool funds for procurement of Misoprostol alongside a longer-term strategy with the MOH for consistent supply of this life-saving drug. As Misoprostol is already a part of Zambia’s essential drugs list, funding is the only issue delaying SFH activities in reducing postpartum hemorrhage. 2. Develop an improved, low-cost Safe Delivery Kit with Misoprostol to pilot through SFH franchise of midwives. This kit would include items such as soap, gloves, a razor, a sterile cloth, along with two notable, very important additions: antiseptic (chlorhexidine) to clean the umbilical cord to avoid newborn infections and Misoprostol, a medication taken after childbirth to prevent severe life-threatening bleeding. Female Condom: 1. Undertake a consumer profiling survey and possibly a cost-effectiveness study to determine how best to target female condoms in future. At present, there is little information on who uses or might use female condoms in Zambia. The program lacks the information required to target the correct messages to the right audience. For sex workers and for young women in particular, this information is essential. A cost-effectiveness study comparing investments and returns on female condoms versus other HIV prevention interventions might also help USAID, the MOH, SFH and partners to know if investments in the female condom make sense in the longer term, or should be reduced or removed in preference to other interventions with proven impact. VCT: 1. Review indicator definition and provide clarification of indicator: “Train 2,300 individuals in HIV counseling and testing.” Since the definition of the indicator is not clear, the evaluation team assumes that the indicator is intended to mean volunteers and outreach workers for CT counseling and testing, not providers of counseling and testing, but as it now reads, it appears PRISM should train thousands of providers, which is disproportionately high both compared to the providers within its networks and the total number of providers country-wide. 2. Scale-up comprehensive service delivery for counseling and testing and CD4 counts. SFH received a donation of mobile CD4 count machines but has since stocked out of its initial purchase of reagents. In line with PRISM contract objectives to increase access to comprehensive services, USAID and SFH should investigate alternative means of funding (or reallocation of program funds) for purchase of reagents. Use of CD4 machines and 44 reagents should be targeted to rural and remote areas where CD4 services are not readily available. 3. Strengthen linkages to care and treatment for HIV. SFH and partners need to work closely with the MOH and other implementing partners to strengthen linkages to follow-on care services for those who test HIV-positive. SFH, using non-PRISM funds, is currently piloting activities with PIMA and lessons learned from this should be built upon and rolled out to improve linkages in other areas, for example, TB testing and treatment. Male Circumcision: 1. Increase funds and focus on training of trainers and supervisors for male circumcision provision. As the national male circumcision program has targets and objectives for coverage and uptake over the next few years, USAID and partners should focus more resources on ramping up training of trainers, doctors, follow-up care providers and supervisors to assist in insuring that the MOH and partners can provide male circumcision services that meet their targets without sacrificing or compromising on quality. 2. Centralize procurement of kits through JSI/Deliver. PRISM and partners have been plagued with stock out problems related to male circumcision kits. JSI/Deliver and SFH both felt strongly that centralized procurement of these through the Deliver program would streamline this process and reduce or eliminate stock outs in future. 3. Conduct Operational research and best practices study on mobile service provision. As the lead partner in assisting the MOH with mobile and campaign-styled male circumcision service provision, many lessons learned and best practices can and should be documented and shared amongst partners. These lessons can also be applied to mobile provision of comprehensive reproductive health and HIV/AIDS services. 4. Increase coordination efforts with partners and communities for mobile and campaign service delivery. With pressure to reach countrywide targets, the evaluation team heard reports of community mobilization activities for mobile male circumcision services that never arrived. This can lead to decreased demand from communities who no longer believe services will be available when promised. More emphasis on careful coordination of efforts is needed. 5. Include Central province in PRISM program male circumcision activities. At present, Central province is lagging behind other provinces in its ability to provide male circumcision services. Central Province is currently not included in PRISM and as such there is no Memorandum of Understanding with the provincial health department for male circumcision service delivery. USAID and SFH should determine if funds are available to expand service delivery to this province and, if so, coordinate with the MOH to get activities underway immediately. 6. Introduce new non-surgical male circumcision kits. Pending outcome of global efforts to introduce new non-surgical male circumcision methods and kits, these should be introduced in Zambia immediately. These kits would increase the numbers and types of service providers able to perform male circumcision and also reduce quality issues and risks associated with minor surgery, such as infection. 45 7. Increase demand-creation activities targeting older men and myths and misconceptions. Uptake of male circumcision is hampered by lack of knowledge and understanding in a number of areas already addressed in this report. Increased depth of information covering the procedure, short- and long-term effects, full range of benefits, and targeted to both men and women (of all ages), using traditional leaders as a key font of information, is recommended. 46 ANNEX A: SCOPE OF WORK FROM THE TASK ORDER DESCRIPTION / SPECIFICATIONS / STATEMENT OF WORK IDENTIFICATION DATA 1. Project Title: Partnership for Integrated Social Marketing (PRISM) Program 2. Project Number: Contract No. GHH-I-00-07-00062 3. Project Dates: August 1, 2009 – September 30, 2014 4. Project Ceiling: $65,454,770 (Base value) or /$73,170,778 (with two optional activities which are yet to be exercised (see below)) 5. Obligated Amount: $43,406,861 6. Implementing Organization: Population Services International (PSI) 7. Target population: a. Male and females of reproductive age group (15-49) for Family Planning (FP); b. Most at risk groups for HIV prevention interventions (sexually-active males and females, discordant couples, intravenous drug users, fishermen, commercial sex workers, and men having sex with men (MSM)); c. HIV-negative males for male circumcision; and d. Children under the age of five years and people living with HIV/AIDS for prevention of diarrheal diseases. 8. Geographic Coverage: National 9. Contracting Officer’s Representative (COR): Dr. George Sinyangwe I. BACKGROUND A. Main Task / aim of the solicitation The USAID/Zambia Population, Health, and Nutrition (PHN) office wishes to contract services of an independent evaluation team to perform a mid-term evaluation of the Partnership for Integrated Social Marketing (PRISM) Program. The evaluation will focus on performance accountability and generating lessons to improve project performance. 47 PRISM has four main objectives: 1. Increase the supply and diversity of health products and services to distribute and deliver through the private sector, in conjunction with the public sector, for disease prevention and control as well as integrated health service delivery; 2. Increase the awareness of and demand for health products and services to emphasize prevention and to build an informed, sustainable consumer base; 3. Establish and develop the ability of a commercial/private sector entity to produce and market at least one currently socially marketed health product or service in a sustainable, self-sufficient manner; and 4. Integrate service delivery and other activities, emphasizing prevention, at national, provincial, district, facility, and community levels through joint planning with the GRZ, other USG, and non-USG partners. Additionally, the Program has two optional tasks, subject to further consultation with the MOH and approval by USAID/Zambia: 1. Increase the awareness of, demand for, and use of Misoprostol through the private sector, in conjunction with the public sector, primarily in the prevention of post￾partum hemorrhage; and 2. Increase the awareness of, demand for, and use of Zinc complementing oral rehydration therapy through the private sector, in conjunction with the public sector, in the management of acute diarrheal diseases among children. Specifically, PRISM distributes through social marketing male and female condoms for HIV prevention, family planning methods (oral and injectable contraceptives and implants) for family planning, insecticide treated bed nets to prevent malaria, and a point-of-use home water treatment solution to disinfect drinking water. PRISM also provides stand-alone testing and counseling for HIV and male circumcision services to prevent HIV infection among HIV-negative males. The aforementioned PRISM activities are positioned to contribute to USAID/Zambia’s Development Objective (DO) of improving the health status of Zambians, and through this goal, contributing to Zambia’s achievement of health-related Millennium Development Goals (MDGs). To date, $43,406,861 has been obligated into the award. PRISM has a presence in all the ten provinces of Zambia. The objectives of this mid-term performance evaluation are as follows: a) to assess progress towards intended project results listed above; b) to recommend modifications to project activities or priorities, as necessary, to address implementation issues, apply lessons learned, or capitalize on new opportunities; and c) to identify relevant issues that require discussion and resolution at a level higher than the PRISM program. 48 B. Key Challenges to achievement of health-related Millennium Development Goals in Zambia Zambia is committed to achieving all the health-related MDGs by the year 2015. However, challenges with community health services delivery, and strategic health sector policy management and implementation, constrain Zambia’s ability to achieve its national health sector goals and the MDGs. Community and household: Use of essential health services and products such as testing and counseling for HIV, family planning, male circumcision, and male and female condoms remains low, especially in rural areas due to low demand (poor health seeking behaviors) and existence of barriers such as impassable roads, limited number of health facilities in Zambia, high population densities in peri-urban areas, gender inequities, traditional healer influence, negative cultural practices, and low household incomes. Table 1: Low uptake of essential services Activity Desired National Target Prevalence Male Circumcision 80% aged 13-49 by 2015 16.5% Counseling and testing for HIV 100% of all adults (13-49) 22% Contraceptive Prevalence Rate 42% 33% Health services delivery: The Ministry of Health (MOH) is supposed to have 30, 000 health care workers on its register. However, to date the MOH only has 15, 000. As a result, Zambia experiences weak technical guidance, program management, and supervision in several health areas. Further, lack of equipment, infrastructure, and medical supplies, and a weak supply chain system for drugs and medical supplies, compound the situation. Health sector policy and strategic management: The burden of health sector financing is largely shouldered by the MOH because the policy environment has not provided adequate incentives to attract sufficient private sector investment in health. For example, employers are not compelled by legislation to contribute to medical insurance schemes of their employees. Public Policies cutting across sectors: Recruitment and placement of government workers is centralized (done in Lusaka by the Public Service Commission). This has led to inequitable distribution of health care workers because the central level does not always understand the unique needs of provinces and districts. Burden of Disease: The high burden of disease is a huge challenge to Zambia’s achievement of the health-related MDGs: 1. HIV/AIDS: Zambia has a generalized HIV epidemic estimated at 14.3% among adults 49 aged 15 – 49 (ZDHS, 2007). Adult HIV prevalence remains higher among women (16.1%) than men (12.3%) and higher in urban areas (19.7%) than rural areas (10.3%). Although HIV incidence may have begun to stabilize, the absolute number of people living with HIV/AIDS and needing care and treatment will continue to grow because of new HIV infections and people living longer because of treatment; 2. Malaria: Despite a lot of progress being made to confront the disease, malaria is still regarded as the number one cause of morbidity and mortality in Zambia. In 2008, 3.2 million cases of malaria (confirmed and unconfirmed) were reported countrywide with 3,871 deaths. The annual malaria incidence was estimated at 252 cases per population of 1,000 people in 2008, a drop from 358 cases per population of 1,000 people in 2007. Malaria accounts for up to 40% of all infant mortality and 20% of all maternal mortality in Zambia and the disease poses a severe social and economic burden on communities living in endemic areas (MOH, 2008); 3. Family Planning/Reproductive Health (FP/RH): Zambia has the world’s third highest fertility rate. The 2007 ZDHS found that a Zambian woman would give birth to 6.2 children (4.3 in urban areas and 7.5 in rural areas) by the end of her childbearing years. This total fertility rate (TFR) represents an increase from 5.9 in 2002, and is attributed to the rise in fertility rate in rural areas (6.9 in 2002) with no change in urban areas. Between 2002 and 2007, approximately 40% of all births still occurred less than three years after the preceding birth, and Zambian women bear their first child around age 19. These fertility trends have generated a youth bulge in Zambia, where individuals under the age of 15 constitute nearly half of the population. Teenage pregnancy rates are also high with more than 20% of all women age 15 – 19 already having had their first child. Moreover, the proportion of these teenage women increased from 6.6% in 2002 to 8.3% in 2007; and 4. Maternal and Child Health (MCH): Between 2002 and 2007, the maternal mortality ratio (MMR) declined from 729 to 591 per 100,000 live births, but it remains well above the MDG target (162). Although over 90% of Zambian women receive some antenatal care (ANC) from a skilled provider, the majority of births, especially in rural areas, still do not occur in health facilities or with assistance from a medically trained provider. To reduce the MMR towards the MDG target, Zambia must improve the quality of ANC, including HIV testing, counseling, and prevention of mother-to-child-transmission (PMTCT) interventions. C. Overview of the Health Sector Response by the Government of the Republic of Zambia (GRZ) The National Health Strategic Plan (NHSP) 2011-2015 identifies child health, integrated reproductive health, HIV/AIDS and sexually transmitted infections (STIs), and malaria as four of the major public health priorities. For these priorities, the MOH aims to do the following: Under-five mortality: Reduce the under-five mortality rate from 168 per 1,000 live births to 134 by 2011 and 63 by 2015. 50 Integrated Reproductive Health and Family planning (FP): Increase access to integrated reproductive health and family planning services that reduce the maternal mortality ratio from 729 per 100,000 live births to 547 by 2011 and 162 by 2015. HIV/AIDS and STIs: Halt and begin to reduce the spread of HIV/AIDS and STIs by increasing access to quality HIV/AIDS and STI interventions. Malaria: Halt and reduce the incidence of malaria by 75% and mortality due to malaria in children under five by 20%. In conjunction with the development of the NHSP, the MOH has restructured the organization of the health sector to support the decentralization of planning and service delivery (Table 2), including authority delegated for key management tasks, and approximately 60% of resources sent to the district level. Table 2: Organization of the health sector in Zambia D. Overview of the Health Sector Response by the United States Government (USG) in Zambia USAID/Zambia’s health sector response aligns with the NHSP and National AIDS Strategic Framework (NASF) through a strategic approach focused on three intermediate results (IRs): Health Service Delivery improved, Health Systems and Accountability improved, and Community Health Practices improved. Current USAID/Zambia-supported health programs include the following: Zambia Integrated Systems Strengthening Project (ZISSP) ZISSP provides technical support to the MOH targeted at strengthening the health system. Illustrative activities implemented by ZISSP include provider training in FP, MCH, HIV, and malaria, retention of health care workers, development of guidelines, and strengthening management skills within the MOH. Level Entity Major role Central Ministry of Health (MOH) Develops national policies, protocols, and guidelines Provincial Provincial Health Offices (PHOs) Oversees public health facilities and services District District Health Management Teams (DHMTs) and Hospital Management Teams (HMTs) Implements health services conforming to national standards Community Neighborhood Health Committees and Health Center Committees Mobilizes communities to identify health needs and use health services 51 Zambia Prevention, Care, and Treatment Partnership (ZPCT) II ZPCT II works with the MOH to strengthen and expand HIV/AIDS service delivery in six of Zambia’s nine provinces (Central, Copperbelt, Luapula, Muchinga, Northern, and North Western). Key intervention areas include HIV counseling and testing (CT), prevention of mother- to-child-transmission (PMTCT), as well as antiretroviral (ARV) treatment and clinical care for HIV/AIDS. Communication Support for Health (CSH) CSH conducts mass media campaigns and develops health promotion materials on family planning/reproductive health, HIV/AIDS, malaria, and maternal, neonatal, and child health at the national level. JSI/DELIVER JSI/DELIVER works with the MOH and Medical Stores Limited (MSL), a parastatal company based in Lusaka, to operate an integrated supply chain management system that includes quantifying, procuring, and distributing health-related commodities, such as test kits, laboratory supplies, artemisinin combination therapies, ARV drugs, and essential medicines. Partnership for Integrated Social Marketing (PRISM) Program PRISM socially markets health services and products in HIV prevention, family planning, malaria, and maternal and child health. E. Detailed description of the PRISM (the activity to be evaluated) PRISM is a $73 million contract between USAID/Zambia and the Society for Family Health (SFH) Zambia and its parent organization, Population Services International (PSI) from August 2009 to September 2014. PRISM applies social marketing approaches and techniques to address child health, integrated reproductive health, HIV/AIDS and STIs, and malaria, as prioritized by the MOH and USAID/Zambia. Currently, the PRISM activity addresses the following program areas: Child Health: PRISM encourages families to use a home water chlorination solution to prevent diarrhea, especially among children. PRISM has been supporting the marketing of Clorin, a branded solution produced by Pharmanova, a Zambian pharmaceutical manufacturing company. PRISM supplies Clorin to disinfect 2 billion liters of water every year. Integrated Reproductive Health: PRISM encourages women and couples to use modern contraceptive methods to space births, prevent unplanned pregnancies, and determine family sizes. In particular, PRISM has been supporting the marketing of SafePlan oral contraceptives, Intra-uterine contraceptive devices (IUDs), and implants. USAID/Zambia procures these contraceptives directly through the Central Contraceptive Procurement Project (CCP). HIV/AIDS and STIs: PRISM encourages sexually active individuals to practice protected sex to prevent the transmission of STIs, including HIV. PRISM has been supporting the marketing of Maximum Classic and Scented male condoms and Care female condoms as part of a balanced approach that promotes abstinence and mutual fidelity in addition to correct and consistent condom use. PRISM has established 2,060 condom outlets and distributes up to 1.3 million pieces of male condoms and 30,000 female condoms per month throughout Zambia. In 2011, PRISM distributed approximately 20 million male condoms. PRISM also encourages sexually active individuals to know where to seek voluntary counseling and testing (CT) services, to learn about their HIV status, and to obtain follow-up care and support. PRISM supports 18 52 New Start centers, consisting of nine fixed sites (two in Lusaka and one in each of the following towns: Chipata, Kitwe, Livingstone, Mansa, Ndola, Mongu, and Solwezi), and nine mobile sites. In addition to CT services, New Start has introduced male circumcision (MC) services. Malaria: PRISM encourages pregnant women and children under five to sleep under insecticide-treated mosquito nets (ITNs) to prevent malaria. As a key implementing partner supported by the U.S. President’s Malaria Initiative and Zambia’s Malaria in Pregnancy Initiative, PRISM has been supporting the marketing of Mama Safenite ITNs in public sector antenatal clinics (ANCs) and child health facilities. PRISM’s Change Theory PRISM has positioned itself to contribute to USAID/Zambia’s DO of improving the health status of Zambians by the year 2015. The intermediate results for this development goal are as follows: 1. Incidence of diarrhea disease reduced: PRISM assumes that the increase in the use of a point-of-use home water treatment solution (Clorin) at household level will reduce the incidence of diarrhea in Zambia. PRISM assumes that the increase in the supply of Clorin and production and dissemination of promotional materials emphasizing the benefits of Clorin will lead to increased use of Clorin. PRISM assumes that the transfer of Clorin from the PRISM to a commercial entity will make Clorin more available. 2. Prevalence of HIV reduced: PRISM assumes that increases in the prevalence of voluntary medical male circumcision, number of Zambians receiving counseling and testing services for HIV, and consistent and correct use of condoms will result in reduced HIV incidence. PRISM assumes that increasing the number of distribution sites for condoms and the actual number of condoms distributed coupled with campaigns to increase the correct and consistent condom use will result in increased use of condoms. PRISM assumes that by increasing service outlets for counseling and testing for HIV coupled with messages encouraging counseling and testing for HIV will increase the number of Zambians testing for HIV. PRISM assumes that increasing service outlets providing voluntary medical male circumcision (VMMC) services coupled with campaigns to create demand will increase the prevalence of VMMC in Zambia. 3. Contraceptive prevalence rate (CPR) increased: PRISM assumes that distribution of a mix of methods, provider training, and communications for behavior change will lead to an increased CPR. 4. Incidence of Malaria reduced: PRISM assumes that widespread net use will reduce the incidence of malaria. PRISM assumes that distributing bed nets will result in widespread use of bed nets. 53 II. EVALUATION A. Evaluation Purpose and Rationale Implementation of the PRISM program reached its mid-point in July 2011. The USAID/Zambia Health Office seeks an independent team to perform a mid-term performance evaluation of the PRISM program. The objectives of this mid-term evaluation are three-fold: Progress towards intended results To evaluate the progress made by PRISM towards intended results and change and describe the context and what other possible factors (such as health programs by other donors or actors, campaigns by a variety of actors, etc.) that are also contributing to the observed changes. Appropriateness To determine whether the activities, techniques, and processes agreed upon between USAID and PRISM and deployed by PRISM are appropriate to achieve the intended results and/or change within the five-year performance period and to recommend modifications to project activities or priorities, as necessary, to address implementation issues, apply lessons learned, or capitalize on new opportunities. Sustainability To determine whether gains made through PRISM are sustainable beyond the life of PRISM and, if not, recommend techniques and/or processes that would ensure sustainability of the gains made. The evaluation will thus provide USAID/Zambia, the Government of Zambia, and other in-country stakeholders with objective information on what has been achieved to date, what is working and why, as well as what is not working. The information will inform decisions to make appropriate modifications. B. Evaluation Questions Progress towards intended results: 1. Are the social marketing activities, techniques, and processes employed by PRISM contributing to the following intended results? a. Increased awareness of socially marketed health services and products? ; b. Increased use of socially marketed health services and products; and Effectiveness of interventions: 2. Are the social marketing, activities, techniques, and processes agreed upon between USAID/Zambia and PRISM and deployed by the latter effective to achieve the intended results? 3. Gender-Are the social marketing activities, techniques, and processes deployed by PRISM addressing the unique needs of men and women? 4. Target population-Are the social marketing activities, techniques, and processes deployed by PRISM equitably addressing the needs of the rural and urban populations? 54 5. Are the protocols and best practices identified by the project being implemented according to the work plan? 6. Is PRISM applying a level of effort that can ensure achievement of the intended results? 7. Does the Government of the Republic of Zambia (GRZ) consider PRISM to be accountable to the country’s strategic objectives? Are there activities that the GRZ would rather see implemented differently? Sustainability: 8. Is PRISM building host country systems for Zambia to continue to implement some or all of the currently socially marketed services and products after the conclusion of the program? a) To what extent is PRISM establishing systems that will make the currently socially marketed products and services continue to be socially marketed in Zambia by a commercial entity after the PRISM Task Order ends? Will the program be able to increase the number of individuals/clients willing to pay for currently socially marketed services and products at unsubsidized rates by 2014? b) Is PRISM generating evidence and lessons learned which could help other implementing partners and/or the MOH? c) Is PRISM providing useful skills to Zambians (judged by locally engaged staff) to improve their ability to socially market health services and products? C. Methodology The Contractor shall employ one or more of the following methodologies, as appropriate. This list is not exhaustive or meant to exclude other methods not described: Evaluation sites: The Contractor shall gather data from a cross-section of the geographical areas in which the programmatic activities are being implemented. The following sites are suggested: 1. PRISM head office situated in Lusaka; 2. PRISM implementation platforms located in the ten provinces of Zambia; 3. PRISM stand-alone male circumcision (MC) and counseling and testing (CT) sites; 4. Ministry of Health headquarters located in Lusaka; 5. Provincial Health Offices located in provincial headquarters; 6. District Health Offices; 7. Other U.S Government-supported and 8. Non-U.S Government-supported organizations implementing similar activities; 55 9. Communities benefiting from PRISM services and products such as counseling and testing and MC clients; and 10. Commercial outlets distributing PRISM health services and products. Sampling: The Contractor shall name the sampling method/s to be used in determining study subjects. Sources of data: The Contractor shall extract data from the following sources: 1. Project Documents : The Contractor shall start the evaluation work with the review of the documents listed below: a) Sections C and F of the Signed contract; b) All relevant contract modifications; c) Quarterly, semi-annual, and annual reports; d) PRISM performance monitoring plan; e) Formative research reports (HIV/AIDS, Family Planning, MCH, and Malaria); f) Portfolio review templates; g) PRISM Self-Assessment Report: Prior to the Contractor beginning the evaluation work, PRISM shall conduct a self-assessment exercise in response to some evaluation questions. PRISM shall submit the report to USAID. USAID will provide this report to the Contractor before the commencement of the evaluation; and h) Any other relevant documents. **The aforementioned documents will be provided electronically by the COR within five days after the award is signed. 2. Key informant interviews: The Contractor shall, where appropriate, conduct qualitative, in-depth interviews with key stakeholders and partners. The Contractor shall conduct face-to-face interviews with informants. When it is not possible to meet with stakeholders in person, telephone interviews shall be conducted. Key informants include the following: a) PRISM program staff (Chief of Party, Deputy Chief of Party, Technical Directors); b) Program staff from ZISSP, CSH, USAID DELIVER, Child Fund, ZPCT II, CIRDZ, and JHPIEGO; c) USAID staff (Contracting Officer’s Representative (COR) for PRISM) and any other staff) Government of the Republic of Zambia (GRZ) staff from Ministry of Health (MOH), National HIV/AIDS/TB/STI Council (NAC), and National Malaria Control Center (NMCC); d) Centers for Disease Control and Prevention (CDC); e) Group Interviews: The Contractor shall conduct group interviews with sub￾populations of interest to the PRISM program. Such populations include 56 service providers, community distributors of products and services, and users of products and services; f) Mini-surveys: The Contractor shall conduct mini surveys among sub￾populations of interest to the PRISM program. Such populations include service providers, community distributors of products and services, and users of products and services; g) Focus groups: The Contractor shall conduct focus group discussions with sub- populations of interest to the PRISM program. Such populations include service providers, community distributors of products and services, and users of products and services; and h) Direct observation: The Contractor shall sample for direct observations platform offices and stand-alone PRISM service delivery points. Data capturing and management The Contractor shall be required to 1. Describe data capture tools, 2. Types of data to be collected, and 3. How data will be managed to generate the required report/s. III. EVALUATION DELIVERABLES The contractor shall produce the following deliverables: Evaluation plan: The Contractor shall submit a detailed work plan before commencement of field work to the COR. The evaluation plan shall include all the activities that are going to be performed by the contractor in order to complete the evaluation, including the following elements: plans and duration for consulting USAID/Zambia and its partners and key stakeholders and dates for all the milestones in the evaluation process such as draft mid￾term evaluation report and final mid-term evaluation report. The COR shall review and provide written feedback within two business days of receiving the plan. Final Evaluation methodology: The contractor shall submit the final evaluation methodology protocol before commencement of fieldwork to the COR. The COR shall review and provide written feedback within two business days of receiving the final evaluation methodology protocol. The final methodology protocol must be accompanied by an annex, detailing the following: protocols for interview and site visits, proposed report outline for the final report, designating individual team member responsibilities, dates for entry, mid-term, and final briefings with the USAID/Zambia, other USG agencies and staff, implementing partners, and host government officials. Briefings: The Evaluation team will organize and provide entry and final briefings for USAID/Zambia staff, other USG agencies and staff, implementing partners and host government officials. 57 Draft mid-term evaluation report: The Contractor shall submit three hard copies and one electronic copy of the draft report to the USAID COR two working days prior to the final de- briefing and Team Leader departure from Zambia. The Contractor shall separate the findings, conclusions, and recommendations for each question. All recommendations included in the report shall be practical, specific, and action-oriented and designate the proposed implementer and timeframe. Final evaluation report: The final evaluation report must incorporate modifications requested by USAID and the GRZ, as agreed by both parties, unless the modifications are designed to alter the findings. However, if USAID has identified factual errors or can provide additional evidence/information to the evaluation team, and they agree to amend the report, then the modification will be accepted. The Contractor shall submit three hard copies and one electronic copy of the final report to the USAID COR. Evaluation report format: The Contractor shall prepare the draft and final evaluation reports in accordance with the following format: 1. The evaluation report must be written in English. 2. The evaluation report must be formatted for size A4 paper. 3. Report contents: a) Executive Summary (6 pages maximum length): Brief description of PRISM Project key results/impacts, and evaluation’s major findings/recommendations and lessons learned; b) Main body (40 pages maximum length): i. Description of the project: Drawing from the PRISM Program, concisely describe the rationale of PRISM’s Social Marketing interventions, what constraints/opportunities they were meant to address, and what, specifically, the program has been trying to accomplish. ii. Evaluation purpose, methodology: Describe, briefly, types and sources of evidence and methodologies employed to complete the evaluation SOW. iii. Findings: Present findings, with supporting evidence, as regards to the questions/issues in the SOW and other pertinent matters that should arise during the course of the evaluation. c) Conclusions: Present conclusions in relation to the findings. d) Recommendations: Present and synthesize pertinent recommendations as they regard ongoing planning, management and implementation of the PRISM Program. Also address matters of long-term sustainability and impact. All recommendations shall be practical, action-oriented, specific, and designate the proposed implementer. e) The evaluation report shall also contain all the data collection instruments used in the evaluation in the appendices. 58 f) When applicable, evaluation reports must include statements regarding any significant unresolved differences of opinion on the part of the funders, implementers and/or members of the evaluation team. 59 ANNEX B: TOOLS/SURVEY INSTRUMENTS, KII GUIDE INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. (IBTCI) MIDTERM EVALUATION OF THE USAID FUNDED PARTNERSHIP FOR INTEGRATED SOCIAL MARKETING (PRISM) PROGRAM Mini-Household Survey Introduction: We are from International Business & Technical Consultants, Inc. (IBTCI), who have been contracted by the United States Agency for International Development (USAID) to evaluate the Partnership for Integrated Social Marketing (PRISM) Program that has been social marketing health products such as male and female condoms, Safe Plan, Clorin, mosquito nets and insertion of intrauterine devices and Jadelle implants for birth spacing or family planning. We anticipate the interview will last about half an hour to an hour. Your answers to the questions we will ask are completely confidential and the information you provide will not be identifiable to you as individuals. Your participation is voluntary. You can refuse to answer any question or all the questions with no penalty. Also, you are free to leave at any time. We appreciate your assistance in this matter. Are you willing to participate? Yes..........1 No............. Interviewer’s signature: _________________________________ 60 Respondent No. Date Key: North western to start with 4 (e.g. - 401) Southern - 501 Lusaka - 6001 A. Demographic Data NO. QUESTIONS ANSWERS CODES CODES 1. How old are you? 1. 15-19 2. 20-24 3. 25-29 4. 30-34 5. 35-39 6. 40-44 7. 45-49 8. Above 50 9. Don’t know 2. What is your level of education? 1. Primary 2. Secondary 3. Tertiary 4. No education 3. What is your marital status? 1. Never Married 2. Married 3. Divorced 4. Widowed 5. Separated 6. Cohabiting (Living together but not married) 7. Polygamous Item Response Enter Number District 1 - Urban 2 - Rural Ward Village/Town Gender Male -1 Female -2 Time Started: Time Completed: 61 NO. QUESTIONS ANSWERS CODES B. Knowledge and awareness about socially marketed products and services 4. Have you heard of the following products and services that are promoted and/or offered in health facilities by the PRISM program in collaboration with the MOH and/or the DHO? (Circle the response in the Codes column) (If person responds NO to any option in question 4, skip that option in questions 5 & 6) a. Safe Plan oral contraceptive (OCs)? Yes ……….1 No.………..2 b. Intrauterine contraceptive device (commonly known as IUCD) offered in health facilities Yes ……….1 No.………..2 c. Jadelle Implants offered in health facilities Yes ……….1 No.……….2 d. HIV counseling and testing? Yes ……….1 No.………..2 e. Maximum Male Condoms? Yes ……….1 No.………..2 f. Care Female condoms Yes ……….1 No.………..2 g. Mama Safe Nite Insecticide￾treated mosquito nets (ITNs) to prevent malaria? Yes ……….1 No.………..2 h. Clorin to make drinking water safe? Yes ……….1 No.………..2 i. Safe water kits Yes ……….1 No.………..2 5. How did you hear about the products and services, which you have just mentioned? Radio – 1 TV – 2 Print media – 3 Community Volunteers – 4 Local traditional leaders – 5 Other community members – 6 IEC materials – 7 Health Facility – 8 New Start Centres – 9 All of the above – 10 None of the above – 11 Outreach - 12 (multiple responses are allowed) Other (specify) – If the answer to the options in question 4 is yes, then ask this a. Safe Plan oral contraceptive (OCs)? b. Intrauterine contraceptive device (commonly known as IUCD) offered in health facilities c. Jadelle Implants offered in health facilities d. HIV counseling and testing? e. Maximum Male Condoms? f. Care Female condoms? g. Insecticide-treated mosquito nets (ITNs) to prevent malaria? h. Clorin to make drinking water safe? i. Safe water kits 62 question 5. NO. QUESTIONS ANSWERS CODES C. Access to socially marketed products and services 6. Do you know where you can get these products and services? If the answer to the options in question 4 is yes, then ask this question 5. a. Safe Plan oral contraceptive (OCs)? Yes …….1 No.……..2 b. Intrauterine contraceptive device (commonly known as IUCD) Yes …….1 No.……..2 c. Jadelle Implants offered in health facilities Yes …......1 No.……..2 d. HIV counseling and testing? Yes ……...1 No…..…..2 e. Maximum Male Condoms? Yes ……..1 No.……..2 f. Care Female condoms Yes ……...1 No...……..2 g. Insecticide-treated mosquito nets (ITNs) to prevent malaria? Yes ……...1 No.……..2 h. Clorin to make drinking water safe? Yes ……..1 No.……..2 i. Safe water kits Yes ……...1 No.………2 NO. QUESTIONS ANSWERS CODES D. Family planning and reproductive health 7. Are you or your partner using any modern contraceptive method? (If no, skip to question 9 below). Yes………...1 No…………2 8. If you are using any modern contraceptive method, which one(s)? (If yes, ask question 11) 1. Oral contraceptive Safe Plan 2. Depo-Provera 3. IUCD 4. Jadelle Implant 5. Female condom 6. Male condom 7. Dual method (condom and another modern FP method) 8. Lactation amenorrhea 9. Withdrawal method 10. Permanent 63 method (tubal ligation) 11. Permanent method (vasectomy) 12. Other oral contraceptive method specify________ __ 9. Over the past 12 months, have you discussed contraceptive use with your partner? Yes ………....1 No.…….……2 10. Are family planning services provided free of charge? 1. In health facilities? Yes ….……….1 No.……..….. .2 Don’t know …3 2. In the community? Yes ……..…....1 No.…………...2 Don’t know ….3 11. If yes to Q8-1 Would you be willing to pay more for Safe Plan? Yes ………......1 No.……….......2 E. STIs/HIV/AIDS 12. Have you been sexually active in the past 12 months? (If No, skip to questions 13 and 15). Yes ………...1 No.………....2 13. Over the past 12 months, have you had more than one sexual partner? Yes ………...1 No.…….…..2 14. Have you tested and received your HIV results over the last 12 months? Yes ………...1 No.………...2 15. Do you know the HIV status of your partner(s)? Yes ………...1 No.………....2 16. Do you use condoms correctly and consistently with all sexual partners including regular, long term and trusted partners? Yes ………...1 No.…..……..2 17. Do you demand that your partner(s) use condom at all times? Yes ………...1 No.…………2 18. The last time you had sex, did you use a condom? Yes ………..1 No.…….…..2 19. What are the benefits of MC? 1. Reduces risk of acquiring HIV in men. 2. Reduces risk of 64 contracting STIs. 3. Reduces risk of cervical cancer in women. 4. Better hygiene. 5. All of the above 6. Don’t know 7. Other specify____________ ___ 20. (Check if response in question 8 is 6 or 7) If the person is using Maximum condom, ask if they will be willing to pay more for Maximum condoms? Yes ………...1 No.………....2 21. (Check if response in question 8 is 5) Would you be willing to pay more for Care condoms? Yes ………...1 No.………....2 F. Malaria 22. Has your house been sprayed over the last 12 months? Yes ………....1 No.………....2 23. Do you have a mosquito net in your household? (If No, skip to question 29) Yes ………...1 No.……........2 24. If you have any children less than 5 years, did they sleep under a mosquito net last night? Yes …..…….1 No.…….…..2 NA…….…...9 25. Did you sleep under a mosquito net last night? Yes ………....1 No.……..…..2 26. May I please see one of the mosquito nets that you are using if possible after the interview? (Check whilst it is hung if this is the case.) a. Mosquito net is an ITN Yes ………....1 No.…….….2 b. Mosquito net is hung properly Yes ………....1 No.……...….2 NA…………3 c. Mosquito net is in a reasonable condition Yes …….…..1 No.…….….2 28. Would you be willing to pay more for an ITN? Yes ………...1 No.………....2 G. CLORIN/SAFE WATER 29. Do you purify drinking water in your household? (If no, skip Q 32) Yes …….....1 No.…....…..2 30. If yes, which method do you use to purify drinking water for you and your family? 1. Clorin 2. Boiling water 3. Water filter 4. Buy bottled water 5. Other (specify) 31. If they use Clorin ask to see the Available……….. 65 bottle. 1 Not available….....2 32. Why do you not use Clorin to purify drinking water for your family? 1. Lack of money/too expensive 2. Don’t like the taste of water treated with Clorin 3. Other specify:___________ 33. Would you be willing to pay more for Clorin? Yes ………...1 No.………....2 34. What suggestions do you have on the socially marketed products and services that we have discussed? _ _ _ THANK YOU FOR YOUR TIME 66 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. (IBTCI) MID-TERM EVALUATION OF THE USAID FUNDED PARTNERSHIP FOR INTEGRATED SOCIAL MARKETING (PRISM) PROGRAM Focus Group Discussion (FGD) Guide Thank you for agreeing to talk with us. International Business & Technical Consultants, Inc. (IBTCI) has been contracted by the United States government through the United States Agency for International Development (USAID) to carry out this midterm performance evaluation of the USAID funded Partnership for Integrated Social Marketing (PRISM) program in Zambia. The purpose of this evaluation is to assess the achievements of the PRISM program to date, the challenges it is facing and how these can be overcome. We believe that you are in a good position to tell us about your organization and what it is doing in relation to the PRISM program, hence this interview. We anticipate the FGDs to last about an hour or less and appreciate any information you can provide. Your answers to the questions we will ask are completely confidential and the information you give us will be reported without names. Your participation is voluntary, and you can refuse to answer any question or all the questions with no penalty. Similarly, the nature of your responses positive or negative will not lead to any benefit or consequence. We appreciate your assistance in this matter. Do you mind if we record this discussion as backup in case we miss important points? Do you have any questions? Can we begin now? Signature of Interviewer (Indicates Participant Agreement to Participate):___________________________ 67 Awareness of socially marketed health services and products: a. What services and products are you aware of that PRISM is providing? Can you mention them by name? (Services: Male Circumcision, New Start VCT, Family Planning; Products: Maximum condoms, Care female condoms, Clorine, Mama Safenite ITNs, Safe Plan contraceptive, Jadelle Implants) b. What have you heard about the product or service and from where? c. What do you think about what you have heard about the products and services? d. How has the community responded to these messages? 2. Availability of socially marketed health services and products: a. Which of the products are available in this area and where exactly do you find them? Are there any products you are aware of that are not available here? b. Are the products and services affordable, in your view? 3. Use of socially marketed health services and products: Reproductive Health & Family Planning: a. Have you noticed any change in use of the different family planning methods being promoted by PRISM? Please give us examples. b. What are the reasons for these changes? HIV/AIDS/STIs a. Have you noticed any change in use of HIV related products and services being promoted by PRISM? Please give us examples. b. What are the reasons for these changes? c. What have you seen to be the challenges that prevent people from accessing HIV products/services, such as testing and counseling, male circumcision, and condoms? Male Circumcision a. How is the community responding to voluntary medical male circumcision messages? b. Has there been any change in demand for male circumcision? If so, about what ages are the men/boys who seek male circumcision? c. What have you seen to be the challenges that prevent people from using MC services? Diarrheal Disease Control &Safe Water a. Have you noticed any change in use of Clorin? Please give us examples. b. What are the reasons for these changes? c. What have you seen to be the challenges that prevent people from using Clorin? Malaria/ITNs a. Have you noticed any change in use of ITNs or insecticide-treated mosquito nets? b. What are the reasons for these changes? c. What have you seen to be the challenges that prevent people from using ITN? d. Who usually sleeps under an ITN in your household? 68 4. Addressing the unique needs of men and women a. What has been the participation of male and females in PRISM activities? b. Do you think that the PRISM program is giving messages in an appropriate manner for both men and women? Please explain offering concrete examples, if you can. c. Do you think the different services and products offered through PRISM address the needs of both men and women adequately? Please explain offering concrete examples, if you can. Probe for participation of different genders in different products and services; constraints that prevent men and women from accessing different products and services; and how the gaps can be addressed. 5. Ability to continue to socially market health services and products a. Which activities in the PRISM program would you like to continue and why? b. What activities would you be able to carry on doing without PRISM support and for how long? c. What training have you had that you feel would be most useful in helping you to carry on with this work? d. What else do you need to be able to carry on doing this work for a long time to come? e. Can you please share your opinions on coordination and collaboration of partners implementing the PRISM Program? f. Would you like to make any recommendations to strengthen the PRISM program? If yes, what are your recommendations? THANK YOU FOR YOUR TIME 69 INTERNATIONAL BUSINESS & TECHNICAL CONSULTANTS, INC. (IBTCI) MIDTERM EVALUATION OF THE USAID FUNDED PARTNERSHIP FOR INTEGRATED SOCIAL MARKETING (PRISM) PROGRAM Key Informant Interview (KII) Schedule Province: District: Organization: Date: Sex: Position: Venue: Time Started: Time Ended: Interviewer: Introductory remarks Thank you for agreeing to talk with us. International Business & Technical Consultants, Inc. (IBTCI) has been contracted by the United States government through the United States Agency for International Development (USAID) to carry out this mid-term performance evaluation of the USAID funded Partnership for Integrated Social Marketing (PRISM) program in Zambia. The purpose of this evaluation is to assess the achievements of the PRISM program to date, the challenges it is facing and how these can be overcome. We believe that you are in a good position to tell us about your organization and what it is doing in relation to the PRISM program, hence this interview. We anticipate the interview will last about an hour or less and appreciate any information you can provide. Your answers to the questions we will ask are completely confidential and the information you give us will be reported without names. Your participation is voluntary, and you can refuse to answer any question or all the questions with no penalty. Similarly, the nature of your responses positive or negative will not lead to any benefit or consequence. We appreciate your assistance in this matter. Do you mind if we record this discussion as backup in case we miss important points? Do you have any questions? Can we begin now? Signature of Interviewer (Indicates Participant Agreement to Participate):___________________________ 70 Introduction 1. Please give us a background of your organization and what it does? 2. How long has your organization been involved with the PRISM program? 3. Specifically, how is your organization involved with the PRISM program? 1. Progress towards intended results: a. Are the social marketing activities, techniques, and processes employed by PRISM contributing to/achieving the following intended results: i. Increased awareness of socially marketed health services and products; ii. Increased availability of socially marketed health services and products; and iii. Increased use of socially marketed health services and products. b. If yes, what products/services/activities? Please give examples. Please provide concrete examples of achievements and areas where the project is not on target/achieving expected results. Include areas where you think there is room for improvement. 2. Appropriateness and effectiveness of interventions: a. Effectiveness: How effective has the PRISM program been (activities, techniques, and processes)? b. Gender: To what extent has the PRISM program addressed the issues of gender (the social marketing activities, techniques and processes)? c. Target population: To what extent has the PRISM program addressed the issues of equity between rural and urban populations (the social marketing activities, techniques, and processes deployed)? d. Are the protocols and best-practices identified by the project being implemented according to the work plan, standards, and guidelines? e. Is the PRISM program on schedule to achieving intended results (applying level of effort to ensure achievement)? f. Does the Government of the Republic of Zambia (GRZ) consider PRISM to be accountable to the country’s strategic objectives? Are there activities that would be more in line with GRZ priorities (alignment to government strategies/objectives)? g. How are the partnerships working within the PRISM program (e.g., local and international successes and constraints)? 71 3. Sustainability: a. Is PRISM building systems and local capacity to enable Zambian counterparts to continue to implement some/all of the services and products currently being socially marketed after the conclusion of the program? If so, how? b. To what extent is PRISM establishing systems to insure that currently socially marketed products and services are sustained by a commercial or public entity after the program ends? c. What is your organization’s involvement with PRISM to insure that currently socially marketed products and services are sustained by a commercial or public entity after the program ends? d. Will the program be able to increase the number of individuals/clients willing to pay for chargeable services or increase the use of currently socially marketed services and products at unsubsidized rates by 2014? How? e. Has PRISM shared evidence and lessons learned, which your organization and other implementing partners, including MOH, could use to improve or plan better strategies and activities? 4. Other: Is there anything else you would like to say or add to what we have discussed? THANK YOU FOR YOUR TIME 72 ANNEX C: DATA COLLECTION TABLE: NUMBER OF HOUSEHOLD INTERVIEWS DONE BY DISTRICT, PROVINCE Province and Districts FGDs & participants KIIs &respondents Households Eastern Province: Chipata &Mambwe 7 FDGs, 84 participants 17 KIIs, 20 respondents 34 Luapula Province: Mansa & Mwense 16 FDGs, 196 participants 14 KIIs, 21 respondents 22 Lusaka National level - 20 KIIs, 30 respondents - Lusaka Province 3FGD, 27 participants 6 KIIs, 6 respondents 125 Northern Province: Kasama & Mungwi 4 FDGs, 39 participants 20 KIIs, 20 respondents 24 North-western Province Solwezi & Kasempa 4FDGs 54 participants 2KIIs 2particpants 19 Southern Province Livingstone & Kazungula 4FDGs, 28 participants 12 KIIs 30 respondents 15 Southern Africa Regional PSI Staff 1 KII Total 38 FDGs, 428 participants 92 KIIs, 130 respondents 240 73 ANNEX D: LIST OF INTERVIEWEES 1. List of Key Respondents in Northern Province List of key informants in Kasama District 1. Dr. Chisenga Kasama DMO 2. Ms Nomsa Saimwanza Kasama Quality Assurance Officer – SFH 3. Lisa Susiku Chompa Kasama MCH Coordinator – Army Clinic 4. Mr Chomba Simon Mulenga Kasama Sales Assistant (SFH) 5. Ms Ivy Chimbano Kasama Community distributor 6. Dr. Kalubi Kasama MC trained officer 7. Ms Silvia Nakazwe Kasama Drug store distributor 8. Ms Julie Chilambwe Kasama Trainer (SFH) 9. Ms Catherine Mwansa Kasama Sister in charge – Chisanga 10. Mr Simon Sampa Kasama Driver –Kayambi 11. .Mr Godfrey Kamanga Kasama Manager – (SFH) List of Key informants in Mungwi 12. Ms Grace Katongo Mungwi CBD 13. Mr George Mofu Mungwi CBD 14. Ms Edwin Silwamba Mungwi Officer in charge – Nseluka RHC 15. Mr Ignatius Bwalya Mungwi Traditional Leader 16. Ms Grace Chonya Mungwi DHO – MCH Nursing officer 17. Mr. Ignitius Bwalya Mungwi DHO –nursing officer, MCH 2. List of Key Respondents in Southern Province List of Key informants in Livingstone 18. 1Mr Sibusenga Livingstone Acting PMO 19. Ms Singoi Livingstone Acting MCH Coordinator 20. Ms Hazemba Livingstone PMO Human Resource Officer 21. Mr John Mapulanga Livingstone SFH Platform Manger 22. Ms Rita Tembo Livingstone SFH-Communication Assistant 23. Mr Misheki Mutamba Livingstone CARE Manger 24. Ms Lapila Chirwa Livingstone New-Start VCT 25. Ms Faith Musaba Livingstone Jolly boys 26. Joseph Miseyu Kazunguala CBD 27. Willie Maxiwell Kazungula CBD List of Key informants in Kazungula 28. Ms. Lapila Chirwa Kazungula DMO Acting officer in charge 29. Ms Anita Siluwahile Kazungula MCH Acting Sister in charge 74 3. List of Respondents in Lusaka National, Province List of Key respondents in Lusaka National 1 Grace Tembo Community mobilization STIs and VMC MCD/MCH 2 3 Hilda Mansa Wina Judy Zenovia Ngombe Family planning specialist FP trainer MCD/MCH (from ZISSP) MCD/MCH 4 5 6 Mr. Jay Goulden Maureen Mubanga Martha Deputy Country Director Project Manager Finance CARE International 7 Jessica Price Country Director Population Council 8 9 Kwame Asiedu/ Dr.Omega Country Director Program Manager, PRISM Project JHPIEGO 10 Dr. Wezi Kaonga, HIV Specialist MOH 11 Dr. Albert Kaonga MC Coordinator MOH 12 Nabanda Mulako and 5 team members Director ZHECT 13 14 Christopher Mazimba Maurice Pengele Country Program Director M & E Team Leader Scaling up Family Planning in Zambia (SUFP) 15 Dr. Namwinga Chintu Kelly Young, and 6 Program Directors Executive Director and National staff SFH 16 Ibrahim Singalika Health Specialist UNFPA/MOH 17 Dr. Kamwinga Director, NMCP National Malaria Control and Prevention, MOH 18 Rakesh N. Shah, CEO Marketing Director Pharmanova 19 Dr. Meena Gandhi Health Advisor DFiD 20 Rabson Zyambo Senior Public Health Logistics Advisor- Malaria & family Planning JSI Deliver 21 Doug Call Regional Director, Southern Africa PSI 4. List of Lusaka Province KIIs 22 Milimo Habensu, Chongwe Site Coordinator (ZHECT) 23 Teddy Nkomeshya Chongwe Outreach Worker (ZHECT) 24 Sister Mabena Chongwe VCT Coordinator 25 Sis. Maureem Chikwa, MC, Chawama Hospital Chawama 26 Sis. Ruth, SFH FP provider, Chawama hospital Chawama *Sis= Sister 75 5. List of Key Respondents for Key informants in Eastern province 1 Modester Chileshe Chipata CARE-PRISM Development Coordinator 2 Jim Ja Chipata SFH - (Team Leader Service Delivery) 3 Francis Chimfwembe Chipata SFH - (Area Sales Representative) 4 Oliver Kazangalale Chipata SFH - (CT Site Manager Chipata) 5 Mukalya Elizabeth Kampamba Chipata ( Communications Coordinator) 6 Memory Mwanza Chipata (Communications Assistant) 7 Sam Mwale (Counsellor Chipata SFH - (Counsellor) Psychosocial & VCT) 8 Bernard Nyirenda Chipata Motala Pharnacy - (Manager) 9 K.V. Ramachandran Chipata Life Pharmacy - (Manager) 10 Jean Kaluba Chipata Pola Pharmacy - (Sales lady) 11 Samuel Chizalika Chipata Acting DDH, Nursing Officer Standards 12 Mrs Manda Chipata MCH Coordinator 13 Bridget Ngoma Chipata Muchini Drug Store - (Sales Lady) 14 JJ Chris Shop Chipata Chiparamba Market 15 Edwin Mkelenga Chipata Nurse, Acting in-charge 16 Emmanuel Thole Chipata EHT, Nsefu Health Centre, PRISM FPP 17 Esther Banda Chipata MCH Coordinator 18 Patrick Mbao Chipata EHT Masumba Health centre, PRISM FPP 19 Alfred Chola Chipata RN, MC Provider 20 Dr. Gideon Zulu Chipata Distirct Director of Health 6. List of Key Respondents in Luapula Province Mansa and Mwense 1 Dr. Elicho Bwalya M PMO Provincial Health Office 2 Regina Chola F Communications Assistant SFH Area Office 3 David Phiri M Development Officer Care International (Luapula and Northern Provinces) 4 John Chanda M Platform Manager SFH 5 Agnes Chansa F CDE/ CBD Matumbusa Community Rural Health Centre 6 Mercy Kanswata F RN/RM Butungwa Community Urban Health Centre 7 Febby Lienje F Certified Midwife Butungwa Community Urban Health Centre 8 Mr. Mwansa M Acting NO/MCH Coordinator Mansa District Community Medical Office 9 Pascoe Salimu M Gender Officer/Team Leader UNFPA 10 Moses Zulu M Executive Director Luapula Foundation 11 Mick Kalubwa M CT Manager Luapula Foundation 12 Geoffrey Zulu Programme Manager Luapula Foundation 13 Ben M. Chembe M Counsellor/Acting Senior Counsellor/Horizons Coordinator Luapula Foundation 14 Webby Kaloba M Counsellor Luapula Foundation 15 Kennedy Kombe M Counsellor Luapula Foundation 76 16 Brenda Chembe F Sex worker Kashiba Community Rural Health Centre 17 Mary Bwalya F Sex worker Kashiba Community Rural Health Centre 18 Theresa Kalunga Miti F CM Musonda Falls Community Rural Health Centre 19 Emmanuel Mwenya M Young man Musonda Falls Community Rural Health Centre 20 Stanley Kapya M Young man Musonda Falls Community Rural Health Centre 21 Probbie Cheelo F EN/EM Chibondo Community Rural Health Centre 77 ANNEX E: LIST OF DOCUMENTS REVIEWED National Level Research and Strategic Documents Zambia DHS 2007 2012 Census of Housing and Population Zambia National Malaria Indicator Survey 2012 National Health Strategic Plan 2011-2015 Zambia HIV/AIDS Country Report 2010-2011 Zambia Monitoring, Research & Evaluation Plan 2011-2015 Zambia Adolescent Health Strategic Plan HIV Sexual and Reproductive Health and Rights Report 2011-2014 Audit of Commodities Funded under The President’s Malaria Initiative in Zambia 2012 Policy Overview and Status of the AIDS Epidemic in Zambia Intervention Specific Research Evaluation of the Informed Consent Process for MC Scale-Up in Zambia Dedicated Providers of Long-Acting Contraception: A New Approach in Zambia Perceptions of the IUD and Implants among Women in Lusaka District Assessing the Safety and Acceptability of Post-Partum IUD Users in Zambia Strategic Document for M-CHIP Zambia Child Survival Analysis Assessing SFH’s Misoprostol Pilot Project in Rural Zambia Male Circumcision Emotional Points of Difference Sex with Stitches: the Resumption of Sexual Activity during the Post-Circumcision Healing Period in Zambia SFH Gender Assessment Report 2012 78 PRISM Contract Related Documentation and Project Reporting USAID PRISM Contract and Contract Modifications Project PMP Years 1-4 work plans with deliverables Years 1-3 Annual reports TRaC surveys, various Delta marketing plans Environmental Risk Review SFH Strategic Plan 2006-2011 KFW Evaluation Report 79 ANNEX F: SUSTAINABILITY OF ITNS, CLORIN AND MALE CONDOMS Handover of ANC Program to NMCC In keeping with USAID’s increasing emphasis on using host-nation systems, USAID/Zambia relies on the Zambian Government to store and distribute the majority of PMI-funded commodities. John Snow International (JSI), through its USAID-funded Deliver program, assists the Medical Stores Limited (MSL) with forecasting, procurement, and delivery of ITNs to SFH. SFH assists the NMCC and MOH with warehousing and storage of ITNs as well as with the distribution to district level (for ANC nets) and to health facilities and communities for other ITN funded programs. Malaria commodities (including drugs and tests) procured through the JSI/DELIVER Project are received and stored through the Zambian Government’s MSL upon arrival. These are then distributed directly to district health offices who are tasked with distributing them throughout their network of health facilities. ITNs are delivered by JSI/DELIVER directly to SFH, who stores and distributes them to district health offices based on instructions received from Zambia’s National Malaria Control Center (NMCC). These instructions are based on forecasted needs by district developed jointly between the NMCC and JSI/DELIVER. Delivery of ITNs, alongside all USID-purchased malaria commodities, are audited annually by the PMI. A number of findings and recommendations from the 2012 PMI audit are relevant to the issue of handover of the ANC ITN program. The audit reported notable problems with MOH and MSL commodities accounting and tracking at district and health facility levels, while it found partner funded programs to be relatively strong in this area: “The mission’s contracts with the partners clearly define a scope of work and other requirements, such as reporting on program activities and allocating and accounting for program costs. The audit found that the partners’ systems for procuring and distributing commodities generally accounted for the commodities under their control. However, once the commodities were transferred to the Zambian Government, its systems for storage and distribution were weak. The audit team found problems at every stage of the process—from the central MSL warehouse that received the commodities to the health facilities that distributed them to beneficiaries. Storage space and personnel training were inadequate, which made it difficult to account for PMI commodities.” The audit found weaknesses across all MSL systems and at all levels for tracking and filling orders for malaria commodities: for example, batch numbers for drugs and kits were not recorded for commodities issued but not received at DHOs and health facilities, thus, these products could not be tracked. MSL’s central warehouse was also found to lack sufficient space, pallets and shelves to store PMI-funded tests, drugs, and insecticide properly. If the MSL is struggling to adequately and safely store these smaller commodities, shifting the responsibility for safe warehousing and storage of millions of ITNs would add significantly to existing 80 challenges and constraints they are facing. Due to these systematic weaknesses, the auditors could not confirm whether all commodities reached their intended beneficiaries. For ITNs specifically, the audit found the following:  Thirty health facilities without documentation to account for 38% of the commodities tracked from MSL and district health offices, and 21 did not have records on hand to document receipt of 80,990 nets (80% of the nets distributed);  Successful tracing of 100,647 nets worth about $372,006 to 28 health facilities from district offices. Twenty-one facilities did not have records on hand documenting receipt of 80,990 nets (80%);  Although many facilities maintained registries with the names of ITN beneficiaries, the records were not organized such that it was possible to compare the total number of beneficiaries with the total nets the facility received. In some cases, total beneficiaries registered exceeded the total nets recorded as received;  At 4 of the 28 facilities, the audit found that the staff members were not distributing nets exclusively to pregnant women and children under five as intended; and  The audit also found nearly 10%/66,185 of the 752,900 PMI-funded nets delivered in March 2012 still in storage three months later at eight health facilities, six district health offices, and in SFH’s warehouse. Officials in one district said they lacked adequate transportation to deliver the nets to health facilities and had not yet finished training health staff on how to distribute them properly.10 Adding to the considerable systematic constraints and challenges the MOH is faced/grappling with already in stock and supply chain management, it is important to consider the considerable financial burden the MOH would have to absorb were it to take over all aspects of the free ITN programs country-wide. To date, SFH has expended approximately $2.5 million in PRISM funds alone on ITN warehousing, distribution, and logistics (approx. cost per ITN: $1.23). This does not include the considerable costs for ITNs stored and distributed for the MOH funded through other donors. It is clear from multiple sources and levels of the MOH that the MSL is not equipped at present to take on the considerable human resources, warehousing, supply chain, and financial responsibilities this would require. While the evaluation team feels strongly that handover of the ANC (and other) ITN programs to the MOH is ill-advised in the short-term, SFH could and should work more closely with the MOH and DHOs to strengthen capacity and systems to better manage these programs now and in the future. Improved forecasting, tracking, monitoring, reporting, and understanding of the MOH ITN program is clearly needed. While SFH should be commended for its considerable efforts to respond to growing requests for ITN logistical support from the MOH, they have failed to build the systems and support required to assist the MOH to monitor and sustain ITN programs as they have done in other intervention areas such as CT and male circumcision. 10 SFH retains a small proportion of ITNs at the bequest of the NMCC to fill gaps and respond to urgent requests as needed. 81 The NMCC confirmed that handover of the ITN program was not desirable in the near future. With mass distribution of an estimated 8 million nets planned in 2014, they are looking for increased numbers of partners to support and assist with logistics for this, not less. However, they did feel strongly that there was considerable scope to begin with systems strengthening at all levels of the MOH, NMCP, and MCDMCH. Handover of Clorin to the Commercial Sector Under PRISM, SFH was tasked with “graduating” one of their socially marketed products, Clorin, to a commercial entity with the goal of insuring sustainable, commercial sector sales without program assistance or price subsidies. To date, the Population Council has undertaken a willingness-to-pay survey for Clorin, in response to which SFH raised its price points, allowing sales of Clorin at a price that is close to full cost-recovery. Booz Allen was scheduled to undertake a market assessment to determine the viability, potential, reach, market size, and target market for Clorin. After numerous delays, Booz Allen has just indicated that they will not be able to carry this out. PSI has already identified another agency to carry this forward. However, it is highly unlikely that SFH will achieve this indicator before end of program. Discussions with Pharmanova, the local Zambian manufacturer of Clorin, raise a number of issues that still need to be examined more closely before handover is advisable. First, Pharmanova indicated that a price increase of at least 18-20% would be required for them to take over the sales and distribution of Clorin. This price increase was calculated only on costs to transport Clorin (which is heavy and voluminous compared to other PRISM products) and only to commercial wholesalers in urban centers. Associated costs for added staffing and management were not assessed. Pharmanova has no access or penetration into rural areas and little incentive to undertake activities to “push” Clorin out to rural markets where it is most needed. In addition, Pharmanova indicated that it would only be interested in undertaking commercial sector distribution and sales of Clorin if SFH and partners were to continue with branded marketing and BCC activities. Furthermore, handover to Pharmanova would affect SFH ability to respond to emergencies and provide the MOH, UNICEF, and others with stocks in times of disease outbreaks. Many questions remain to be answered as to whether or not handover of Clorin to a commercial entity is prudent: who is the current consumer of Clorin, and are a significant proportion of current subsidies being wasted on urban populations who could pay more? Findings from this evaluation suggest that rural populations are those who are most aware of the need to treat water and most in demand of Clorin. They are also the populations with least access to MOH services and treatment for diarrheal disease, so arguably the most important target group for this intervention. Price was given as the primary reason for not using Clorin amongst household respondents interviewed. A significant increase in price through handover of this life-saving product is likely to result in significant losses in access and use amongst these high-risk populations. A market survey and consumer profiling are needed in order to better understand the risks and benefits of this activity before proceeding. 82 Condom Social Marketing PSI is focused on developing the total market for condoms and not exclusively the brand Maximum. While Maximum is the primary male condom brand of the PRISM program, the overall strategy for male condoms is to increase use regardless of whether it is a branded condom, subsidized condom, or fully commercial condom. In order to further the sustainability of condom use in Zambia (i.e., decrease total subsidy in condom procurement and distribution), PSI will soon extend Trust condoms into the Zambian market at a price level that positions it between Maximum and the higher-end commercial brands. The goal is to provide consumers with greater choice. More importantly, this brand launch will further segment the market so that consumers with some ability to pay, and less in need of a subsidy, have a viable condom brand that is more affordable than Durex and other premium condom brands that are commercially available in Zambia. The price point for Trust will be sufficiently high to cover the full cost of the condom, packaging, and its relevant distribution and promotional expenses. PSI manages Trust brand condoms from Johannesburg given the regional nature of the program and in order to achieve greater economies of scale. SFH will be a partner in assuring that Trust condoms are promoted appropriately. 83 ANNEX G: EVALUATION TEAM BIOS Jennifer Peters has twenty years’ experience in management and consulting in public health in the HIV/AIDS, MCH and reproductive health sectors, with experience in 19 countries. Her technical expertise includes evidence based social marketing and behavior change communications (BCC), monitoring and evaluation, quantitative and qualitative research, mass media and community mobilization campaigns, program design and implementation, and report writing. She has provided technical support in social marketing, including the development of marketing plans and communication strategies targeting vulnerable populations, in support of HIV/AIDS, family planning, child health, and malaria projects in southern African countries, including in Zambia. She has also assisted in the development of a BCC and service delivery network for a youth reproductive health program. Her recent evaluations include the midterm evaluation of USAID/Jordan’s Health Systems Strengthening II Project and a midterm evaluation of USAID/Palestine’s Health Sector Reform and Development Project, on which she served as MCH and BCC expert and BCC and community mobilization expert, respectively. In addition to relevant sector and evaluation expertise, Ms. Peters is a strong writer and has produced reports for a number of donor agencies. She was the lead consultant on the preparation of Botswana’s Millennium Development Goals 2009 report and has prepared reports for UNAIDS, UNICEF, and UNFPA. Ms. Peters holds an MSc in in Health Services Management from the London School of Hygiene and Tropical Medicine. Milka Juma has over 12 years of experience in monitoring and evaluation, implementation, research, and data collection in the health sector, with expertise in health interventions for HIV/AIDS, sexual reproductive health, and child health within Africa. Her recent evaluations include an evaluation of the USAID-funded APHIA II Health Communication and Marketing (HCM) project in Kenya for which she wrote the report and made recommendations for follow￾on activities. She is also evaluating a Red Cross project which aims to increase community awareness on HIV and AIDS, preparedness, access and adherence to antiretroviral therapy. During six years with the Population Council/Horizons, Ms. Juma worked with implementing partners and government departments to develop and implement operations research studies on HIV/AIDS prevention, treatment, and mitigation/care in Kenya, Uganda, Zambia and Zimbabwe. She also monitored the program’s Periodic Presumptive Treatment (PPT) of STD project in Zambia, Zimbabwe and South Africa, and worked with local partners to develop the monitoring and evaluation tools used for the project. Ms. Juma holds an MA in Population Studies and Demography from the University of Nairobi and is a PhD candidate for Health Promotion Research and Interventions at Maastricth University, Netherlands. Phoebe Bwembya has over 20 years’ experience working for government, bilateral and multi￾lateral institutions in the field of health, food security, and nutrition. She has extensive experience at the national and regional level and has demonstrated her analytical and conceptual abilities through program/project monitoring and evaluation, research, project formulation, and policy reviews. Her PhD research focused on “Factors Influencing Reproductive Decision Making of HIV Positive Zambian Women.” Her evaluation experience includes serving as Team 84 Leader on the HODI evaluation of an HIV and Gender-based Violence Project in Zambia and consulting as part of the evaluation team for an evaluation of the SIDA-funded Strengthening Mitigating Mechanisms Amongst Smallholder Farmers project and for the mid-term evaluation of the Southern Province Household Food Security Program in Zambia. Dr. Bwembya is proficient in a number of computer applications for data analysis, including SPSS, Atlas-ti, and MS Excel. She holds a PhD in Public Health from the University of Melbourne and an MSc in Human Development from Southern University of Illinois at Carbondale. Peggy Chibuye is a Zambian health professional with 18 years of experience in health development programs, possessing skills in the design planning, implementing, managing, monitoring, and evaluating of child health, reproductive health, and HIV/AIDS programs in Africa. Recent evaluation consultancies included evaluations of the Joint Sixth Country Program for UNFPA and of the performance of Our Lady’s Hospice, both in Zambia. While serving as Technical Advisor for the Basic Support for Accelerating Child Survival project in Malawi, she contributed to evaluations of the Global Fund assistance provided to the Malawi National AIDS Commission. Dr. Chibuye has previously served as a Senior Public Health Advisor for USAID/Zambia, where she helped organize the 1999 evaluation of Zambian Health reforms. She also contributed to the social marketing of female and male condoms in Zambian supermarkets and pharmacies. Dr. Chibuye holds a PhD in Public Health from the London School of Hygiene and Tropical Medicine and an MSc from the University of Manchester. Moses Simuyemba is a Medical Doctor and Public Health Specialist with over 11 years of experience managing HIV/AIDS and other related health programs. He specializes in research design and monitoring and evaluation and has managed, coordinated, and reported on health programs funded by many development agencies such as USAID, EC, DFID, IFRC, and the UN. Dr. Simuyemba has successfully worked on thirteen other development projects in Zambia between 2010 and 2012; facilitating the development of several NGO’s strategic plans, doing technical writing for various evaluations including the Futures Group, performing research for many works including Women and Law in Southern Africa (a situational analysis of health services in a specific Zambian district,) and conducting case studies for evaluations where he used qualitative and quantitative data collection and research methods. As the Operations Director and Special Programs Manager at Afya Mzuri in Zambia, between 2008 and 2010, Mr. Simuyemba headed the M&E and Special Programs units where he was in charge of implementation, management and technical oversight of all special health programs. Dr. Simuyemba received his Masters for Public Health from the University of Western Cape, South Africa, and two Bachelor’s degrees in Medicine and Surgery, and Human Biology from the University of Zambia in addition to holding a certificate in planning, monitoring and evaluation from the University of Zambia. Dr. Simuyemba is fluent in English and Nyanja with basic skills in Bemba and French.