PERFORMANCE EVALUATION OF USAID/ZIMBABWE’S IMPROVING FAMILY PLANNING SERVICES (IFPS) ACTIVITIY Task Order: AID-613-TO-16-00005 Contract No: AID-OAA-I-15-00028 EVALUATION REPORT March 27, 2017 This publication was produced for review by the United States Agency for International Development. It was pre￾pared by The Mitchell Group, Inc. with assistance from JIMAT. The core evaluation team was comprised of Dr Elvira Beracochea, Evaluation Team Leader and Dr Naomi Wekwete, Family Planning Specialist. PERFORMANCE EVALUATION OF USAID/ZIMBABWE’S IMPROVING FAMILY PLANNING SERVICES (IFPS) ACTIVITY A FINAL EVALUATION REPORT OF THE PROGRESS TOWARDS ACHIEVING ITS GOALS BY USAID ZIMBABWE’S IFPS PROJECT FOR USAID/ZIMBABWE’S REVIEW. Performed under Task Order NO. AID-613-TO-16-00005 PPL/LER Monitoring & Evaluation IDIQ Contract NO. AID-OAA-I-15-00028 Evaluation Report March 27, 2017 ZIMBABWE FACE-HIV EVALUATION II DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ZIMBABWE FACE-HIV EVALUATION III TABLE OF CONTENTS ABBREVIATIONS & ACRONYMS ................................................................................ iv EXECUTIVE SUMMARY ................................................................................................. vi Findings and Conclusions ............................................................................................... vii Recommendations ............................................................................................................. ix INTRODUCTION AND CONTEXT .............................................................................. 1 EVALUATION QUESTIONS AND METHODOLOGY ........................................... 2 1.1 Evaluation Questions ................................................................................................. 2 1.2 Evaluation Methodology and Data Limitations ...................................................... 2 1.3 District Selection for Evaluation Site Visits ............................................................ 3 1.4 Data Analysis ............................................................................................................... 3 FINDINGS AND CONCLUSIONS ................................................................................. 4 2.1 Access to FP services ................................................................................................. 4 2.2 Coverage of IFPS services ......................................................................................... 6 2.3 Demand ........................................................................................................................ 6 2.4 Quality .......................................................................................................................... 7 2.5 Sustainability .............................................................................................................. 10 2.6 Findings related to alternative approaches and strategies .................................... 11 2.7 Challenges faced by IFPS ......................................................................................... 12 2.8 FP Challenges and Priorities .................................................................................... 13 2.9 Perceptions by national and local stakeholders ..................................................... 15 RECOMMENDATIONS......................................................................................................................17 ZIMBABWE IFPS EVALUATION IV ABBREVIATIONS & ACRONYMS AIDS Acquired Immune Deficiency Syndrome ANC Antenatal Care CBD Community-Based Distributor CPR Contraceptive Prevalence Rate COP Country Operational Plan CPI Client Provider Interaction DEC Development Experience Clearinghouse DfID Department for International Development, UK FGD Focus Group Discussion FCH Family and Child Health FP Family Planning FY Financial Year GOZ Government of Zimbabwe HIV Human Immunodeficiency Virus IFPS Improving Family Planning Services Project IP Implementing Partner IUCD Intrauterine Contraceptive Device KII Key Informant Interview LAPM Long Acting and Permanent Methods MCHIP Maternal and Child Health Integrated Program MDRTB Multiple Drug Resistant Tuberculosis M&E Monitoring and Evaluation MICS Multiple Indicator Cluster Survey MNCH Maternal, Neonatal and Child Health MOHCC Ministry of Health and Child Care ZIMBABWE IFPS EVALUATION V MSI Marie Stopes International NGO Non-Governmental Organization PEPFAR President's Emergency Plan for AIDS Relief PLHIV People Living with HIV PSZ Population Services Zimbabwe QA Quality Assurance QI Quality Improvement SF Social Franchise SOW Scope of Work SRH Sexual and Reproductive Health TA Technical Assistance TL Tubal Ligation UNFPA United Nations Population Fund USAID United States Agency for International Development USG United States Government VHW Village Health Worker WHO World Health Organization ZDHS Zimbabwe Demographic and Health Survey ZNFPC Zimbabwe National Family Planning Council ZIMBABWE IFPS EVALUATION VI EXECUTIVE SUMMARY The purpose of this report is to present the findings, conclusions and recommendations of an end-of-project performance evaluation of the USAID-supported Improved Family Planning Services (IFPS) Activity in Zimbabwe. IFPS is a five-year project implemented by Population Services Zimbabwe (PSZ), a Zimbabwe￾an non-governmental organization (NGO) affiliate member of Marie Stopes International (MSI). IFPS has two main objectives: 1. Increase the awareness of voluntary FP methods, particularly LAPMs by 20% from 17% among un￾derserved women and men aged 15-49 years in all provinces by 2017, and 2. Provide comprehensive voluntary FP serves to 589,506 poor and underserved women and men aged 15-49 through mobile clinical outreach and a social franchising network in all provinces by 2017 The evaluation took place from December 10, 2016 until March 31, 2017. Lessons from this evaluation will help inform efforts to fine tune and improve on the current activity that will end in September, 2017 and de￾sign new activities for improved effectiveness and efficiency towards reaching USAID/Zimbabwe's FP objec￾tives. The evaluation was organized to respond to a number of evaluation questions posed by USAID/Zimbabwe. The Evaluation Team (ET) addressed the evaluation questions and sub-questions about the performance of the IFPS activity, as follows: 1. How can the IFPS activity design and implementation be enhanced to increase efficiency and effectiveness towards meeting the project goal of reducing maternal mortality and morbidity? 1.1 What design and implementation aspects of the IFPS project are critical to achieving intended results? Why? 1.2. What design and implementation aspects of the project are not achieving intended results? 1.3. What alternative approaches and strategies could achieve better results? Justify' proposed approaches and strategies. 2. What are the major challenges faced by the IFPS activity in delivering comprehensive FP services and how can USAID address these challenges? 3. What are the major national FP challenges and priorities and how can USAID assist in addressing these challenges through the IFPS project or other future USAID FP support? 4. What are the perceptions of national and local stakeholders towards USAID FP support and why? How can negative perceptions be addressed? The evaluation questions were further refined to explore IFPS achievements in terms of access, demand, cov￾erage, quality, and sustainability based on discussions with USAID/Zimbabwe. ZIMBABWE IFPS EVALUATION VII FINDINGS AND CONCLUSIONS The principal finding is that IFPS is on track to achieve all the objectives for which it was designed. IFPS is implemented by a local NGO, Population Services Zimbabwe (PSZ), which has nationwide coverage through its clinical outreach teams, social franchise network of private health service providers, and static clinics that are supported through other sources of funding in addition to the funding provided by USAID. This makes investing in PSZ’s development and strengthening a likely sustainable investment. IFPS supports nine out￾reach teams reaching all 10 provinces, serving 950 government clinic sites and 249 medical tent sites, and 49 of the PSZ’s BlueStar franchise clinics. Through these channels, IFPS has served a total of 475,775 clients as of December 2016, and is perceived as having been instrumental in Zimbabwe having achieved a high CPR of 66 in the 2015 DHS survey. USAID, consequently, is perceived as a key partner of the Ministry of Health and Child Care (MOHCC), as well. Below are selected findings by evaluation question: 1. How can the IFPS activity design and implementation be enhanced to increase efficiency and ef￾fectiveness towards meeting the project goal of reducing maternal mortality and morbidity? The service delivery channels of IFPS have been effective in reaching the target number of clients. Despite having and enforcing quality standards, there are some aspects of the IFPS service delivery that do not seem to be enhancing its effectiveness. The quality of group counseling is not consistent and there is lack of conti￾nuity in client care that would help clients choose permanent methods. There is limited personalized relation￾ship between provider and client or provider-initiated follow up. Despite having access to clients’ mobile phone numbers, there is not enough staff time to provide a personalized follow up. The number of outreach teams only allows teams to visit every site once in every three months on average and does not allow for continuous follow up of clients’ side effects and changes in FP methods to meet their reproductive health needs as their age and circumstances change. In addition to increasing the number of outreach teams and designing a continuous “client for life” approach to follow-up clients, below are a num￾ber of recommendations to enhance further the design of the already effective model. 1.1 What design and implementation aspects of the IFPS project are critical to achieving in￾tended results? Why? By having trained staff who can provide all FP methods, IFPS offers clients the method mix that they need and want. IFPS has quality standards and consistently provides care in the same way in all its outreach sites. Staff are supervised and supported. However, the nature of the work that keeps health teams on the road eve￾ry day makes it hard and there is high staff turnover. The Medical Tents were reported to reach very isolated areas of the country where there are no government facilities. In addition to outreach services, the IFPS pro￾ject voluntary FP clients through PSZ’s BlueStar network of social franchise providers. The reported quality and quantity of services provided by BlueStar Franchisees may not be what is expected by a private healthcare provider that serves a wealthier population. The number of FP clients is two or three per day and some fran￾chisee staff reported to be trained “on-the-job.” Some BlueStar franchise clinics seem more successful and reported that providing FP services attracts more clients. The charged prices for private FP services have gone down as a result of receiving free FP commodities and training through IFPS. 1.2. What design and implementation aspects of the project are not achieving intended results? IFPS is achieving the intended results for which it was designed. However, due to current design aspects, it is not providing continuous care or community based service delivery. IFPS was designed to complement the public provision of FP services and is not integrated into the FP program of the government facilities. Alt￾hough government health staff are appreciative of the contribution of PSZ/IFPS, they lack the training and ZIMBABWE IFPS EVALUATION VIII tools to follow up clients and their side effects in between visits of the IFPS outreach teams and do not seem to take ownership of clients served by IFPS. IFPS was designed to reach a certain number of clients and was not designed specifically to target unserved groups such as youth, males, and religious groups that do not come to the clinic on the day the outreach team comes. BlueStar Franchisees seem to reach those that afford the services in urban areas but are not designed to reach the urban poor. 1.3. What alternative approaches and strategies could achieve better results? Justify' proposed approaches and strategies. Key informants reported that the public health sector had improved its delivery of FP services through a “Results-Based Financing” activity funded by the World Bank and the Health Development Fund, a basket fund set up by a number of donors including DFID. Also, the ET found that there is a high rate of teen pregnancies and HIV infections and that the DREAMS program, with USAID funding, has been reported to be successful in reaching youth. Key informants also reported that the MOHCC with support from UNFPA has developed a new adolescent reproductive health program to meet the educational needs of youth nation￾wide. However, a service delivery program of FP to youth to meet their needs is not available at this time. 2. What are the major challenges faced by the IFPS activity in delivering comprehensive FP ser￾vices and how can USAID address these challenges? The main challenge is that there are only nine teams to serve the whole country and government clinics do not have the capacity to provide care in between visits of the teams. With a community-based approach and trained staff in the public sector, IFPS could deliver integrated FP services that meet the needs of every citi￾zen, being able to reach youth, and religious leaders and educate men about the benefits of FP services, but engaging public sector counterparts represents a critical challenge. 3. What are the major national FP challenges and priorities and how can USAID assist in address￾ing these challenges through the IFPS project or other future USAID FP support? The main chal￾lenge reported by most informants is that FP commodities are almost 100% dependent on donor support. Zimbabwe has a comprehensive FP policy and strategy called FP2020, but depends on the support of donors to implement it. The leadership of the MOHCC contributes by coordinating the effort but progress towards the objectives of FP2020 reported to be limited due to the lack of trained staff and funding to achieve these objectives. Another challenge is the lack of timely and reliable information to make management decisions due to the existence of a fragmented health information system that relies on numerous registers to keep track of the MOHCC programs, including FP. A new register for FP is being introduced that places more record keeping burden on government staff. IFPS has a computerized information system that could avoid that burden and empower staff to make informed decisions by having system that gathers patient as well as program data and provides reports to inform clinical and management decisions. The main consequence of these challenges is that the Zimbabwe public sector is simply unable to afford to continue providing FP ser￾vices to its growing population without donor support. 4. What are the perceptions of national and local stakeholders towards USAID FP support and why? How can negative perceptions be addressed? USAID’s FP support is perceived as indispensable and crucial for Zimbabwe to provide FP services to its people. There are some political leaders that seem to see this as western influence to limit population growth. However, the key informants interviewed at national, provincial and district levels all are aware of the health benefits of an effective FP program and of having a broad method mix. All informants had stories about pregnant adolescents that should not be and of women that have reached their desired family size and are not ZIMBABWE IFPS EVALUATION IX allowed to stop getting pregnant. All interviewed health care providers wish they could provide FP services to more of these women and girls. Interviewed clients and FGD participants valued having access to free FP services. USAID’s contribution is much appreciated. RECOMMENDATIONS 1. IFPS is a successful activity that has achieved its objectives and should continue providing free ser￾vices in rural areas through existing channels and with the comprehensive method mix as it does now. 2. IFPS should continue increasing awareness and specially target unserved groups, particularly rural poor and increase access to underserved groups: urban poor, youth, religious groups, people with disability and males. IFPS should design new channels to meet the needs of special target groups such as youth, peri-urban poor, religious groups, people with disability and men building on effective programs such as the DREAMS project that targets youth 3. IFPS should improve counseling on and management of side effects through effective CPI. IFPS should ensure that communication and client counseling to dispel myths and misconceptions is a continuous and ongoing process, particularly regarding LAPM 4. IFPS needs to be redesigned to include public sector nurses to be trained on FP, especially on LAPMs and management of side effects. This will involve integrating FP services in the existing clin￾ics, as well as integrating the Village Health Workers in reaching the unserved. 5. IFPS should advocate for change of the law that gives restrictions on access to FP information and services among school-going youth and address existing political barriers as well as. The Parliamen￾tary Portfolio on Health is a possible platform to leverage. 6. PSZ should facilitate a business assessment of the BlueStar SF channel and build on successful SF practices to develop a business and marketing model that improves the financial viability of the pri￾vate sector providers 7. USAID should consider conducting a study of the contribution of RFB and IFPS to overall CPR and analyze the lessons learned to contribute to advance FP 2020 in an integrated way. 8. USAID should continue supporting the FP program and the FP2020 strategy because it is perceived as an important and instrumental partner to sustain and expand FP services in Zimbabwe, targeting the hard to reach through existing and new channels. A sustainability plan that builds on existing health system structures and IFPS achievements should be part of the design of future FP activities. ZIMBABWE IFPS EVALUATION 1 INTRODUCTION AND CONTEXT The purpose of this report is to present the findings of an end-of-project performance evaluation of the USAID-supported Improved Family Planning Services (IFPS) Activity in Zimbabwe. IFPS is a five-year project implemented by Population Services Zimbabwe (PSZ), a Zimbabwean non-governmental organiza￾tion (NGO) affiliate member of Mary Stopes International (MSI). IFPS has two main objectives: 1. Increase the awareness of voluntary FP methods, particularly LAPMs by 20% from 17% among un￾derserved women and men aged 15-49 years in all provinces by 2017, and 2. Provide comprehensive voluntary FP serves to 589,506 poor and underserved women and men aged 15-49 through mobile clinical outreach and a social franchising network in all provinces by 2017 An evaluation work plan was submitted on December 30, 2016, revised after USAID’s briefing and approved by USAID/Zimbabwe (see annex D). ZIMBABWE IFPS EVALUATION 2 EVALUATION QUESTIONS AND METHODOLOGY The evaluation was organized to respond to a number of evaluation questions posed by USAID/Zimbabwe. Lessons from this evaluation helped inform efforts to fine tune and improve on the current activity and de￾sign new activities for improved effectiveness and efficiency towards reaching USAID/Zimbabwe's FP objec￾tives. 1.1 EVALUATION QUESTIONS The Evaluation Team (ET) addressed the evaluation questions and sub-questions about the performance of the IFPS activity, as follows: 1. How can the IFPS activity design and implementation be enhanced to increase efficiency and effectiveness towards meeting the project goal of reducing maternal mortality and morbidity? 1.1 What design and implementation aspects of the IFPS project are critical to achieving intended results? Why? 1.2. What design and implementation aspects of the project are not achieving intended results? 1.3. What alternative approaches and strategies could achieve better results? Justify' proposed approaches and strategies. 2. What are the major challenges faced by the IFPS activity in delivering comprehensive FP services and how can USAID address these challenges? 3. What are the major national FP challenges and priorities and how can USAID assist in addressing these challenges through the IFPS project or other future USAID FP support? 4. What are the perceptions of national and local stakeholders towards USAID FP support and why? How can negative perceptions be addressed? The evaluation questions were further refined to explore IFPS achievements in terms of access, demand, cov￾erage, quality, and sustainability based on discussions with USAID/Zimbabwe. 1.2 EVALUATION DESIGN, METHODOLOGY AND DATA LIMITATIONS Using a cross-sectional performance evaluation design, the ET examined current project implementation ap￾proaches, strategies, and activities and identified areas for improvement and ways to improve efficiency and effectiveness towards meeting Mission’s development objective 2: “Increased number of Zimbabweans live a longer and healthier life.” The ET made use of available activity performance data and secondary data from national health surveys and the MOHCC to assess direct project results and the project’s contribution to na￾tional results. The ET conducted a review of the national FP context with proposed key focus areas for fu￾ture USAID FP support. The ET employed a variety of quantitative and qualitative primary data collection methods as well as a de￾tailed review of available secondary data to generate the evidence required to answer the evaluation questions ZIMBABWE IFPS EVALUATION 3 adequately. The ET interviewed relevant individuals from the list of key stakeholders, who were identified in discussions with the PSZ team and USAID/Zimbabwe. The list of key informants is in Annex C. The data collection methods also included a desk review of all the project’s reports and documents developed by the project (Annex B). The ET also conducted 12 Focus Group Discussions with clients and non-users, visited BlueStar social franchise facilities, observed mobile outreach sessions, interviewed with healthcare providers, and reviewed client records and statistics. Available IFPS staff was interviewed in person or by email. There are a couple of limitations that were mitigated. This is a cross-sectional assessment of the progress achieved by the IFPS activity and may not reflect all project’s achievements so the ET reviewed all project documentation to analyze progress and changes throughout the life of the project. Second, the IFPS activity has worked in over 1200 sites and the ET visited less than 20 of these sites. However, the sampled sites were selected in discussions with the USAID/Zimbabwe and the IFPS team to ensure they were representative of all the sites and the challenges the project has faced. 1.3 DISTRICT SELECTION FOR EVALUATION SITE VISITS During the past six years, USAID support has increased FP service coverage through the present configura￾tion of nine outreach teams working in all provinces of Zimbabwe. In addition, the project has established a network of 50 BlueStar Social Franchise Network partners. The ET conducted site visits to a purposive sam￾ple of 5 districts, including a representative mix of urban, peri-urban and rural sites, selected in such a manner so as to accompany outreach teams as they conduct their previously scheduled visits, and also to visit BlueStar Social Franchisees. See Annex D for a detailed explanation of the sites. 1.4 DATA ANALYSIS Qualitative data from FGDs was analyzed according to the themes related to the evaluation questions. Quan￾titative data were analyzed using Excel and EpiInfo. The project’s monitoring data were analyzed using Excel to ascertain coverage of objective 2 of the project. ZIMBABWE IFPS EVALUATION 4 FINDINGS AND CONCLUSIONS In this section, we present the evaluation findings and conclusions by evaluation question. 1. How can the IFPS activity design and implementation be enhanced to increase efficiency and ef￾fectiveness towards meeting the project goal of reducing maternal mortality and morbidity? 1.1 What design and implementation aspects of the IFPS project are critical to achieving in￾tended results? Why? 1.2. What design and implementation aspects of the project are not achieving intended results? The findings related to these evaluation questions indicate that IFPS has a number of critical characteristics that make it effective at delivering FP services such as its nationwide coverage of free services and other de￾sign aspects that are not achieving the intended results in regards to quality and coverage as follows: 2.1 ACCESS TO FP SERVICES 2.1.1 Finding: As of December 2016, IFPS has provided services to 475,775 clients, and projections indicate it will reach its target of 589,506 clients by end of project (EOP) (table 1). IFPS provides these services in all 10 provinces through three main channels: 940 “outreach” sites, 259 medical tent sites and 49 BlueStar Social Franchise clinics. Outreach refers to mobile teams that visit rural government, that is public, clinic facilities to provide a complete method mix of FP services about every three months in average. Table 1. Total number of clients served by channel. Clients by channel and year CHANNEL Year 1 Year 2 Year 3 Year 4 Outreach 76,136 75,058 106,653 67,696 Medical tents 0 0 0 32,351 Social Franchise 14,501 18,777 50,450 33,153 Total 90,637 93,835 157,103 134,200 Conclusion: IFPS has increased access to FP services and is on track and will achieve the objectives for which it was designed. Table 1 also shows a decline in the number of clients in PY4 suggesting that it is get￾ting harder to find new clients through the current strategies and service delivery channels. 2.1.2 Finding: The comparison of data from various sources shows a decline in maternal mortality ratio and infant and under-five mortality rates with an increase in contraceptive prevalence rate (CPR) in the last 5 years (Table 2). ZIMBABWE IFPS EVALUATION 5 Conclusion: The current USAID FP strategy that includes IFPS seems to be having a positive impact on maternal and child health indicators at population level. Table 2. Trend of various FP related indicators over the last 20 years 1 Year Selected FP Indicators 1994 1999 2005-6 2010-11 2015 Total Fertility Rate (TFR) (Children/ woman) 4.3 4 3.8 4.1 4 Modern Contraceptive Prevalence Rate (CPR) (%) 42 50 58 57 66 Unmet Need (%) 19 13 10 Neonatal Mortality (%) 24 29 24 31 29 Infant Mortality Rate (IMR) (%) 53 65 60 57 50 Under-five Mortality Rate (%) 77 102 82 84 69 Maternal Mortality Ratio (MMR) (Deaths/100,000 live births) 578* 555* 612 960 651 Source: Zimbabwe Demographic and Health Survey 2.1.3 Finding: Clients report to value having free access to IFPS services: “We are also happy with the FP methods offered by PSZ because these services are offered for free. The teaching we got also helped us in dispelling the myths and misconceptions that were levelled against some methods.” FGD women clients “The service (by PSZ) is for free. We only pay a dollar for other administrative purposes at the clinic (to the government clinic) like payment of the guard.” FGD women clients Conclusion: IFPS free services attract clients in rural areas while government clients charge a fee to cover their costs. 2.1.4 Finding: The ET observed that 37% of interviewed clients had to travel more than 45 minutes. “Distance will not hinder us from getting FP services, if you are serious about FP you will come.” FGD women clients.” FGD women clients Conclusion: When clients are aware of the availability of FP service and their benefits, clients will make the effort to access them and even travel long distances. 2.1.5 Finding: Without IFPS, clients only can get services from government clinics, where only a few FP methods are available, mostly oral contraceptives, or at one of the 13 clinics managed by Zimbabwe National Family Planning Council (ZNFPC), and youth may not have access to all FP services as shown by this quote from one of the FGD: 1 Source: CSO and Macro International (1995, 2000, 2006) ZDHS; ZIMSTAT and ICF International (2012, 2016) ZDHS; ZIMSTAT (2016) ZIMBABWE IFPS EVALUATION 6 “Nurses at the clinic, if you are below 14 they deny you to receive other FP services but give you condoms.” Female youth FGD Conclusion: IFPS is providing services that no one else provides but youth are not reached consistently yet. 2.1.6 Finding: IFPS has increased access to those that come to the outreach clinics and tents. The project was designed to reach a total target of clients and has done that. The total target was based on historical data of those that come for services. The target has been increased in the life of the project in light of the high demand and unmet need. IFPS was designed to cover the whole country but it was not designed to have a denominator population to serve as a whole. Conclusion: This design aspect means that IFPS focuses on reaching its target. It was not designed to follow a population of women throughout their reproductive life and retain them as “clients for life” providing ser￾vices since teenagers to menopause and eventually, for mothers to bring their daughters. 2.2 COVERAGE OF IFPS SERVICES 2.2.1 Findings: Project statistics show that IFPS provides a full method mix of services, particularly long￾acting and permanent methods (LAPM), in all provinces, through fixed clinics, outreach teams at government clinics, tents and BlueStar Franchisees. Key Informants believe IFPS has reached many clients and has been instrumental for Zimbabwe to reach the high CPR it has, but it has not reached the hard to reach populations such as youth and religious groups yet. “The apostolic church namely the Johane Marange does discourage its congregants from FP services.” FGD Women Clients Conclusion: IFPS service delivery channels have expanded coverage of FP services, particularly LAPM, but has not reached the harder to reach yet 2.3 DEMAND 2.3.1 Findings: Despite having achieved a high CPR, there were FGD participants, health workers and most key informants reported that there still remain women who want FP services to space births, but their hus￾bands do not let them. Religious groups are reported to prevent women to access FP knowledge and services although some find ways to get them. “There has been an increase with various religious sectors in appreciating FP services with only a few Apostolic churches that are still discouraging their members from accessing FP services”. FGD Women clients “The three of us, we are from Johane Mugodhi church, we are not allowed to use FP methods but as a person, you make your own decision and come unseen to receive FP methods.” FGD Women Clients “We also belong to this church (Apostolic sect); we sneak to access FP services.” FGD Women Client “People are accepting the (FP) services because it helps us space our children.” FGD Non- User Conclusion: IFPS has helped achieve high CPR and is reaching to some religious groups but it was not de￾signed to reach special groups systematically. Involving men to support women’s FP choices might increase CPR even further. 2.3.2 Finding: IFPS clients report having knowledge to dispel myths and misconceptions. “The education we receive helps us to dismiss some myths and misconceptions, like Jadelle was associated with cancer and contin￾uous bleeding while depo was associated to barrenness after Depo use.” FGD Women clients ZIMBABWE IFPS EVALUATION 7 Conclusion: IFPS is helping address myths and misconceptions about FP methods. 2.3.3 Finding: IFPS has increased awareness of and demand for FP through various channels. IFPS has par￾ticipated in radio programs that reach over two million people; and has trained community mobilizers and over 500 “Choice Champions” that mobilize communities to come for FP services. Conclusion: IFPS has an effective community mobilization strategy that raises awareness of FP methods and explains how the project met its objectives 2.3.4 Finding: FGD participants reported side effects, particularly bleeding with Jadelle and misconceptions that need to be addressed by IFPS: “Some of the stories we tell each other in the communities cause us not to access these FP services, like some say if you get Jadelle when you want to have a baby the baby will be deformed.” FGD women clients “Some people are crying with Jadelle because they end up falling pregnant and they bleed continuously.” FGD Women clients “Men do not like Jadelle, most women have problems with their husbands because Jadelle may cause them to bleed which affects their sex life.” FGD Women client Conclusion: Counseling about side effects needs to be strengthened as well as active follow up of implant clients. 2.3.5 Finding: Key informants reported that FP education is not permitted in schools and youth have limited access leading to high HIV infection and teen pregnancy rate of 22%2 . Youth are not getting the information they need from their parents to make FP decisions: “Parents do not want us to use contraceptives but we decide on our own.” FGD female “Adults tell us to abstain.” FDG female “Adults do not want to hear about contraceptives.” FGD female “ My mother did not want me to use contraceptives but after I got pregnant she actually gave me.” FGD female “I think peer pressure hinders FP especially the use of condoms because someone may tell you about their good experiencing with￾out the condom and you end up wanting to try it too.” FGD Male Conclusion: The demand for FP services has increased but it is still limited by gender and normative rules that prevent youth, particularly young women from receiving FP information and services 2.4 QUALITY 2.4.1 Findings: IFPS has written quality standards for FP service delivery, and staff have job descriptions and are supervised regularly. Despite that, the ET observed some factors that affect the quality of services such as clients reporting having to wait long for services, up to 5 hours and some have been on a waitlist for a 2 Naomi, please insert source of this figure. ZIMBABWE IFPS EVALUATION 8 tubal ligation for several months. Regarding long-acting methods, we observed preference for implants as IUCDs are reported to be “invasive” and women do not feel comfortable to have them inserted in the cur￾rent settings with a provider they do not know. On the other hand, despite the high preference for implants, there is high rate of removal of Jadelle implants and nurses in government clinics report not to know how to manage side effects of Jadelle implants and do not know how to remove them after the IFPS team has left. Conclusion: There is need for IFPS to review the FP service delivery process from a one-time service to a continuous client-centered point of view that ensures continuous care and follow up. Having rapport and trust, clients would increase use of IUCDs and improve management of side effects. IFPS needs strengthen￾ing voluntary FP counselling skills so providers are able to address barriers to access of IUCD and also as a way to make sure clients receive information on potential side effects of Jadelle. 2.4.2 Finding: IFPS group counseling is in person, lasting 45 minutes to an hour, and content was observed to be consistent. We observed differences in delivery among staff, though. Some staff were more engaging and elicited more questions from their audience. Time of individual counseling was not measured but seemed to be brief in most cases. The ET did not observe counseling to respect privacy. However, table 3 shows that 12% of the clients interviewed after their appointment reported not having been told about side effects. May￾be they did and they did not remember, but the result is the same. Conclusion: Counseling on side effects needs improvement. Table 3. Percentage of clients that report being told about side effects in client exit interviews Told what to do if side effects Frequency Percent No 12 12.24% Yes 86 87.76% Total 98 100.00% 2.4.3 Finding: About half of the clients reported to have received about the right amount of information (table 4), 39% reported to have received too much and 6% too little. Table 4. Reported amount of information received on the day of the survey Amount of information received Frequency Percent Too little 6 6.12% Too much 39 39.80% About right 51 52.04% Don’t know 2 2.04% Total 98 100.00% ZIMBABWE IFPS EVALUATION 9 Conclusion: Client counseling is a continuous process and trying to give it all in one opportunity may be too much and not effective. A continuous approach to counseling may benefit IFPS. 2.4.4. Finding: Despite having been presented with all methods, clients remembered a few only (table 5). This may prevent helping clients make an informed decision and deserves further study. Table 5. Reported knowledge of FP methods learned on day of survey. What methods clients learned today Frequency Percent Female sterilization 54 55.10% Male sterilization 39 39.80% IUCDs 68 69.39% Implants 88 89.80% Injectable 63 64.29% OC 74 75.51% Male condoms 66 67.35% Female condoms 62 63.27% Emergency contraception 13 13.27% Breastfeeding 14 14.29% Other 2 2.04% 2.4.5. Finding: Table 6 shows that most of the clients had already decided what method they wanted sug￾gesting that they may have attended other FP group counseling sessions or other services. Also, 30% do not want another child which would make them a candidate for a permanent method. However, IFPS has only two trained providers that can do a TL in the whole country, and no client chose this method. It was interest￾ing to note that almost ¾ of the clients had a mobile phone which would make it easier for IFPS providers to follow up their clients after a procedure. ZIMBABWE IFPS EVALUATION 10 Table 6. Selected characteristics of clients interviewed on the day of the survey. Selected Characteristic Frequency Percent Already knew what method she wanted 84 85.71% Has a mobile phone 74 75.51% Wants to wait > 2 years for next baby 41 41.83% Does not want another child 30 30.61% Conclusions: Despite having and enforcing quality standards, there are some aspects of the IFPS service delivery that do not seem to be enhancing its effectiveness. The quality of group counseling is not consistent and there is lack of continuity in client care that would help clients choose permanent methods. There is lim￾ited personalized relationship between provider and client or provider-initiated follow up. Despite having access to clients’ mobile phone numbers, there is not enough staff time to provide a personalized follow up. 2.5 SUSTAINABILITY USAID/Zimbabwe is in search for more sustainable approaches toward FP programming and here are se￾lected findings related to the sustainability of IFPS services. 2.5.1 Findings: There are three important findings that influence the sustainability of IFPS. First, in the pub￾lic sector, despite some lack of political support, Zimbabwe has a comprehensive FP policy and a strategic plan and idle capacity to do more. For example, the infrastructure of ZNFPC has the capacity to do trainings, and deliver services with additional support. Second, PSZ is a Zimbabwean and well respected NGO affiliat￾ed with MSI with nationwide coverage. And thirdly, most FP commodities are provided by DfID and USAID. Conclusions: The investment of IFPS is likely to be sustained by PSZ and there is support in the public sec￾tor for expanding coverage. However, Zimbabwe is almost 100% reliant on donor support for commodities at this time and that concerns some key stakeholders. 2.5.2. Finding: IFPS has an information system that was started about a year ago. The system allows IFPS to track clients and possibly sustain a continuous program of care for each client. Figure 1. Is one of the screen￾shots of its report menus. ZIMBABWE IFPS EVALUATION 11 Figure 1. CLIC system. IFPS/PSZ client information system reports Conclusion: IFPS has an effective information system that allows it to track clients no matter where they are served and that produces effective management reports. 2.5.3 Finding: There is no requirement to develop a sustainability plan of IFPS activity in the cooperative agreement. Conclusion: Planning for sustainability was not part of the design of IFPS activity. 2.5.4. Finding: IFPS staff report working conditions are hard having to be on the road all the time. There is high staff turnover. Conclusion: More teams and smaller areas to cover would allow staff from the same area to work in the same area. 2.6 FINDINGS RELATED TO ALTERNATIVE APPROACHES AND STRATEGIES 1.3. What alternative approaches and strategies could achieve better results? Justify' proposed ap￾proaches and strategies. 2.6.1 Finding: On alternative approaches to increase access. Results-based financing (RBF) is reported to be effective in motivating the integrated delivery of FP services in public health sector rural facilities. In urban areas, city clinics run by local city councils provide FP services but charge a fee and this was reported by key informants to limit access to peri-urban poor populations. ZIMBABWE IFPS EVALUATION 12 Conclusion: There is need to expand free access to FP services to urban poor and to explore ways to com￾bine IFPS and the role of NGOs and the government in the provision of FP services into a coordinated na￾tional program. 2.6.2 Finding: The Village Health Worker (VHW) program is to substitute the “old” community-based dis￾tributors (CBD) of FP commodities and provide FP counseling. They are not part of the IFPS service deliv￾ery channels yet. Conclusion: The VHW program is reported to be an effective approach to reach the underserved and the poor in every village in Zimbabwe. The VHW program might be integrated as part of IFPS and its follow-on activities using a community-based approach to FP service delivery. 2.6.3 Finding: Teen pregnancy and HIV infection are high in 15-24 age group, and the DREAMS program is reported to be effective at addressing the social and cultural determinants of both. Conclusion: DREAMS may be a strategy to be integrated as part of the IFPS. This needs further research. 2.6.4 Finding: IFPS/PSZ are the only ones with trained staff in modern contraception. ZNPFC does not have the capacity to provide in-service training and pre-service training at undergraduate and graduate levels lacks courses in modern FP methods. Conclusion: There is need to establish a sustainable mechanism to ensure public and private FP service pro￾viders meet training and competence requirements. 2.6.5 Finding: BlueStar Franchisees report to need more marketing and business training to be viable as businesses. The ET observed very small number of clients (2 to 3 per day) in some BlueStar franchisees. Small numbers of clients can be the cause for these clinics not to be financially viable. They also seem to be outside of the national FP program as they report not to report their clients to the local health authorities but they do for EPI and other public health programs. Conclusion: Some franchisees may not be a viable vehicle to expand FP access. IFPS was not designed to figure out the role of the private sector in an integrated national FP program but to increase FP access, which it did. However, given the role BlueStar franchises have played in IFPS, it may be necessary to define their role so quality private provision of FP services is also reported to and supervised by the MOHCC as the im￾munization program and other public health programs. 2.7 CHALLENGES FACED BY IFPS 2. What are the major challenges faced by the IFPS activity in delivering comprehensive FP services and how can USAID address these challenges? 2.7.1 Finding: Lack of access by youth is one of the main challenges of IFPS. Key informants reported that teens (10 -19) are not allowed to receive information about FP in schools, and key informants also reported ZIMBABWE IFPS EVALUATION 13 that teens are not served in a timely manner in public health facilities. Teens cannot afford to use private facil￾ities. This limited access to FP to teens may explain reported high pregnancy and HIV infection. Conclusion: IFPS service channels were not designed to attract and reach youth. 2.7.2 Finding: Males are reported not be consistently involved yet. However, there is some initial evidence with Choice Champions and vasectomies are increasing. IFPS reported 23 vasectomies in the last two months. Conclusion: IFPS channels were designed to deliver FP services to mostly women who attend health cen￾ters. IFPS has looked into other approaches to involve and serve men as well as women but these have had limited success. 2.7.3 Finding: Staff working conditions are hard with high staff turnover and IFPS does not have a succes￾sion pool from which to recruit. Conclusion: There are gaps in service due to lack of trained staff. IFPS, and the whole country need a strate￾gy to produce qualified health care providers that can deliver quality FP services and find ways to monitor and improve working conditions. 2.8 FP CHALLENGES AND PRIORITIES 3. What are the major national FP challenges and priorities and how can USAID assist in addressing these challenges through the IFPS project or other future USAID FP support? 2.8.1 Finding: Political support for FP services is not homogeneous. IFPS was not designed to include a program to continuously educate leaders on the benefits of FP to the people and the nation as a whole. Conclusion: IFPS was not designed to have a public advocacy program. 2.8.2 Finding: Key informants in urban areas report that fees prevent access to FP services as municipal clin￾ics charge fees. Even one dollar maybe too much for urban poor. Conclusion: There is limited access to FP services to urban poor who cannot afford to pay 2.8.3 Finding: IFPS has 9 outreach teams; each covers one whole province (Bulawayo is covered by Matabe￾leland South team and Harare by Mashonaland provinces). Outreach teams visit government clinics every three months. The teams are not able to follow up clients from previous visits but focus on the clients they have that day. IFPS has a large coverage area and clients have limited access to their providers in between visits as they can only call the toll-free number and not get in touch with “their” provider. Conclusion: IFPS has an insufficient number of outreach teams to for the large coverage area. At least two teams in each province would be required to ensure continuous client follow up and support clients to stay in touch with “their” provider. A personalized approach and closer client provider interaction would improve quality of care. ZIMBABWE IFPS EVALUATION 14 2.8.4 Finding: IFPS serves mostly women and has developed ways to involve men but still most women re￾port to need permission from their husband. IFPS staff reported that the project has also integrated gender￾based violence (GBV) prevention and detection and referral activities. At this time, the GOZ only has two clinics where GBV victims get care. Conclusion: IFPS needs to increase male involvement and continue to expand its GBV activities. 2.8.5 Finding: Informants report that youth in schools are afraid to be seen coming to IFPS points of ser￾vice and most do not have access to FP until after they get pregnant. Conclusion: IFPS does not reach youth schools and who are not receiving FP services. 2.8.6. Finding: Adolescents aged 10-19 years who are out of school can only be reached through DREAMS or through youth centers. The MOHCC with UNFPA support have developed an Adolescent Reproductive Health Program that will be launched shortly but it is not part of IFPS yet. Out of school youth are not reached by IFPS. Those who come for FP services, are pregnant or have already had children. IFPS does not have an objective of reaching youth. Conclusion: IFPS does not have youth-specific activities; IFPS aims to reach clients of reproductive age, and youth fall within that group, but there aren’t clear strategies or activities to reach a subpopulation within the reproductive age such as youth besides the work with the DREAMS program. 2.8.7. Finding: FGDs showed that there is still an unmet need and demand for permanent methods. There is need to counsel on permanent methods. The project by the NGO Cordaid also reported that they gave vouchers and paid for TL but was not able to meet its targets either due to lack of acceptance of TL and Vas￾ectomies “You can do TL secretly without telling your husband because men think like children, he might wake up saying that he wants children, so if you tell him that you have TL, he might wake up marrying someone else”. (FDG women clients) “It’s okay for old women and those who feel they have had enough children”. “What will I do when I realize that I made a mistake and I need a child”. “What if all my children die?”. “What will other men say to me, they will say I am not a man”. “I wouldn’t want my husband to have vasectomy because I don’t think he will still perform the same in bed”. “Men will never go for vasectomy. How can one be called a man if he has no power to have children?” Conclusion: TLs and vasectomies are not accepted yet. Other projects have faced the same challenge and there is need to raise awareness and provide effective counseling to those that have reached desired family size so they can access LAPM. ZIMBABWE IFPS EVALUATION 15 2.9 PERCEPTIONS BY NATIONAL AND LOCAL STAKEHOLDERS 4. What are the perceptions of national and local stakeholders towards USAID FP support and why? How can negative perceptions be addressed? 2.9.1 Finding: Key informants report IFPS has significantly increased CPR and CYP in Zimbabwe and PSZ/IFPS is perceived as a very important partner of the MOHCC and ZNFPC Conclusion: IPFS has developed a very useful partnership with MOHCC and ZNFPC and achieved its ob￾jectives. The strategies used by IFPS to increase access to voluntary FP services have played a key role in in￾creasing CPR and CYPs in Zimbabwe 2.9. 2 Finding: IPFS is perceived as instrumental in providing nationwide FP coverage, particularly in long￾term and permanent methods in rural areas Conclusion: IFPS and USAID are perceived as important partners and their support is highly valued 2.9.3 Finding: IFPS has increased the demand for FP services but not reached some underserved groups such as, the urban poor and adolescents Conclusion: IFPS has done what it was designed to do and there is still work to achieve universal coverage by 2030. 2.9.4 Finding: IFPS has quality standards and SOPs and supervises quality. However, quality of follow up is reported not to be consistent and there is concern for the high rate of removal of Jadelle. Conclusion: IFPS quality of FP services is good but client-provider interaction (CPI) and follow up need improvement. 2.9.5. Finding: Sustainability as a strategy “Towards more sustainable FP interventions and programs” was not part of IFPS design. However, PSZ is perceived as a good Zimbabwean NGO that will sustain achieve￾ments. Conclusion: Investing in sustaining PSZ might sustain FP services 2.9.6 Finding: Informants are concerned by the lack of commodity security in an FP program that is almost 100% donor funded and wonder if Zimbabwe has the capacity to meet the objectives of FP2020 Conclusion: Predictability of partners’ contribution to support FP commodities would help reduce concerns about the future of FP2020. 2.9.7 Finding: Informants perceive USAID as a very important partner without which Zimbabwe would not have the CPR it has. No negative perceptions were reported. ZIMBABWE IFPS EVALUATION 16 Conclusion: USAID has the opportunity to build on its achievements and influential role to take its FP pro￾gram further. 2.9.8 Finding: The sustainability of the IFPS program may be affected by the lack of support by policy mak￾ers and leaders. On February 15, there was an article about a motion made by the Mashonaland West Senator, Mike Byton Musaka calling for a plan to incentivize large families, at least eight children per woman, as a strategy to grow Zimbabwe’s population and attract foreign investment. The article ignores the impact on the health system and the high cost of the healthcare needs this motion would have and the impact on women’s health and mortality if proper birth spacing is not followed as well. Conclusion: This article indicates the need of a continuous program to educate leaders on the need of effec￾tive maternal and FP services and on the impact of the IFPS project. ZIMBABWE IFPS EVALUATION 17 RECOMMENDATIONS 1. IFPS is a successful project that has increased access and demand for FP services and should be con￾tinue. PSZ should continue providing free services in rural areas through existing channels and with all the complete mix as it does now. In addition, PSZ should consider increasing the number of teams and their coverage to allow teams to follow up clients in between visits, particularly those who have received an implant. 2. IFPS should continue increasing awareness and specially serving the unserved groups, particularly the rural poor and increase access to undeserved groups: urban poor, youth, religious groups, people with disabilities and males. IFPS should develop and test new strategies to reach these unserved groups, including provision of public sector support by building the capacity of VHWs and govern￾ment clinic staff as part of an integrated FP program that is owned by the local clinic and health au￾thorities. Integrating the lessons learned of others projects such, such as the DREAMS project that targets youth, and integration with other public health programs such HIV, TB, MNCH and EPI that have nationwide coverage would allow the delivery of an integrated package of services to those that need it when and where they need it. 3. IFPS healthcare delivery model should be redesigned to improve counseling on and management of side effects through effective and continuous CPI. The new delivery model should strive to reduce the high rate of removals though effective counseling and management of side effects. By developing rapport and trust through effective client provider interactions, IFPS staff should be able to support clients and help manage side effects as well as increase demand for IUCDs and permanent methods. IFPS should continue expanding its communication and client counseling to dispel myths and mis￾conceptions. This must be a continuous and ongoing process, particularly about LAPM. 4. A community-based approach to planning and delivering FP services would allow IFPS to reach eve￾ryone in every village with the correct information about the benefits of FP and schedule services. This approach should be considered in order to expand coverage to those that are not served and reach local political and religious leaders and inform them about the benefits of FP and advocate for more government support. 5. IFPS should also be redesigned to integrate the role of public sector nurses and other staff in gov￾ernment clinics. Public health staff needs to be trained on modern FP methods, especially on LAPMs and government clinics need to be provided with supplies to deliver services and follow up clients in between visits of outreach teams. This will involve integrating FP services in the existing clinics, and integrating the role of VHW in managing the demand for FP services. 6. IFPS should consider using the Parliamentary Portfolio on Health as a platform to address the politi￾cal barriers to access to FP services as well as advocate for the benefits of FP and for change of the law that gives restrictions on access to FP information and services among school-going youth. Ad￾dressing the causes of high HIV and pregnancy in teens and the related high mortality should be a priority to leaders. 7. PSZ should facilitate a business assessment of the BlueStar SF channel and build on successful SF practices to develop a business and marketing model that improves the financial viability of the pri- ZIMBABWE IFPS EVALUATION 18 vate sector providers. The definition of the role of quality private provision of FP services needs to be defined, particularly in cities, where they should be under the supervision of the city council and report to the local health authorities as they do for other public health programs. 8. USAID should consider studying the contribution of RFB towards FP coverage objectives and the complementary role IFPS where the private nonprofit and for profit provision of FP services con￾tribute to the country’s overall CPR and analyze the lessons learned to contribute to advance FP 2020 9. USAID should continue supporting the FP program and FP2020 because it has demonstrated and is perceived as important and instrumental partner to sustain and expand FP services in Zimbabwe, targeting the hard to reach through new specially designed channels. A sustainability plan should be included in the design of future FP activities in order to build on existing health system structures and IFPS achievements. ZIMBABWE IFPS EVALUATION 20 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 Tel: (202) 712-0000 Fax: (202) 216-3524 www.usaid.gov