INTEGRATED FAMILY PLANNING PROGRAM MIDTERM PERFORMANCE EVALUATION MAY 17, 2019 This publication was produced for review by the United States Agency for International Development. It was prepared by the Mozambique Monitoring and Evaluation Mechanism and Services Project (MMEMS) for Management Systems International (MSI), A Tetra Tech Company. INTEGRATED FAMILY PLANNING PROGRAM Midterm Performance Evaluation May 17, 2019 Contracted under AID-656-C-17-00002 Mozambique Monitoring and Evaluation Mechanism and Services (MMEMS) Project DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | i CONTENTS ACKNOWLEDGMENTS .............................................................................................II ACRONYMS...................................................................................................................III EXECUTIVE SUMMARY........................................................................................ 1 EVALUATION PURPOSE AND EVALUATION QUESTIONS .................. 9 2.1 EVALUATION PURPOSE .............................................................................................9 2.2 EVALUATION QUESTIONS (EQS) .............................................................................9 BACKGROUND ..................................................................................................... 11 3.1 HEALTH SYSTEM IN MOZAMBIQUE .......................................................................11 3.2 IFPP IN MOZAMBIQUE..............................................................................................11 EVALUATION METHODS AND LIMITATIONS ......................................... 14 4.1 DATA COLLECTION METHODOLOGY AND ANALYSIS ....................................14 4.2 LIMITATIONS .............................................................................................................19 FINDINGS AND CONCLUSIONS.................................................................... 20 5.1 PROFILE OF SURVEY RESPONDENTS .....................................................................20 5.2 EVALUATION QUESTION 1: DEMAND CREATION.............................................22 5.3 EVALUATION QUESTION 2: ACCESS AND QUALITY .........................................35 5.4 EVALUATION QUESTION 3: COMMODITY MANAGEMENT..............................46 5.5 EVALUATION QUESTION 4: SUSTAINABILITY .....................................................49 RECOMMENDATIONS........................................................................................ 52 ANNEX A: FINDINGS AND CONCLUSIONS MATRIX............................ 56 ANNEX B: EVALUATION STATEMENT OF WORK................................. 74 ANNEX C: GETTING TO ANSWERS MATRIX ........................................... 88 ANNEX D: DATA COLLECTION INSTRUMENTS..................................... 90 ANNEX E: WORK PLAN................................................................................... 126 ANNEX F: KAP SURVEY METHODOLOGY............................................... 139 ANNEX G: SOURCES OF INFORMATION................................................. 143 ANNEX H: LIST OF INTERVIEWS ................................................................ 145 ANNEX I: COMPLETE FOCUS GROUP LIST ............................................ 152 ANNEX J: QUANTITATIVE DATA RESULTS............................................ 154 ANNEX K: DEFINITIONS AND SYNTAX FOR CALCULATION OF DEMAND AND NEED FOR FAMILY PLANNING............................................ 183 ANNEX L: DISCLOSURE OF CONFLICTS OF INTEREST .................... 193 ANNEX M: REFERENCE FOR ADS CHAPTER 201................................... 200 ii | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ACKNOWLEDGMENTS The Mozambique Monitoring and Evaluation Mechanism and Services (MMEMS) Project would like to thank the Integrated Family Planning Program (IFPP) and USAID for their support during the conceptualization, planning and execution phases of this evaluation. This report would not have been possible without their help and dedication. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | iii ACRONYMS APE Agente Polivalente Elementar – Ministry of Health Community Health Worker CBO Community-Based Organization CDCS Country Development Cooperation Strategy CHW Community Health Worker CMAM Central de Medicamentos e Artigos Médicos – National Drugs, Commodities and Supplies Warehouse COC Combined Oral Contraceptive Pill CPR Contraceptive Prevalence Rate CYP Couple-Years of Protection DFID United Kingdom Department for International Development DHS Demographic Health Survey DPS Direcção Provincial de Saúde – Provincial Health Directorate DQA Data Quality Assessment EA Enumeration Area EQ Evaluation Question FCR Findings, Conclusions and Recommendations FGD Focus Group Discussion FP Family Planning FP/RH Family Planning/Reproductive Health FTP First-Time Parent GHSC-PSM USAID Global Health Supply Chain Program-Procurement and Supply Management GRM Government of the Republic of Mozambique HC Health Center HF Health Facility HH Household HMIS Health Management Information System HoH Head of Household HRH Human Resource for Health ICAP International Center for AIDS Care and Treatment Program IEC Information Education Communication IFPP Integrated Family Planning Program IHO Integrated Health Office IMASIDA Inquérito de Indicadores de Imunização, Malária e HIV/SIDA em Moçambique – National Malaria and HIV Indicator Survey INE Instituto Nacional de Estatística – National Institute of Statistics IP Implementing Partner IPC Interpersonal Communication Agent IUD Intrauterine Device KAP Knowledge, Attitude and Practices KII Key Informant Interview LAM Lactational Amenorrhea Method LARC Long-Acting Reversible Contraceptives LTM Long-Term Method MCH Mother and Child Health mCPR Modern Contraceptive Prevalence Rate M&E Monitoring and Evaluation MMEMS Mozambique Monitoring and Evaluation Mechanism and Services MNCH Maternal Newborn and Child Health iv | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV MOEHD Ministry of Education and Human Development MOH Ministry of Health MSI Management Systems International NGO Non-Governmental Organization NHS National Health Services OVC Orphans and Vulnerable Children PAC Post-Abortion Care PHC Primary Health Care POP Progestogen-Only Pill PSI Population Services International RH Reproductive Health SBCC Social and Behavioral Change Communication SDP Service Delivery Point SDSMAS Serviço Distrital de Saúde Mulher e Acção Social – District and City Health Directorate SES Socioeconomic Status SI Strategic Information SIGLUS Sistema de Informação de Gestão Logística das Unidades Sanitária – Logistical Inventory Management System for Health Units SOW Scope of Work TA Technical Assistance TBA Traditional Birth Attendant TFR Total Fertility Rate TWG Technical Working Group UATAF Unidade de Assistência Técnica para a Educação Funcional – Technical Assistance Unit for Functional Education, a local NGO UNDP United Nations Development Program UNFPA United Nations Fund for Population Activities UNICEF United Nations Children’s Fund USAID United States Agency for International Development USG United States Government WB World Bank USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 1 EXECUTIVE SUMMARY This report summarizes the midterm performance evaluation of the Integrated Family Planning Program (IFPP) from its inception through March 2019. Completed by the Mozambique Monitoring and Evaluation Mechanism and Services project (MMEMS), the report describes the findings and conclusions of the evaluation team’s desk review of key documents and field research and offers recommendations to improve activity performance. EVALUATION PURPOSE The purpose of this midterm performance evaluation is to determine how well IFPP is meeting its intended objectives halfway through its implementation period and to make recommendations on approaches and management for the final years of the activity. The evaluation’s findings will be used to (1) contribute to learning that will have a positive impact on activity implementation and (2) supply timely feedback that will inform the design of future USAID/Mozambique activities. The intended audience for this evaluation is primarily USAID/Mozambique. Other key interested parties include Mozambique’s Ministry of Health (MOH) and other donors with an interest in health activities. ACTIVITY BACKGROUND IFPP is a $34,560,000, five-year USAID/Mozambique-funded initiative to increase use of modern contraceptive methods by target populations in all 36 districts in Nampula and Sofala provinces, led by Pathfinder International with a team of global and local partners – N’weti, Population Services International (PSI), and Abt Associates. The overall goal of the IFPP activity is to increase the use of modern contraceptive methods by target populations. Its aim is to reach an additional 565,000 new contraceptive users (“initiators”1) and spur a 15-percentage point increase in the Modern Contraceptive Prevalence Rate (mCPR), as well as offer diverse and effective contraceptive methods in intervention areas. EVALUATION DESIGN AND METHODS The evaluation methodology consisted of three phases, including a thorough review of IFPP documentation; field data collection of quantitative and qualitative data; and analysis and reporting. The evaluation team collected data in Maputo (Mozambique’s capital) and Nampula, one of IFPP’s two provinces. 2 The quantitative field team visited 16 districts in Nampula and surveyed 816 women ages 15 to 49. The areas in which the survey took place (enumeration areas) are the same ones that were covered in Nampula during the baseline study conducted by IFPP in 2017. MMEMS only analyzed data from Nampula to compare baseline and midline survey results. The qualitative team visited 3 of Nampula’s districts and the Nampula provincial capital to conduct 22 Key Informant Interviews (KIIs) and 13 group interviews with 50 participants across a variety of categories, including IFPP staff, USAID/Mozambique, MOH, other donor-funded programs, and community leaders. The evaluation team also visited 6 health facilities and 4 supply warehouses. Finally, the team conducted 22 Focus Group Discussions (FGDs) with 185 male and female adolescents and adults. Women accounted for 52% of interviewees and 54% of FGD participants. After fieldwork was completed, the evaluation team analyzed the data to triangulate findings by reviewing, refining and comparing quantitative and qualitative results. 1 Initiators are defined as clients who begin using a contraceptive from a Service Delivery Point. 2 The baseline evaluation included data from Nampula and Sofala. The evaluation team initiated midline data collection in both provinces. However, Cyclone Idai hit as midline data collection was beginning. With USAID’s approval, data collection was suspended and efforts were refocused solely on Nampula province. The survey results are representative of the target population in IFPP’s target areas in Nampula. 2 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV MMEMS conducted an initial briefing with USAID to discuss preliminary results; a validation workshop with IFPP to review findings and conclusions; and a final outbrief presentation with USAID to disseminate evaluation results. FINDINGS AND CONCLUSIONS EVALUATION QUESTION 1: DEMAND CREATION To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? While this study cannot definitively attribute increased demand to specific IFPP family planning interventions, the evaluation team observed that knowledge increased, demand and unmet need increased, demand satisfied and met need decreased, utilization rates remained stagnant, and the community environment improved.3 Survey results indicate an increase in the target population’s knowledge of any contraceptive method from 80% at baseline to 92% at midline. The largest percentage gains in knowledge were for long-acting reversible contraception (LARC), which includes implants (35% to 59%) and injectables (66% to 84%). Knowledge of pills and male condoms also increased (63% to 78% and 48% to 62%, respectively). However, knowledge of some alternative methods, like male sterilization, LAM4 and emergency contraception remain low (9%, 7% and 6%, respectively) at midline. Qualitative evidence drawn from KIIs and FGDs also show that knowledge has increased. Demand for FP services increased from 43% at baseline to 56% at midline. Unmet need for FP services also increased from 23% at baseline to 39% at midline. Demand satisfied decreased from 47% to 32%, and met need decreased from 20% to 17%. The mCPR – the proportion of women ages 15 to 49 who are using (or whose partners are using) a modern method of contraception – has remained stagnant at 17% at midline. 5 Qualitative evidence drawn from KIIs and FGDs also show that demand has increased. IFPP improved the community environment in relation to female empowerment, male acceptance of family planning, and the reduction of stigma related to contraceptive use through the deployment of mobile brigades, youth couples counseling, Community Health Workers (CHWs), and home visits. Acceptance of family planning among Heads of Household (HoHs, 81% of whom are male) increased from 53% at baseline to 65% at midline. In addition, joint decision making between couples increased from 37% at baseline to 44% at midline. However, 24% of men still make all family planning decisions without input from their partner. Female confidence in making family planning decisions also improved. Respondents who feel confident in their ability to suggest to their husbands to wait a few months before having another baby increased, from 44% at baseline to 63% at midline. Confidence in suggesting family planning methods increased from 47% to 60%. Confidence in obtaining the method respondents want to use increased from 41% to 59%. Qualitative evidence drawn from KIIs and FGDs also show that acceptance and confidence has increased. 3 The technical terms Demand, Unmet Need, Demand Satisfied, and Met Need have precise definitions to ensure accurate measurement. These terms are defined in Annex K: Definitions and Syntax for Calculation of Demand and Need for Family Planning. 4 Lactational Amenorrhea Method, or breastfeeding as birth control. 5 Compared to 20% at baseline. The 3% decrease is not statistically significant. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 3 Finally, IFPP deployed innovative approaches to improve family planning service delivery, spur demand creation and support the health system. Innovations include deeper integration of family planning with existing health services; improved training for all facility-based health providers; improving the privacy of adolescent corners at health facilities6; Agente Polivalente Elementar (APE) home visits; and deployment of mobile brigades in rural areas and secondary schools. EVALUATION QUESTION 2: ACCESS AND QUALITY To what degree have IFPP interventions increased access and quality of FP/RH services? Access to family planning service providers increased. Of the 51% of women respondents who went to a health appointment in the past 12 months (up from 35% at baseline), 67% of them discussed family planning at midline compared to 57% at baseline. IFPP eliminated the urban-rural gap in respondents who discussed family planning in a health facility, from a 35% gap in favor of urban respondents at baseline to -5% gap in favor of rural respondents7 at midline). Women mostly attend medical appointments for Triage (appointment with general practitioner, 25%) and Child Health Consultations (25%), which may indicate increased integration of family planning into existing health services. Access to referrals and certain types of family planning messaging (radio broadcasts and participation in community events about family planning) decreased or remained stagnant. According to IFPP data, IFPP referrals in Nampula increased from 38,424 in Year 1 to 80,076 in Year 2. However, survey data indicate that the percentage of respondents referred by CHWs decreased, from 94% to 74%, possibly because IFPP reduced their emphasis on referrals from Interpersonal Communication Agents (IPCs) in order to focus on return visits to target households. In addition, participation in community events in the past 12 months where FP was discussed remained static – 14% at baseline and 17% at midline. 8 Respondents that heard a radio program discussing family planning in the past 6 months also decreased, from 32% at baseline to 25% at midline. This may be reflective of broader trends in the population rather than because of IFPP intervention, as there was a corresponding decrease in radio ownership among respondents between baseline and midline, from 38% to 23%. Access to accurate information about family planning increased. At midline, 76% of respondents were informed by health providers about the side effects or complications of contraceptive methods, compared to 40% at baseline. Seventy-one percent of respondents at midline are informed about what to do if side effects occur, compared to 40% at baseline. Survey respondents were also better able to discern between true and false statements relating to myths about contraceptive methods. Respondents improved the frequency of correctly identifying misleading statements, like “men can use a condom more than once” (21% increase from baseline to midline) and “the pill is effective even if a woman forgets to take it” (14% increase). Qualitative data collected suggest that the quality of family planning services is good and that the perceived climate at health facilities targeted by IFPP has improved and is more respectful and client￾friendly. Seventeen out of 22 focus groups report that the quality of family planning services is good, and 6 Adolescent corners are not new to Mozambique, but the project has adapted them in innovative ways. This evaluation notes that adolescent corners in Nampula are generally seen as positive and helpful, according to youth and health facility staff interviewed (the evaluation team conducted 11 focus groups with male and female adolescents - 36 female and 30 male, representing 50% of all focus groups conducted and 36% of all participants in focus groups. They were asked to provide different reasons for good/poor access to contraceptives specifically for young people.). Interviewees generally believe that private spaces for youth have decreased the stigma of acquiring contraceptives. The 15-19 age cohort was also the only group to see contraceptive use increase between baseline and midline. However, IFPP and USAID should be aware that a large study on this topic concludes that service spaces designated for youth are not an effective way to increase mCPR among youth. 7 -5% signifies that at midline, rural respondents were actually more likely to discuss family planning at health appointments than urban respondents. 8 This percentage change is not statistically significant. 4 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV that patients are treated with friendliness and respect and get the services they need free of charge. Community Scorecards9 show that health facilities in Nampula scored best on access to FP services, freedom to choose FP methods, availability of FP methods, and information about advantages and disadvantages of certain FP methods. Health facilities scored worst on healthcare worker punctuality (8/8 health facilities scored “bad”); charging free services (3/8 “bad”); and waiting time (7/8 “bad”). However, qualitative data reveal a different perspective on wait times and free services – the preponderance KIIs and FGDs show that services are free of charge and wait times are low.10 Despite the fact that the use of any modern contraceptive method has increased among the 15 to 19 age cohort, there continue to be significant barriers to reaching youth with family planning services. Restrictive school policies, parental pushback, health facility limitations, community stigma and cultural norms reduce the effectiveness and reach of family planning services for adolescents. (1) Parents quashed IFPP’s initiative to bring more long-term contraception methods into schools. In Year 1, IFPP, in conjunction with MOH and MOEHD, made amendments to legislation regarding sexual and reproductive health, thus creating an enabling environment at schools to support long-term FP service provision. However, this legislation was later reversed by the MOH and MOEHD after parents pushed back. (2) Interviews with adolescents show that while there have been improvements in friendliness and youth acceptance at health facilities, some local facilities continue to lack private adolescent corners. In addition, some adolescents in FGDs signaled that service continues to be unfriendly. (3) Fifty-five percent of young women ages 15 to 19 consider distance to the health facility to be a “big problem”. Fear of being served by a male health provider increased from 15% at baseline to 30% at midline. This is not a particular preoccupation of young women, but rather of women in the lowest two quintiles of income (34%). Fear of being treated badly by a health provider when asking for contraceptives increased from 19% at baseline to 37% at midline. Finally, fear that the health facility will not have the desired contraceptive available increased from 17% at baseline to 37% at midline. (4) Qualitative data suggest that community norms continue to favor early child marriage and encourage childbearing at a young age. Parents harbor misconceptions about modern contraceptives and discourage their children from using them. In addition, adolescents do not want to be seen by community members at health facilities because they fear being labeled promiscuous. EVALUATION QUESTION 3: COMMODITY MANAGEMENT To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? IFPP is relatively effective at managing commodities at service delivery points despite stockouts11 of injectables. The forecasting and quantity of commodities received is adequate and stable for all modern contraceptives except for injectables. Quantitative data from the Sistema de Informação de Gestão 9 Community Scorecards are tools used by communities and providers to assess the quality of family planning services. Community Scorecards are operational in 14 out of 385 health facilities (4% of facilities, 8 in Nampula and 6 in Sofala). 10 One FGD with women in Nacala complained that women had to pay for implant removal. 11 Stockout is defined as the lack of availability of any contraceptive method that the health facility is expected to provide. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 5 Logística das Unidades Sanitária (SIGLUS) 12 show a minor increase in stockout levels of all commodities except injectables, from 0% to 3% in Q3 of 2018 to 4% to 9% in Q1 of 2019. Stockout rates for injectables are much higher. Due to a national shortage of injectables, stockouts of at least one commodity have increased from 8% during Q3 of 2018, to 23% during Q4 of 2018, and finally to 39% from January-March (Q1) of 2019. This shortage beyond IFPP’s control has supposedly been alleviated in late April 2019 but continues to cause local-level stockouts across Nampula province. IFPP combines a patchwork of solutions – technical assistance, SIGLUS rollout assistance, stock verification, and transportation – to improve the short-term functioning of the current commodities delivery and warehousing system. IFPP is not responsible for the overall functioning of this system and has performed well within parameters established in agreement MOH and the Global Health Supply Chain Procurement and Supply Management (GHSC-PSM) project. In health facilities where SIGLUS is used, interviews with warehouse staff and service providers show that the tool is intuitive and helpful at maintaining stocks of family planning commodities. However, it is a duplicative and inefficient system. IFPP continues to use physical paper stock cards and requisition forms, and costly onsite verification is required to confirm commodity stockpiles. 13 Finally, staff do not collect accurate data concerning the duration of stockouts experienced by health facilities because they do not know how to input these data into SIGLUS. 14 EVALUATION QUESTION 4: SUSTAINABILITY How has the project ensured that there is local ownership of interventions that have been introduced? What is the likelihood of project achievements continuing after the project ends? The project’s efforts to align and blend with MOH plans and activities since inception improve the chances of transitioning project activities to local ownership after the project ends, even though IFPP has no written transition document describing how this would be accomplished. The likelihood of project achievements continuing after the project ends depends to a large extent on the size of the local budget, political commitment, technical knowledge, and the nature of different IFPP interventions. While technical knowledge and political commitment at MOH exist, there is no budget allocated to continue most IFPP tasks. The contraceptive supply chain can only continue with donor support. In addition, the rollout of SIGLUS and commodity stock verification and data quality assurance are imperiled if MOH does not assume responsibility for these tasks after project closedown. Service providers have received long-term benefits from training and learning materials, but additional training and supervision may be discontinued. Finally, some community-level visits from CHWs will come to an end, but the evaluation team is cautiously optimistic that knowledge and cultural changes will remain. Demand generation interventions have led to significant gains in knowledge and positive cultural changes. These changes are likely sustainable. However, many demand-generation activities will discontinue in the absence of MOH commitment to sustain them. IPC visits, youth couples counseling 12 SIGLUS is the Logistics Management Information System for Health Units in Mozambique. It is a digital commodity stock management system used by the MOH to track and deliver commodities from warehouses to health facilities across the country. IFPP is helping to implement this system and train staff to operate it in 140 facilities in Nampula province. 13 Interviews with MOH, Medical Supply Warehouse Managers, IFPP and GHSC-PSM. 14 Understanding the length of facility stockouts is a critical indicator for the success of the commodity management system that is not currently captured in SIGLUS data. 6 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV sessions and mobile brigades are not likely to continue. Messaging tools like radio broadcasts and printed FP materials will also stop without further funding. APE activities will likely continue because they are funded by the United Nations Children’s Fund (UNICEF), and Traditional Birth Attendant (TBA) home visits could likely continue because of the small amount of money needed to sustain these activities. RECOMMENDATIONS MMEMS developed the following recommendations that should be considered by IFPP and USAID to improve performance and enhance the sustainability prospects of project efforts. 1.1.1 DEMAND CREATION • IFPP should increase contraceptive utilization rates through intensified efforts in Nampula province. The youth cohort has shown the strongest growth in contraceptive use from baseline to midline. This trend may continue if the project remains focused on their needs. The project should consider other ways to further deepen the integration of health services to attract women from many different groups, especially poor and rural women. • IFPP should continue and intensify mobile brigades, youth couples counseling, CHW deployments, and IPC home visits to reach more adolescents and men in both rural and urban areas. IFPP should disaggregate the CHW indicator (which tracks visits to beneficiaries) to show the individual contribution of IPCs. • Evidence shows that knowledge has increased, but this has not always translated into increased contraceptive use. IFPP should assess whether efforts can be refocused on confronting barriers that could be blocking women from using family planning methods. A possible area of focus could be the 24% of men who exclusively make household family planning decisions. Designing a strategy focused on convincing these men to loosen their grip on FP decision-making could help to spur more women to seek out family planning services. • IFPP should assess the extent to which community platforms like churches, mosques, political parties (including women’s leagues), and sports clubs could be leveraged to drive demand for family planning services and increase community acceptance of family planning. 1.1.2 ACCESS AND QUALITY • IFPP should develop specific activities that engage parents of adolescents as change agents for increasing the demand and use of contraceptives among adolescents. Consider collaboration with other USAID-funded activities, like the Unidade de Assistência Técnica para a Educação Funcional (UATAF) “Strengthening School Council Capacity for Sustainable Organizational Development” project, to work in conjunction with school councils, community leaders, parents and teachers to mobilize and sensitize family planning in schools. IFPP should also consider whether religious leaders, matronas15 and extended family members have a role to play in increasing parents’ acceptance of contraceptive use. • Use marketing materials to reinforce the fact that contraceptives are free. 15 Matronas perform the same functions as the Traditional Birth Attendants (TBAs), similar to a midwife. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 7 • Increase the use of Community Scorecards to assess health facility performance. Reinforce the service-oriented role of health facility staff and the need to arrive on time to attend to clients. • Continue efforts to de-stigmatize family planning. Deeply held socio-cultural beliefs remain persistent and are a major barrier to overall acceptance, access, and use of modern contraceptive methods. IFPP should continue to build on the successes of mobile brigades, IPC agents, and Community-Based Organizations (CBOs) to reduce stigma related to family planning and build community trust, promote open discussion and encourage uptake of family planning services. • IFPP should explore ways to maintain the clinical skills of trained staff working in small health centers where client case load is low. One such method is called “visitas de troca de experiencia”, in which staff could rotate to high case load facilities for a finite period to increase exposure to more clients who need implants, sterilization, and other minor surgical procedures. • Explore efforts to increase access for young, poor, rural and uneducated women. This is the demographic that lags behind the average across most indicators related to contraceptive access. Efforts may include additional home-based services through TBAs and other community members, as well as a pilot deployment of male CHWs to further increase acceptance of FP methods among the predominantly male Head of Household (HoH) population. Expansion of private adolescent corners could also help to increase access for young people, although the project should further study the overall effectiveness of adolescent corners prior to any buildout of these services. • Assess whether the referral system, radio broadcasts, and community events are effective methods for increasing contraceptive access young women. For example, radio ownership is decreasing while television ownership is increasing, so a shift to more modern messaging techniques may be necessary. 1.1.3 COMMODITY MANAGEMENT • IFPP should collaborate directly with MOH and GHSC-PSM improve the overall functionality and utility of the SIGLUS system. IFPP should jointly plan a workshop to discuss several important issues with key partners including: (1) a plan for the rapid transition of all health facilities from Comm-Care to SIGLUS; (2) specific SIGLUS performance enhancements to improve system utility; (3) a plan to transition to a digital system without reliance on on-site verification of paper stock cards and requisition forms; and (4) development of a risk mitigation plan to reduce the likelihood of future injectables stockouts due to failed international shipments of critical supplies. • IFPP should undertake additional training for SIGLUS users. Specifically, SIGLUS users must be trained to accurately input stockout start and end dates. Without these data, stockout durations cannot be calculated and the severity of stockouts cannot be determined. 1.1.4 SUSTAINABILITY • Develop a transition plan with MOH. This should begin immediately, as the execution of the transition plan will take significant time and effort. The plan should also address the role local communities will play throughout the transition process and after closedown. • Search for other sources of funding to continue trainings, CHW visits, and effective messaging strategies after the project ends. IFPP should consider coordinating with the MOH through the FP Technical Working Group (TWG) to advocate for a budget dedicated solely to family planning. In addition, IFPP could develop an advocacy toolkit to support resource mobilization towards FP programming post-2020. 8 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 1.1.5 USAID • USAID should closely monitor the following IFPP indicators – mCPR, unmet need, meet need, demand satisfied and demand – to ensure that indicator targets are being met. USAID should examine underlying supporting data contributing to these indicators to further understand how the project defines, monitors and calculates them. A data quality assessment (DQA) could help to shed light on the discrepancies observed between IFPP data and MMEMS survey data concerning the usage, demand and need for family planning services. • IFPP deployed the mCENAS and Connecting with Sarah mobile applications. According to IFPP, these have been successful. However, due to a lack of data about usage of these platforms, the evaluation team could not say if these methods have been successful. USAID should request usage statistics for these platforms to determine how widely they are being used and what can be done to improve their effectiveness. • USAID should collaborate with IFPP on the best approach for a sustainable transition of project activities to MOH before project closedown. USAID may be able to serve as a bridge between MOH and IFPP to jumpstart the transition, and USAID could help to smooth out problems if difficulties arise during the transition period. • USAID and IFPP should ensure that the activity is including adaptive management strategies to respond to changing socioeconomic circumstances among target households. There is some evidence that communities have decreased ownership of household goods since the baseline evaluation was conducted (15% fewer own a radio at midline compared to baseline, 19% fewer own a mobile phone, 12% fewer own a bicycle, 7% fewer own a motorcycle). The project must proactively plan and adapt strategies to confront these and other constantly evolving challenges. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 9 EVALUATION PURPOSE AND EVALUATION QUESTIONS 2.1 EVALUATION PURPOSE The purpose of this midterm performance evaluation is to determine how well IFPP is meeting its intended objectives halfway through its implementation period and to make recommendations on approaches and management for the final years of the activity. EVALUATION OBJECTIVES. (1) To examine how IFPP has increased demand for modern contraceptives methods; (2) To assess how IFPP interventions have increased access and improved quality of FP service delivery; and (3) To determine how commodity management has improved following IFPP support of select service delivery points. This midterm evaluation examines the extent to which IFPP is achieving the objectives above and offers analysis to deepen understanding of activity progress to date. Conclusions and recommendations stemming from this report will be used to support further refinement of IFPP’s existing implementation approach, as well as increase the likelihood of sustainability of activity outcomes after project closedown. The key audiences for this evaluation are IFPP consortium partners and USAID/Mozambique (specifically, the Mother and Child Health Team, DO4, Program Office, Front Office, and the Integrated Health Office). Other external audiences are outlined below. EXTERNAL AUDIENCES. The Government of the Republic of Mozambique (GRM); MOH; and the Global Health Supply Chain Procurement and Supply Management (GHSC-PSM) project. OTHER DONORS. UK Department for International Development (DFID); the United Nations Fund for Population Activities (UNFPA) and the World Bank (WB). External audiences will use the evaluation to inform health policy, support FP service delivery and improve access to modern contraceptives. Internal audiences, in addition to using evaluation results to improve IFPP implementation, will also use findings, conclusions and recommendations to build greater synergy and improve coordination across USAID FP/RH programs. Similarly, other donors can learn about the efficacy of IFPP’s approach to improve FP/RH programming in Mozambique. 2.2 EVALUATION QUESTIONS (EQS) This section presents an overview of the evaluation questions, guided by the following themes related to family planning: (1) demand, (2) access and quality of services, (3) commodity management and (4) sustainability. The main evaluation questions are supported by a set of sub-questions to ensure that key issues within each of the thematic areas are sufficiently addressed. 1. Demand To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health?16 16 The question does not specifically ask about modern contraceptive usage. The evaluation team considered that USAID’s interests would be best served if the evaluation also discussed contraceptive usage. IFPP’s mandate to is to increase the use of modern contraceptive methods by target populations to achieve the following: ● Output 1. Increased access to a wide range of modern contraceptive methods and quality FP/RH services; ● Output 2: Increased demand for modern contraceptive methods and quality FP/RH services; and ● Output 3: Strengthened FP/RH health systems. 10 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV a. How has the knowledge and awareness of modern contraceptive methods increased? How has that change contributed to demand increase? b. How has the community environment contributed to support FP/RH service-seeking behaviors? c. Which local innovations, if any, bolstered FP/RH service delivery, demand creation and health systems quality? The evaluation team sought to understand how knowledge and awareness of modern contraceptive methods had changed over IFPP’s period of performance. Evaluators also sought to assess whether there were any changes in demand of modern contraceptive services in catchment areas surrounding target health facilities. In each case, the team tried to determine whether any changes could be linked to project interventions. 2. Access and Quality of Services To what degree have IFPP interventions increased access and quality of FP/RH services? a. How has access to modern contraceptive methods and Family Planning/Reproductive Health services increased? To what extent have IFPP interventions contributed to that increase? b. How has the quality of Family Planning/Reproductive Health services improved? How have IFPP interventions contributed to that improvement? c. What are the main barriers to reaching youth and adolescents with FP/RH? (Learning Question) The evaluation team sought to identify specific IFPP activities that aimed to improve access to FP services in locations visited for the midterm evaluation. Issues related to access (or perceptions of access) relied on self-reporting drawn from community member experiences, including households that fall within health facility catchment areas where the interventions have been rolled out. In addition, the evaluation team documented respondent perspectives and noted similarities and differences in experiences when seeking FP services. The evaluation team also documented successes and challenges in ensuring good delivery of quality FP services. Finally, the evaluation team examined the efficacy of activity interventions geared to improve quality of FP services. 3. Commodities Management To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? a. To what extent have the forecast and quantities of commodities received met the needs at service delivery point? b. At which level of commodities supply management chain is there a high frequency of commodities stockouts of FP/RH? The evaluation team documented how stock management (forecasting and requisitions) of FP commodities occurs at the national and sub-national level. Experiences drawn from health service providers and supply chain actors provided the team with insights on the process and highlighted perceived bottlenecks in the supply chain management of FP commodities. Accounts from health providers and MOH staff provided the team with information on availability of FP commodities at health facilities. The team also explored the frequency of stockouts and what measures, if any, providers take when faced with this challenge. 4. Sustainability How has the project ensured that there is local ownership of interventions that have been introduced? What is the likelihood of project achievements continuing after the project ends? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 11 The evaluation team used specific criteria to determine whether IFPP was prepared to ensure the sustainability of outcomes after project closedown. The evaluation team assessed whether the activity had a formal exit strategy in place and how it planned to deliver on its mandate to improve and sustain demand for FP services. In addition, the team sought to develop a comprehensive picture of whether key strategies used by the activity are aligned to national FP policy and whether there is sufficient political will in the MOH to support a sustainable transition. BACKGROUND 3.1 HEALTH SYSTEM IN MOZAMBIQUE Mozambique is moving towards a more decentralized health system “that favours partnerships to maximize the health and well-being of all Mozambicans”. 17 The MOH is organized into three administrative levels: (1) the Central Agencies, (2) the Provincial Health Directorate (DPS), and (3) the District and City Health Directorate (SDSMAS). Health services in Mozambique are largely provided through the public sector by the National Health System (NHS). Ninety-five percent of healthcare is provided through NHS government services, but only 60% of the population can reach these services. Public sector facility service provision is organized into four levels of care. 18 Family planning services are integrated into health facilities alongside reproductive health services (i.e. antenatal care, maternal health and neonatal child health services). These services are offered across all care levels. Front-line service providers tend to be facility-based nurses in maternal and child health units or departments at Level 1 (health centers and health posts) and Level 2 (district hospitals). The range of FP services provided at health facilities includes counseling and access to both short and long-acting reversible contraceptives (LARC). At the community level, mobile brigades, APEs and/or NGO￾supported Community Health Workers (CHWs, locally known as “activistas”) provide contraceptive services. The private sector is largely confined to major cities, and only 16% of users access contraceptive methods through the private sector via pharmacies and grocery stores. 3.1.1 FAMILY PLANNING IN MOZAMBIQUE Mozambique has improved its reproductive health indicators in recent years. From 2011 to 2015, the total fertility rate (TFR) decreased from 5.9 to 5.3 births per woman, the Modern Contraceptive Prevalence Rate (mCPR) among women in union increased from 11% to 25%, and unmet need for FP decreased from 29% to 23%.19 Antenatal care during pregnancy is high, with 93% of women having attended at least one visit. Seventy percent of women deliver in a health facility. However, this overall progress masks substantial regional variations in reproductive health indicators. According to the 2015 National Malaria and HIV Indicator Survey (IMASIDA), Nampula and Sofala provinces have mCPRs lower than the national average, at 22% overall and 14% among women in union. They are in need of program interventions to increase access to reproductive health services. 3.2 IFPP IN MOZAMBIQUE IFPP is a $34,560,000, five year USAID/Mozambique-funded initiative to increase use of modern contraceptive methods by target populations in all 36 districts of Nampula20 and Sofala21 provinces. 17 Health Sector Strategic Plan 2014 – 2019. 18 Level 1: Health centers and health posts; Level 2: District hospitals; Level 3: Provincial hospitals and/or rural hospitals; Level 4: Central hospitals. 19 IDS 2011, IMASIDA 2015. 20 Nampula: Angoche, Eráti, Ilha de Moçambique, Lalaua, Larde, Liúpo, Malema, Meconta, Mecubúri, Memba, Mogincual, Mogovolas, Moma, Monapo, Mossuril, Muecate, Murrupula, Nacala, NacalaVelha, Nacarôa, Nampula Rapale, Ribáuè 21 Sofala: Nhamatanda, Muanza, Marínguè, Sofala, Marromeu, Machanga, Gorongosa, Beira, Búzi, Caia, Chemba Cheringoma, Chibabava, Dondo 12 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV IFPP responds to the United States Government (USG) strategy for development and foreign assistance in Mozambique through the Country Development Cooperation Strategy (CDCS). The USAID/Mozambique CDCS outlines an overarching development objective health goal to “Improve the Health Status of Target Populations” through three results: (1) Increased coverage of high impact health and nutrition services, (2) Increased adoption of positive health and nutrition behaviors, and (3) Strengthened systems to deliver health, nutrition, and social services.22 The project is led by Pathfinder International with a team of global and local partners – N’weti, Population Services International (PSI), and Abt Associates. Pathfinder International is the prime implementing partner, leading the team in family planning service delivery. N’weti’s main objective is to increase demand among the project’s target population, address gender norms that impede contraceptive use, and cultivate an improved enabling environment for FP services. Abt Associates strengthens FP/RH provincial and district systems. Finally, PSI expands access to private sector services. IFPP aims to reach an additional 565,000 new contraceptive users (“initiators”23) and spur a 15- percentage point increase in mCPRs as well as offer a more diverse and effective contraceptive methods in the intervention areas. IFPP’s results framework is shown in the figure below. 22 Country Development Cooperation Strategy, 2013. 23 Initiators are defined as clients who begin using a contraceptive from a Service Delivery Point. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 13 Figure: IFPP Results Framework 14 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Key outcomes anticipated at the end of the project performance period include positive changes in the perceived quality of FP services (IR2); increased uptake of FP services (IR1); and marked improvements in availability of modern contraceptives at service delivery points (IR3). The IFPP activity design provides a mix of both demand-side and supply-side interventions. On the demand-side, interventions are geared towards improving uptake of FP services among the following five target categories of beneficiaries: post-partum women; women living with HIV; adolescents including orphans and vulnerable children (OVC); medium and high-parity women; young couples; and, post-abortion women. On the supply-side, the service delivery model focuses on the following service delivery points: health facilities, APEs, mobile brigades, FP commodity warehouses, and schools. EVALUATION METHODS AND LIMITATIONS This evaluation used a mixed methods approach, combining both qualitative and quantitative techniques across four research phases: (1) document review; (2) quantitative survey, (3) qualitative fieldwork, and (4) data analysis. These evaluation tools and research phases are described in more detail below. 4.1 DATA COLLECTION METHODOLOGY AND ANALYSIS The evaluation team gathered qualitative and quantitative data using the following techniques: document review, quantitative survey, key informant interviews (KII), Focus Group Discussions (FGDs), and facility-based observations. A brief description of each method is presented below: DOCUMENT REVIEW. The evaluation team systematically gathered data from 50 USAID documents, IFPP documents, and other outside sources pertinent to this evaluation. The evaluation team also analyzed Community Scorecards to determine the quality of services provided at health facilities.24 The evaluation team also reviewed additional documents provided by MOH and other implementing partners (IPs), including commodity stock data from the SIGLUS system. QUANTITATIVE SURVEY. The evaluation team surveyed 816 women in Nampula province. The same quantitative survey tool was used for both baseline and midline measurements. This enabled the team to assess any changes or variations in overall respondent knowledge, awareness and attitudes of modern contraceptive methods. The baseline and midline measurements only compare responses from Nampula province – the evaluation team could not visit Sofala during midline data collection because of Cyclone Idai. KEY INFORMANT INTERVIEWS (KIIS). The evaluation team used semi-structured interview guides to facilitate discussions with 50 stakeholders and beneficiaries at the national and subnational levels. These interviews provided insights on the overall perceived effectiveness of IFPP approaches, as well as highlighted Activity successes and gaps in implementation. Interview guides were used to facilitate discussions with a wide range of stakeholders and beneficiaries at national and subnational level. The list of key informants interviewed is found in Annex H. FOCUS GROUP DISCUSSIONS (FGDS). The evaluation team conducted 22 semi-structured FGDs with a range of beneficiaries – adult males, adult females, adolescent males and adolescent females – to gain in-depth understanding of their experiences. Specifically, the evaluation team gathered qualitative data on knowledge, attitudes, beliefs and practices regarding family planning, accessibility to FP services, and their overall experience with modern contraceptive use. The list of FGDs is in Annex I. 24 Community Scorecards tools to bring the communities and providers together to assess the quality of services. Based on the results, they jointly select 5 priority areas to improve services and develop an action plan. Community Scorecards are operational in 14 out of 385 health facilities (4% of facilities, 8 in Nampula and 6 in Sofala). USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 15 OBSERVATIONS. The team visited 6 health facilities and 4 supply warehouses to confirm the presence of an FP-trained health provider who had received family planning training through the Activity. Using an observation checklist, the evaluation’s family planning expert observed FP-related consultations and documented whether providers had the resources necessary to provide quality FP services. In addition, the team sought to determine whether “adolescent corners” – areas where youth can receive family planning services away from other community members – were available and in use for adolescents. 4.1.1 PHASE 1: DOCUMENT REVIEW Prior to launching fieldwork, the evaluation team conducted a desk review of 50 key IFPP documents and extracted data relevant to the evaluation. The list of all documents reviewed is in Annex G. 4.1.2 PHASE 2: QUANTITATIVE SURVEY The evaluation team conducted 816 surveys in 16 districts (spanning 34 enumeration areas) in Nampula province. The quantitative survey is representative of areas that receive IFPP intervention and allows for comparison of the treatment areas in Nampula province between baseline and midline.25 It is helpful to note, however, that the sample design is not sufficient to conduct analyses at the district or catchment area level. Analysis of rural and urban areas is possible with this sample design. The target population included households (HHs) located within catchment areas surrounding service delivery points (health facilities) where IFPP interventions were implemented. Probability sampling was used to inform the survey sampling in Nampula. The sampling approach is multi-stage (three tiered) to ensure that every HH had an equal chance of being selected. The profile of individuals targeted by the survey were women of reproductive age, between ages 15 to 49 years. This is in line with the demographic characteristics of IFPP beneficiaries who participated in the baseline survey. The evaluation team selected one woman for participation in each HH. A Kish Grid was used to select the respondent in HHs where more than one woman met the eligibility criteria for participation.26 More information about the sampling approach and sample frame can found in Annex F. Descriptive statistics include the means/percentage for 2017 and 2019, their standard errors, and the mean differences between 2017 and 2019. The standard error of the mean difference was computed with the following formula: SE=(SE2 17+SE2 19) 1/2, assuming independence of the variables between 2017 and 2019. The statistical significance of p<0.05 was considered if the difference taken in absolute terms was more than twice its standard error (SE) (95% confidence of the difference between baseline and midline). Finally, Chi-square tests and T-test were used to test differences between sub-groups. For 2x2 tables, p-values from Fisher’s Exact Test (2-sidd) was used. All survey data discussed in this document is statistically significant (p<0.05) unless otherwise stated. The following table shows the number of surveys conducted in each district in Nampula province. 25 The 2007 census data were used to estimate the proportion of the population of target reproductive age women and the average household size. The sample design was developed by the National Institute of Statistics. According to the baseline report, a sample of 816 households and 34 enumeration areas serves as a representative sample based on the following assumptions: providing indicators with acceptable level of accuracy with coefficient of variation below 17% and a confidence interval of 95%, for each representative domain (the province). Sample size was estimated to detect a 10% change. In addition, it assumed a non-response rate of 10%, with a design effect of 2 and power of 0.8. 26 Residents of collective housing estates and the homeless were excluded from the sample. 16 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Table : Quantitative Survey Sample Distribution, Nampula Province Districts # of Surveys Conducted Angoche 72 Cidade Nampula 96 Iha de Moçambique 24 Meconta 48 Memba 24 Mogovolas 72 Mogincual 24 Mecuburi 24 Monapo 96 Mossuril 24 Murrupula 24 Nacala porto 48 Nacala Velha 48 Nacarroa 48 Namapa-Erati 72 Nampula 72 Total 816 4.1.3 PHASE 3: QUALITATIVE FIELD RESEARCH The main qualitative data collection methods consisted of KIIs from the national, district and health facility levels, as well as FGDs at the community level. The evaluation team collected data in one province, Nampula, and in 3 districts (Monapo, Mecuburi and Nacala-Porto). The field research initially included interviews and surveys in Sofala province as well. However, fieldwork in Sofala was canceled due to Cyclone Idai. The evaluation team completed a total of 50 KIIs (17 national, 7 provincial, 19 district, 7 community), 22 FGDs (including 55 male adults, 30 male adolescents, 64 female adults and 36 female adolescents), 6 health facility observations and 4 commodity warehouse observations. A map of the fieldwork and summary table of the qualitative field data collection are below. Annexes H and I provide full details of all interviews and FGDs conducted. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 17 Table: Full Qualitative Data Collection Summary Category # of Individual Interviews # of Group Interviews Total number of Interviewees (% Female) # of Focus Groups # of Focus Group Participants (% Female) USAID 4 - 4 - - MOH (National￾Level) 1 1 3 - - MOH (Province￾Level) 2 - 2 - - MOH (District￾Level) 4 7 19 - - IFPP Consortium Partners 5 1 8 - - Other Donors 2 1 4 - - Community Leaders 1 2 5 - - Others International Programs 3 1 5 - - Female adults - - - 6 64 Female adolescents - - - 5 36 Male adults - - - 6 55 Male adolescents - - - 5 30 TOTAL 22 13 50 (52%) 22 185 (54%) 18 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV The following map shows all areas visited for the quantitative and qualitative fieldwork. Figure: Map of Districts Visited for Fieldwork 4.1.4 PHASE 4: QUALITATIVE AND QUANTITATIVE ANALYSIS The evaluation used a combination of thematic and pattern analysis to synthesize qualitative data. The evaluation team conducted a descriptive analysis and an inferential statistical analysis of HH survey data. Descriptive statistics yielded information that enabled the evaluation team to describe the sample while inferential statistics allowed the evaluation team to make inferences and predictions about the survey population as it relates to FP service-seeking behaviors. To answer evaluation questions and formulate definitive conclusions at midline, evaluators triangulated results across all primary and secondary sources. More details on how the evaluation team approached analysis for both qualitative and quantitative data sets are presented below. DAILY INTERPRETATIVE ANALYSIS. At the end of each day of fieldwork, evaluation team members reviewed their interview notes and recordings, discussed them with each other, and filed interview memos. CONTENT ANALYSIS AND TRIANGULATION OF EVIDENCE. The evaluation team used a content analysis approach to identify themes and trends relevant to each evaluation question and better understand the meaning of statements and the context in which they were made. The team reviewed interview notes and transcriptions to discern trends in data to begin the development of key findings. Areas of high convergence were used to develop the Findings and Conclusions Matrix (see Annex A). USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 19 QUANTITATIVE ANALYSIS. Survey analysis used descriptive statistics and statistical tests (mentioned above) to determine whether the differences observed between the baseline and midline surveys were statistically significant (p<0.05). The evaluation team disaggregated key indicators into the following relevant demographic categories: age, marital status, religion, location (urban-rural) and socioeconomic status (SES). Where survey findings differed from data collected by IFPP, the evaluation team suggests reasons for why there may be discrepancies in findings. Specifically, this additional analysis was necessary for findings relating to mCPR and unmet need/demand calculations in EQ1. VALIDATION WORKSHOP. Once preliminary analysis was complete, the evaluation team met with IFPP to review the evaluation’s findings and conclusions. This workshop was important to ensure that the evaluation team understood the context for all findings, but most importantly to discuss areas where evaluation findings diverged from IFPP reporting. The evaluation team documented all areas of divergence to provide context for any follow-on actions USAID requests as a result of this report. 4.2 LIMITATIONS Key limitations of the evaluation are presented below, along with a brief description of mitigation strategies that were used to the ensure validity of evaluation results. OMISSION OF SOFALA. The original scope of the evaluation aimed to include Sofala province, as the province was surveyed as part of the baseline study. However, the evaluation team was unable to visit Sofala, which was hit by Cyclone Idai when fieldwork commenced. The human toll exacted by the storm was catastrophic, with 598 people killed and approximately 130,000 people displaced from their homes across the country. In addition, organizing logistics in such an environment would have been dangerous and ill-conceived. Mitigation: USAID/Mozambique approved the omission of Sofala because the need to adhere to the fieldwork schedule despite the adverse circumstances. Following consultations with USAID, the midterm evaluation presents findings and conclusions drawn from quantitative and qualitative data gathered from Nampula. To ensure that data is consistent across the baseline and midline measurements, comparisons with baseline data in the evaluation report only include observations from Nampula province. RECALL BIAS. Interview data are subject to cognitive biases, including recall bias – error caused by the incompleteness or inaccuracy of memories about their experiences. Mitigation: The evaluation team ensured reliability of findings through systematic triangulation of interview and document sources. This approach reduced the potential for bias in the midterm evaluation. ASSESSMENT OF CLINICAL QUALITY. The evaluation did not test the individual competency of trained staff to perform clinical procedures, as this was outside the scope of the evaluation. Mitigation: In lieu of a clinical competency test, the evaluation team interviewed staff to inquire about quality of care using the data collection tool in Annex D. In addition, the evaluation team analyzed Community Scorecards – IFPP-led assessments of the quality of provider services based on community and health facility feedback – to measure general perceptions of service quality. 20 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV FINDINGS AND CONCLUSIONS 5.1 PROFILE OF SURVEY RESPONDENTS The purpose of this section is to help the reader understand the profile of the baseline and midline survey respondents, as the majority of the analysis of survey results is presented according to respondent socio-demographic characteristics. The midline survey collected demographic and family planning data from a total of 816 women between the ages of 15 and 49 in Nampula province.27 Key characteristics to describe the population surveyed include household location (urban-rural), household characteristics (type of housing and household makeup), age, marital status, religion and level of education. These characteristics are statistically similar between baseline and midline measurements in Nampula province. The following table outlines the demographic profile of the baseline and midline survey respondents. 27 IFPP conducted the initial (baseline) survey with 802 respondents in 2017. The midline survey with 816 respondents took place in February￾March 2019. The survey instrument was identical to the one used at baseline. While the survey at baseline took place in both Nampula and Sofala, the midline data collection in Sofala was canceled due to Cyclone Idai. Only data from Nampula was used to compare baseline and midline data. The survey methodology was nearly the same, with a small increase in survey respondents in Nampula to ensure the validity and reliability of results. For more information on how the survey was conducted, please see Annex F. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 21 Table: Profile of Baseline and Midline Survey Respondents Respondent Profile Baseline Midline Statistically Significant (Y/N) Household Characteristics Electricity 22% 25% Y Natural Roof 72% 73% Y Natural Floor 80% 83% Y Improved Source of Water 62% 69% Y Toilet 31% 32% N Television 20% 20% N Radio 38% 23% Y Male Head of Household 81% 78% N Median Age First Sexual Experience 14 15 N Mean Age at First Use of Contraceptives 23.5 22.8 N Household Size 4.7 4.2 N Background Characteristics Rural 67% 71% N Urban 33% 29% N Age 15-19 15% 24% Y Age 20-24 22% 22% N Age 25-29 18% 20% N Age 30-34 12% 14% N Age 35-39 15% 10% Y Age 40-44 10% 5% Y Age 45-59 9% 4% Y Marital Status Never Married 7% 8% N Married 65% 37% Y Living Together 17% 41% Y Divorced/Separated 8% 11% N Widowed 3% 2% N Education Level No Education 47% 41% Y Primary 41% 45% N Secondary or Higher 11% 14% N Religion Catholicism 53% 53% N Islam 42% 36% Y 22 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 5.2 EVALUATION QUESTION 1: DEMAND CREATION EQ 1: To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? • EQ 1.1: How has the knowledge and awareness of modern contraceptive methods increased? How has that change contributed to the demand increase? • EQ 1.2: How has the community environment contributed to support FP/RH service seeking behaviors? • EQ 1.3: Which local innovations, if any, bolstered FP/RH service delivery, demand creation and health systems quality? 5.2.1 EQ1 FINDINGS 5.2.1.1 EVALUATION QUESTION 1.1 – KNOWLEDGE AND AWARENESS Knowledge and awareness of modern contraceptives is high and has increased among most women surveyed. Demand and unmet need increased, but demand satisfied and met need have decreased. Increased demand does not translate into actual modern contraceptives usage – while both demand and unmet need have increased, overall use of contraceptives has remained stagnant. 5.2.1.1.1 KNOWLEDGE OF CONTRACEPTIVE METHODS Knowledge of all specific modern methods increased, as shown below. Graph: Knowledge of Modern Contraceptive Methods Survey data show a significant increase in knowledge of any modern contraceptive method among survey respondents. Knowledge of any modern contraceptive method among women ages 15 to 49 increased from 80% at baseline to 92% at midline. Knowledge of all modern methods increased, with the largest percentage gains for long-acting reversible contraception which include implants (35% to 59%) 80 35 66 63 48 92 59 84 78 62 0 10 20 30 40 50 60 70 80 90 100 Knowledge of any modern contraceptive method among women aged 15 to 49 Knowledge of Implants Knowledge of Injectables Knowledge of Pills Knowledge of Male Condoms Baseline Midline USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 23 and injectables (66% to 84%). Similarly, increases were noted for pills (63% to 78%) and male condoms (48% to 62%).28 However, knowledge of some alternative methods, like male sterilization, LAM29 and emergency contraception remain low, at 9%, 7% and 6%, respectively at midline. As show below, women ages 15 to 19, women in the lowest two income quintiles (hereafter known as Q1 and Q2) and women residing in rural areas had lower levels of knowledge of modern contraceptive methods than women ages 24 to 49, women in the highest income quintile (hereafter known as Q5), and women in urban areas. Graph: Knowledge of Modern Contraceptive Methods by Group Women ages 15 to 19 had the least knowledge of any modern method (88%), while women ages 20 to 24 had the highest (93%). Women in the two lowest income quintiles had less knowledge than women in the highest quintile of income (88% vs. 96%). Rural women had less knowledge than women in urban areas (92% vs. 98%). While the gap between younger/older women and rural/urban women has remained small from baseline to midline, there has been enormous progress in closing the knowledge gap between the richest and the poorest women (23% gap to 8% gap). Qualitative data support these findings, showing that the majority of focus group participants had knowledge of contraceptive methods. In both urban and rural areas, participants in 20 out of 22 adolescent and adult focus groups had knowledge of at least one modern contraceptive method. 5.2.1.1.2 DEMAND GROWTH Survey data show that demand and unmet need for contraceptive services increased, while met need and demand satisfied decreased. Women with unmet need for family planning increased, from 23% at baseline to 36% at midline, which may indicate increased knowledge translating into increased demand. 28 IFPP staff explain that while condoms are widely used to prevent HIV transmission, they are not commonly used within the household and are not generally considered a family planning method. 29 Lactational Amenorrhea Method, or breastfeeding as birth control. 88 65 87 80 75 80 96 88 98 92 88 93 0 10 20 30 40 50 60 70 80 90 100 Q5 Q12 Urban Rural 15-19 24-49 Wealth Location Age Baseline Midline 24 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Wealthier women (Q5) have a lower unmet need (30%) than poorer women (Q1&2, 40%), women ages 15 to 19 have lower unmet need (32%) than women ages 25 to 49 (38%) and urban women have lower unmet need (30%) than rural women (39%). Demand also increased from 43% at baseline to 53% at midline. Demand is highest among urban (58%), wealthier (Q5, 57%), and married (56%) women ages 25 to 49 (56%). Demand is lowest among rural (51%), poorer (Q1&2, 51%), younger (ages 15 to 19, 52%), and women who are not union (24%). At the same time, met need and demand satisfied decreased from 20% to 17% and 47% to 32%, respectively. The following graph – and a box explaining these terms – show these trends. Graph: Need and Demand for Modern Contraceptives30 Interviews with USAID and MOH officials also reflect this increased demand for FP services. Three out of 4 KIIs with USAID reported increased demand in Nampula province based on increased utilization of FP services at service delivery points. One service provider at a health facility in Mecuburi echoes these 30 Please see Annex K to see how demand and need were calculated. Also see “Revising Unmet Need for Family Planning.” DHS Analytical Studies 25, January 2012. For definitions of unmet need, met need and demand, see pages 11 and 39. KEY DEFINITIONS mCPR: Women of reproductive age who are using (or whose partner is using) a modern contraceptive method at a particular point in time. Unmet Need: Women who are fecund and sexually active but are not using any method of contraception, and report not wanting any more children or wanting to delay the next child. The concept of unmet need points to the gap between women's reproductive intentions and their contraceptive behavior (WHO, 2015). Met Need: women of reproductive age either married or in a union, who want to stop or delay childbearing and are using contraception. Demand: Unmet need for family planning + current contraceptive use (any method). Demand Satisfied: Current contraceptive use (modern method) / Unmet need for family planning + current contraceptive use (any method) 23 20 43 47 36 17 53 32 0 10 20 30 40 50 60 70 80 90 100 Unmet need for Modern Methods FP Met need for Modern Methods of FP Demand for Modern Methods of FP Demand satisfied for Modern Methods Baseline Mideline USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 25 sentiments: “Demand has increased significantly…The health center received training in February 2017 and began seeing increases in FP clients in April 2017.” Finally, 7 of 9 MOH provincial and district-level MOH directors interviewed also expressed a broad increase in demand for FP services. 5.2.1.1.3 SERVICE UTILIZATION According to survey data, overall use of modern contraceptives has remained stagnant, as shown in the graph below. Graph: Use of Modern Methods – Ever and mCPR The percentage of women who have ever used a modern contraceptive method has remained static – the change from 42% at baseline to 40% at midline is not statistically significant. The modern contraceptive prevalence rate (mCPR) has also remained static (the change from 20% to 17% is not statistically significant). The graph below provides a demographic breakdown of the mCPR data. Graph: Current Use of Modern Methods (mCPR) by Group 42 20 40 17 0 10 20 30 40 50 60 70 80 90 100 Ever Use Modern Methods Current Use of Modern Methods Baseline Midline 28 13 27 18 13 21 28 11 27 16 18 18 0 10 20 30 40 50 60 70 80 90 100 Q5 Q12 Urban Rural 15-19 24-49 Wealth Location Age Baseline Midline 26 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV These data show the following: • Women in the highest quintile of income (Q5) are using contraceptives at a higher rate compared to the two lowest income groups (Q1&2) at midline (28% vs. 11%). The usage gap between high-income and low-income earners is wide and has not changed. • Rural women use modern contraceptives at a lower rate than urban women (16% vs 27%). This gap has remained unchanged from baseline to midline. • Young people are using modern contraceptives at a higher rate than at baseline (13% to 18%). Other age groups have seen a percentage decrease in modern contraceptive use between baseline and midline. The usage gap between younger and older women seen at baseline has been eliminated. The following demographic categories are not shown in the graph above but are equally important: • Married women are using contraceptives at a lower rate than at baseline, although the decrease is not statistically significant. They use implants and Intra-Uterine Devices (IUDs) more than other methods – 10% and 4% respectively at midline – reflecting their preference for long-term contraceptive methods. • Muslim women use modern contraceptives at a higher rate than non-Muslim women, although the difference is not statistically significant (19% vs. 16% at midline). • Women with a secondary or higher education are more likely to use modern contraceptives than women with less or no education (27% vs 15% at midline). Finally, knowledge of certain contraceptive methods correlates with the types of methods respondents currently use. Injectables, male condoms and pills are the types about which respondents have the most knowledge and are the most popular methods used, as shown in the graph below. Graph: Type of Method Ever Used (Baseline and Midline) and Current Use (Grey Bars) 0.7 19 28 6 7 0.7 1 14 21 4 16 12 5 10 2 0 10 20 30 40 50 60 70 80 90 100 Sterilization Pills Injectables Implants Male Condom Traditional methods Baseline Midline Current Use USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 27 The graph shows the following: • Male and female sterilization is rarely used (<1% for baseline and midline). • The pill decreased in usage from 19% at baseline to 14% at midline. It is currently used by 5% of women. • Injectables decreased from 28% at baseline to 21% at midline and is currently used by 10% of women. • Implants also decreased from 6% at baseline to 4% at midline and is currently used by 2% of women. For women in union, current use increases to 10%. • Male condom use has increased from 7% at baseline to 16% at midline. • Traditional methods increased from 1% at baseline to 12% at midline, although current use of traditional methods is negligible. The evaluation team expected the FP utilization rate to be higher than reported in the KAP survey, considering that IFPP registered 278,144 new users of modern contraceptive methods (188,906, or 68% of whom are in Nampula) in FY1, according to the project’s Performance Indicator Matrix. Despite these gains in new users in Nampula, survey data show that utilization is flat (20% to 17%). The evaluation outlines the following considerations to put these divergent findings into context. 1. How were data on IFPP’s contraceptive users collected to inform this result? MMEMS surveyed women ages 15-49 in households within the same enumeration areas and with similar demographic characteristics used at baseline (see section 6.1 for a profile of survey respondents). 2. How many of IFPP’s users are from Nampula province? 68% of IFPP’s new users are from Nampula. The survey took place 16 of Nampula’s districts, and only respondent data from Nampula province is used to compare baseline and midline results. 3. Could an overall increase in young people ages 15-19 in the population be responsible for the stagnant % change between baseline and midline? Possible, but unlikely. With only two years between baseline and midline measurements, the population is not likely to have undergone significant change. 4. Did any other survey respondent background characteristics differ in a significant way between baseline and midline? Overall, midline respondents owned fewer key household goods compared to baseline respondents (23% vs. 38% own a radio, 31% vs. 50% own a mobile phone, 19% vs. 31% own a bicycle, and 11% vs. 18% own a motorcycle). Ownership of these goods - or lack thereof - could reflect a decrease in household wealth, which may have implications for access to some IFPP services. For example, it may be harder for women at midline to get transport to health facilities or to receive information through mobile applications. Please see table on p. 21 for additional detail. Percentages of married individuals or those living separately also differed significantly between baseline and midline (65% vs. 37% and 17% vs. 41% respectively), but it is unlikely that these characteristics fully explain the overall differences seen between the baseline and midline survey respondents. 5. Could discontinuation be responsible for the stagnant utilization rate? Possibly. The survey instrument did not include any questions about whether methods like injectables had been discontinued by users. Additional investigation by IFPP could shed light on this issue. 6. What else could have contributed to the stagnant usage rate between baseline and midline? A national level shortage of injectables has resulted in stockouts of this commodity in 31% (43) of health facilities in Nampula province on which IFPP collects data. The height of this stockout, March 2019, coincided with the timing of the midline survey. The evaluation team speculates that the lack of injectables – the most widely used modern contraceptive in Nampula – may have contributed to depressing the utilization rate. 7. Are the data reliable? Yes, the data are reliable and were collected, validated and analyzed using internationally accepted methods. This survey data is based primarily on perceptions. There may be recall errors or comprehension errors. However, there is no reason to suggest that these issues are not present in equal measure in the baseline study. 28 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Qualitative interviews reflect the prevalent use of injectables and pills. In the case of injectables, many women use that method because it is easy, and easier to hide from their husbands. Use of pills is also prevalent among women interviewed, along with implants, although this is not widely reflected in the survey data. 5.2.1.2 QUESTION 1.2 – COMMUNITY ENVIRONMENT According to survey data, the community and domestic environments for discussing family planning have improved. IFPP has deployed various strategies to educate and inform the public about modern contraceptive methods. Interviewees and focus group respondents applaud IFPP’s efforts at improving the environment surrounding seeking family planning services through the use of mobile brigades, youth couples counseling, Community Health Workers, and home visits. These efforts have proven fruitful. Overall, survey data show that domestic attitudes about gender norms and family planning are improving. While significant barriers persist, respondents are more confident in proposing the use of contraceptives with their husbands, and men are more accepting of family planning practices. 5.2.1.2.1 FACTORS IMPROVING THE ENVIRONMENT FOR SEEKING FP SERVICES IFPP deployed the following strategies to improve the community environment for seeking family planning services. MOBILE BRIGADES IN SCHOOLS AND COMMUNITIES. According to interviews with USAID, MOH, and IFPP, mobile brigades of health providers successfully disseminated family planning information through urban and rural secondary schools and at households. One interviewee from USAID said that mobile brigades “have done a good job” in creating an enabling environment.31 YOUTH COUPLES COUNSELING IN RURAL AREAS. Community members report that six￾week counseling sessions for young couples ages 15 to 30 in rural and “combo” areas32 – provided by educators or facilitators in conjunction with community leaders or volunteers working with community based organizations (CBOs) – reduced the stigma surrounding family planning and led to the ability to discuss and seek out contraceptive services. Ten out of 12 focus groups with adults affirm the positive changes in discussing family planning because of these counseling efforts. 31 IFPP deployed 1,639 mobile brigades including contraceptive services in FY1 and exceeded the FY2 target of 2,528 mobile brigades deployed. 32 “Combo” areas combine aspects of both urban and rural areas. Also defined as peri-urban areas. “I use depo [injectable] without my husband knowing. It is easier because e.g. having pills might come to the husband´s knowledge. It is more difficult.” (Focus Group in Nacala) “I’m 22 years old and have been doing injection for the past 3 years because I have 2 children. I used to use Pills but because I used to forget to take them at the same time every day my sister advised me to do injections because that I could easily control.” (Focus Group in Nacala Porto) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 29 COMMUNITY HEALTH WORKER TRAINING AND DEPLOYMENT. Interviews show that Community Health Workers 33, especially APEs, improve the environment for seeking FP services because they visit (mostly rural) homes to provide family planning services and help to reduce taboos and address misconceptions surrounding modern contraceptive methods. URBAN HOME VISITS BY COMMUNICATION AGENTS. Interviews with IFPP and Focus Group Discussions show that face-to-face counseling between Interpersonal Communication Agents (IPCs) and women at their homes in urban areas reduce misconceptions about family planning methods and help women to talk more openly about family planning with their husbands.34 5.2.1.2.2 FEMALE EMPOWERMENT AND MALE ACCEPTANCE OF FP IFPP’s strategies for making the community environment more hospitable for family planning services – home visits, Community Health Worker deployment, couples counseling and mobile brigades – have improved female empowerment and male acceptance of family planning. The graph below shows that HoHs (81% of whom were men in the baseline compared to 76% at midline) are increasingly in favor of using family planning to avoid pregnancy. 33 Community Health Workers comprised of Agente Polivalente Elementar (APEs) and Traditional Birth Attendants (TBAs), also called Activists, are trained in family planning by MOH and IFPP. 32,433 beneficiaries in Nampula received contraceptive services from APEs in FY1 and exceeded the FY2 target of 56,388 beneficiaries in Nampula in FY2. 34 According to IFPP’s Performance Indicator Matrix, data on IPC visits is not disaggregated from Community Health Worker visits (1,763 CHWs provided services in FY1). “Before the project we never had too much information about FP because it was a taboo.” (Focus Group in Mecuburi District) “Since the activists started with the community sessions and community dialogues about FP, both with women and men, they started learning and having information about FP.” (Focus Group at Monapo Sede) 30 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Graph: Acceptance of Family Planning by Head of Household Fifty-three percent of households at baseline were in favor of using family planning to avoid pregnancy, compared to 65% at midline, representing a 12% increase. Community leaders across Nampula province echo this finding, saying that men have accepted the need for modern contraceptives. Female empowerment regarding family planning issues also shows signs of improvement. As the graph below demonstrates, shared decision-making about family planning increased and is the principal way couples discuss family planning. Graph: Decision Maker About the Use of Contraception 26 20 37 25 24 44 0 10 20 30 40 50 60 70 80 90 100 Woman alone Husband alone Both Woman and Husband Baseline Midline “There is a great acceptability of modern contraceptive methods, even by the male partners / husbands. A lot of sensitization has been done in the communities, among the families, about…intervals between children. (Community Leaders, Mecuburi District) 53 65 0 10 20 30 40 50 60 70 80 90 100 Baseline Midline USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 31 Joint decision-making between couples increased from 37% at baseline to 44% at midline.35 However, challenges remain. Almost one-quarter of men make all of the family planning decisions for the household, and one-quarter of women also make decisions without consulting their partners. Confidence to make family planning decisions is another indicator of female empowerment. As shown below, a majority of survey respondents “strongly agree” with their ability obtain contraceptives, to make suggestions to their husbands about the family planning method they should use, and to suggest waiting before having another baby. Graph: Ability to Negotiate Contraceptive Use with Partner Respondents who feel confident in their ability to suggest to their husbands to wait a few months before having another baby increased, from 44% at baseline to 63% at midline. Confidence in suggesting family planning methods increased from 47% to 60%. And confidence in obtaining the method she wants to use increased from 41% at baseline to 59% at midline. Poorer and more rural women reported lower confidence than the mean across all of these indicators (12% to 16% gap for women in the lowest two quintiles of income compared to Q5 women, and a 9% to 10% gap for rural women compared to urban women). Qualitative data also support the conclusion that female empowerment and acceptance of family planning has improved. FGD participants from Nacala Porto discussed this dynamic as follows: “Before talking about FP, it was a taboo, nobody used to talk about it because it was shameful, but now things changed. You can find a group of women sitting together and talking about different methods.” According to community leaders in Monapo Sede, the community talks more about family planning compared to several years ago. They particularly emphasize the positive change seen in men over this period. 35 7% of respondents responded “Other” to this question. 63 60 59 44 47 41 0 10 20 30 40 50 60 70 80 90 100 I feel confident in my ability to suggest to my husband to wait a few months before having another baby I feel confident in my ability to suggest to my husband the FP method that we should use If I want to use FP method to avoid pregnancy, I feel confident in my ability to obtain the method I want to use Baseline Midline “The attitudes, principally among the men, improved…People talk more about FP compared with just a couple of years ago. Although, still the woman has to convince her husband to use FP method.” (Community Leaders, Monapo Sede) 32 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Despite these advances, 33% of survey respondents discussed family planning with someone, but only 10% discussed family planning with their partners over the past 6 months. Instead, they rely on community members outside of the family. Among women who use contraceptives, 18% spoke with their partner in the past 6 months about family planning. Women who do not use contraceptives were less likely to have spoken to their partner in the past 6 months (8%). 5.2.1.3 EVALUATION QUESTION 1.3 – INNOVATIONS IFPP successfully innovated across the three principal activity areas: service delivery, demand creation and health system support. 5.2.1.3.1 SERVICE DELIVERY The most significant service delivery innovations are: TRAINING FOR ALL HEALTH FACILITY STAFF. IFPP sought to improve the quality of family planning service delivery at health facilities through sensitization and clinical training of facility staff.36 Health providers state that sensitization activities targeted all health personnel, from doctors and nurses to receptionists and cleaners. As a result, IFPP created a friendlier environment for adolescent and female clients seeking FP services. Interviews and observations at health facilities revealed that staff are friendly and attentive. In addition, interviewees say that nurses “take time to explain all the different methods” before being provided services.37 ADOLESCENT CORNERS AT HEALTH FACILITIES. IFPP adapted existing “adolescent corners” at health facilities – private areas reserved for adolescents – to improve the use of family planning services among the youth. In general, adolescents were reluctant to visit health facilities to ask for family planning services because of the shame and stigma surrounding adolescent sexual activity. Qualitative evidence shows that adolescent corners have improved the usage of family planning services for young people. As seen previously, the survey data corroborate this finding: use of any modern contraceptive method has increased among the 15 to 19 age cohort, while it has dropped for other age segments. In Focus Group Discussions, young people expressed the view that services offered are of high quality and have helped to reduce the individual and shared stigma associated with adolescent use of modern contraceptives. AGENTE POLIVALENTE ELEMENTAR (APE) HOME VISITS. While APEs existed prior to IFPP intervention, the training they received by the project improved their knowledge in family planning and equipped them with modern contraceptives. Qualitative data show that APEs are an innovative resource for women because APEs conduct home visits and provide family planning services directly to women. According to beneficiaries, health providers and IFPP documents, APEs are also responsible for 36 As stated in the Limitations section, the evaluation team did not assess the clinical skills of service providers. However, MMEMS did conduct interviews with service providers and clients to gauge service delivery improvement. 37 Focus Group at Monapo Sede and at Namina Health Center in Mecuburi District. “With assistance from Pathfinder, we relocated the corner in a different area in the facility. Now the services are used a lot more. We provide counseling and all short-term and long￾term methods for the adolescents. The boys are using the corner as well as the girls.” (Health provider, Monapo Health Center) “The service is friendly. There are always commodities at the HF, especially condoms.” (Focus Group, Male Adolescents, Monapo Sede) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 33 reducing social stigma surrounding the use of family planning. The importance of APEs is growing – survey data reveal a significant increase in family planning visits over the past year, from 12% at baseline to 24% at midline. APEs account for the majority of this increase. Survey data show that APE household visits increased from 1% to 11% over this time period. 38 MOBILE BRIGADES IN RURAL AREAS AND SECONDARY SCHOOLS. Mobile brigades existed prior to IFPP, mostly to provide immunizations in remote areas. The project expanded their role by trained teams of nurses and technicians to provide on-site, routine family planning services in rural areas and secondary schools. Despite a lack of strong qualitative evidence showing their effectiveness (no beneficiaries commented on mobile brigades in rural areas), evaluation team observations, desk review research and interviews with health providers note that they are innovative because they provide family planning information in hard-to-reach areas and secondary schools. 5.2.1.3.2 DEMAND CREATION The most significant demand creation innovations are: YOUTH COUPLES COUNSELING. 39 IFPP innovated by integrating previously existing couples counseling teams into the project by providing them with family planning training. Qualitative data suggest that the counseling teams are effective because they improve knowledge of modern contraceptive methods and increase demand for family planning services through a series of 6 home visits. 40 TRADITIONAL BIRTH ATTENDANTS (TBAS). Originally trained by MOH to help women give birth at home, TBAs have been recognized by IFPP as a valuable entry point to reach women at home. IFPP, in conjunction with the MOH, has adapted the role of the TBA to also provide family planning services to expectant women and provide referrals to health facilities. Interviews with USAID, other donors and IFPP show that TBAs are effectively increasing local demand by improving knowledge of family planning, countering misconceptions, and conveying the importance of healthy timing and spacing of pregnancies. 38 The increase in family planning visits will be covered in more detail under Evaluation Question 2. 39 Youth couples counseling is discussed in more detail in Question 1.2: Community Environment. 40 Interviews with USAID, MOH, UNFPA, the Dutch Embassy, Health Providers, and IFPP. “Pathfinder has decreased the pregnancies and there is more information. The mobile brigades also serve the adolescents. They go to the schools. Those brigades were working also previously but not as much as today.” (Health Director, Monapo District) 34 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 5.2.1.3.3 HEALTH SYSTEM INTEGRATION OF FAMILY PLANNING WITH EXISTING HEALTH SERVICES. IFPP expanded family planning integration with other primary health care services, including Mother and Child Health (MCH). Integration affords a larger number of women the opportunity to get information and counseling about family planning services. This is innovative because women can receive family planning services at any public health facility at no cost, as well as attend to other medical needs. Qualitative data indicates that women and health providers are satisfied with the integrated service. SIGLUS.41 Introduced by the MOH and mainly supported by USAID’s Global Health Supply Chain Procurement and Supply Management (GHSC-PSM) project, IFPP leveraged the SIGLUS platform to ensure improved stock management of FP commodities at target health facilities with the aim of minimizing stockouts. While this is not an innovation developed by IFPP, the project identified SIGLUS as a useful tool to track FP supplies in real-time and decrease the occurrence of stockouts. Interviews with MOH, medical supply warehouse managers, IFPP and GHSC-PSM suggest that the platform is useful for notifying when supplies are needed at health facilities. Because this innovation was only recently introduced in June 2018 to select health facilities, more time is needed to determine how effective this innovation is in improving supply management. SIGLUS’s current capability to reduce stockouts is discussed more in EQ3. 5.2.2 EQ1 CONCLUSIONS KNOWLEDGE. Survey data indicate that there is a significant increase in knowledge of any modern contraceptive method among adult and adolescent females surveyed. Knowledge of all specific modern methods increased, but knowledge of some methods, like male sterilization, LAM and emergency contraception remain low. Young, poor, and rural women had lower levels of knowledge than other women, but the gap between wealthy and poor women has shrunk significantly from baseline to midline. DEMAND GROWTH. Interviews with USAID and MOH officials indicate that there is increased demand for family planning services. Survey data show that demand and unmet need for contraceptives have increased while demand satisfied and met need have decreased between baseline and midline. Wealthier and younger women have lower unmet need for contraceptives while older and poorer women have higher unmet need for contraceptives. SERVICE UTILIZATION. Overall use of modern contraceptives has remained stagnant. When women do use contraceptives, they vastly prefer modern methods over traditional methods, which are rarely used. Injectables, male condoms and pills are the most likely to be used. FACTORS INFLUENCING THE ENVIRONMENT FOR SEEKING FP SERVICES. Interviews with IFPP and service providers, and focus groups with beneficiaries, show that mobile brigades are 41 In addition to SIGLUS, IFPP mentioned the Clinical Mentorship application, the partnership between MOH and the OES “nudge” team, and the mCENAS SMS communications platform as innovative ways to increase uptake of family planning services (mCENAS sends free SMS messages to adolescents to generate interest in and provide accessible to family planning information through novela-style storylines). While these efforts are documented to some extent in IFPP reporting, they lacked any quantitative data – usage statistics, for example – and are therefore not highlighted as innovations in the midterm performance evaluation report. “The integration of FP services with other services has been useful. Now FP is integrated with all other MCH services. They are able to direct women to FP when women come in for other services, for example vaccinations, or antenatal care. Post-partum FP services are also provided at the maternity.” (Health Provider at Monapo Health Center) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 35 successful in targeting schools and households with family planning services. Community members report that couples counseling reduced the stigma surrounding family planning and led to the ability to discuss and seek out contraceptive services. FGDs with beneficiaries and interviews with community leaders reveal support for Community Health Workers, especially APEs, because they visit homes to provide family planning services and help to reduce taboos surrounding modern contraceptive methods. Finally, interviews with IFPP and FGDs should that IPC home visits with women reduce misconceptions and help women to talk more openly about family planning with their husbands. FEMALE EMPOWERMENT AND MALE ACCEPTANCE OF FAMILY PLANNING. Female empowerment and male acceptance of family planning is improving the enabling environment for family planning services. Survey data show that Heads of Households, most of whom are male, are increasingly in favor of using family planning to avoid pregnancy. Male acceptance of modern contraceptives has increased, as has female empowerment regarding family planning issues. Shared decision-making about family planning increased and is the principal way couples decide family planning. A majority of women feel confident in their ability to suggest to their husbands to wait before having another baby and to use FP methods. Poorer and rural women consistently report lower confidence than the mean across all of these indicators. Despite these successes, obstacles to female empowerment remain. Approximately one-quarter of men still make the family planning decisions for the household and a minority of respondents have discussed family planning with their partner over the past six months, relying instead on community members outside of the family. INNOVATIONS. IFPP successfully innovated across the three principal activity areas: service delivery, demand creation and health system support. • Service Delivery: Improved training for all health facility staff; improving the privacy of adolescent corners at health facilities42; Agente Polivalente Elementar (APE) home visits; and mobile brigades in rural areas and secondary schools have improved family planning service delivery. • Demand Creation: Youth couples counseling and Traditional Birth Attendants (TBAs) improve attitudes towards family planning and drive demand for services. • Health System: Integration of family planning with existing health services and the SIGLUS commodity inventory monitoring system improve the functionality of the health system. 5.3 EVALUATION QUESTION 2: ACCESS AND QUALITY EQ 2: To what degree have IFPP interventions increased access and quality of FP/RH services? • EQ 2.1: How has access to modern contraceptive methods and Family Planning/Reproductive Health services increased? To what extent have IFPP interventions contributed to that increase? • EQ 2.2: How has the quality of Family Planning/Reproductive Health services improved? How have IFPP interventions contributed to that improvement? • EQ 2.3: What are the main barriers to reaching youth and adolescents with FP/RH? (Learning Question) 42 Adolescent corners are not new to Mozambique, but the project has adapted them in innovative ways. This evaluation notes that adolescent corners in Nampula are generally seen as positive and helpful, according to youth and health facility staff interviewed. Specifically, interviewees believe that private spaces for youth has decreased the stigma of acquiring contraceptives. The 15-19 age cohort was also the only group to see contraceptive use increase between baseline and midline. However, IFPP and USAID should be aware that a large study on this topic concludes that service spaces designated for youth are not an effective way to increase mCPR among youth. 36 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 5.3.1 EQ2 FINDINGS 5.3.1.1 EVALUATION QUESTION 2.1 Access to facilities, providers, supplies and information for women and marginalized groups has improved in some ways and worsened in others. While the percentage of referrals to family planning services and radio broadcasts and community events about family planning decreased or remained stagnant, discussions between women and service providers about family planning increased. Access to modern contraceptive supplies also increased alongside access to accurate health information from health providers. Despite the fact that the use of any modern contraceptive method has increased among the 15 to 19 age cohort, some barriers to contraceptive access remain problematic for young people. 5.3.1.1.1 ACCESS THROUGH REFERRALS AND MESSAGING Referrals, community events and radio broadcasts are important elements in the overall IFPP messaging strategy to improve access to family planning. IFPP also uses digital applications, like mCENAS and Connecting with Sarah43, to increase access. Referrals44 to health facilities for family planning services by Community Health Workers are an essential indicator of the ability to access health facilities. According to IFPP data, IFPP referrals in Nampula have increased from 38,424 in Year 1 to 80,076 in Year 2. However, survey data indicate that the percentage of respondents referred by activists actually decreased, from 95% to 74%. Despite this decrease, the next section demonstrates that respondents who spoke to a health provider in the last 6 months actually increased from 3% to 45%.45 To explain the conflicting results between IFPP indicator data and survey results on referrals, IFPP posits that the decrease in referrals may be due to a programmatic shift. While IPCs were originally intended to visit many beneficiaries to provide counseling sessions and referrals, their mandate is now more focused on repeat visits to fewer clients.46 This strategy reduces the total number of referrals made by each IPC and therefore may contribute to the reduction in referrals seen in the midline survey results. It is important to note that the evaluation team was unable to corroborate this statement through other sources and therefore cannot determine the extent to which the overall decrease in referrals can be attributed to IFPP. Trends concerning referrals, participation in community events, and radio programs are shown in the graph below. 43 mCENAS sends free SMS messages to adolescents to generate interest in and provide accessible family planning information through novela￾style storylines. Connecting with Sarah is a performance management tool used by IFPP to customize follow-up visits from service providers. Due to a lack of data concerning usage of these platforms, the evaluation team could not determine if these methods have been successful. 44 Referrals provide an indication of the number of women who were reached by the project who go on to seek family planning services at health facilities. 45 These findings are seemingly contradictory, as one would expect that a decrease in referrals would go hand in hand with a decrease in respondents who spoke to a health provider about FP. One possible explanation is that more people are being visited but fewer referrals are being made. 46 This is also likely not due to an issue with timing, as the survey question asks whether the woman respondent had been referred by an activist, without specifying a time period. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 37 Graph: Referrals, Participation in Community Events, and Radio Programs Participation in community event in the past 12 months where FP was discussed remained static from 14% at baseline to 17% at midline (the percentage change is not statistically significant). Younger respondents ages 15 to 19 (21%), Muslim women (19%), wealthier women (18%) and unmarried women (20%) were more likely to participate in community events than other respondents. Respondents that heard a radio program discussing family planning in the past 6 months also decreased, from 32% at baseline to 25% at midline. This may be reflective of broader trends in the population rather than because of IFPP intervention, as radio ownership decreased among households between baseline and midline, from 38% to 23%. 5.3.1.1.2 ACCESS TO HEALTH PROVIDERS – CONTACT AT HOME AND HF As shown in the graph below, more respondents – including an increasing share of rural women – are attending medical appointments and discussing family planning with a health provider during appointments. 95 14 32 74 17 25 0 10 20 30 40 50 60 70 80 90 100 Woman referred to a health facility Participated in community event in past 12 months where FP was discussed Heard radio program discussing FP/HTSP in past 6 months Baseline Midline 38 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Graph: Access to Health Providers Of the 51% of respondents who went to a health appointment in the past 12 months (up from 35% at baseline), 67% of them discussed family planning, compared to 57% at baseline. Rural respondents were equally as likely as urban respondents to discuss family planning (69% and 63% respectively, although this change is not statistically significant). The urban-rural gap in respondents who discussed family planning in a health facility has been eliminated (35% at baseline to -5%47 at midline). Survey respondents who spoke to a health provider within the past 6 months about family planning also increased, from 3% at baseline to 45% at midline. Finally, women mostly attend medical appointments for Triage (general practitioner appointment, 25%) and Child Health Consultations (25%), which may indicate increased integration of family planning into existing health services. IFPP reports, interviews with MOH, health providers, community leaders, and Focus Group Discussions with beneficiaries suggest that IFPP contributed to increases in family planning discussions with health providers through access expansion efforts like mobile brigades48, APE home visits, improving the privacy of adolescent corners at health facilities, and integration of family planning into local health systems (as discussed in EQ1.3). 5.3.1.1.3 ACCESS TO MODERN CONTRACEPTIVE SUPPLIES IFPP has provided broad access to modern contraceptive supplies through accurate forecasting and delivery of commodities to local health facilities. This is explored in-depth in EQ3. Respondents rely on the public sector (hospitals, health centers, health posts and mobile brigades) to access the most commonly used contraceptives, like pills, injectables and condoms. Ninety-five percent 47 -5% signifies that at midline, rural respondents were actually more likely to discuss family planning at health appointments than urban respondents. 48 Although, as previously noted in EQ1.3, attendance of mobile brigades has remained static (the change from 10% to 8% is not statistically significant). 35 57 3 51 67 45 0 10 20 30 40 50 60 70 80 90 100 Women who had any health appointment in past 12 months Women who discussed FP (among women who had an appointment) Spoke to a Health provider in the past 6 months about FP Baseline Midline USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 39 of respondents got injectables at public facilities, 80% got condoms (up from 43% at midline) from public facilities, and 86% got pills at public facilities. Qualitative data support this finding – reports from Focus Group Discussions and interviews with service providers show that supplies of modern contraceptives at these facilities are readily available. Quantitative data generally align with these interviewees, with several caveats. First, according to SIGLUS data, most methods were readily available except for injectables, with stockout rates rising to 23% in Q4 of 2018 and 39% in January to March of 2019. Second, stockout levels of other contraceptives (not injectables) ranged from 0% to 3% in Q3 of 2018, but increased slightly to between 4% and 9% during Q1 of 2019. IFPP is partially responsible for maintaining access to contraceptive supplies because their monitoring, training and data collection efforts are aimed at reducing the likelihood and duration of stockouts. However, IFPP does not control the distribution of contraceptive supplies to health facilities and therefore is not fully responsible for these improvements. 5.3.1.1.4 ACCESS – CHANGES TO RESTRICTIVE POLICIES AND REDUCTION OF SOCIO-CULTURAL BARRIERS As highlighted in the Activity Results Framework, IFPP also addresses cultural and social barriers to contraceptive access, especially for adolescents, post-abortion women and women with HIV.49 Some of these efforts, like the push for long-term contraceptives in schools to improve access to modern contraceptives for adolescents, have not achieved desired results. This is discussed in detail below. SCHOOL CONTRACEPTIVE POLICIES. IFPP attempted to reduce barriers facing adolescent access to contraceptives. During the period under review, the Activity helped to pass legislation making it easier for adolescents to access long-term contraceptives at secondary schools. In Year 1, IFPP, in conjunction with MOH and MOEHD, made amendments to legislation regarding sexual and reproductive health, thus creating an enabling environment at schools to support FP service provision. However, this legislation was reversed by the MOH and MOEHD on March 4, 2019 after parents pushed back. Interviews with IFPP, MOH and USAID described the sense of discomfort and unease among parents worried that long-term contraceptive methods harmed the future fertility of their children. Now, only counseling, pills, and condoms are available at secondary schools. As discussed in Evaluation Question 1.3, IFPP also deployed mobile brigades to secondary schools to improve adolescent access to services. They also installed private adolescent corners at health facilities to reduce the shame adolescents experienced during visits. REDUCING SOCIO-CULTURAL BARRIERS FOR WOMEN. IFPP sought to address socio￾cultural barriers that are known to limit women’s access to FP services. Focus Group Discussions with beneficiaries and interviews with community leaders demonstrate that IFPP is successfully using a mix of 49 IFPP focuses on marginalized groups (including post-abortion women and post-partum women) through the Maternity Emergency Wards at health facilities. These women are given care particular to their needs, including counseling services. “We already use condoms, we got the condoms in the hospital, it was not difficult to get one, and the nurses did not ask many questions.” (Focus Group Participants, Nacala Porto District Hospital) “Previously, adolescents would receive any available method. But now nurses say that the adolescents only can get pills and condoms [at school]...The MOH went back in their strategy. That is definitively a handicap, a problem.” (IFPP) 40 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV approaches to reduce these barriers. Examples include youth couples counseling involving community elders; home visits from Community Health Workers and mobile brigades; broad integration of family planning with general health services; and improvements to service provider training. Each of these aim to improve community dialogue about sexual health, as well as de-stigmatize and encourage open conversations regarding sexual and reproductive health with health providers, between couples, and between parents and children. 5.3.1.1.5 ACCESS TO HEALTH INFORMATION Survey respondents have better access to information about the side effects or complications of family planning methods, as seen in the graph below. Graph: Prevalence of Misconceptions (% Correct Response to Misconception) Seventy-six percent of female respondents (aged 15-49 who ever used a contraceptive method in Nampula) at midline were informed by health providers about the side effects or complications of contraceptive methods, compared to 40% at baseline. Seventy-one percent of respondents at midline were informed about what to do if side effects occur, compared to 40% at baseline. In addition, survey respondents were better able to discern between true and false statements relating to widespread myths about contraceptive methods. Respondents improved the frequency of correctly identifying misleading statements, like “men can use a condom more than once” (a 21% increase from baseline to midline) and “the pill is effective even if a woman forgets to take it” (a 14% increase). However, not all types of knowledge increased. There was almost no change in knowledge of good child spacing practices. The evaluation team did not observe a statistically significant change concerning (1) the correct minimum time to wait to conceive after having a birth for the health of the mother and baby (87% at baseline and 86% at midline) and (2) the correct minimum time to wait to conceive after abortion before conceiving again (79% at both baseline and midline). 40 40 39 27 47 42 76 71 53 51 68 55 0 10 20 30 40 50 60 70 80 90 100 Women informed by health providers about side effects or complications of methods Women informed about what to do if side effects or complications occur Pill is effective even if a woman forgets to take it IUD can be stuck forever inside the woman Men can use a condom more than once A woman who uses injectables will never again have children in her life Midline Baseline Access to accurate health information increased Prevalence of misconceptions decreased USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 41 5.3.1.2 EVALUATION QUESTION 2.2 According to qualitative sources and IFPP documents, the quality of family planning services has improved. Overall, beneficiaries report that they receive high quality services from health providers. IFPP training may be responsible for the perceived high service quality, as observations of training and interviews with IFPP staff indicate that trainings are thorough and improve health provider performance. 5.3.1.2.1 QUALITY OF FAMILY PLANNING SERVICES AND MODERN CONTRACEPTIVES Overall, qualitative data collected suggest that the quality of family planning services is good. Patients are treated with friendliness and respect and get the services that they need free of charge. Seventeen out of 22 focus groups (77%) report that the quality of family planning services is good. Family planning service quality can also be measured by limited quantitative data from Community Scorecards.50 Health facilities in Nampula scored best on access to FP services, freedom to choose FP methods, availability of FP methods, and information about advantages and disadvantages of certain FP methods. Health facilities scored worst on healthcare worker punctuality (8/8 health facilities in Nampula scored “bad”); charging free services (3/8 “bad”); and waiting time (7/8 “bad”).51 Despite these data, the preponderance of Focus Group Discussion participants say that services are free of charge and wait times are low.52 5.3.1.2.2 QUALITY OF PROVIDERS – IFPP TRAINING Qualitative data show that providers were trained to provide high quality services. Review of project documents and interviews with IFPP and health facility staff suggest that trainings provided by IFPP improved health worker knowledge and skills in family planning. In addition, qualitative data suggests that IFPP is responsible for tangible improvements in service quality and access to information about contraceptive methods.53 As noted in the “Limitations” section of this report, the evaluation did not test the individual competency of trained staff to perform clinical procedures. However, interviews with IFPP staff do reveal a concerted effort to improve provider quality through time-intensive, hands-on training courses and routine supervision by health facility managers. 50 Community Scorecards tools to bring the communities and providers together to assess the quality of services. Based on the results, they jointly select 5 priority areas to improve services and develop an action plan. Community Scorecards are operational in 14 out of 385 health facilities (4% of facilities, 8 in Nampula and 6 in Sofala). 51 Relatório “A perspectiva dos utentes sobre a qualidade e humanizaçao dos serviços de planeamento familiar.” October 2018. 52 One Focus Group Discussion with women in Nacala complained that they had to pay for implant removal. 53 During the first year of implementation, IFPP implemented an 8-day on-site training to improve the skills of health providers. Trainings took place in high-volume health facilities, led jointly by IFPP and MOH. The course content covered: promotion of integrating FP with other services, sensitization of HF staff (clinical and non-clinical); counseling; and practical skills on how to administer LT methods. During the second year of implementation, IFPP implemented two 8-day trainings held in high-volume health facilities, targeting newly assigned providers. (IFPP Reports, Interviews with IFPP, MOH Managers and Staff, Health Facility Observation) “The service is friendly. There are always commodities at the HF, especially condoms.” (Male Adolescents, Monapo Sede) “At the hospital they treat us well when we go to get condoms and do HIV testing. They always give us for free.” (Female Adolescents, Nacala-Porto) “There is routine follow-up, mentoring and supervision. All trainees are followed up on after 2-3 months following the training… Every month a team comprised of regional coordinator, technical coordinator and ME advisor visits the district for one week.” (Health Officer, Mecuburi District) 42 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV One trainee from a low-volume health facility reported that because she has few clients, she lacks confidence in performing some clinical tasks (“I tried once to insert an IUD and I wasn’t successful. I am not going to try it again”). 5.3.1.3 EVALUATION QUESTION 2.3 Despite the fact that the use of any modern contraceptive method has increased among the 15 to 19 age cohort, there continue to be significant barriers to reaching youth and adolescents with family planning services. Restrictive school policies, parental pushback, health facility limitations and community stigma and cultural norms reduce the effectiveness and reach of family planning services for adolescents. 5.3.1.3.1 PARENTAL DISAPPROVAL OF FAMILY PLANNING SCHOOL RESTRICTIONS. As detailed in EQ2.1, IFPP documents and interviews show that parents petitioned schools to restrict the types of contraceptives students can receive from school nurses. As a result, schools no longer provide long-term contraceptive methods to students. SEXUAL EDUCATION. Qualitative data show that, in general, parents do not provide sexual education to their children. It is considered taboo for parents and children to discuss sex or family planning. Similarly, interviews with IFPP, MOH, community leaders and others mention that parents of girls do not generally consent to contraception use because they want them to marry early and bear children. Parental disapproval of family planning also means that they are not giving their children the money they need to get to the health facility, making it even harder for adolescents to access contraceptives. The graph below illustrates these barriers. Graph: Youth Barriers – Transportation and Permission54 54 Responses for this question also included a significant number of “I don’t know” answers – “Obtain permission to go to a health facility” had 19% at baseline and 5% at midline. “Obtain money for transportation” had 9% at baseline and 4% at midline. “There are parents who want their daughters to get pregnant… to profit from their daughter's marriage. Other parents do not approve to get contraceptive methods because of religion rules.” (Male Adolescents, Nacala-Porto District) 13 50 28 50 30 55 0 10 20 30 40 50 60 70 80 90 100 Obtain permission to go to health facility Obtain money for transportation Youth Midline Baseline USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 43 Fifty percent of survey respondents cited obtaining money for transportation as a barrier to access. This increases to 55% at midline for adolescents ages 15 to 19. Respondents also signaled that obtaining permission to go to the health facility was a “big problem” for 28% at midline, compared to 13% at baseline. This barrier is more acute for adolescents – 30% cite permission as a barrier to obtaining contraceptives. 5.3.1.3.2 HEALTH FACILITY LIMITATIONS UNFRIENDLY SERVICE. Interviews with adolescents, IFPP, USAID, health providers and community leaders reveal that while there have been noteworthy improvements in friendliness and youth acceptance at health facilities, some lower-level facilities continue to lack private adolescent corners and some adolescents continue to feel that service is unfriendly. DISTANCE AND FEAR. As shown in the graph below, 54% of survey respondents, and 55% of young women ages 15 to 19, consider distance to the health facility a “big problem”. Graph: Youth Barriers – Distance and Fear of Facility/Provider55 In addition, fear of being served by a male health provider increased from 15% at baseline to 30% at midline. This is not a particular preoccupation of young women, but rather of women in the lowest two quintiles of income (34%). Women with no education also considered this to be a “big problem” compared to women with a secondary education or higher (30% vs. 23%). Fear of being treated badly by the health provider when asking for contraceptives increased from 19% at baseline to 37% at midline. Young women, women in the lowest two quintiles of income and rural women consider this to be more of a “big problem” than other groups (41%, 42% and 39% respectively). Finally, fear that the health facility will not have the desired contraceptive available increased from 17% at baseline to 37% at midline. This does not seem to be a particular preoccupation for young women 55 Responses for this question also included a significant number of “I don’t know” answers, ranging from 18% to 21% at baseline for all questions to 10% to 12% for all questions at midline. Overall, there was a decrease in the “I don’t know” responses between baseline and midline. 17 19 15 48 37 37 30 54 35 41 27 55 0 10 20 30 40 50 60 70 80 90 100 Afraid health facility will not have the contraceptive method she wants Afraid of being treated badly by health provider when asking for contraception Afraid of being served by a male health provider Distance to health facility Youth Midline Baseline “In terms of barriers, I know that the main one is that the services are not adolescent-friendly. There are “adolescent corners” at the HFs but they are not functional.” (USAID) 44 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV (35%). However, unmarried women and women in the poorest two quintiles of income perceive this as a bigger problem than other groups (42% and 38% respectively).56 5.3.1.3.3 COMMUNITY STIGMA AND MISCONCEPETIONS According to Focus Group Discussions with adolescents, young people do not want to be seen by community members at health facilities because of the stigma and shame surrounding family planning. This is primarily a consideration for young women, who fear being labeled promiscuous, or prostitutes, by their families and community members. FGDs with women highlight religious stigma related to reproductive health, including the misconception that family planning is killing unborn children. Other false beliefs include the notion that use of implants, pills, and IUDs can harm women’s future fertility. Survey data support the qualitative evidence that community stigma is a barrier for large numbers of young women. As show in the graph below, one quarter of respondents, and 31% of young women, are afraid that someone will see them at the health facility and they will be judged by community members. Graph: Youth Barriers – Community Stigma Women in the lowest two quintiles of income, unmarried women, rural women and women with no education are more likely to fear this community stigma than other groups (29%, 29%, 27% and 26% respectively). 5.3.2 EQ2 CONCLUSIONS ACCESS THROUGH REFERRALS AND MESSAGING. According to IFPP data, IFPP referrals have increased, but survey data indicate that the percentage of women referred actually decreased. 56 This fear is likely not based on actual stockouts because, as will be seen in EQ3, stocks of the majority of modern contraceptives (with the exception of injectables) are readily available. 10 24 31 0 10 20 30 40 50 60 70 80 90 100 Afraid someone will see her, and she will be judged by the community Youth Midline Baseline “Our fathers and the rest of the community should never know or find out about an adolescent doing FP because its considerate shameful. In this community an adolescent who is known using contraceptive is excluded from the society, discriminated, they consider her a girl with bad reputation, a prostitute.” (Female Adolescents, Nacala-Porto District) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 45 Radio programming decreased from baseline to midline and community events where FP was discussed remained static. ACCESS TO HEALTH PROVIDERS. Women, particularly rural women, are increasingly likely to attend medical appointments and discuss family planning with a health provider during appointments. Women mostly visit health facilities seeking medical appointments for Triage and Child Health Consultations, which may signify increased integration of FP into existing MCH services. Furthermore, qualitative data and IFPP reports indicate that IFPP has expanded access to services through mobile brigades, APE home visits, and private adolescent corners at health facilities. ACCESS TO MODERN CONTRACEPTIVE SUPPLIES. IFPP increased access to modern contraceptive supplies through improved management of the contraceptive supply chain. Women rely on the public sector (hospitals, health centers, health posts and mobile brigades) to access the most commonly used contraceptives, like pills, injectables and condoms. This is explored in-depth in EQ3. ACCESS – CHANGES TO RESTRICTIVE POLICIES AND REDUCTION OF SOCIO￾CULTURAL BARRIERS. IFPP is addressing socio-cultural barriers to contraceptive access. Some of these efforts, like the push for long-term contraceptives in schools, have not achieved substantial results. IFPP deployed mobile brigades to schools to improve adolescent access to services, and installed private adolescent corners at health facilities to reduce the shame experienced by adolescents seeking contraceptive services. ACCESS TO HEALTH INFORMATION. IFPP has improved access to information about the side effects or complications of various family planning methods. According to survey data, adolescent and adult women were better able to discern between true and false statements relating to widespread myths about contraceptive methods at midline than at baseline. However, knowledge of child spacing practices did not change from baseline to midline. QUALITY OF FAMILY PLANNING SERVICES AND MODERN CONTRACEPTIVES. Overall, Focus Group Discussions with beneficiaries show that the quality of family planning services is good and that the perceived climate at health facilities targeted by IFPP is more respectful and client￾friendly. A majority of interviewees also note that local family planning services are free of charge (with one group saying they had to pay to have implants removed). Community Scorecards show generally good service, but also leave room for improvement in waiting times and health worker punctuality. QUALITY OF PROVIDERS – IFPP TRAINING. Qualitative data show that providers were trained to provide high quality services. Review of project documents and interviews with IFPP and health facility staff suggest that trainings provided by IFPP improved health worker skills and knowledge and may be responsible for improvements in service quality and access to information about contraceptive methods. Service providers with few clients may lack the clinical skills necessary to perform more complicated clinical tasks, like sterilization and implant insertion. BARRIERS – PARENTAL DISAPPROVAL. Parents quashed IFPP’s initiative to bring more long￾term contraception methods into schools. Qualitative data show that, in general, parents do not provide sexual education to their children, and most do not allow their children to use contraception for a variety of reasons related to socio-cultural norms and commonly held misconceptions about modern contraceptive methods. BARRIERS – HEALTH FACILITY LIMITATIONS. Interviews with adolescents show that while there have been improvements in friendliness and youth acceptance at some health facilities, some 46 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV lower-level facilities continue to lack private adolescent corners and some adolescents continue to feel that service is unfriendly. Finally, a majority of young respondents consider distance to the health facility as a big problem to be overcome. Other young respondents fear different aspects of health facility services. BARRIERS – COMMUNITY STIGMA AND MISCONCEPTIONS. Qualitative data suggest that community norms continue to favor early child marriage and encourage childbearing at a young age. Parents hold misconceptions that discourage use of modern contraceptives. Adolescents do not want to be seen by community members at health facilities because they fear being labeled promiscuous. 5.4 EVALUATION QUESTION 3: COMMODITY MANAGEMENT EQ 3: To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? • EQ 3.1: To what extent have the forecast and quantities of commodities received met the needs at service delivery point? • EQ 3.2: At which level of commodities supply management chain there is high frequency of commodities stockouts of FP/RH? 5.4.1 EQ3 FINDINGS 5.4.1.1 EVALUATION QUESTION 3.1 IFPP is relatively effective at managing commodities at service delivery points despite recurrent stockouts of injectables. The forecasting and quantity of commodities received is adequate and stable for all other modern contraceptives. The national shortage of injectables, beyond IFPP’s control and only alleviated in late April 2019, continues to cause shortages in health facilities across Nampula province. Thirty-nine percent of facilities in Nampula have a stockout of at least one commodity, of which the vast majority are injectable stockouts. 5.4.1.1.1 SERVICE DELIVERY NEEDS MET With the exception of injectables, IFPP has succeeded in forecasting and delivering commodities to local health facilities. According to SIGLUS57 and IFPP quarterly reporting, 8% of health facilities in Nampula during Q3 experienced a stockout58 of one commodity sometime during that period. This stockout rate increased the following quarter (Q4 of 2018) to 23% and rose again to 39% from January-March 2019, as seen in the graph below. 57 SIGLUS is the Logistics Management Information System for Health Units in Mozambique. It is a digital commodity stock management system used by the MOH to track and deliver commodities from warehouses to health facilities across the country. IFPP is helping to implement this system and train staff to operate it in 140 facilities in Nampula province. 58 “Stockout” is defined as the lack of availability of any contraceptive method that the health facility is expected to provide. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 47 Graph: Contraceptives Stockouts in Nampula 2018-2018 This increase in stockouts across Nampula has one root cause – a national shortage of injectables due to a delayed international shipment that has affected the availability of this contraceptive method at many local-level facilities. The graph above shows that all stockouts of other commodities – implants, IUD, pills/COC,59 and pills/POP60 – did not surpass 9% over this time period. Stockouts of injectables, therefore, are by far the most significant contributing cause to the drastic increase in stockouts in one commodity over the activity implementation timeframe. Finally, stockouts of 2 commodities from January to March 2019 were 11%. Stockouts of more than 2 commodities at the same time are negligible. Focus Group Discussions with beneficiaries show that contraceptives are generally available at service delivery points. Men, women and adolescents who participated in FGDs did not face any stockouts at health facilities. Additionally, only 1 of the 6 health facilities visited by the evaluation team reported experiencing any contraceptive stockouts at all since they began working with IFPP. The exception was the Metuchiria Health Center, which did not receive any pills, condoms or injectables over the past year.61 5.4.1.1.2 PATCHWORK SYSTEM The MOH employs an inefficient patchwork of commodity verification tools to supply service delivery points with necessary contraceptives. In the absence of a single, unified logistics system, IFPP has had to develop piecemeal solutions like technical assistance; assistance in the rollout of SIGLUS; management of 59 Combined oral contraceptive. 60 Progestogen-only pill. 61 IFPP explains that they did not receive calls or visits from the project over the last year because IFPP community assistance only began in September 2018. The Metuchiria Health Center is a rural health post which was upgraded to a health center one year ago. Back then, patient demand was low. 6% 9% 4% 36% 8% 23% 39% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Q3 2018 Q4 2018 Q1 2019 IUD Pills/POP Pills/COC Implants Injectables Stockout of 1 Commodity “There are no shortages of any contraceptive methods at the health post. We experienced no stockouts since I was trained.” (Health Provider at Unidade Moçambicana Health Post) 48 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV the old Comm-Care stock verification system; physical on-site stock verification; and transportation to improve the short-term functioning of the medical supply procurement and distribution system.62 IFPP’s commodity management efforts are limited to forecasting, requisitioning and facilitating delivery of stock to select facilities in Nampula and Sofala. The project is not responsible for the maintenance or quality of the digital stock management tool, SIGLUS. That responsibility falls to the USAID Global Health Supply Chain Program-Procurement and Supply Management (GHSC-PSM) project. Further, GHSC-PSM works directly with the MOH and the Central de Medicamentos e Artigos Médicos (National Drugs, Commodities and Supplies Warehouse, or CMAM) which control management and distribution of key commodities nationwide. Based on activity reports, IFPP oversees SIGLUS data collection in 140 of 232 facilities (60%) in Nampula province. In health facilities where SIGLUS is used, interviews with staff show that the tool is intuitive and helpful at maintaining stocks of family planning commodities. However, it is a duplicative and inefficient system, as physical paper stock cards and requisition forms are still in use which require onsite verification.63 In addition, staff turnover requires continuous training on how to use the SIGLUS system. In one recent example cited by IFPP staff, 10 out of 17 facilities in Menapo district needed training for new staff who did not know how to order needed quantities of commodities through the SIGLUS system. Finally, staff do not collect accurate data concerning the duration of stockouts experienced by health facilities because they do not know how to input this data into SIGLUS. 64 In addition to SIGLUS training, qualitative data show that IFPP provides technical assistance to health facility staff on how to complete stock cards and requisition forms.65 This assistance is complemented by weekly supervision from the Commodity Management Task Force (made up of IFPP, DPS and District Medical Store [DPM] staff) to check on commodity stocks. IFPP also assists with the monthly transportation of contraceptives from the districts to health facilities because the district warehouses do not have funding for transportation. 5.4.1.2 EVALUATION QUESTION 3.2 For all commodities except for injectables, there is no evidence to suggest that national, provincial, district, or local-level facilities have a high frequency of commodities stockouts. The recent shortage of injectables affected all levels, from the national level down to the local level. At the local level, evaluation team observations and SIGLUS data show that stockouts are relatively uncommon and stockout rates for all commodities except injectables have remained relatively low since IFPP intervention began. 62 The goal of the MOH is to eventually transfer all stock data collection from Comm-Care to SIGLUS by August 2019. SIGLUS is considered by all partners to be more accurate than Comm-Care, although IFPP notes that SIGLUS is also frequently inaccurate. Another layer of difficulty includes the procurement of commodities from outside Mozambique. Sometimes these shipments are delayed, complicating efforts to distribute commodities to health facilities before stockouts occur. The evaluation did not receive qualitative data about this issue and therefore cannot weigh in on IFPP’s role in this process. 63 Interviews with MOH, Medical Supply Warehouse Managers, IFPP and GHSC-PSM. It is unclear if the task of physical stock verification is only a short-term remedy, or if this will be required in conjunction with SIGLUS data entry. 64 Understanding the length of facility stockouts is a critical indicator for the success of the commodity management system that is not currently captured in SIGLUS data. 65 Interviews with IFPP, MOH, District Warehouse Managers, and Health Providers. “No Pathfinder, no transport, no contraceptives.” (District Chief Medical Officer) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 49 Besides injectables, there are no data showing frequent stockouts of contraceptives at the national, provincial or district levels. However, interviews with MOH, Provincial and District Warehouse Managers, other donors, GHSC-PSM and International Center for AIDS Care and Treatment Program (ICAP) reported occasional shortages (imminent stockouts) at the local level which were resolved.66 As seen in the previous section, quantitative data from SIGLUS show low stockout levels of all contraceptives except injectables, ranging from 0% to 3% in Q3 of 2018. Stockout levels increased during Q1 of 2019, where stockouts levels for contraceptives other than injectables ranged from 4% to 9%. 5.4.2 EQ3 CONCLUSIONS SERVICE DELIVERY NEEDS MET. IFPP is relatively effective at managing commodities at service delivery points despite high stockouts of injectables. The forecasting and quantity of commodities received is adequate and stable for all other modern contraceptives, although there has been a small increase in stockouts of most commodities between from Q3 of 2018 to Q1 of 2019. The national shortage of injectables, only alleviated in late April 2019, continues to cause stockouts across Nampula province. This shortage, outside of project control, accounts for the vast majority of stockouts at Nampula’s health facilities. PATCHWORK SYSTEM. IFPP has had to combine a patchwork of solutions – technical assistance, SIGLUS rollout assistance, stock verification, and transportation – to improve the short-term functioning of the existing contraceptives procurement system. IFPP is not responsible for the overall functioning of this system and has performed well within its mandated parameters. 5.5 EVALUATION QUESTION 4: SUSTAINABILITY EQ4: How has the project ensured that there is local ownership of interventions that have been introduced? What is the likelihood of project achievements continuing after the project ends? 5.5.1.1 EVALUATION QUESTION 4 The project’s efforts to align and blend with MOH plans and activities since inception improves the chances of transitioning project activities to local (Mozambican) ownership after the project ends, even though IFPP currently has no written transition document describing how this would be done. The likelihood of project achievements continuing after the project ends depends to a large extent on the size of the local budget, political commitment, technical knowledge, and the nature of different IFPP interventions. While technical knowledge and political commitment at the MOH exist, there is no budget allocated to continue most IFPP tasks. The contraceptive supply chain will only survive with donor support but can continue without direct IFPP intervention. Service providers have received a long-term benefit from training and learning materials, but additional training and supervision may be discontinued. Finally, some community-level visits from health workers will come to an end, but the evaluation team is cautiously optimistic that knowledge and cultural changes will remain. 66 Interviews with MOH, Provincial and District Warehouse Managers, Other Donors, GHSC-PSM and ICAP. “Other than the occasional stockouts at the SDP [Service Delivery Point], there are no stock outs at any other level of the supply chain. There have been delays due to transportation issues, but no stockouts.” (GHSC-PSM Provincial Representative) 50 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 5.5.2 EQ4 FINDINGS 5.5.2.1.1 LOCAL OWNERSHIP ALIGNMENT WITH MOH STRATEGY. While the project cannot assure that there will be local ownership of interventions after the project ends, IFPP has improved the chances of a successful transition through a close alignment of activity interventions and integration of family planning strategies with MOH. According to interviews with MOH, IFPP, and a review of IFPP/MOH documents, Pathfinder collaborated with the MOH during the development of the National Family Planning Strategy prior to IFPP startup. Pathfinder’s close relationship with MOH continued after IFPP was awarded to the current consortium partners, of which Pathfinder is the prime implementing partner. The integration of family planning with health services and the training of service providers are the two key strategic elements that align IFPP and MOH priorities. As a result of this collaboration, IFPP efforts have the support of MOH officials at all levels and have strong potential to transition to local ownership after the activity ends. INTEGRATION. Furthermore, IFPP did not create a parallel system of tools and actors to implement activity interventions. Instead, the project integrated with existing MOH efforts such as Community Health Worker deployments, training guidelines and data collection systems. For example, IFPP is using the MOH Health Management Information System (HMIS) to collect and report FP data. Similarly, the activity uses the training guidelines and standards approved by the MOH for IFPP’s training programs of Community Health Workers. TRANSITION DOCUMENT. Despite significant IFPP efforts to support a sustainable transition through alignment of priorities and activities with the MOH, IFPP does not currently have a written transition strategy in place that outlines how and when the activity interventions and achievements will fully transition to local control once IFPP ends. This document is essential to describe the steps that each partner must take, within a reasonable timeframe, to ensure that a smooth transition takes place. 5.5.2.1.2 CONTINUATION OF PROJECT ACHIEVEMENTS POLITICAL WILL AND FUNDING. With the exception of one interviewee from USAID, no respondents interviewed supported the view that MOH and the government lack political will to continue IFPP activities after the project closes.67 In fact, Mozambique published a “Family Planning 2020 Commitment” document in 2017, in which the country commits to “1) increase the use of modern contraceptive methods for adolescents (15-19 years old) from 14.1% to 19.3% in 2020; 2) provide FP services (information and contraceptives) in all secondary schools by 2020; and ensure that 30% of all health public facilities use 67 The USAID interviewee stated: “FP is not a priority for the MOH even if it is for the government. One important thing: there will be a national FP meeting we will see that every province should have one FP focal point. There is no one now.” “IFPP is good in helping in the provinces, they don’t organize parallel programs but work in the existing structure – this is the difference with IFPP compared with other implementing partners”. (MOH) “I believe what Pathfinder doing is a continuity of what has been implemented by the MOH strategic plan.” (MOH) “One of the biggest challenges is that the government does not have resources for FP.” (IFPP) “Budgeting is very complicated, and we don’t really know how to do it.” (MOH) USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 51 electronic stock management information system including contraceptives by 2020.” 68 These written commitments align with IFPP strategic objectives. However, nearly all interviewees have highlighted that funding is the single greatest limitation to building on IFPP’s achievements after the project closes. MOH does not have the budget to cover most of the activities currently funded by the project. This has implications for each aspect of the project, as described below. CONTRACEPTIVE SUPPLY CHAIN. The contraceptive supply chain will only continue with donor support. At the national level, an estimated 95% of contraceptives are obtained through donations, especially from USAID and UNFPA. The GRM purchases only 5% of contraceptives supplied through public health facilities. 69 According to IFPP, at present, MOH does not assign any budget towards contraceptives purchases. Given these circumstances, it is highly likely that the contraceptive supply chain will not continue in the absence of donor support. In addition, IFPP is responsible for helping the MOH implement the SIGLUS system at all health facilities in Nampula. Sixty percent of facilities (140/232) have transitioned to SIGLUS, staff are being trained to use the system, and IFPP deploys teams to facilities to conduct on-site verification of commodity stocks. Once IFPP closes, the SIGLUS rollout and the verification of commodity data will likely discontinue without additional financial support. SERVICE PROVIDER CAPACITY. Service providers have improved technical capacity through training provided by IFPP and “training of trainers” sessions to build additional technical resilience into the health system. Health providers who have undergone training have gained new skills and knowledge in family planning. Trained staff are able to build capacity of their peers through on-the￾job training, thus making a positive contribution to future generations of service providers.70 However, this capacity improvement is not permanent. The current training model relies on continued supervision and refresher training. Physical infrastructure will also remain in place, including the adolescent corners at health facilities. Because MOH and IFPP jointly conducted training sessions and use the same training manuals, this material can contribute to continued capacity building among health providers. However, this training model is only likely to work after the activity closes if MOH absorbs existing costs to ensure that trained staff receive continued support and that new staff are trained. This essentially requires the government to allocate human and financial resources that are currently provided by IFPP. COMMUNITY-LEVEL DEMAND GENERATION. Demand generation interventions implemented by IFPP through the deployment of Community Health Workers, mobile brigades, and others have led to significant gains in knowledge and positive cultural changes within communities. The evaluation team is cautiously optimistic that knowledge and cultural changes are sustainable in the short and medium-term. However, the future of demand generation interventions themselves is less clear due to budgetary concerns. Visits from TBAs and APEs may continue, as TBAs receive small stipends (which will need to be covered by MOH or another entity), and APEs are currently paid by UNICEF. IPCs, couples counselors 68 See GRM 2012 commitment to FP 2020. 69 Interviews with Other Donors, MOH and IFPP. 70 Interviews with MOH, IFPP and Other Donors. “The people trained will be here and they stay. This will increase the likelihood of sustainability. The same people can stay on and continue to train others.” (MOH) 52 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV and mobile brigades may discontinue their work if MOH does not provide transport or does not pay them to visit households to discuss family planning. Messaging tools will also likely discontinue if MOH does not assume the cost of radio broadcasts, printed FP materials and IFPP’s digital messaging platforms, mCENAS and Connecting with Sarah. 5.5.3 EQ4 CONCLUSIONS LOCAL OWNERSHIP. IFPP has aligned its interventions and integrated family planning policy strategies with MOH. As a result, IFPP efforts have the support of MOH officials at all levels and have strong potential to transition to local ownership after the activity ends. While IFPP supports a sustainable transition through training and transferring of processes, IFPP does not have a written transition strategy outlining how and when the activity interventions and achievements will transition to local control once IFPP ends. CONTRACEPTIVE SUPPLY CHAIN. The contraceptive supply chain will not continue without support from donors. In addition, the rollout of SIGLUS and commodity stock verification and data quality assurance are imperiled if MOH does not assume responsibility for these tasks after project closedown. SERVICE PROVIDER CAPACITY. Enhanced service provider capacity will likely sustain after the end of the activity, and trained experts can also pass technical knowledge on to the next generation of service providers. Physical infrastructure, like private adolescent corners at health facilities, will remain in place. However, the costs of trainers, trainees, supervisors, transport, and MOH visits to facilities – all currently covered by IFPP – will need to transition to MOH control. COMMUNITY-LEVEL DEMAND GENERATION. Demand generation interventions have led to significant gains in knowledge and positive cultural changes. These changes are likely sustainable. However, many demand-generation activities will discontinue in the absence of MOH commitment to sustain them. IPCs, couples counselors and mobile brigades are not likely to continue; neither are messaging tools like radio broadcasts and printed FP materials. APE activities can continue because they are funded by UNICEF, and TBA activities may continue if they receive small stipends from MOH or another donor. RECOMMENDATIONS MMEMS developed the following recommendations that should be considered by IFPP and USAID to improve performance and enhance the sustainability prospects of project efforts. 6.1.1 DEMAND CREATION • IFPP should increase contraceptive utilization rates through intensified efforts in Nampula province. The youth cohort has shown the strongest growth in contraceptive use from baseline to midline. This trend may continue if the project remains focused on their needs. The project should consider other ways to further deepen the integration of health services to attract women from many different groups, especially poor and rural women. • IFPP should continue and intensify mobile brigades, youth couples counseling, CHW deployments, and IPC home visits to reach more adolescents and men in both rural and urban areas. IFPP should USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 53 disaggregate the CHW indicator (which tracks visits to beneficiaries) to show the individual contribution of IPCs. • Evidence shows that knowledge has increased, but this has not always translated into increased contraceptive use. IFPP should assess whether efforts can be refocused on confronting barriers that could be blocking women from using family planning methods. A possible area of focus could be the 24% of men who exclusively make household family planning decisions. Designing a strategy focused on convincing these men to loosen their grip on FP decision-making could help to spur more women to seek out family planning services. • IFPP should assess the extent to which community platforms like churches, mosques, political parties (including women’s leagues), and sports clubs could be leveraged to drive demand for family planning services and increase community acceptance of family planning. 6.1.2 ACCESS AND QUALITY • IFPP should develop specific activities that engage parents of adolescents as change agents for increasing the demand and use of contraceptives among adolescents. Consider collaboration with other USAID-funded activities, like the Unidade de Assistência Técnica para a Educação Funcional (UATAF) “Strengthening School Council Capacity for Sustainable Organizational Development” project, to work in conjunction with school councils, community leaders, parents and teachers to mobilize and sensitize family planning in schools. IFPP should also consider whether religious leaders, matronas71 and extended family members have a role to play in increasing parents’ acceptance of contraceptive use. • Use marketing materials to reinforce the fact that contraceptives are free. • Increase the use of Community Scorecards to assess health facility performance. Reinforce the service-oriented role of health facility staff and the need to arrive on time to attend to clients. • Continue efforts to de-stigmatize family planning. Deeply held socio-cultural beliefs remain persistent and are a major barrier to overall acceptance, access, and use of modern contraceptive methods. IFPP should continue to build on the successes of mobile brigades, IPC agents, and Community-Based Organizations (CBOs) to reduce stigma related to family planning and build community trust, promote open discussion and encourage uptake of family planning services. • IFPP should explore ways to maintain the clinical skills of trained staff working in small health centers where client case load is low. One such method is called “visitas de troca de experiencia”, in which staff could rotate to high case load facilities for a finite period to increase exposure to more clients who need implants, sterilization, and other minor surgical procedures. • Explore efforts to increase access for young, poor, rural and uneducated women. This is the demographic that lags behind the average across most indicators related to contraceptive access. Efforts may include additional home-based services through TBAs and other community members, as well as a pilot deployment of male CHWs to further increase acceptance of FP methods among the predominantly male Head of Household (HoH) population. Expansion of private adolescent corners could also help to increase access for young people, although the project should further study the overall effectiveness of adolescent corners prior to any buildout of these services. • Assess whether the referral system, radio broadcasts, and community events are effective methods for increasing contraceptive access young women. For example, radio ownership is decreasing while 71 Matronas perform the same functions as the Traditional Birth Attendants (TBAs), similar to a midwife. 54 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV television ownership is increasing, so a shift to more modern messaging techniques may be necessary. 6.1.3 COMMODITY MANAGEMENT • IFPP should collaborate directly with MOH and GHSC-PSM improve the overall functionality and utility of the SIGLUS system. IFPP should jointly plan a workshop to discuss several important issues with key partners including: (1) a plan for the rapid transition of all health facilities from Comm-Care to SIGLUS; (2) specific SIGLUS performance enhancements to improve system utility; (3) a plan to transition to a digital system without reliance on on-site verification of paper stock cards and requisition forms; and (4) development of a risk mitigation plan to reduce the likelihood of future injectables stockouts due to failed international shipments of critical supplies. • IFPP should undertake additional training for SIGLUS users. Specifically, SIGLUS users must be trained to accurately input stockout start and end dates. Without these data, stockout durations cannot be calculated and the severity of stockouts cannot be determined. 6.1.4 SUSTAINABILITY • Develop a transition plan with MOH. This should begin immediately, as the execution of the transition plan will take significant time and effort. The plan should also address the role local communities will play throughout the transition process and after closedown. • Search for other sources of funding to continue trainings, CHW visits, and effective messaging strategies after the project ends. IFPP should consider coordinating with the MOH through the FP Technical Working Group (TWG) to advocate for a budget dedicated solely to family planning. In addition, IFPP could develop an advocacy toolkit to support resource mobilization towards FP programming post-2020. 6.1.5 USAID • USAID should closely monitor the following IFPP indicators – mCPR, unmet need, meet need, demand satisfied and demand – to ensure that indicator targets are being met. USAID should examine underlying supporting data contributing to these indicators to further understand how the project defines, monitors and calculates them. A data quality assessment (DQA) could help to shed light on the discrepancies observed between IFPP data and MMEMS survey data concerning the usage, demand and need for family planning services. • IFPP deployed the mCENAS and Connecting with Sarah mobile applications. According to IFPP, these have been successful. However, due to a lack of data about usage of these platforms, the evaluation team could not say if these methods have been successful. USAID should request usage statistics for these platforms to determine how widely they are being used and what can be done to improve their effectiveness. • USAID should collaborate with IFPP on the best approach for a sustainable transition of project activities to MOH before project closedown. USAID may be able to serve as a bridge between MOH and IFPP to jumpstart the transition, and USAID could help to smooth out problems if difficulties arise during the transition period. MMEMS developed the following recommendations that should be considered by IFPP and USAID to improve performance and enhance the sustainability prospects of project efforts. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 55 • USAID and IFPP should ensure that the activity is including adaptive management strategies to respond to changing socioeconomic circumstances among target households. There is some evidence that communities have decreased ownership of household goods since the baseline evaluation was conducted (15% fewer own a radio at midline compared to baseline, 19% fewer own a mobile phone, 12% fewer own a bicycle, 7% fewer own a motorcycle). The project must proactively plan and adapt strategies to confront these and other constantly evolving challenges. 56 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX A: FINDINGS AND CONCLUSIONS MATRIX EVALUATION QUESTION 1. DEMAND: To what extent has IFPP increased demand for modern contraceptive methods for family planning and reproductive health?72 1.1 How has the knowledge and awareness of modern contraceptive methods increased? How has that change contributed to the demand increase? FINDINGS CONCLUSIONS KNOWLEDGE OF CONTRACEPTIVE METHODS • Knowledge of any modern contraceptive method among women ages 15 to 49 increased from 80% at baseline to 92% at midline. Knowledge of all modern methods increased, with the largest percentage gains for implants (35% to 59%), injectables (66% to 84%), pills (63% to 78%) and male condoms (48% to 62%). Knowledge of certain methods remains low, including male sterilization (9%), LAM (7%) and emergency contraception (6%). Women ages 20-24 had the highest knowledge of any modern method (93%) while women 15-19 had the lowest (88%). Women in the lowest 2 quintiles had less knowledge (88%) than women in the highest quintile of income (96%). Rural women had less knowledge than women in urban areas (92% vs. 98%). In terms of change between baseline and midline, while the gap has remained the same from baseline to midline between younger and older women (5% gap vs. 5% gap) and rural and urban women (7% gap to 6% gap), there has been enormous progress in closing the knowledge gap between q5 women and q1&2 women (23% to 8%) • Modern contraceptive methods about which women have the most knowledge are the most likely to be used. Top 3 modern methods used ever (injectables 21%, male condom 16%, and pill 14%) are the same as the methods that women have most knowledge of (injectables 84%, pill 78%, and male condom 62%). By this metric, we could expect that implants will also be used more frequently in the future, as 59% have knowledge of them. • In both urban and rural areas, 20 out of 22 adolescent and adult focus groups had knowledge of some contraceptive method. Women also reported that they were more willing to seek FP services than men. (Focus Group Discussions) • In peri-urban and urban areas, adolescent girls and boys had good knowledge and awareness of modern contraceptives (7 out of 10 focus groups). A proportion of girls and boys mentioned that they had started learning about modern contraceptives two years prior to IFPP, probably through an HIV/AIDS program. In rural areas, girls were less knowledgeable and less willing to discuss health seeking behavior or modern contraceptive use. (Focus Group Discussions) • The evaluation noted community leaders’ awareness of the IFPP Activity and attributed any positive changes linked to improvements in knowledge and demand for FP services to Pathfinder International. “The attitudes, principally among the men, improved. The couples also go to the family planning consultation together. People talk more about FP compared with just a couple of years ago. Although, still the woman has to convince her husband to use FP method”. (Community leaders in Monapo Sede) (Community Leaders) DEMAND GROWTH EVALUATION QUESTION 1.1 Knowledge and awareness of modern contraceptives and has increased among most women surveyed. This improvement is correlated with an increase in demand, but a large proportion of this demand remains unsatisfied. Increased demand is also uncorrelated with actual contraceptives usage – while both met and unmet demand has increased, overall use of contraceptives has remained stagnant. KNOWLEDGE OF CONTRACEPTIVE METHODS Survey data indicate that there is a significant increase in knowledge of any modern contraceptive method among adult and adolescent females surveyed. Knowledge of all specific modern methods increased, but knowledge of some methods, like male sterilization, LAM and emergency contraception remain low. Young, poor, and rural women had lower levels of knowledge than other women, but the gap between wealthy and poor women has shrunk significantly from baseline to midline. Qualitative data also show that the majority of focus group participants had knowledge of contraceptive methods. DEMAND GROWTH Both qualitative and quantitative data show that demand for contraceptive services increased. Interviews with USAID and MOH officials indicate that there is increased demand for family planning services. Survey data show that unmet need for contraceptives has increased significantly. Wealthier, younger women have lower unmet need for contraceptives while older and poorer women have higher unmet need for contraceptive services. SERVICE UTILIZATION Increased demand is not correlated with actual modern contraceptives usage – while demand has increased, overall use of modern contraceptives has remained stagnant. Demographic breakdown of use data shows the following: 72 The quantitative data examined to answer all questions outlined in this FCR matrix are exclusively from Nampula province. Survey work in Sofala was halted because of Cyclone Idai. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 57 • Three out of four KIIs with USAID reported to have increased demand in Nampula and Sofala, based on increased utilization of FP services at service delivery points.73 (Interviews with USAID) • Out of the 9 respondents interviewed at MOH provincial and district level directors, 7 expressed increase in demand for FP services. Reports suggest greater FP service uptake (utilization) at IFPP service delivery points. However, for adolescent target groups, 11 out of 15 health providers reported that more intensified efforts will be required to encourage FP service uptake. It was noted that there was improved uptake among boys. (Interviews with MOH) • Women with unmet need for family planning increased significantly from 23% to 37% baseline to midline, which may indicate increased knowledge translating into increased demand. Wealthier women have a lower unmet need (32%) than poorer women (42%). Younger women 15-19 have lower unmet need (32%) than older women 25-49 (41%). Rural women have higher unmet need than urban women (40% vs. 35%). SERVICE UTILIZATION • % of women who have EVER used a modern contraceptive method has decreased, although not statistically significant (42% to 40%). Modern methods - sterilization was and is rarely used (male and female 0.4% baseline and 0.7% at midline), use of pill and injectables decreased (14% and 21% at midline compared to 19% and 28% at baseline), implants also decreased (6% baseline to 4% midline); but male condom use has increased (7% to 16% baseline to midline). Injectables, male condoms and the pill are the three most commonly used methods, comprising 51% of use of all modern contraceptives. Use of traditional method increased from 1% to 12%. Withdrawal increased from 0.3% baseline to 9.3% midline, and periodic abstinence increased from 0 to 4%. Use of traditional methods remains low. • % of all women CURRENT use of modern contraceptive – decreased from 20% baseline to 17% midline, although this decrease is not statistically significant. The most common modern methods currently used are injectables (10% midline, 13% baseline, not statistically significant decrease), pill (5% midline, increase from 4% baseline, not statistically significant increase, and implants (2% baseline, 2% midline). Overall current use of modern contraceptives is quite low. Traditional methods are not used at all. • Compare this data to CURRENT use of a modern contraceptive method for women in UNION: use of any modern method (19% baseline 16% midline). Implants (10% midline, 2% baseline) and IUD (4% midline, 1% baseline) are the most common modern methods for the married demographic. Traditional methods don’t feature at all among this group. Long-term methods are more currently used than any short-term method for women in union • Women ages 30-34 are most likely to use any modern method (37% baseline, 21% midline) followed by 25-29 age group (24% baseline, 19% midline) and 15-19 (9% baseline, 18% midline). Use of any modern method among 15-19 has increased (13% baseline to 18% midline). Overall decrease in use among all age groups except for youth. Women age 20-24 are least likely to use a modern method (14%). The usage gap between younger and older women seen at baseline has essentially been eliminated. • Education: Women with a secondary education or higher are most likely to CURRENTLY use contraceptives (27% midline vs 46% baseline), while those with no education least likely to currently use contraceptives (15% midline vs 15% baseline). • Women in highest 20% income use compared to poorest two quintiles at midline (28% vs. 11%). Gap has remained unchanged (15% at baseline vs. 17% at midline). • Current use of modern contraceptive: 19% for Muslim women versus 16% for others at midline. • Rural women use modern contraceptives at a lower rate than urban women. This gap has remained unchanged from baseline to midline. • Young people are using modern contraceptives at a higher rate than at baseline, yet other age groups have seen a percentage decrease in modern contraceptive use between baseline and midline. The usage gap between younger and older women seen at baseline has essentially been eliminated. • Married women are using contraceptives at a lower rate than at baseline, although the decrease is not statistically significant. They are mostly using implants and IUDs (while unmarried women use injectables and pills) reflecting their preference for more long-term contraceptive methods. • Women in the highest quintile of income are using contraceptives at a higher rate compared to all other income groups. The usage gap between high-income and low-income earners is wide and has not changed. • Muslim women use modern contraceptives at a higher rate than non￾Muslim women, although the difference is not statistically significant. • Women with a secondary or higher education are more likely to use modern contraceptives than women with less or no education. When women do use contraceptives, they vastly prefer modern methods over traditional methods, which are rarely used. Finally, knowledge of certain contraceptive methods does correlate with the types of methods women use. Injectables, male condoms and pills are both the most likely to be used and the types about which women have the most knowledge. 73 The evaluation did not include field visits to Sofala and therefore will not be able to establish whether there was a marked increase from Year 1 to Year 2 in that province. All three central MOH officers the team met also reported increased demand for FP in Sofala. 58 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • Rural 16% vs 27% urban for current use. Gap between urban and rural has not closed between baseline and midline. This gap has remained unchanged from baseline to midline. • Current use of long term/permanent methods by all women: female sterilization (0.3 %) IUDs (0.7%) implants (2%) baseline; female sterilization (0.8%) IUDs (0.2%) implants (1.4%) midline (KAP Survey) • Couple-Years of Protection (CYPs)74 have been increasing steadily in both Nampula and Sofala between Oct 2016 and December 2018. CYPs increased from 5,000 to 14,000 in Nampula and from 7,000 to 13,000 in Sofala between FY1Q1and FY3Q1 (MOH HMIS and IFPP M&E Documents) 1.2 How has the community environment contributed to support FP/RH service-seeking behavior? FINDINGS CONCLUSIONS FEMALE EMPOWERMENT AND MALE ACCEPTANCE OF FP • Acceptance of FP by the Head of Household (HoH)75 In favor of using FP to avoid pregnancies: 53% baseline, 65% midline. • “There is a great acceptability of modern contraceptive methods, even by the male partners / husbands. A lot of sensitization has been done in the communities, among the families, about to have intervals between children. First there were problems, with delays among men, they didn’t accept the messages right away. They wanted to have more children – even if they didn’t take care of them – but later on they started to understand and changed their consciousness” (Community leaders in Namina, Mecuburi district) • Women alone comprise 25% of main decision makers about the use of contraception (compared to 26% at baseline, although decrease is not statistically significant), followed by husband alone (24% vs 20%), Both women and husband joint decision making increased from 37% to 44%. • Ability to negotiate contraceptive use with the partner: Strongly agree. I feel confident in my ability to suggest to my husband to wait a few months before having another baby: 44% baseline, 63% midline. Suggest to my husband the FP method that we should use: 47% to 60%. Convince my husband on which FP method to use – 37% to 58%. If I wanted to use a FP method to avoid pregnancy, I feel confident in my ability to obtain the method I want to use: 41% baseline to 59% midline. I am able to ask my husband to use a condom if I want to: 25% baseline to 48% midline. Poorer, younger and more rural women consistently report lower confidence than the mean across all of these indicators. • Spoke to a health provider in the past 6 months about FP: 3% at baseline, 45% at midline. Discussed FP with someone else – partner, family member, friend, neighbor: baseline: 16%, midline: 33%. Most discussion are with friends and neighbors (13%), followed by “Other” (13%) and then family member (13%). Partner comes in last (10%). • The attitudes, principally among the men, improved. The couples also go to the family planning consultation together. People talk more about FP compared with just a couple of years ago. Although these men still say that “the woman has to convince her husband to use FP method” (Community Leaders, Monapo Sede) • Yes, the family planning methods are accepted in our community. (FGD male adults Unidade Mocambicana in Mecuburi district). FACTORS INFLUENCING THE ENVIRONMENT FOR SEEKING FP SERVICES MOBILE BRIGADES IN SCHOOLS AND COMMUNITIES EVALUATION QUESTION 1.2 The community environment is an important driver of FP service-seeking behavior. Overall, domestic attitudes about gender norms and family planning are improving. While significant barriers persist, women are more confident in proposing the use of contraceptives with their husbands, and men are more accepting of family planning practices. Outside of the household, women are more likely to seek out discussions about family planning than in the past, both with a health provider and with non-family community members. There is a strong correlation between the improved community and domestic environment for discussing family planning and the various strategies that IFPP has deployed to educate and inform the public about modern contraceptive methods. Interviewees and focus group respondents broadly applaud IFPP’s efforts at improving the environment surrounding seeking family planning services. FEMALE EMPOWERMENT AND MALE ACCEPTANCE OF FP Female empowerment and male acceptance of family planning is improving the enabling environment for family planning services. Survey data show that Heads of Households, most of whom are male, are increasingly in favor of using family planning to avoid pregnancy. Community Leaders across Nampula province echo this finding, saying that men have accepted the need for modern contraceptives. Increasing male acceptance of modern contraceptives correlates with female empowerment regarding family planning issues. Nearly half of women have spoken to a health provider in the past six months about family planning, a significant increase from the baseline. Shared decision-making about family 74 CYP is the estimated protection provided by contraceptive methods during a one-year period, based upon the volume of all contraceptives distributed to clients during that period. 75 81% of the HH heads were men in the baseline compared to 76% at midline. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 59 • Pathfinder and PSI created an enabling environment by targeting urban and rural secondary schools and households through the use of mobile brigades of health providers that spread family planning information. (Interviews with USAID, GRM MOH, IFPP) • “There have been approaches to create an enabling environment. They are working with the family members and at the schools through mobile brigades. I think, yes, they have done a good job.” (USAID) • Attended mobile brigade in the past 6 months: 10% to 8% - no statistically significant change. YOUTH COUPLES COUNSELING IN RURAL AREAS • Run by N’Weti, this activity targets young couples for counseling in rural and combo areas. Peer educators, or facilitators team with community leaders and community-based organizations to provide this counseling. In rural areas, de￾stigmatization of modern contraception was addressed by IFPP using CBOs to provide group counseling for young couples (ages 15-30) involving influential leaders in sessions over a six-week period. (Interviews IFPP) • Community members reported positive changes in being able to talk openly about FP services and use of modern contraception. Many attributed this to Pathfinders efforts (citing couple counselling). This was mentioned in 10 out of 12 focus groups with adults, ages 15 to 49. (Focus Group Discussions) • ¨There is an increase in acceptability in the communities both by us women and our husbands. Our husbands were difficult to convince or to talk to about FP because it was believed to cause infertility. But since the activist started with the community sessions and community dialogues about FP both with women and men they started learning and having information about FP”. (Women at Monapo Sede) COMMUNITY HEALTH WORKER TRAINING AND DEPLOYMENT • Community Health Workers comprised of Agente Polivalente Elementar (APEs) and Traditional Birth Attendants (TBAs), also called Activists, are trained in family planning by MOH and IFPP. • “Before the project we never had too much information about FP because it was a taboo, some considerate as secret and shameful to talk about FP. The people were not comfortable talking about it or coming to the HF to request for contraceptive and at times we only knew about 2 contraceptive which were DEPO and Pills only, so the options were limited.” (FGD female adults Namina in Mecuburi district) URBAN HOME VISITS BY COMMUNICATION AGENTS • This face-to-face counseling run by PSI deploys Interpersonal Communication agents (IPCs) to primarily urban areas to speak with women at home. IFPP recruited and trained IPC agents in urban areas to address misconceptions of modern contraception through household counseling within Activity catchment areas. (Interviews IFPP) • “Before it was difficult to convince our husbands to allow us to do FP. But now our husbands accept, because it’s something they have been sensitized to do as well. When the activists come to our house, they sometimes find our husbands and they are always willing to participate and learn.” (FGD Nacala Porto Urbano Health Center) planning increased and is the principal way couples decide family planning. A majority of women feel confident in their ability to suggest to their husbands to wait before having another baby, to use FP methods, and to use a condom with their husbands. Poorer, younger and more rural women consistently report lower confidence than the mean across all of these indicators. Despite these successes, obstacles to female empowerment remain. One￾quarter of men still make the family planning decisions for the household. A minority of women have discussed family planning with their partner over the past six months, relying instead on community members outside of the family. FACTORS INFLUENCING THE ENVIRONMENT FOR SEEKING FP SERVICES • MOBILE BRIGADES IN SCHOOLS AND COMMUNITIES. According to qualitative data, mobile brigades are considered successful in targeting schools and households with family planning services. However, survey data show no increase in attendance at mobile brigade sessions. • YOUTH COUPLES COUNSELING IN RURAL AREAS. Community members report that couples counseling reduced the stigma surrounding family planning and led to the ability to discuss and seek out contraceptive services. • COMMUNITY HEALTH WORKER TRAINING AND DEPLOYMENT. Qualitative data reveal support for Community Health Workers, especially APEs, because they visit homes to provide family planning services and help to reduce taboos surrounding modern contraceptive methods. • URBAN HOME VISITS BY COMMUNICATION AGENTS. Qualitative data suggest that urban home visits with women by IPCs reduce misconceptions and help women to talk more openly about family planning with their husbands. 1.3 Which local innovations, if any, bolstered FP/RH service delivery, demand-creation, and health systems? FINDINGS CONCLUSIONS 60 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • A majority of Key Informants believe that IFPP innovated within the FP arena. (National level interviews with USAID, MOH, UNFPA, Dutch Embassy) SERVICE DELIVERY • Key examples of service delivery innovations include integration of FP with other existing services in health facilities, onsite training for the entire health facility staff, provision of adolescent friendly services, use of APEs in providing injectables, pills and condoms, and mobile brigades for rural communities and secondary schools. (National level interviews with USAID, MOH, UNFPA, Dutch Embassy) INTEGRATION OF FAMILY PLANNING WITH EXISTING HEALTH SERVICES • Family planning is offered across all health facility services (antenatal, post-partum, HIV-counseling etc.), increasing the number of women who have the opportunity to get information and counseling. Innovative because women may not know how to seek out counseling or information regarding FP, but if they go to any health facility service, they also have the ability to receive family planning services. This integration allows women to receive all the services at the same time from the same HF instead in disparate places at different times. The effect of this innovation has been to bolster FP/RH service delivery, demand-creation, and the health system as a whole. • Beneficiaries did not comment on the integration of these services, but they did mention that service was of high quality: “The information about FP they reach us very well, we understand clearly and the nurses at the HF they are always available to help if we have any doubts, we address to them and they are always willing to help.” (FGD female Metuchiria health center in Monapo district). • “The services are great, first because it’s free we don’t pay for it and when we go to the HF to do FP the nurses take time to explain all the different methods and they give us an opportunity to choose which is best for us. And even if we chose a specific method and it doesn’t react well in our organism, we are able to go back to the HF and ask to change and they do it always. All the methods that we want are always available specifically talking about those that most of us use - DEPO/Injection.” (FGD female Monapo Sede in Monapo district). • IFPP has integrated FP services into Mother and Child Health Care (MCH) services at 6/6 observed HFs, levels 1 to 4, targeted by the Activity in urban, peri-urban and rural areas. (Health Facility Observations) • “We observe increasing numbers of clients due to the community work. I think that the integration of FP services with other services has been useful. Now FP is integrated with all other MCH services. They are able to direct women to FP when women come in for other services, for example vaccinations, or antenatal care. Post- partum FP services are also provided at the maternity.” Health provider at Monapo Health Center IMPROVED TRAINING FOR ALL HEALTH FACILITY STAFF • All health facility staff – cleaners, receptionists, nurses, etc. – are trained to provide friendly service. All health workers are trained to provide family planning services. Wait times for receiving family planning services are not very long. “The prescriptions, we get them with the facilitators, when they do the sessions and whoever is interested is given a slip and directed to the HF to go and the contraceptive, and the process is very fast because they are lots of nurses in the HF facility and thus we don’t wait too long on the queues to be attended.” (Female adults at Namina Health Center, Mecuburi). This is innovative because the training of the entire staff improves the enabling environment for community members and adolescents. • “All staff has been trained in and oriented to FP—total 56. The number includes the doctor, health technicians, general practitioners, MCH nurses, health agents, laboratory technicians and pharmacists. Entire staff receives orientation and clinical staff receives training on counseling and provision of short- and long-term contraceptives”. Health Provider Monapo Health Center EVALUATION QUESTION 1.3 IFPP successfully innovated across the three principal activity areas: service delivery, demand creation and health system support. SERVICE DELIVERY The most significant service delivery innovations are: Integration of Family Planning with Existing Health Services Family planning is integrated with other health facility services, including Mother and Child Healthcare (MCH). Women can receive family planning services at any health facility at no additional cost. Qualitative data show that women and health providers are satisfied with the integrated service. Improved Training for All Health Facility Staff IFPP innovated at health facilities by training all facility staff members. Health providers state that training for all health facility staff, from doctors to receptionists and cleaners, improves the friendliness and quality of family planning services provided to women and adolescents. Adolescent Corners at Health Facilities IFPP developed “Adolescent Corners” at health facilities – private areas reserved for adolescents – to improve the use of family planning services among the youth. Qualitative evidence shows that the “Adolescent Corners” have increased youth family planning services among youth, and the survey data corroborate this finding: use of any modern contraceptive method has increased among the 15-19 age cohort while it has dropped for other age segments. Young people believe that the services are high quality and reduce the stigma associated with adolescent use of modern contraceptives. Agente Polivalente Elementar (APE) Home Visits While APEs existed prior to IFPP intervention, the project improved their training and equipped them with modern contraceptives. Qualitative data show that APEs are an innovative resource for women because they conduct home visits and provide family planning services directly to women. According to beneficiaries, APEs are also responsible for reducing social stigma surrounding the use of family planning. Survey data reveal a significant increase in family planning visits over the past year, with APEs contributing the largest percentage increase in visits to households. Mobile Brigades in Rural Areas and Secondary Schools While mobile brigades existed prior to IFPP, the project trained teams of nurses and technicians to provide on-site, routine family planning services in rural areas and secondary schools. Despite a lack of strong qualitative evidence showing their effectiveness (no beneficiaries commented on mobile brigades in rural areas), evaluation team observations, desk review research and interviews with health facilities note that they are innovative because they provide family planning information in hard-to-reach areas. DEMAND CREATION The most significant demand creation innovations are: USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 61 ADOLESCENT CORNERS AT HEALTH FACILITIES • In general, adolescents don’t like to go to the health facilities because they don’t want to be seen seeking FP services by adult community members. Adolescent corners were previously created in the HF but mostly for the HIV&AIDS services. For some period, they were not used, but now those corners are being revitalized, in order to better serve the youth. Adolescents are more comfortable receiving FP services at school. “There is an adolescent “corner” at the facility. In the past the corner was located within the area where all other services were provided and wasn’t used much because it didn’t provide much confidentiality and adolescents didn’t want to be seen there. With assistance from Pathfinder we relocated the corner in a different area in the facility. Now the services are used a lot more. We provide counseling and all short term and long-term methods for the adolescents. The boys are using the corner as well as the girls¨. (Health provider, Monapo Health center) • “The service is friendly. There are always commodities at the HF, especially condoms.” (FGD Male adolescents, Monapo Sede) • We visited the room assigned for adolescents which was in a secluded area of the health center. It is staffed by a nurse. The nurse told us that they are receiving increasing numbers of adolescents, both girls and boys. Adolescents are mostly referred from the schools, but they also receive referrals from the community activists. The nurse provides counseling and both short term and long￾term methods for the adolescents. (Observation from Nacala Porto Urbano Health Center) • Use of any modern method among 15-19 has increased (13% baseline to 18% midline). In fact, overall decrease in use among all age groups except for youth. AGENTE POLIVALENTE ELEMENTAR (APE) HOME VISITS • Previously, APEs were working at the community level with different programs. Now, they have received training on FP by the MOH and IFPP. They also have supplies of commodities, such as pills, condoms and injectables. As the APEs live at the communities, they are easy to reach, they are well known and well respected. Instead of walking long distances to the HF, the people get service at home. They help in creating the enabling community environment and help to give the correct knowledge about the FP methods, thus reducing stigma and myths. (IFPP project documents, MMEMS observations) • “Should continue training APEs” (Male FGD participants, Monapo Sede) • “PF also trains the TBAs, APEs and activists in communities which helps to increase demand. One important thing is the myths/misconceptions about FP in the communities.” (Health Provider at Namina Health Center) • There is this school counter at the school and there the youth can ask any questions, about family planning, or diseases or get counseling. They also have almost any kind of methods available at the school counters. Activists, who are students themselves, are very active and attend the students 2 days a week. (FGD female adolescents Monapo Sede) • Visited by and discussed FP with a Traditional Birth Attendant, or TBA; Agente Polivalente Elementar, APE; (Activists) in the past 12 months: 12% baseline, 24% midline. Biggest contributor was APE, from 1% to 11%. MOBILE BRIGADES IN RURAL AREAS AND SECONDARY SCHOOLS • Mobile brigades already existed prior to IFPP. They provided only immunizations and were not functional due to lack of funding. IFPP leveraged mobile brigades, which include health facility Maternal and Child Health nurses and health technicians, to visit communities on a routine basis and provide FP counseling, and both short-term and long-term contraceptives. Mobile brigades are innovative because they offer FP services in remote areas and secondary schools. This Youth Couples Counseling IFPP innovated by integrating previously existing couples counseling teams into the project by providing them with family planning training. They provide consistent face-to-face communication with young married couples over the course of six visits, involving local community leaders. Qualitative data suggest that the counseling teams are effective because they improve knowledge of modern contraceptive methods and actually increase demand for family planning services. Mobile Platforms (SMS Messaging) The mCenas SMS communications platform is an innovative way to increase youth uptake of family planning services. Anonymized targeting of adolescents via free SMS delivery and the use of novela-style storylines has promise to drive demand for modern contraceptives. Traditional Birth Attendants (TBAs) Originally intended to help women who give birth at home, IFPP has trained TBAs to provide family planning services at the local level directly with women. Interviews with USAID, other donors and the IP show that TBAs are effectively increasing local demand through referrals to health facilities. HEALTH SYSTEM SIGLUS IFPP improved the Logistics Management and Information System (LMIS) used by MOH to track medical supplies by adapting and supporting the rollout of family planning inventory tracking software, called SIGLUS. SIGLUS has improved the data quality of stocks in health facilities, which in turn has contributed to improved management of stock levels. While the evaluation team did not conduct an audit of the health system inventory, IFPP reports that stockouts have decreased and MMEMS only encountered one stockout at the six facilities visited. However, because this innovation was only recently introduced in June 2018 to select health facilities, more time is needed to determine how effective this innovation is in improving supply management. This innovation is discussed more in EQ3.77 77 IFPP mentioned the Clinical Mentorship application and partnership with the OES “nudge” team working with the MOH as other innovations. While these efforts are well-documented in IFPP reporting, they were not present to a significant extent in the qualitative data and are therefore not mentioned in the evaluation report. 62 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV is important because the catchment areas for health facilities are large (average of 14 km per catchment area), and those who live farther away are less likely to make the trip. Transport costs are problematic - in general, only about half of women give birth at the HF due to long distances from home to the nearest HF. The school youth are served by the mobile brigades, giving them an alternative as they in general are not very willing to go to the HF and been seen by people they know, especially if the HF doesn’t have any adolescent corner. (IFPP Documents and MMEMS Observation) • “Pathfinder has decreased the pregnancies and there is more information. The mobile brigades also serve the adolescents. They go to the schools. Those brigades were working also previously but not as much as today. The transportation is being facilitated by Pathfinder, in order to reach out to the schools.” (Monapo Health Director). Unfortunately, there is no mention of mobile brigades from beneficiaries. DEMAND CREATION YOUTH COUPLES COUNSELING • N’weti’s peer education through facilitators, one male and one female, who address young couples directly, face-to-face sensitization, together with community leaders and other influential persons. N’weti previously addressed HIV using the same method. The innovation is to use it for family planning, which has not been done by any other organization. Face-to￾case communication helps in increasing the information, knowledge and acceptance, as people have opportunity to ask questions and be addressed individually or in small groups. Six sessions help to consolidate both knowledge and understanding. (IFPP Reports and MMEMS Observation) • Both rural and urban youth couples counseling teams are viewed to have helped increased demand (Interviews with USAID, MOH, UNFPA, Dutch Embassy, Provincial and District MOH, Health Providers, and IFPP) • “They have constructed an interactive approach in which pairs of professional coordinators work conduct sessions on family planning with married couples and influential individuals in the communities. Couples are then referred to the health facilities.” (Health Director, Mecuburi District) MOBILE PLATFORMS (SMS MESSAGING) • mCenas platform in some districts and secondary schools. It was launched together with other partners, such as DKT, education district directorate, and health district directorate. Through SMS the adolescents receive information when they enroll in the service. Which improves access to information and knowledge regarding FP and RH. • Spread messaging for adolescents in urban areas. Objective is to create demand among the youth in urban areas: generate interest, register the users, give counseling for those interested and refer them to the health facilities. The mobile platform system also strengthens the registration, reference and monitoring mechanisms of the demand generation activities. (Interviews with USAID, MOH, UNFPA, Dutch Embassy, Provincial and District MOH, Health Providers, and IFPP) • The IPC agents (Interpersonal Communication Agents by PSI) register the client and document the different interactions carried out during the year which will strengthen client follow-up, improve registration of FP needs, collect their feedback on the services provided, and track referrals. The app works on Android-equipped smartphones. Geo-localization of households is possible, as well as ease in internal audits and return visits. (IFPP 2nd Annual Report) • Connecting with Sarah – performance management – customized follow up for individuals based on data collection. TRADITIONAL BIRTH ATTENDANTS • Innovative use of TBAs for FP counseling and promoting referrals for FP services. Effective in increasing demand by improving knowledge of family planning, countering prevailing misconceptions and biases, conveying the importance of USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 63 healthy timing and spacing of pregnancy (HTSP), increasing self-efficacy, and promoting linkages with contraceptive service delivery points (IR1). (Interviews with USAID, MOH, UNFPA, Dutch Embassy, Provincial and District MOH, Health Providers, and IFPP). “Pathfinder also trains the TBAs, APEs and activists in communities which helps to increase demand. One important thing is the myths/misconceptions about FP in the communities (meaning that these CHW help to contradict these myths). (Health provider at Namina Health center) • TBAs are trained and supervised by the HF trainers, in partnership with the IFPP district coordinators. TBAs are expected to reach all women and adolescents of reproductive age, specifically targeting first-time parents (FTPs) who are pregnant or postpartum and medium- and high-parity women (defined by IFPP as woman with three or more children). TBAs also engage household influencers and gatekeepers (for example, male partners and mothers-in-law). (IFPP Documents) • TBAs live in the communities and are well known. They have been trained previously by MOH. They conduct family planning counseling at the community level, which is cost-efficient, relevant, and sustainable, as they will continue to work when IFPP ends. HEALTH SYSTEM SIGLUS • Logistics management and information system used to track and report stock data for MOH. Introduced by PSM, IFPP leveraged use of this platform to ensure improved stock management of FP commodities at target facilities, with the aim of minimizing stockouts. SIGLUS is a mobile platform to reduce the occurrence of stockouts of critical medical commodities through timely and accurate communication. (Interviews with Provincial and District MOH, Medical Supply Warehouse Managers, IFPP and PSM) • The evaluation team found (in facilities visited) that SIGLUS is being used, but this is a very recent tool (introduced in June 2018). (Interviews with Provincial and District MOH, Medical Supply Warehouse Managers, IFPP and PSM) • Reports from warehousing staff suggest that the platform is useful in letting them know commodities are needed at health facilities. (Interviews with Provincial and District MOH, Medical Supply Warehouse Managers, IFPP and PSM) • No evidence to show that SIGLUS improved warehouse responsiveness to commodity requisitions, with no clear indication (i.e. from health providers and warehouse staff) on whether timely delivery of commodities had improved (Interviews with Provincial and District MOH, Medical Supply Warehouse Managers, IFPP and PSM) • The warehouse manager is trained by IFPP to manage the contraceptive supply change. He enters the data also keeps paper files. He thinks that SIGLUS is very useful, easy to upload data which is visible to everyone. (Observation, Nacala Porto Urbano Health Center) • According to IFPP M&E reports, stock outs76 at service delivery points have decreased from 14% to 6% over the last two years (from 2016 to 2018). Qualitative data: In only one facility was there a stockout of contraceptives. EVALUATION QUESTION 2. ACCESS AND QUALITY: To what degree has IFPP increased access and quality of FP/RH services? 2.1 How has access to modern contraceptive methods and Family Planning/Reproductive Health services increased? To what extent have IFPP interventions contributed to that increase? 76 Stock out is defined as the lack of availability of any contraceptive method that the health facility is expected to provide. 64 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV FINDINGS CONCLUSIONS ACCESS THROUGH REFERRALS AND MESSAGING • Referrals78 increased from Year 1 to Year 2 in Nampula, from 38,424 to 80,076. Both in Nampula and Sofala confirmed referrals made by TBAs and CFs have increased. However, referrals made by IPCs (PSI urban model) decreased over time. (MOH-HMIS data on referrals). • More women overall have had contact with health workers in general (95 baseline vs. 199 midline), while the proportion of those women who were referred to a health facility decreased (95% baseline to 74% midline, or 90 baseline vs. 148). • Participated in community event in past 12 months where FP was discussed: 14% to 17%, not statistically significant increase. 15-19 women were more likely to participate (21%), Muslims were more likely to participate (19%), unmarried women more likely to participate (20%), and wealthier women more likely to participate (18%). • Heard radio program discussing FP/HTSP in the past 6 months – 32% baseline to 25% midline. ACCESS TO HEALTH PROVIDERS – CONTACT AT HOME AND AT HF • Of the 51% of women surveyed who went to a health appointment in the past 12 months (up from 35% at baseline), 67% of them women who went to a health appointment in the past 12 months discussed Family Planning (compared to 57% in the baseline survey). Most health appointments were Triage (25%) and Health Child Consultations (25%). Rural women were more likely to discuss FP than urban women (69% to 63%, not statistically significant), although the gap between rural and urban women has decreased, from 35% at baseline to -6% at midline. • q1/q2 women were less likely to discuss FP than q5 women (68% to 69%), although the difference is not statistically significant. • To expand the geographic coverage of FP services, especially in the remote rural areas, IFPP revitalized the mobile brigades. Communities visited on a routine basis for provision of FP counseling, and both ST (short term) and LT (long term) contraceptives. IFPP contributed by training the mobile team staff and providing transportation. (Interviews with MOH, HF Managers and Providers, IFPP Reports) ACCESS TO MODERN CONTRACEPTIVE SUPPLIES • Among current users, women overwhelmingly rely on the public sector for their modern contraceptive methods - pills and injectables and condoms (86%, 95% and 80% midline), a marked shift for condoms from private to public sector (38% baseline to 0% from private sector in midline, 10% other and 10% missing). • “We already use condoms, we got the condoms in the hospital, it was not difficult to get one, and the nurses did not ask many questions. The services are good.” (Focus Group Participants, Nacala Porto District Hospital) • Contraceptive supplies are available. “They had no stock out of any contraceptives except for shortages in emergency pills and female condoms over the past year. When they have low stocks for any of the contraceptives, they contact PF and commodities are delivered shortly.” (Namina health center, Mecuburi) EVALUATION QUESTION 2.1 Access to facilities, providers, supplies and information for women and marginalized groups has improved in some ways and worsened in others. While the percentage of referrals to family planning services, radio broadcasts and community events about family planning decreased, discussions between women and service providers about family planning increased. Access to modern contraceptive supplies also increased alongside access to accurate health information from health providers. These changes are sometimes the result of broad systemic changes over which IFPP only exerts partial control (like referrals and community events), but other changes – like access to accurate information and health providers – are more directly linked to IFPP’s policies to reduce barriers to access among young and marginalized groups. ACCESS THROUGH REFERRALS AND MESSAGING Referrals to health facilities for family planning services by Community Health Workers are an indicator of the ability to access health facilities. According to IFPP data, IFPP referrals have increased, but survey data indicate that the percentage of women referred from any source actually decreased.79 Overall contact with radio and community events to encourage family planning – not specifically IFPP messaging – also show a decrease: radio programming decreased from baseline to midline and community events remained static. IFPP does not control all referrals and messaging related to family planning services. Although it should be mentioned that referrals from IPCs decreased, the evaluation cannot determine the extent to which the overall decrease in referrals can be attributed to IFPP programming. ACCESS TO HEALTH PROVIDERS – CONTACT AT HOME AND HF Despite overall reductions in referrals and lagging progress for rural and poor women, more women – including an increasing share of rural women – are both attending medical appointments and discussing family planning with a health provider during appointments. Women mostly attend medical appointments for Triage and Child Health Consultations – not specifically for family planning consultations – indicating increases in the integration of family planning into existing health services. 78 Referrals provide an indication of the number of women who were reached by the project who go on to seek family planning services at health facilities. 79 According to IFPP, the decrease in referrals may be due to a programmatic shift. While IPCs were originally intended to visit many beneficiaries to provide counseling sessions and referrals, their mandate is now focused on repeat visits to fewer clients. This strategy reduces the total number of referrals made by each IPC and therefore may contribute to the reduction in referrals seen in the midline survey results. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 65 ACCESS – CHANGES TO RESTRICTIVE POLICIES AND REDUCTION OF SOCIO-CULTURAL BARRIERS Support for Adolescents • For a short time, adolescents were able to access all forms of contraceptives at schools. In year 1, IFPP, in conjunction with MOH and MOEHD, made amendments to legislation regarding SRH (sexual and reproductive health), thus creating an enabling environment at schools to support FP service provision. However, the MOH and MOEHD decided that only counseling, pills and condoms were allowed. The evaluation noted complaints and general sense of discomfort and unease among parents linked to widespread myths that LT methods harm the future fertility of the girls resulted in reversal of official legislation. (Interviews with IFPP, MOH, and USAID). “Previously adolescents would receive any available method. But now nurses say that the adolescents only can get pills and condoms – to whoever young girl independently whether she has got kids or not. The MoH went back in their strategy. That is definitively a handicap, a problem.” (Pathfinder) • Letter signed on 4th March 2019 by the Minister of Health Dr Nazira Vali Abdula. Assunto: Oferta de Contracepção nos Cantos de Saúde das Escolas Secundarias, especificando o pacote de intervenções nas escolas. • The provision of youth-friendly FP services was regarded to be a positive step in encouraging better uptake of FP services among adolescents. (Interviews with IFPP, MOH, and USAID) • “Mobile teams go into the communities, provide counseling and contraceptives-condoms, pills, injectables and implant.” (Mecuburi Sede, health district directorate) • Quote about adolescent-only corners reducing barriers to access. “We are receiving increasing numbers of adolescents, both girls and boys. Adolescents are mostly referred from the schools, but they also receive referrals from the community activists. The nurse (at the corner) provides counseling and both short term and long-term methods for adolescents.” (Urbano Nacala Porto health center, Nurse in charge of adolescent corner) Support for all women • Integration of FP services with all health services delivered by MOH improves services to all women, including postpartum women, post abortion women, women with HIV. Trained staff provided to support these groups. “PF helped them to integrate FP in all sections/wards of the hospital. It includes all sections of the hospital except for the laboratory and pharmacy sections. It includes all MCH services, maternity, and other general services of the hospital such as the pediatric ward, general medicine, surgical services, gynecology, services for the HIV+ clients and the emergency services.” (Health provider at Nacala￾Porto district hospital) • IFPP sought to address social and cultural barriers that are known to limit access to FP services for all women, including some from marginalized groups. This was done through implementation of couples counseling and group counseling involving community elders. (Focus Group Discussions, Community Leaders) ACCESS TO HEALTH INFORMATION Health information • Percent of women informed by health providers about side effects or complications of methods: 40% baseline; 76% midline (KAP Survey) • Percent of women informed about what to do if side effects or complications occur: 40% baseline; 71% midline (KAP Survey) Qualitative data and IFPP reports indicate that IFPP may be responsible for increases in family planning discussions with health providers due to their efforts to expand access to services through mobile brigades, APE home visits, adolescent corners at HFs, and integration of family planning into local health systems (as discussed in question 1.3). However, the extent to which IFPP contributed to this increase cannot be determined. ACCESS TO MODERN CONTRACEPTIVE SUPPLIES IFPP increased access to modern contraceptive supplies through management of the contraceptive supply chain. This is explored in-depth in EQ3. Women overwhelmingly rely on the public sector (hospitals, health centers, health posts and mobile brigades) to access the most commonly used contraceptives, like pills, injectables and condoms. Qualitative data suggest that supplies of modern contraceptives are readily available. IFPP is partially responsible for many of the improvements in contraceptive supplies due to their efforts to monitor and respond to low supply levels and stockouts. However, because IFPP does not fully control the distribution of contraceptive supplies to health facilities, the extent of their contribution to supply improvements cannot be determined. ACCESS – CHANGES TO RESTRICTIVE POLICIES AND REDUCTION OF SOCIO-CULTURAL BARRIERS IFPP is also addressing cultural and social barriers to contraceptive access, especially among adolescents and other marginalized groups, such as post abortion women and women with HIV.80 Some of these efforts, like the push for long-term contraceptives in schools, have not achieved expected results. Support for Adolescents IFPP reduced barriers facing adolescents in accessing contraceptives. The Activity helped to pass legislation making it easier for adolescents to receive long-term contraceptives at school (this was later rolled back by the MOH and MOEHD after parent pushback). As discussed in Evaluation Question 1.3, IFPP also deployed mobile brigades to schools to improve adolescent access to services, and installed adolescent-only corners at HFs to reduce the shame experienced by adolescents when they visited HFs. Support for All Women IFPP also supported women, including post-abortion women and women with HIV. As with adolescents, IFPP deployed mobile brigades to reach them in their homes. ACCESS TO HEALTH INFORMATION 80 IFPP focuses on marginalized groups (including post-abortion women and post-partum women) through the Maternity Emergency Wards at health facilities. These women are given care particular to their needs, including counseling services. 66 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • Knowledge concerning specific contraceptive methods Pill: Pill is effective even if a woman forgets to take it: Correct answer: 39% baseline, 53% midline IUDs: The IUD can be stuck forever inside the woman: Correct answer: 27% baseline, 51% midline Condom: Men can use a condom more than once: 47% baseline to 68% midline Injectables: A woman who uses injectables will never again have children in her life: Correct answer: 42% baseline, 55% midline. (KAP Survey) • Among women ages 15 to 49: Accurate knowledge on minimum time to wait to conceive after having a birth, for health of mother and baby: 87% baseline, 86% midline, not statistically significant. (KAP Survey) • Minimum time to wait to conceive after abortion before conceiving again: 79% to 79%. In addition to increased knowledge noted in Evaluation Question 1.1, women also have improved access to information about the side effects or complications of family planning methods. According to survey data, both adolescent and adult women were better able to discern between true and false statements relating to widespread myths about contraceptive methods at midline than at baseline. However, not all knowledge increased – there was almost no change in knowledge of good child spacing practices. 2.2 How has the quality of Family Planning/Reproductive Health services improved? How have IFPP interventions contributed to that improvement? FINDINGS CONCLUSIONS • IFPP’s major contributions to improve quality of FP services are (1) training of providers, (2) ensuring availability of contraceptives and (3) integration of FP with other services. (IFPP Reports, Interviews with MOH and Health Providers) QUALITY OF FAMILY PLANNING SERVICES – BENEFICIARY PERSPECTIVE • The majority (17/22) of focus group participants note that the quality of family planning services is good. A minority (3/22) say that service quality was not good. In Nacala, women FGD participants complained that there were required to make out of pocket payments for implant removal—which should be free. (Interviews with Health Providers, Focus Group Discussions, Health Facility Observations) • IFPP introduced Community Score Cards tools to bring the communities and providers together and “score” the quality of services. Based on the results, they jointly select 5 priority areas to improve services and develop an action plan. (Interviews with N’weti, Pathfinder, IFPP Reports). It is operational in 14 facilities out of 385 (4% of facilities, 8 in Nampula and 6 in Sofala). • Of those 7 FGDs with adolescents who know about contraceptives and use them, all said that there are contraceptives available at the HF, mostly condoms for boys and both condoms and injectables for girls and there is no shortage. 6 out of 7 groups mentioned that the service at the HF is friendly or good, even when there was no specific youth corner available at the HF. (Focus Groups, Monapo Sede and Nacala-Porto) • “The service is friendly. There are always commodities at the HF, especially condoms”. (Male adolescents at Monapo Sede) • “At the hospital they treat us well when we go to get condoms and do HIV testing. They always give us for free”. (Female adolescents Nacala-Porto district hospital catchment area) • “We got the condoms in the hospital, it was not difficult to get one, and the nurses did not ask many questions. The services are good”. (Male adolescents Nacala-Porto district hospital catchment area) • “Yes, commodities are always available. For example, I once went to the HF with a girl, she wanted to adhere the contraceptive method. First, we were afraid, but on the corridor, we found a nurse, we talked to her and she directed us to a room where she explained everything. The girl adhered to the injection contraceptive method and until today every three months she gets an injection… The service is so good. We have liked it, there is no complication.” (Male adolescents at Urban Health center in Nacala-Porto) EVALUATION QUESTION 2.2 According to qualitative sources and IFPP documents, the quality of family planning services has improved. Overall, beneficiaries report that they receive high quality services from health providers. IFPP training may be responsible for the perceived high service quality, as observations of training and interviews with IFPP staff indicate that the training is thorough and improves health provider performance. QUALITY OF FAMILY PLANNING SERVICES AND MODERN CONTRACEPTIVES Overall, qualitative data collected suggest that the quality of family planning services is good. Patients are treated with friendliness and respect and get the services that they need free of charge. QUALITY OF PROVIDERS – IFPP TRAINING Qualitative data show that providers were trained to provide high quality services. Review of project documents and interviews with IFPP and health facility staff suggest that trainings provided by IFPP improved health worker skills and knowledge and may be responsible for improvements in service quality and access to information about contraceptive methods. While the evaluation did not test the individual competency of trained staff to perform clinical procedures, interviews with IFPP staff reveal a concerted effort to improve provider quality through time-intensive, hands-on training courses and routine supervision by health facility managers. However, some trainees from low-volume health facilities reported that because of a lack of clients, they lack confidence in performing some clinical tasks. IFPP is generally responsible for improvements to provider quality because they lead service provider trainings and service quality monitoring. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 67 QUALITY OF FAMILY PLANNING SERVICES – IFPP TRAINING • During the first year of implementation, IFPP implemented a “8-days on-site training” program to improve the skills of health providers. Trainings took place in high-volume HFs, led jointly by IFPP and MOH. Training courses on FP were incorporated into existing programs, all HF staff were trained. The course content covered: promotion of integrating FP with other services, sensitization HF staff (clinical and non-clinical) of the value of FP and to encourage clients to access FP services, as needed; for clinical staff, training also covered counseling, and practical skills on how to administer LT methods. (IFPP Reports, Interviews with IFPP, MOH Managers and staff, Health Facility Observation) • During the second year of implementation, two 8-day trainings held in large health centers took place, targeting newly assigned providers. (IFPP Reports, Interviews with IFPP, MOH Managers and staff, Health Facility Observation) • Providers from level 1 facilities also underwent training but it is important to note that small client caseloads were insufficient to support opportunities for clinical practice (IFPP Reports, Interviews with IFPP, MOH Managers and staff, Health Facility Observation) • In one small HF, a trained nurse reported that she is not able to provide LT methods because the client load is very low, and she didn’t have confidence in her skills. Trained provider: There are about two LT FP clients per month at our health post. I tried once to insert an IUD and I wasn’t successful. I am not going to try it again; I don’t have the confidence in my skills. (Interviews with IFPP, HF Trainers, FP Trainees, Health Facility Observations) • IFPP is training large numbers of providers in a short timeframe, which may lead to inadequate acquisition of new clinical knowledge and skills (Interviews with IFPP, HF Trainers, FP Trainees, Health Facility Observations) • IFPP established routine supervision and mentoring of trained staff in the HFs. Teams consisted of MOH and IFPP staff. The HF managers and staff are content with the mentoring and follow-up they have been receiving from IFPP. (Interviews with MOH, Health Providers, IFPP) “There is routine follow-up, mentoring and supervision. All trainees are followed up on after 2-3 months following the training. PF has a “cluster approach” There is an PF coordinator for 3 districts, Mecuburi one of them. Every month a PF team comprised of regional coordinator, technical coordinator and ME advisor visits the district for one week.” (Health officer in Mecuburi district) 2.3. What are the main barriers to reaching youth and adolescents with FP/RH? (Learning Question) FINDINGS CONCLUSIONS Restrictive school policies • Youth who access FP services are offered a limited number of family planning methods, unfortunately this choice is made for them by school-based nurses. Need parental approval to seek FP services, parent hesitation of having adolescents access contraception at schools without consultation. (Interviews with Pathfinder, N’weti, PSI, MOH, Health Providers, Community Leaders, Focus Group Discussions) • “There was a problem with the parents. It is not easy for a parent to see that the daughter has an implant or using pills. So it is a barrier but on the other hand it is for to take care (of the adolescent) but yes, it is still a big barrier. The school is doing this together with the parents, reuniting them and informing them”. (MOH). • “Ministry of Education… there are school councils at each level – all (contraceptive) methods were available – and then the parents stopped that”. (USAID) Parental pushback EVALUATION QUESTION 2.3 There continue to be significant barriers to reaching youth and adolescents with family planning services. Restrictive school policies, parental pushback, health facility limitations and community stigma and cultural norms reduce the effectiveness and reach of family planning services for adolescents. Restrictive school policies IFPP documents and interviews show that schools actively restrict the types of contraceptives students can receive from school nurses, and many require parental permission. Parental pushback 68 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • Taboo to openly discuss FP or sex with children. Difficult to provide information to children – mothers and daughter, fathers and sons don’t talk about FP or sex. Low level of parental permission/ consent to allow girls to use contraception. Disapproval of women and girls to decide when and what contraceptive to use. Parents want children to prove fertility by getting pregnant. Girls are married off early, parents do not want financial burden to care for their daughters (Interviews with Pathfinder, N’weti, PSI, MOH, Health Providers, Community Leaders, Focus Group Discussions) • Obtain permission to go to health facility big problem – 13% baseline to 28% midline, 15-19 • “Not all parents support their children in adhering to the contraceptive methods. Because there are parents who want their daughters to get pregnant to charge something - they want to profit from their daughter's marriage. Other parents do not approve to get contraceptive methods because of religion rules”. (Male adolescents, Nacala-Porto district hospital catchment area) • Afraid someone will see her, and she will be judged by the community big problem – 10% baseline 24% midline. 15-19 women perceive this as a bigger problem than 20-24 women – 31% to 18%. Q1 and q2 women perceive this as a bigger problem than q5 women (29% to 13%). Unmarried women consider this a bigger problem than married women (29% vs 23%). Rural women perceive this to be more of a problem than urban women (27% to 15%) Health facility limitations • Training all staff (HRH) in youth friendly service provision. Most lower level HFs do not have functional youth corners. No safe space to openly discuss FP /RH needs. (Interviews with Pathfinder, N’weti, PSI, MOH, Health Providers, Community Leaders, Focus Group Discussions). Limited number of HRH to provide FP services for youth, particularly at lower level facilities, is not available. Some adolescents do not go to HFs – due to lack of friendly services and in some cases, nurses were not available. • “I don’t know about IFPP interventions but in terms of barriers I know that the main one is that the services are not adolescent friendly. There are “adolescent corners” at the HFs but they are not functional, and they are not being used, probably due to the MCH work load. It is possible that there are no assigned staff to the adolescents. It is also possible that adolescents don’t know about the presence of such a “corner”.” (USAID) • Afraid health facility will not have the contraceptive method she wants big problem – 17% baseline, 37% midline. Women in the poorest 2 quintiles perceive this as a bigger problem than q5 women (38% vs. 27%), and unmarried women consider this a bigger problem than married women (42% vs 35%). 15-19 • Afraid of being treated badly by health provider when asking for contraception big problem – 19% baseline, 37% midline. 15-19 women perceive this to be a bigger problem than 20-24 women (41% vs. 33%). Poorer women (q1-2) perceive this to be more of a problem than wealthier women q5 (42% to 24%). Rural women perceive this to be more of a problem than urban women (39% to 31%) • Afraid of being served by a male health provider big problem – 15% baseline, 30% midline. 15-19 women perceive this as a bigger problem than 20-24 women – 26% to 27%. Q1 and q2 women perceive this as a bigger problem than q5 women (34% to 27%) • Distance to health facility big problem – 48% baseline, 54% midline, 15-19 • Obtain money for transportation big problem – 50% baseline, 50% midline, 15-19 Community stigma and cultural norms • Stigma associated with child (youth, adolescent) using contraception (Interviews with Pathfinder, N’weti, PSI, MOH, Health Providers, Community Leaders, Focus Group Discussions). Socio cultural norms (expectation/pressure) to bear children. Stigma attached to wanting access to FP. Stigma includes religious beliefs that FP is killing unborn kids. Girls known to use contraceptives are called prostitutes. Fear of being seen by community members is another factor. Qualitative data show that, in general, parents do not provide sexual education to their children, and most do not allow their children to use contraception for a variety of reasons. Health facility limitations Interviews with adolescents show that while there have been improvements in friendliness and youth acceptance at health facilities, most lower-level facilities continue to lack youth corners and some adolescents continue to feel that service is unfriendly. Finally, a majority of women consider distance to the health facility and associated costs as barriers to service. Community stigma and cultural norms Qualitative data suggest that community norms continue to favor marriage and childbearing at a young age. Adolescents do not want to be seen by community members at health facilities because they fear being labeled promiscuous. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 69 Contraceptives are believed to harm future fertility. “There is a slight difference compared to 2-3 years ago, because now we are aware and have more information about FP. We have the information, but we are unable to share and to use the contraceptives and if we use them it has to be in secret like some of us do. Our fathers and the rest of the community should never know or find out about an adolescent doing FP because its considerate shameful. In this community an adolescent who is known using contraceptive is excluded from the society, discriminated, they consider her a girl with bad reputation, a prostitute etc... So, all negative criticism goes to her.” (Female adolescents, Nacala-Porto district hospital catchment area) • “In this community the atmosphere is very bad regarding the acceptance for adolescents using contraceptives. People see badly at girls who they know are using contraceptives, and these girls many times are called prostitutes, they get a very bad reputation, sometimes people don’t even talk to them anymore. The adults try to justify the non-use by saying that the contraceptives give problems”. They say, “you are killing your babies¨ or ¨the God said to have babies”. (Female adolescents, Nacala-Porto Urban health center catchment area) EVALUATION QUESTION 3. COMMODITIES MANAGEMENT: To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? 3.1 To what extent have the forecast and quantities of commodities received met the needs at service delivery points? FINDINGS CONCLUSIONS BACKGROUND • The medical drugs & commodities supply chain in Mozambique uses a mixed push and pull mechanism. • Contraceptives are distributed on a quarterly basis to health facilities from the districts using a push mechanism and health facilities are supposed to request commodities on a monthly basis (pull-requisition system) (Interviews with IFPP Staff, MOH, Provincial and District Warehouse Managers, and Health Providers) • Forecasting is conducted at the facility level and districts are supposed to distribute the requested number of contraceptives. (Interviews with IFPP Staff, MOH, Provincial and District Warehouse Managers, and Health Providers) • Contraceptive supply chain management capacity at the service delivery points has been weak prior to the launch of IFPP activity. Reportedly, frequent stockouts occurred in the health facilities. (Interviews with IFPP and MOH) SERVICE DELIVERY NEEDS MET • According to beneficiaries, contraceptives have been available at the service delivery points. Men, women and adolescents participated in FGDs have not noticed/faced any stock outs at the HFs. “There are no shortages of any contraceptive methods at the health post. Experienced no stock outs since I was trained”. (Health Provider at Unidade Mocambicana health post) • According to IFPP M&E reports, stock outs at service delivery points have decreased from 14% to 6% over the last two years (from 2016 to 2018). The percentages are based on the number of health facilities monitored by IFPP. (130 facilities in last quarter of 2016 and 209 in 2018) (IFPP M&E Reports). It is not accurate to compare stock out rates based on different sample sizes. However, the evaluation team used this quantitative data to present an estimate on the occurrence of stockouts at the service delivery points. (IFPP M&E Reports) EVALUATION QUESTION 3.1 The forecast and quantities of commodities received have met the needs at service delivery points, despite the lack of a unified technological logistics solution across the health system. SERVICE DELIVERY NEEDS MET IFPP has mostly succeeded in accurately forecasting and delivering commodities to local health facilities. IFPP data show that only 6% of facilities experience stockouts, down more than 50% from two years ago. In addition, interviews at health facilities visited by the evaluation show that only one of six facilities experienced a stock out. However, preventing stock outs at health facilities depends to an outsize extent on IFPP intervention. PATCHWORK SYSTEM In the absence of a complete Logistics Management Information System, IFPP has had to combine a patchwork of solutions – technical assistance, rollout of MOH technology (SIGLUS), stock verification, and transportation – to improve the short-term functioning of an inefficient system.81 81 Another layer of difficulty includes the procurement of commodities from outside Mozambique. Sometimes these shipments are delayed, complicating efforts to distribute commodities to health facilities before stockouts occur. The evaluation did not receive qualitative data about this issue and therefore cannot weigh in on IFPP’s role in this process. 70 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • Out of the 6 health facilities visited by the team only one experienced contraceptive stockouts. Metuchiria Health Center didn’t have any pills, condoms or injectables over the past year. The other five HFs reported that IFPP followed up on contraceptive availability in their facility on a weekly basis. Metuchiria health center didn’t receive any calls or visits from IFPP over the last year because IFPP assistance to this center had begun only recently. (Facility Observations, HF Managers and Staff). Metuchiria Health Center was a rural health post which was upgraded and became a health center one year ago. All staff have been assigned recently. IFPP began assisting the center in September 2018. So far, 2 staff have been trained in FP and the pharmacist received training in commodity management. Community level interventions were not launched at the time of the evaluation team’s visit. (Facility Observations, HF Managers and Staff). Staff report requests for contraceptives from the district warehouse but there has been no response. The center asked for IFPP assistance and were informed that they will be receiving contraceptives soon. (Facility Observations, HF Managers and Staff) • “I am not sure how useful it (SIGLUS system) has been so far because the HF has stockouts. For the entire year I have been at the HC, we didn’t have condoms, pills and Depo Provera. We only have implants and IUDs”. (Health Provider at Metuchiria Health center) PATCHWORK SYSTEM • IFPP’s role in improving contraceptive management is to provide technical assistance to the health facility staff (health providers and pharmacy workers) on how to manage contraceptive inventories-use of stock cards and requisition forms. (Interviews with IFPP, MOH, District Warehouse Managers, and Health Providers) • It is tedious for the facility staff to fill out the forms manually, but it is the requirement of the current MOH system. IFPP is assisting in how to use the forms accurately and in a timely manner. (Interviews with IFPP, MOH, District Warehouse Managers, and Health Providers) • IFPP also assists with the monthly transportation of contraceptives from the districts to the health facilities because the district warehouses don’t have funding for transportation of contraceptives to the health facilities. (Interviews with IFPP, MOH, District Warehouse Managers, and Health Providers). District Chief Medical Officer: “No Pathfinder, no transport, no contraceptives.” • SIGLUS is a mobile technology introduced to MOH by the PSM project. It is used to manage primary health care commodities (not only FP) at the health centers. (MOH, District Warehouse Staff, Health Providers) • IFPP included contraceptives into the SIGLUS platform to improve management of FP commodities at facilities and to minimize stockouts. (MOH, District Warehouse Staff, Health Providers) • A major weakness of the current supply chain is that the MOH LMIS system has not been rolled out to the health facility level. Only central, provincial and district warehouses have the computerized system, thus creating data gaps in commodity supplies in the health facilities. (Interviews with IFPP, MOH, Warehouses, and Health Providers) • Inclusion of contraceptives in the system began in May 2018 and it is not rolled out to all facilities yet. Currently it is used in all facilities in 13 districts of Nampula and Sofala. In those facilities where SIGLUS exists, staff find it easy to use and helpful. However, stock cards and requisition forms are still in use, so it is duplicative. (MOH, District Warehouse Staff, Health Providers) • IFPP conducts weekly follow up visits at health facilities to check FP commodities. In doing so, they are able to know early on of any imminent or total stockouts identified by IFPP district coordinators as part of routine TA. (Interviews with IFPP and Health Providers) • In addition to the above, IFPP also supports regular Commodity Management Task Forces in districts to analyze consumption data and requisitions from each HF to ensure enough stocks are ordered. The task forces are also used as a USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 71 training platform for MOH staff on how to manage contraceptive supply chain. (Interviews with IFPP, MOH District Officers, and District Warehouse Managers) • Health providers find the tool to be easy to use; but it is important to note the MOH requires health providers to maintain paper (hard copy) registers to manage commodities (Interviews with Provincial and District MOH, Medical Supply Warehouse Managers, IFPP and PSM) 3.2 At which level of commodities supply management chain there is high frequency of commodities stockouts of FP/RH? FINDINGS CONCLUSIONS • There haven’t been any significant stock outs of contraceptives at the national, provincial or district levels. There have been occasional shortages (imminent stockouts) which were easily resolved. However, stock outs occur at the services delivery points. (Explained in detail above) (Interviews with MOH, Provincial and District Warehouse Managers, Other Donors, PSM and ICAP) • “Other than the occasional stock outs at the SDP, there are no stock outs at any other level of the supply chain. There have been delays due to transportation issues, but no stockouts.” (PSM Provincial Representative) • The management of medical commodity supply chain receives considerable assistance from other USAID funded projects in addition to IFPP: PSM82 assists with the training of provincial and district warehouse staff in management of primary health care commodities. ICAP83, another USAID funded initiative to strengthen HIV services assists with the transportation of contraceptives from the central level to the regional and districts warehouses along with supplies required for diagnosis and treatment of HIV. IFPP works closely with PSM and ICAP to ensure that the district warehouses are supplied with contraceptive commodities. (Interviews with Provincial and District Warehouse Managers, Other Donors, PSM, ICAP and IFPP) EVALUATION QUESTION 3.2 There is no level (national, provincial, district, local) at which there is high frequency of commodities stockouts. The evaluation team did not observe any stockouts of contraceptive commodities at the national, provincial or district levels. At the local level, stockouts are relatively uncommon and stockout rates have fallen significantly since IFPP intervention began (as mentioned above). EVALUATION QUESTION 4. SUSTAINABILITY: How has the project ensured that there is local ownership of interventions that have been introduced? What is the likelihood of project achievements continuing after the project ends? FINDINGS CONCLUSIONS LOCAL OWNERSHIP • Pathfinder worked closely with the MOH during the development of the National Family Planning Strategy. This was prior to the launch of the IFPP Activity, during the implementation of previous USAID-funded Pathfinder activities. (MOH FP Strategy Documents, Interviews with MOH and Pathfinder) • Building on past knowledge and familiarity with the MOH FP priorities, Pathfinder sought to implement some key strategies – integration of FP with other health services – training of trainers and providers. (MOH FP Strategy Documents, Interviews with MOH and Pathfinder) • Social mobilization to increase demand of FP services, management of contraceptive commodities, which are all priority IFPP interventions. (MOH FP Strategy Documents, Interviews with MOH and Pathfinder) EVALUATION QUESTION 4 The project’s efforts to align and blend with MOH plans and activities since inception improves the chances of transitioning project activities to local (Mozambican) ownership after the project ends, even though IFPP has no written transition document describing how this would be done. The likelihood of project achievements continuing after the project ends depends to a large extent on the size of the local budget, political commitment, technical knowledge, and the nature of different IFPP interventions. While technical knowledge and political commitment at the 82 Procurement and Supply Management. 83 International Center for AIDS Care and Treatment Programs. 72 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV • IFPP did not create “parallel” systems while implementing activity interventions. For example, IFPP has been using the MOH HMIS in collection and reporting FP data. Similarly, the activity has been using the standards approved by the MOH for training programs. (Interviews with MOH and Health Providers) • The IFPP Activity was lauded by several MOH respondents that “IFPP is working with us to implement the interventions outlined in the national strategy” (Interviews with MOH and Health Providers) • “At the national level we have IFPP partners to help to implement the program in the provinces. Some provinces receive help, but others don’t. IFPP is good in helping in the provinces, they don’t organize parallel programs but work in the existing structure – this is the difference with IFPP compared with other implementing partners”. (MOH) • “I am aware of this IFPP because it’s a part of the department of maternal and child health. And I have been working here since 2016. I Believe what Pathfinder doing is a continuity of what has been implemented by the MOH strategic plan after other programs finished. (MOH) • Difficulties of the transition: “The ownership from the MOH is very important! FP is not a priority for the MOH even if it is for the government. One important thing: there will be a national FP meeting we will see that every province should have one FP focal point. There is no one now.” (USAID) • “From project design phase, sustainability was targeted from the beginning – we engaged with the Government about the priorities and how to align them with the IFPP – we engaged directors in the provinces in the beginning. One of the biggest challenges is that the Government does not have resources for FP. IFPP’s strategy is not to replace the government or create a parallel system but to work together and support them”. (Pathfinder) CONTINUATION OF PROJECT ACHIEVEMENTS Contraceptive Supply Chain • At national level, an estimated 95% of the contraceptives are obtained through donations. USAID and UNFPA are the primary sources of donations. (Interviews with Other Donors, MOH and IFPP) • The GRM is reported to purchase only 5% of the contraceptives. According to IFPP, at present, the MOH does not assign any budget towards purchase of contraceptives. (Interviews with Other Donors, MOH and IFPP) • Preferred future method of contraception among women in union who are not pregnant and not currently using contraceptive midline – any method (52%), injectables (28%), and pills (20%). Maintaining Service Providers • Health providers are likely to stay within the MOH system; minimal attrition of doctors and nurses to pursue opportunities abroad. (Interviews with MOH, IFPP, and Other Donors) • Locally, few opportunities exist for health providers to leave the public sector to seek employment in the private sector. (Interviews with MOH, IFPP, and Other Donors) • IFPP and MOH trainers and supervisors train and supervise providers as a team. IFPP provides transport for all training and supervisory/mentoring activities. (IFPP Reports, Health Facility Observations, and Interviews with Health Providers) • The current National Strategy does not have a budget. Senior MOH officer: “We have a complete strategic plan and we are happy with it. Our challenge is that the plan does not have a budget. Budgeting is very complicated, and we don’t really know how to do it” • While the MOH is content with the National FP Strategy, they do not have a budget to support the implementation of the strategy. To implement the FP strategy the MOH relies heavily on donor assistance, specifically IFPP. (Interviews with MOH) MOH exist, there is no budget allocated to continue most IFPP tasks. The contraceptive supply chain will only survive with donor support but can continue without direct IFPP intervention. Service providers have received a permanent benefit from training and learning materials, but additional training and supervision will likely end. Finally, community-level visits from health workers will probably come to an end, but the mentality change of the population will remain. LOCAL OWNERSHIP While the project has not assured that there will be local ownership of interventions after the project ends, IFPP has aligned its interventions and integrated with MOH family planning policy strategies. As a result, IFPP efforts have the support of MOH officials at all levels and have strong potential to transition to local ownership after the activity ends. While IFPP supports a sustainable transition through training and transferring of processes, IFPP does not have a written transition strategy outlining how and when the activity interventions and achievements will fully transition to local control once IFPP ends. CONTINUATION OF PROJECT ACHIEVEMENTS Contraceptive Supply Chain There is no national budget to support the contraceptive supply chain or project activities. Political commitment exists, as does technical knowledge to replace IFPP’s interventions in the contraceptive supply chain once the program ends. Reliance on donor assistance to provide contraceptive commodities for the national FP program has worked well in the short￾term but without deliberate plans for budget allocations towards FP programming gains made under the IFPP, the Activity will not be sustained in the absence of donor support. Maintaining Service Providers Enhanced local capacity created by training of trainers and providers in FP will likely sustain after the end of the activity. IFPP has also established quality tools and materials that provide clinical and non-clinical guidance. Continuation of training programs and supervision of trained staff, however, will be jeopardized since IFPP covers all costs of the trainers, trainees and supervisors. Community-Level Demand Generation The positive community environment created to support FP will likely sustain because the demand creation interventions have led to a “mentality change” within the communities and individual attitudes. However, continuation of community dialogues is unlikely because community level demand generation interventions are conducted by facilitators/ activists who are paid stipends by IFPP. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 73 • “About sustainability… the project has supported capacity building of government staff; health providers are now more skilled to provide services and the beneficiaries have more knowledge. So, I think without the funds, gains will sustain”. (MOH) • “The people trained will be here and they stay. This will increase the likelihood of sustainability. The same people can stay on and continue to train others”. (MOH) Community-Level Demand Generation • IFPP has created a positive community environment to support FP service seeking behaviors (See conclusions provided under EQ1) (Interviews with Health Provider, Community Leaders, and Focus Group Discussions with Beneficiaries) • Community leaders in Mecuburi: “There is increased acceptability of modern contraceptive methods, even by the male partners / husbands. A lot of sensitization has been done in the communities, among the families, about to have intervals between children. We see attitudes changing rapidly.” • “Previously FP was a big problem, really serious, the women couldn’t talk to their husbands about FP. Pathfinder brought a different way of thinking and has changed the mentality. Now a peasant, or a worker goes to the health center with his wife for FP.” (District Director) 74 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX B: EVALUATION STATEMENT OF WORK Overview Mozambique is a low-income country located in south-east Africa. It is ranked 181 of 188 countries on the human development index (UNDP, 2016) and has experienced rapid population growth in recent years, with the total population increasing from 16.1 million in 1997 to an estimated 27.1 million in 2017 (INE, 2017). Overall, Mozambique has improved its reproductive health indicators in recent years. From 2011 to 2015, the total fertility rate decreased from 5.9 to 5.3 births per woman, the modern contraceptive prevalence rate (mCPR) among women in union increased from 11.3% to 25.3%, and unmet need for family planning (FP) decreased from 29% to 23% (IDS 2011, IMASIDA 2015). Antenatal care during pregnancy is high (92.6% of women attend at least one visit), and 70.3% of women deliver in a health facility (IMASIDA 2015). However, this overall progress masks substantial regional variations in reproductive health indicators. Nampula and Sofala provinces have modern CPRs of only 21.8% and 14.4% among women in union, respectively (IMASIDA 2015), and remain in need of program interventions to increase access to reproductive health services. Several factors have hindered family planning efforts in Mozambique over the past decade. Health systems challenges, traditional beliefs favoring large families, myths and misconceptions about contraception, poor access to services, challenging policy environments, and low funding commitments by both the government and donors, have all contributed to low contraceptive use, high fertility rates, and high unmet need for family planning throughout the country. IFPP activity description The Integrated Family Planning Program (IFPP) is a five-year USAID/Mozambique-funded initiative to increase use of modern contraceptive methods by target populations in all 36 districts in Nampula and Sofala provinces, led by Pathfinder International with a team of global and local partners— N’weti, Population Services International (PSI), and Abt Associates. IFPP responds to the United States Government (USG) strategy for development and foreign assistance in Mozambique through the Country Development Coordination Strategy (CDCS). The USAID/Mozambique CDCS outlines an overarching development objective health goal to “Improve the Health Status of Target Populations” through three results: 1) Increased coverage of high impact health and nutrition services, 2) Increased adoption of positive health and nutrition behaviors, and 3) Strengthened systems to deliver health, nutrition, and social services (CDCS, 2013). In alignment with this goal and these results, IFPP aims to support the Government of the Republic of Mozambique (GRM) and Ministry of Health (MISAU) priorities and increase the use of modern contraceptive methods by target populations through three intermediate results. In figure 1, we present the activity’s results framework: USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 75 Figure 1: IFPP’s results framework84 84 Drawn based on Project document - AID-656-A-16-00005 - Pathfinder International, June 07, 2016 – pgs 22-36 – project approach 76 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV The three intermediate results (IRs) are designed to be integrated and mutually reinforcing. IFPP aims to reach women with a particularly high unmet need for family planning (FP), namely: ▪ postpartum women; ▪ women living with HIV; ▪ adolescents, including orphans and vulnerable children (OVC); ▪ medium- and high-parity women; and ▪ post-abortion women. Geographic focus IFPP is being implemented in all 36 districts Nampula and Sofala provinces, as detailed in Figure 2. Figure 2: Districts per province Provinces Priority districts Sofala Nhamatanda, Muanza, Marínguè, Sofala, Marromeu, Machanga, Gorongosa, Beira, Búzi, Caia, Chemba Cheringoma, Chibabava, Dondo Nampula Angoche, Eráti, Ilha de Moçambique, Lalaua, Larde, Liúpo, Malema, Meconta, Mecubúri, Memba, Mogincual, Mogovolas, Moma, Monapo, Mossuril, Muecate, Murrupula, Nacala, NacalaVelha, Nacarôa, Nampula Rapale, Ribáuè, Evaluation Purpose The purpose of the IFPP Mid-Term Performance Evaluation is to assess the effectiveness of IFPP’s approach to increasing use of family planning and reproductive health services, as well as its efforts at strengthening FP/RH health systems. The findings and recommendations from this performance evaluation will inform USAID/Mozambique, the Implementation Partner (IP), GRM, and other stakeholders about the best ways to use IFPP technical assistance to maximize results over the remaining life of the activity. Learning will also inform the design of future FP/RH projects that support the national FP program. Evaluation areas & questions The evaluation areas and questions are presented in Figure 3. Figure 3: IFPP evaluation areas and questions Areas of Evaluation Evaluation questions & sub-questions A. Demand EQ1: To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? Sub-Questions: ▪ How has the knowledge and awareness of modern contraceptive methods increased? How has that change contributed to demand increase? ▪ How has the community environment contributed to support FP/RH service seeking behaviors? ▪ Which local innovations, if any, bolstered FP/RH service delivery, demand creation and health systems quality? B. Access & Quality of Services EQ2: To what degree has IFPP interventions increased access and quality of FP/RH services? Sub-Questions: ▪ How has access to modern contraceptive methods and Family Planning/Reproductive Health services increased? To what extent have IFPP interventions contributed to that increase? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 77 Areas of Evaluation Evaluation questions & sub-questions ▪ How has the quality of Family Planning/Reproductive Health services improved? How have IFPP interventions contributed to that improvement? ▪ What are the main barriers to reaching youth and adolescents with FP/RH? (learning Question) C. Health System Strengthen - Commodities EQ3: To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? Sub-Question: ▪ To what extent have the forecast and quantities of commodities received met the needs at service delivery point? ▪ At which level of commodities supply management chain there is high frequency of commodities stockouts of FP/RH? Methodological approach The evaluation will use qualitative and quantitative methods. The qualitative portion will collect and analyze secondary sources (IFPP records, reporting documents, and useful technical references), key informant interviews (KII), and group interviews. Qualitative data sources will include USAID, the implementation partners (Pathfinder, N’weti and Abt), health facility professionals, including Agentes Polivalentes de Saúde (APEs), as well as MISAU (Ministério da Saúde – Ministry of Health), DPS (Direcção Provincial de saúde – Provincial Directorate of Health) and DDSAS (District Directorate of Health and Social affairs) representatives. Community members – such as values’ gatekeepers, leaders and others – a s well as other relevant FP/RH stakeholders and partners at central, provincial and local levels will also be interviewed. The identification of the principal barriers and successes during implementation will be gauged through KIIs and by analyzing the MEL plan and related data, quarterly reports, and other secondary data. The quantitative approach will use a household survey to follow Knowledge, Attitude, and Practices (KAP) survey principles to measure understanding of general family planning & reproductive health (FP/RH) concepts and specific knowledge, attitude, and practices of specific FP/RH subjects. It will also explore their stance on barriers to the adoption and practice of that knowledge, such as gender norms and socio-cultural values. The KAP survey will use probability methods to target households with women in reproductive age and adolescents from the sample districts. The quantitative data will be complemented with qualitative data using focus groups (FGs) technique to better understand the reasons for existing perceptions, opinions, beliefs, and attitudes related to FP/RH. Survey protocol, sampling design and selection of districts The evaluation team will develop a KAP survey protocol that will assess the degree to which FP/RH messages and approaches in influenced target audience knowledge and practices in communities targeted by IFPP. The KAP survey protocol will be developed by a statistician and will include a sampling plan aligned with the one used by IFPP for its baseline. Since IFPP categorized its intervention districts according to the “package of services” to be delivered, the evaluation’s sample selection for the KAP survey will follow suit: a. Rural districts – the rural package – is directed to districts with limited access to any service delivery entry point; low demand for, and significant sociocultural barriers to, contraception; relatively low uptake; and high ideal number of children. This package couples significant community-based service delivery with intensive SBCC interventions at individual and community levels to generate demand. This package 78 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV focuses on postpartum women, adolescents and youth first-time parents (FTP), and medium- and high- parity women. b. Combo districts with high internal variation in access, demand, and contraceptive uptake due to the presence of corridor areas with increasing economic growth and mobile coverage, the combo package combines elements of the rural and the urban packages. This package will focus on postpartum women, adolescents and youth (particularly FTPs), and medium- and high-parity women, as well as reaching PAC clients through service delivery. c. Urban districts with high demand (including high unmet need) and physical access to services (but where services are of inadequate quality and the method mix is limited), high mobile and media coverage, a higher concentration of adolescents, secondary schools, and higher rates of sexual and gender-based violence (SGBV) and HIV. The urban package uses existing networks and platforms to increase linkages between priority populations and contraceptive and FP/RH services. This package focuses on postpartum women, HIV- positive women, and adolescents, and will reach post-abortion care (PAC) clients through health facilities (HFs). It is expected that this evaluation takes place in thirty-six (36) districts reaching around 2000 households and 100 catchment areas. Data Collection and tools The team will develop data collection tools and instruments for all data sources, including semi structured KII guides, KAP survey, FGD guides for specific groups, and the facility observation checklist. All data collection tools are focused on relevant evaluation questions and will be tested revised and approved by USAID prior to data collection. Tablets might be used for rapid validation and monitoring of survey process. Analysis methods and data disaggregation Gender Considerations The evaluation team will ensure that the evaluation approach incorporates gender analysis throughout all levels of the evaluation and analyze variables that might place certain individuals or populations at a disadvantage. Specific attention should be given to gender considerations while collecting and analyzing data. Where possible, sex-disaggregated data should be evaluated for outputs and outcomes to determine the degree to which the project affected both women and men. Specifically, the team should make sure that gender is incorporated in (1) evaluation design – both women and men should be interviewed and participate in KIIs and FGDs; (2) the data collection tools, which include specific questions on equal access to both sexes; and (3) data tabulation, analysis of results, and the conclusions and recommendations. Since the primary target for the household survey is women, all survey interviewers will be women. Recommended composition of team The evaluation team will be comprised of short-term evaluators, short-term fieldwork staff, and MMEMS full-time staff. MMEMS will also provide the required support staff, transport, office space, and other logistical support. The evaluation and level of effort assume that all tasks listed in the previous section will be conducted. ▪ Team Leader - An international researcher with deep experience in Family Planning, M&E health systems and conducting social and behavior change communication evaluations. He/she will be the technical lead responsible for the evaluation design and implementation of the methodologies to be applied at field level, overseeing data quality gathered at field level, providing analysis and leading report writing. ▪ Health Sector Specialist - A professional experienced in assessing the Mozambican health sector, particularly the area of social and behavior change. This professional will participate USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 79 in data collection and analysis and will participate in reporting under the lead of the Team Leader. ▪ Statistician - A Mozambican professional experienced in developing survey protocols, particularly sampling plans taking into the account rural conditions. ▪ Local Consultants – three individuals (two female and one male) to assist Team Leader and Health Sector Specialists and provide translation (English-Portuguese) and facilitation services. The KAP survey and related analysis, will be provided by a subcontractor. Period of performance and expected level of effort This SOW is designed for a three-month period, starting in February 2019. The total level of effort by the evaluation team will be 36 days in evaluation oversight by MMEMS staff, 64 days for the Team Leader investigator, 45 days for the Health Sector Specialist, and 15 days for the Statistician. Prior to contracting the evaluation team: (February 8) 1. Evaluation draft Scope of Work (SOW) submitted to USAID. Prior to Team Leader arriving in the country: (February 15) 2. Evaluation’s SOW finalized – MMMEMS will require USAID approval of the document; 3. Preliminary household survey protocol, sample design and respective instruments drafted; 4. Draft final household survey instruments (Portuguese and in local languages); First week in-country of the Team Leader: (February 18 to February 23) 5. Team Planning Meeting (TPM), resulting in a shared sense of the way forward and production of a Work Plan (WP), evaluation methodology, implementation timeline, finalization of the survey protocol, qualitative evaluation tools and instruments, and report outline; team meets with USAID to discuss evaluation; 6. Household survey training, pilot test and finalization of the survey instruments (Portuguese and in local languages); Second week in-country of the Team Leader: (February 25 to March1) 7. USAID meeting to present, discuss, fine tune, and approve the WP; 8. Household survey fieldwork in Nampula and Sofala; 9. KII fieldwork in Maputo by the consultants; Third & fourth week in-country of the Team Leader: (March 4 to March 16) 10. Household survey fieldwork in Nampula and Sofala; 11. Household survey DB clean and tabulation plan; 12. Fieldwork in, Sofala and Nampula by the consultants; Fifth & sixth week in-country of the Team Leader: (March 18 to March 30) 13. Data analysis; 14. Findings, Conclusions, and Recommendations table (FCR matrix). 15. Preliminary presentation of the preliminary results to USAID: 16. Report drafting will begin but will not be completed by 30 March. PowerPoint and oral presentations, including methodology, data analysis methods and data disaggregation, findings, conclusions, and recommendations with occur; After the departure of the leader evaluator: (April 1 to May 31) 17. Draft report to be delivered by April 22; 18. Final report after incorporation of USAID comments; and 19. Dissemination events with USAID, IP, and other stakeholders in format TBD (Maputo, Beira, Nampula). Deliverables Deliverables are listed in the Figure 4, by evaluation phase: 80 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Figure 4: Deliverables table Evaluation phase Deliverables Preparation Survey protocol, including the KAP sample design (draft and final versions) Overall planning Team planning meeting Work plan, including tool, development In-brief to Mission/Kick-off meeting Design of tools and instruments Tools and instruments (draft and final versions) Tools in local languages Fieldwork Weekly evaluation field reports/ internal Reporting Preliminary FCR presentation to MMEMS Out-brief to Mission Draft report Comments to report Final report Knowledge sharing Dissemination/learning activities/event Overview Mozambique is a low-income country located in south-east Africa. It is ranked 181 of 188 countries on the human development index (UNDP, 2016) and has experienced rapid population growth in recent years, with the total population increasing from 16.1 million in 1997 to an estimated 27.1 million in 2017 (INE, 2017). Overall, Mozambique has improved its reproductive health indicators in recent years. From 2011 to 2015, the total fertility rate decreased from 5.9 to 5.3 births per woman, the modern contraceptive prevalence rate (mCPR) among women in union increased from 11.3% to 25.3%, and unmet need for family planning (FP) decreased from 29% to 23% (IDS 2011, IMASIDA 2015). Antenatal care during pregnancy is high (92.6% of women attend at least one visit), and 70.3% of women deliver in a health facility (IMASIDA 2015). However, this overall progress masks substantial regional variations in reproductive health indicators. Nampula and Sofala provinces have modern CPRs of only 21.8% and 14.4% among women in union, respectively (IMASIDA 2015), and remain in need of program interventions to increase access to reproductive health services. Several factors have hindered family planning efforts in Mozambique over the past decade. Health systems challenges, traditional beliefs favoring large families, myths and misconceptions about contraception, poor access to services, challenging policy environments, and low funding commitments by both the government and donors, have all contributed to low contraceptive use, high fertility rates, and high unmet need for family planning throughout the country. IFPP activity description The Integrated Family Planning Program (IFPP) is a five-year USAID/Mozambique-funded initiative to increase use of modern contraceptive methods by target populations in all 36 districts in Nampula and Sofala provinces, led by Pathfinder International with a team of global and local partners— N’weti, Population Services International (PSI), and Abt Associates. IFPP responds to the United States Government (USG) strategy for development and foreign assistance in Mozambique through the Country Development Coordination Strategy (CDCS). The USAID/Mozambique CDCS outlines an overarching development objective health goal to “Improve the Health Status of Target Populations” through three results: 1) Increased coverage of high impact health and nutrition USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 81 services, 2) Increased adoption of positive health and nutrition behaviors, and 3) Strengthened systems to deliver health, nutrition, and social services (CDCS, 2013). In alignment with this goal and these results, IFPP aims to support the Government of the Republic of Mozambique (GRM) and Ministry of Health (MISAU) priorities and increase the use of modern contraceptive methods by target populations through three intermediate results. In figure 1, we present the activity’s results framework: 82 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Figure 1: IFPP’s results framework85 85 Drawn based on Project document - AID-656-A-16-00005 - Pathfinder International, June 07, 2016 – pgs. 22-36 – project approach. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 83 The three intermediate results (IRs) are designed to be integrated and mutually reinforcing. IFPP aims to reach women with a particularly high unmet need for family planning (FP), namely: ▪ postpartum women; ▪ women living with HIV; ▪ adolescents, including orphans and vulnerable children (OVC); ▪ medium- and high-parity women; and ▪ post-abortion women. Geographic focus IFPP is being implemented in all 36 districts Nampula and Sofala provinces, as detailed in Figure 2. Figure 2: Districts per province Provinces Priority districts Sofala Nhamatanda, Muanza, Marínguè, Sofala, Marromeu, Machanga, Gorongosa, Beira, Búzi, Caia, Chemba Cheringoma, Chibabava, Dondo Nampula Angoche, Eráti, Ilha de Moçambique, Lalaua, Larde, Liúpo, Malema, Meconta, Mecubúri, Memba, Mogincual, Mogovolas, Moma, Monapo, Mossuril, Muecate, Murrupula, Nacala, NacalaVelha, Nacarôa, Nampula Rapale, Ribáuè, Evaluation Purpose The purpose of the IFPP Mid-Term Performance Evaluation is to assess the effectiveness of IFPP’s approach to increasing use of family planning and reproductive health services, as well as its efforts at strengthening FP/RH health systems. The findings and recommendations from this performance evaluation will inform USAID/Mozambique, the Implementation Partner (IP), GRM, and other stakeholders about the best ways to use IFPP technical assistance to maximize results over the remaining life of the activity. Learning will also inform the design of future FP/RH projects that support the national FP program. Evaluation areas & Questions The evaluation areas and questions are presented in Figure 3. Figure 3: IFPP evaluation areas and questions Areas of Evaluation Evaluation questions & sub-questions D. Demand EQ1: To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? Sub-Questions: ▪ How has the knowledge and awareness of modern contraceptive methods increased? How has that change contributed to demand increase? ▪ How has the community environment contributed to support FP/RH service seeking behaviors? ▪ Which local innovations, if any, bolstered FP/RH service delivery, demand creation and health systems quality? E. Access & Quality of Services EQ2: To what degree has IFPP interventions increased access and quality of FP/RH services? Sub-Questions: 84 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Areas of Evaluation Evaluation questions & sub-questions ▪ How has access to modern contraceptive methods and Family Planning/Reproductive Health services increased? To what extent have IFPP interventions contributed to that increase? ▪ How has the quality of Family Planning/Reproductive Health services improved? How have IFPP interventions contributed to that improvement? ▪ What are the main barriers to reaching youth and adolescents with FP/RH? (learning Question) F. Health System Strengthen - Commodities EQ3: To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? Sub-Question: ▪ To what extent have the forecast and quantities of commodities received met the needs at service delivery point? ▪ At which level of commodities supply management chain there is high frequency of commodities stockouts of FP/RH? Methodological approach The evaluation will use qualitative and quantitative methods. The qualitative portion will collect and analyze secondary sources (IFPP records, reporting documents, and useful technical references), key informant interviews (KII), and group interviews. Qualitative data sources will include USAID, the implementation partners (Pathfinder, N’weti and Abt), health facility professionals, including Agentes Polivalentes de Saúde (APEs), as well as MISAU (Ministério da Saúde – Ministry of Health), DPS (Direcção Provincial de saúde – Provincial Directorate of Health) and DDSAS (District Directorate of Health and Social affairs) representatives. Community members – such as values’ gatekeepers, leaders and others – a s well as other relevant FP/RH stakeholders and partners at central, provincial and local levels will also be interviewed. The identification of the principal barriers and successes during implementation will be gauged through KIIs and by analyzing the MEL plan and related data, quarterly reports, and other secondary data. The quantitative approach will use a household survey to follow Knowledge, Attitude, and Practices (KAP) survey principles to measure understanding of general family planning & reproductive health (FP/RH) concepts and specific knowledge, attitude, and practices of specific FP/RH subjects. It will also explore their stance on barriers to the adoption and practice of that knowledge, such as gender norms and socio-cultural values. The KAP survey will use probability methods to target households with women in reproductive age and adolescents from the sample districts. The quantitative data will be complemented with qualitative data using focus groups (FGs) technique to better understand the reasons for existing perceptions, opinions, beliefs, and attitudes related to FP/RH. Survey protocol, sampling design and selection of districts The evaluation team will develop a KAP survey protocol that will assess the degree to which FP/RH messages and approaches in influenced target audience knowledge and practices in communities targeted by IFPP. The KAP survey protocol will be developed by a statistician and will include a sampling plan aligned with the one used by IFPP for its baseline. Since IFPP categorized its intervention districts according to the “package of services” to be delivered, the evaluation’s sample selection for the KAP survey will follow suit: a. Rural districts – the rural package – is directed to districts with limited access to any service delivery entry point; low demand for, and significant sociocultural barriers to, contraception; USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 85 relatively low uptake; and high ideal number of children. This package couples significant community-based service delivery with intensive SBCC interventions at individual and community levels to generate demand. This package focuses on postpartum women, adolescents and youth first-time parents (FTP), and medium- and high- parity women. b. Combo districts with high internal variation in access, demand, and contraceptive uptake due to the presence of corridor areas with increasing economic growth and mobile coverage, the combo package combines elements of the rural and the urban packages. This package will focus on postpartum women, adolescents and youth (particularly FTPs), and medium- and high-parity women, as well as reaching PAC clients through service delivery. c. Urban districts with high demand (including high unmet need) and physical access to services (but where services are of inadequate quality and the method mix is limited), high mobile and media coverage, a higher concentration of adolescents, secondary schools, and higher rates of sexual and gender-based violence (SGBV) and HIV. The urban package uses existing networks and platforms to increase linkages between priority populations and contraceptive and FP/RH services. This package focuses on postpartum women, HIV- positive women, and adolescents, and will reach post-abortion care (PAC) clients through health facilities (HFs). It is expected that this evaluation takes place in thirty-six (36) districts reaching around 2000 households and 100 catchment areas. Data Collection and Tools The team will develop data collection tools and instruments for all data sources, including semi structured KII guides, KAP survey, FGD guides for specific groups, and the facility observation checklist. All data collection tools are focused on relevant evaluation questions and will be tested revised and approved by USAID prior to data collection. Tablets might be used for rapid validation and monitoring of survey process. Analysis methods and data disaggregation Gender Considerations The evaluation team will ensure that the evaluation approach incorporates gender analysis throughout all levels of the evaluation and analyze variables that might place certain individuals or populations at a disadvantage. Specific attention should be given to gender considerations while collecting and analyzing data. Where possible, sex-disaggregated data should be evaluated for outputs and outcomes to determine the degree to which the project affected both women and men. Specifically, the team should make sure that gender is incorporated in (1) evaluation design – both women and men should be interviewed and participate in KIIs and FGDs; (2) the data collection tools, which include specific questions on equal access to both sexes; and (3) data tabulation, analysis of results, and the conclusions and recommendations. Since the primary target for the household survey is women, all survey interviewers will be women. Recommended composition of team The evaluation team will be comprised of short-term evaluators, short-term fieldwork staff, and MMEMS full-time staff. MMEMS will also provide the required support staff, transport, office space, and other logistical support. The evaluation and level of effort assume that all tasks listed in the previous section will be conducted. ▪ Team Leader - An international researcher with deep experience in Family Planning, M&E health systems and conducting social and behavior change communication evaluations. He/she will be the technical lead responsible for the evaluation design and implementation of the methodologies to be applied at field level, overseeing data quality gathered at field level, providing analysis and leading report writing. 86 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ▪ Health Sector Specialist - A professional experienced in assessing the Mozambican health sector, particularly the area of social and behavior change. This professional will participate in data collection and analysis and will participate in reporting under the lead of the Team Leader. ▪ Statistician - A Mozambican professional experienced in developing survey protocols, particularly sampling plans taking into the account rural conditions. ▪ Local Consultants – three individuals (two female and one male) to assist Team Leader and Health Sector Specialists and provide translation (English-Portuguese) and facilitation services. The KAP survey and related analysis, will be provided by a subcontractor. Period of performance and expected level of effort This SOW is designed for a three-month period, starting in February 2019. The total level of effort by the evaluation team will be 36 days in evaluation oversight by MMEMS staff, 64 days for the Team Leader investigator, 45 days for the Health Sector Specialist, and 15 days for the Statistician. Prior to contracting the evaluation team: (February 8) 1. Evaluation draft Scope of Work (SOW) submitted to USAID. Prior to Team Leader arriving in the country: (February 15) 2. Evaluation’s SOW finalized – MMMEMS will require USAID approval of the document; 3. Preliminary household survey protocol, sample design and respective instruments drafted; 4. Draft final household survey instruments (Portuguese and in local languages); First week in-country of the Team Leader: (February 18 to February 23) 5. Team Planning Meeting (TPM), resulting in a shared sense of the way forward and production of a Work Plan (WP), evaluation methodology, implementation timeline, finalization of the survey protocol, qualitative evaluation tools and instruments, and report outline; team meets with USAID to discuss evaluation; 6. Household survey training, pilot test and finalization of the survey instruments (Portuguese and in local languages); Second week in-country of the Team Leader: (February 25 to March1) 7. USAID meeting to present, discuss, fine tune, and approve the WP; 8. Household survey fieldwork in Nampula and Sofala; 9. KII fieldwork in Maputo by the consultants; Third & fourth week in-country of the Team Leader: (March 4 to March 16) 10. Household survey fieldwork in Nampula and Sofala; 11. Household survey DB clean and tabulation plan; 12. Fieldwork in, Sofala and Nampula by the consultants; Fifth & sixth week in-country of the Team Leader: (March 18 to March 30) 13. Data analysis; 14. Findings, Conclusions, and Recommendations table (FCR matrix). 15. Preliminary presentation of the preliminary results to USAID: 16. Report drafting will begin but will not be completed by 30 March. PowerPoint and oral presentations, including methodology, data analysis methods and data disaggregation, findings, conclusions, and recommendations with occur; After the departure of the leader evaluator: (April 1 to May 31) 17. Draft report to be delivered by April 22; 18. Final report after incorporation of USAID comments; and 19. Dissemination events with USAID, IP, and other stakeholders in format TBD (Maputo, Beira, Nampula). Deliverables Deliverables are listed in the Figure 4, by evaluation phase: USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 87 Figure 4: Deliverables table Evaluation phase Deliverables Preparation Survey protocol, including the KAP sample design (draft and final versions) Overall planning Team planning meeting Work plan, including tool, development In-brief to Mission/Kick-off meeting Design of tools and instruments Tools and instruments (draft and final versions) Tools in local languages Fieldwork Weekly evaluation field reports/ internal Reporting Preliminary FCR presentation to MMEMS Out-brief to Mission Draft report Comments to report Final report Knowledge sharing Dissemination/learning activities/event 88 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX C: GETTING TO ANSWERS MATRIX Data Required Data Sources Data Collection Methods Data Analysis Methods Evaluation Question #1: To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? Data on trends in target audiences’ knowledge, attitudes and demand for contraceptive services Data on trends in community environment to support family planning service seeking behavior Data on any local innovations to support FP service delivery, demand creation and health systems quality USAID, IFPP and MOH staff and other FP program stakeholders at the central, provincial and districts levels Health care providers at the service delivery points Reproductive age women and their partners Adolescents Community leaders KIIs IFPP records and reports KAP survey FGDs Analyses of qualitative and quantitative data Data triangulation Descriptive statistics and data visualization of key indicators of KAP survey Estimation of IFPP interventions’ effect Evaluation Question #2: To what degree has IFPP interventions increased access to and quality of FP/RH services? Data on trends in access to modern contraceptive methods USAID, IFPP and MOH staff and other FP program stakeholders at the central, provincial and districts levels KIIs IFPP records and reports Analyses of qualitative and quantitative data Data triangulation USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 89 Data Required Data Sources Data Collection Methods Data Analysis Methods Data on IFPP’s interventions which has contributed to increasing access to FP services Data on barriers to reaching youth and adolescents with FP services Health care providers at the service delivery points Reproductive age women and their partners Adolescents Community leaders KAP survey FGDs Descriptive statistics and data visualization of key indicators of KAP survey Estimation of IFPP interventions’ effect Evaluation Question #3: To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? Data on stockouts and availability of contraceptives at the service delivery points Data on management and functionality of contraceptive supply chain USAID, IFPP and MOH staff and other FP program stakeholders at the central, provincial and districts levels Health care providers at the service delivery points Reproductive age women and their partners Adolescents Community leaders KIIs IFPP records and reports FGDs Facility observations Analyses of qualitative and quantitative data Data triangulation 90 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX D: DATA COLLECTION INSTRUMENTS KAP SURVEY QUESTIONNAIRE QUESTIONNAIRE FOR WOMEN BETWEEN 15-49 of age IDENTIFICATION DISTRICT VILLAGE/SETTLEMENT/ NEIGHBORHOOD NAME AND NUMBER OF ENUMERATION AREA SERIAL NUMBER (S2#) URBAN/RURAL (URBAN = 1; RURAL = 2) NAME OF THE HEAD OF THE HOUSEHOLD NAME AND LINE (refer to questionnaire Agregado familiar – HH survey) OF WOMAN BETWEEN 15- 49 years of age VISITS FROM ENUMERATOR Date Starting time Ending time ENUMERATOR Outcome* Next Visit Date Time Final Visit Day Month Year 201 Code Outcome Total number of visits *Codes: 1 Complete 2 Not at home 3 Postponed 4 Refused 5 Refusal during the interview 6 Partially completed 7 Disabled 8 Others (Specify) ENUMERATOR Name Date SUPERVISOR Name Date TYPIST Name USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 91 Date QUESTIONNAIRE FOR WOMAN BETWEEN 15-49 years of age Module 1. SOCIO-DEMOGRAPHIC CHARACTERISTICS 101 On which month and year were you born? Month Don’t know the month……… Year Don’t know the year……… 102 What age did you complete on your last birthday? Compare and correct 101 and/or 102 if inconsistent Age in completed years 103 Have you ever been to school? Yes…….1 No……..2 - go to 106 104 What is the highest level of education you completed? Literacy…………………01 Primary school (EP1)………02 Primary school (EP2)………03 High school (1st cycle)……..04 High school (2nd cycle)…...05 Technical education……….06 Higher education……………07 Don’t know……………………08 105 What is the highest grade/year you completed at this level? Grade……. 106 What is your religion? Catholicism……….01 Protestant/Evangelical….02 Zionism…………03 Islam……………04 Animist………..05 No religion….06 Other (Specify) 107 What is your nationality? Mozambican……….01 South African……..02 Zimbabwean………03 Malawian…………..04 Tanzanian…………05 Nigerian……………06 92 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Other (Specify) 108 What is your mother tongue? Portuguese………..01 Emakhuwa………..02 Xichangana……….03 Elomwe……………04 Cisena……………..05 Echuwabo………..06 Shona………………07 Other (Specify) Module 2. REPRODUCTION 201 Now I would like to ask you questions about births you had in your life. Have you ever given birth? Yes……………..1 No………………2 go to 206 202 Do you have children of your own, children that you have given birth to, that live with you currently? Yes……………..1 No………………2 go to 204 203 How many male children live with you? How many female children live with you? If none, record ‘00’ Male children at home……… Female children at home……… 204 Do you have any born alive son or daughter who currently doesn’t live with you? Yes…. No…..go to 206 205 How many alive male children do not live with you? How many alive female children do not live with you? IF NONE, RECORD ‘00’ Male children away from home…. Female children away from home…. 206 Did you give birth to any son or daughter who was born alive but died afterwards? IF NO, INSIST: Any child who cried or has given any other sign of being alive at the moment of delivery but who didn’t survive? Yes…. No…..go to 208 207 How many boys died? How many girls died? IF NONE, RECORD ‘00’ Boys who died …. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 93 Girls who died…. 208 SUM UP THE ANSWERS TO QUESTIONS 203, 205 AND 207 AND MARK THE TOTAL. IF NONE, RECORD ‘00’ Total… 209 CHECK 208: I would like to check if I understood properly: In TOTAL you had … children born alive. Is that number correct? Yes…. No….. INSIST AND CORRECT 201-208 IF NECESSARY 210 CHECK 208: One or more births NONE GO TO 216 211 Now I would like to ask questions about your last delivery, no matter if this child is alive or not. RECORD THE NAME OF THE 3 LAST BIRTHS IN 212. IN CASE OF TWINS RECORD IN SEPARATE LINES (QUESTIONS 212-215) 216 Are you pregnant currently? Yes……………..1 No………………2 219 I Don’t Know……8 - go to 219 217 When you got pregnant, did you want to get pregnant at that moment? Yes……………..1 - go to 301 No………………2 - go to 301 218 Did you want to have a child later on or you didn’t want to have any (other) child? Later on………1 go to 301 Didn’t want to have (another) child…..2 - go to 301 219 When did your last menstruation start? DATE IF GIVEN Days ago Weeks ago Months ago Is in menopause/hysterectomized Before the last pregnancy Never had menstruation Module 3. FAMILY PLANNING ON ANTE-NATAL, DELIVERY AND POST-PARTUM 301 Check 214: One or more births in 2011 or later No birth in 2011 or later - go to 401 302 Check 212: Record in Table the name of the last childbirth 94 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Now I would like to ask you questions about your last child’s health (the one that was born in the last 5 years). 302A When you got pregnant of (name), did you want to have a child at that moment? Yes……………..1 -go to 303 No………………2 302B Did you want to have a child later on or you didn’t want to have any (other) child? Later on………1 Didn’t want to have any (other) child……2 - go to 303 302C How long you wanted to wait? Months…….1 Years………..2 Don’t know……….. 303 When you were pregnant of (name), did you go to any antenatal care appointment? If yes, who examined you? Anyone else? Record all mentioned answers Healthcare professional: Doctor……….A Nurse…………B Midwife……...C Other people: Traditional Birth Attendant……D Community health agent……E Relative/neighbor………….F Other (Specify)……….X Didn’t go to an antenatal care appointment 304 In which place did you have the antenatal care appointment for this pregnancy? In any other place? Record all mentioned answers If not able to identify a Hospital, health center or if it is a public or private clinic, write the name of the place. (Name of the place(s)) Public Sector: Central hospital……A General hospital……B Rural hospital………C District hospital……D Heath center………..E Health post…………F Mobile brigades…..G Other (Specify)………H Private Sector: Clinic……….I Doctor’s office……J Pharmacy…………K USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 95 Other (Specify)………L Other: Market/fair………..M Church………………N Traditional healer.. O Other (Specify)………X 305 How many antenatal care appointments did you have during this pregnancy? Number…… Don’t know……….. 311 Did you get any advice or information about the use of post-partum family planning methods during any of the antenatal care appointments for this pregnancy? (several ways or methods couples use to avoid or space post-partum pregnancy) Yes……………..1 No………………2 - go to 314 Don’t Know…8 - go to 314 313 What was the method recommended for you? Record all mentioned answers Sterilization……..A Vasectomy….B Contraceptive pill……………..C IUD ………….D Contraceptive Injections…………….E Birth Control Implant……………..F Male condom………..G Female condom…….H Lactation Amenorrhea….I Periodic Sexual Abstinence…….J Coitus Interruptus……….K Others (Specify)…………….X 314 Who delivered your last child (name)? Anyone else helped? Record all mentioned answers If the answer is that no one delivered her child, ask if some adults have been present during the delivery. Healthcare professional: Doctor……….A Nurse…………B Midwife……...C Other people: Traditional Birth Attendant……D Activist……….E Friends/Relatives……..F Others: (Specify)…………….X Nobody Y 315 Where did you give birth to (name)? Record all mentioned answers 96 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV If not able to identify a Hospital, health center or if it is a public or private clinic, write the name of the place. Public Sector: Central hospital……11 General hospital……12 Rural hospital………13 District hospital……14 Heath center………..15 Health post…………16 Mobile brigades…..17 Other (Specify)……18 Private Sector Clinic……….21 Doctor’s office……22 Pharmacy…………23 Other (Specify)………28 House Own house…..31 Traditional Birth Attendant’s house…..32 Midwife/Nurse’s house…….33 316 After giving birth and after leaving the hospital/sanitary unit, did you get any advice or information about family planning? Yes……………..1 No………………2 - go to 322 Don’t Know…8 - go to 322 317 Who gave you that advice or information? Record all mentioned answers Healthcare professional: Doctor……….A Nurse…………B Midwife……...C Other people: Traditional Birth Attendant……D Activist……………………E Friends/relatives……..F Others (Specify)………X 318 Did they give you advice about any specific family planning method? Yes……………..1 No………………2 - go to 320 Don’t Know…8 - go to 320 319 Which method did they recommend you to take? Record all mentioned answers Sterilization ……..A Vasectomy….B Contraceptive pill ……………..C IUD………….D USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 97 Contraceptive Injections…………….E Birth Control Implant .……………..F Male condom………..G Female condom…….H Lactation Amenorrhea….I Periodic Sexual Abstinence…….J Coitus Interruptus……….K Others (Specify)…………….X 320 Did you leave the hospital/sanitary unit with a family planning method? Yes……………..1 No………………2 go to 322 Don’t Know…8 go to 322 321 Which method(s) did you receive? Record all answers Sterilization ……..A Vasectomy….B Contraceptive pill ……………..C IUD………….D Contraceptive Injections…………….E Birth Control Implant……………..F Male condom………..G Female condom…….H Lactation Amenorrhea….I Periodic Sexual Abstinence…….J Coitus Interruptus……….K Others (Specify)…………….X Answering to any of these questions, go to 323 322 (If the response is that she didn’t leave the hospital/sanitary unit with a family planning method): Did you receive advice about when you needed to return to the sanitary unit to start the use of a contraceptive method? Yes……………..1 No………………2 Don’t Know… 8 323 After giving birth how many days or weeks did you have the first post-partum care appointment? If it was the same day record ‘00’ days Days later…………1 Weeks later………2 Never had…………3 go to 401 Don’t Know……….8 go to 401 324 In which place did you have the first post-partum care appointment? Home……………1 Sanitary unit…2 Others (Specify)……6 98 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 325 Did anyone give you advice or information about family planning in that first post￾partum care appointment? Yes……………..1 No………………2 go to 401 Don’t Know…8 go to 401 326 Who gave you that advice or information? Record all mentioned answers Healthcare professional: Doctor……….A Nurse…………B Midwife……...C Other people: Traditional Birth Attendant……D Community health agent……E Others (Specify)…………….X 327 Did they give you advice about the use of any specific family planning method? Yes……………..1 No………………2 go to 329 Don’t Know…8 go to 329 328 Which method did they advice you to take? Record all mentioned answers Sterilization ……..A Vasectomy….B Contraceptive pill ……………..C IUD………….D Contraceptive Injections…………….E Birth Control Implant……………..F Male condom………..G Female condom…….H Lactation Amenorrhea….I Periodic Sexual Abstinence…….J Coitus Interruptus……….K Others (Specify)…………….X 329 Did you leave that first post-partum care appointment with a family planning method? Yes……………..1 No………………2 go to 401 Don’t Know…8 go to 401 330 Which method(s) did you receive? Record all mentioned answers Sterilization……..A Vasectomy…..….B Contraceptive pill……………..C IUD ………….D USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 99 Contraceptive Injections…………….E Birth Control Implant……………..F Male condom………..G Female condom…….H Lactation Amenorrhea….I Periodic Sexual Abstinence…….J Coitus Interruptus……….K Others (Specify)…………….X 331 After giving birth to (name) did your menstruation come back? Yes……………..1 No………………2 332 Usually who decide about how the money you receive will be used: you, your husband/partner or both together? The respondent…………1 Husband/partner………2 Both………………………….3 Other………………………..6 333 Usually who decide about your healthcare: you, your husband/partner or both together? The respondent…………1 Husband/partner………2 Both………………………….3 Other………………………..6 Module 4. CONTRACEPTION 401 Now I would like to talk a little about ways or methods of family planning – several ways or methods couples use to avoid or to space the pregnancy. What methods or ways of family planning do you know or already heard about? - PUT A CIRCLE AROUND THE CODE (FOR EXEMPLE 01 FOR STERILIZATION) IN 401, REPEAT IT FOR A METHOD SPONTANEOUSLY MENTIONED. - FOR EACH METHOD NON-SPONTANEOUSLY MENTIONED PROCEED IN COLUMN 401, READING THE NAME, THE DESCRIPTION OF EACH METHOD. - FOR EACH METHOD NON SPONTANEOUSLY MENTIONED, ASK: Have you ever heard about this method? 402 Have you ever used it (METHOD) ASK THIS QUESTION ONLY FOR METHODS SHE KNOWS OR ALREADY HEARD ABOUT 01 STERILIZATION (tubal ligation). Women can make small surgery to stop having children. Yes……………..1 No………………2 Have you ever had any small surgery to stop having children? Yes……………..1 No………………2 100 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 02 VASECTOMY. Men can make small surgery to stop having children Yes……………..1 No………………2 Have you ever had a husband or partner who made a surgery to stop having children? Yes……………..1 No………………2 03 CONTRACEPTIVE PILL. Women can take a pill every day to avoid pregnancy Yes……………..1 No………………2 Yes……………..1 No………………2 04 IUD (Intrauterine Device). A midwife or doctor can insert a device in a woman uterus to avoid pregnancy. Yes……………..1 No………………2 Yes……………..1 No………………2 05 CONTRACEPTIVE INJECTIONS. Every three months women can receive an injection to avoid pregnancy. Yes……………..1 No………………2 Yes……………..1 No………………2 06 BIRTH CONTROL IMPLANT. Women can have several small sticks inserted by a doctor or nurse in the arm which can prevent pregnancy for one or more years Yes……………..1 No………………2 Yes……………..1 No………………2 07 MALE CONDOM. Men can use a condom (JEITO) during sexual intercourse Yes……………..1 No………………2 Yes……………..1 No………………2 08 FEMALE CONDOM. A woman can insert a female condom inside her vagina before sexual intercourse Yes……………..1 No………………2 Yes……………..1 No………………2 09 LACTATION AMENORRHEA METHOD. After a birth often a woman would be protected from pregnancy while breastfeeding until menstruating again. Yes……………..1 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 101 No………………2 Yes……………..1 No………………2 10 PERIODIC SEXUAL ABSTINENCE. Couples can avoid having sexual relations during the days of the month when the wife has a greater risk of getting pregnant Yes……………..1 No………………2 Yes……………..1 No………………2 11 WITHDRAWAL METHOD (coitus interruptus). During sexual intercourse men can be careful and withdraw the penis from the vagina before ejaculation. Yes……………..1 No………………2 Yes……………..1 No………………2 12 EMERGENCY CONTRACEPTION. Women can take some pills up to five days after a non-protected sexual intercourse to avoid pregnancy Yes……………..1 No………………2 Yes……………..1 No………………2 13 Have you ever heard about other methods or ways a woman or a man can use to avoid pregnancy? Yes……………..1 (Specify) No…………….2 (Specify) 403 Check 402: None - “Yes” (Never used) At least one - “Yes” - go to 406 (Has used) 404 Have you ever did or used something to avoid pregnancy? Yes……………..1 No………………2 go to 407 405 What did you do or use to avoid pregnancy? Correct 402 and 403 (and 401 if necessary) 406 Check 402 (01): Non-sterilized woman Sterilized woman – go to 409A 407 Check 216: 102 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Non-pregnant woman Nor sure Pregnant woman go to 411 408 Are you or your husband/partner currently using any method to avoid pregnancy? Yes……………..1 No………………2 go to 411 409 Which method are you using? RECORD ALL MENTIONED ANSWERS IF ONE OR MORE ARE MENTIONED, THE INSTRUCTIONS OF “GO TO” THE FIRST METHOD MENTIONED IN THE LIST. Sterilization……..A Vasectomy….B Contraceptive pill……………..C IUD ………….D Contraceptive Injections…………….E Birth Control Implant…………….. E Male condom………..F Female condom…….G Lactation Amenorrhea….H Periodic Sexual Abstinence…….I - go to 413 Coitus Interruptus……….J - go to 413 Others (Specify)…………….X 409A Put a circle around letter ‘A’ for Sterilization 410 Where did you get the (CURRENT METHOD) last time? If more than one method was recorded in 409, ask about the first method recorded in the list If not able to identify a hospital, health center or if it is a public or private clinic, write the name of the place (Name of the local(s)) If the method is sterilization don’t record pharmacy, market/fair, Friend/Relative Public Sector: Central hospital……11 General hospital……12 Rural hospital………13 District hospital……14 Heath center………..15 Health post…………16 Mobile brigades…..17 Other (Specify) Private Sector: Clinic……….21 Doctor’s office……23 Pharmacy…………24 Other (Specify)………25 Other: Market/fair………..31 Church………………32 Traditional healer…33 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 103 Friends/Relatives….34 Community health agent….35 Pair educators …………36 Others (Specify) 96 Answering any of these – go to 413 411 Do you know where you can get a family planning method? Yes……………..1 No………………2 go to 413 412 In which place? Some other place? Record all mentioned answers If not able to identify Public Sector: Central hospital……A General hospital……B Rural hospital………C District hospital……D Heath center………..E Health post…………F Mobile brigades…..G Other (Specify)………H Private Sector: Clinic……….I Doctor’s office……J Pharmacy…………K Other (Specify)………L Other: Market/fair………..M Church………………N Traditional healer...O Friends/Relatives….P Community health agent….Q Pair educators …………R Others (Specify) X Check 402: At least one “Yes” - (Has used) None “Yes” - (Never used contraceptives) go to 414 413A In which year did you start using family planning for the first time or how old were you? Starting year Age 413B Have you ever been informed by the healthcare provider about the side effects or problems that you could have when using the method? 104 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Yes……………..1 No………………2 413C Did they inform you what to do in case you feel those side effects or problems? Yes……………..1 No………………2 414 In the last 12 months did you have any medical appointment at the health center/ health post/hospital for your own healthcare (or that of your children)? Yes……………..1 No………………2 – go to 418 415 If yes, what services did you look for? Appointment for healthy children (vaccination) ……1 Triage…….2 Appointment for reproductive health…….3 TARV……4 Others 5 416 Did they ever mention, in any of these services, something about contraceptive methods? Yes……………..1 No………………2 – go to 418 417 In which of these services did they say something about contraceptive methods? Appointment for healthy children (vaccination) ……1 Triage…….2 Appointment for reproductive health…….3 TARV……4 Others 5 418 Would you say that the contraceptive method is mainly decided by you, by your husband/partner or by both of you? By the respondent.1 By husband/partner….2 Both….3 Others….6 MODULE 5. MARRIAGE AND SEXUAL ACTIVITY 501 Are you currently married or live in a union with a man as if you were married? Yes currently married…..1 go to 504 Yes living in a marital union...2 go to 504 No……3 502 Have you ever been married or lived in a union with a man as if you were married? Yes, been married…..1 Yes, lived in a marital union …..2 No……3 go to 506 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 105 503 What are your marital status currently: are you a widow, divorced, separated? Widow.....1 Divorced…..2 Separated…..3 Any of these – go to 506 504 Are your husband/partner currently living with you or in another place? Lives with him…..1 In another place…..2 505 Does your partner or husband approve that you use a contraceptive method to prevent pregnancy? Yes………1 No……….2 Don’t Know………8 505A Check 501 and 502 Got married/lived with a man once Got married/lived with a man more than once In which year and month did you start living with your husband/partner? In which year and month did you start living with your first husband/partner? Month……. Doesn’t know the month………98 Year……… go to 507 Doesn’t know the year………9998 - go to 507 505B How old were you when you started living with him? Age……….. 506 Check the presence or other people before proceeding to make sure there is privacy 507 Now I would like to ask you some questions about your sexual activity for a better understanding of some life problems. How old were you when you had sexual relations for the first time? Never had sexual relations…….00 – go to 601 Age in years……… Started sexual life for the first time with the first husband/partner…..95 508 Now I would like to ask you some questions about your sexual activity. But before I would like to assure you again that all your answers are absolutely confidential and that they will not be disclosed to anyone. If I ask you a question that you don’t want to answer, tell me and I will go to the next question. When did you have sexual relations for the last time? IF LESS THAN 12 MONTHS, THE ANSWER SHOULD BE RECORDED IN DAYS, WEEKS OR MONTHS. IF 12 MONTHS OR MORE THE ANSWER SHOULD BE CONVERTED AND RECORDED IN YEARS. ….days ago…….1 ….weeks ago……2 106 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV …..months ago……3 ……years ago……..4 Any of these – go to 510 509 When did you have sexual relations with this person for the last time? Last sexual partner Penultimate sexual partner ….days ago…….1 ….weeks ago……2 …..months ago……3 Third to last sexual partner ….days ago…….1 ….weeks ago……2 …..months ago……3 510 The last time you had sexual relations the condom was used? The same question for all of these 3 last partners Yes………1 No……….2 – go to 512 511 Did you use condom for ALL sexual relations during the last 12 months? The same question for all of these 3 last partners Yes………1 No……….2 Yes………1 No……….2 Yes………1 No……….2 512 What kind of relationship do you have/had with this person you had sexual relations with? IF BOYFRIEND: Did you live together as if you were married? IF YES, PUT A CIRCLE AROUND ‘2’ IF NO, PUT A CIRCLE AROUND ‘3’ The same question for all of these 3 last partners Husband…..1 Partner……2 Boyfriend not living with the respondent…….3 Occasional date………4 Other (specify)……6 Husband…..1 Partner……2 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 107 Boyfriend not living with the respondent…….3 Occasional date ………4 Other (specify)……6 Husband…..1 Partner……2 Boyfriend not living with the respondent…….3 Occasional date ………4 Other (specify)……6 513 Apart from this/these person(s) did you have sexual relations with other person during the last 12 months? YES…….1 (GO BACK TO 509 NEXT COLUMN) NO……..2 (GO TO 601) YES…….1 (GO BACK TO 509 NEXT COLUMN) NO……..2 (GO TO 601) 514 How many different persons did you have sexual relations with during the last 12 months altogether? INSIST TO OBTAIN THIS ESTIMATE Number of partners during the last 12 months….. Don’t know……98 Module 6. PREFERENCES REGARDING FERTILITY 601 CHECK 402 – 01/02 she/he is not sterilized/didn’t do vasectomy she/he is sterilized/did vasectomy GO TO 700 602 CHECK 216: Is not pregnant or is in doubt: Now I would like to ask you some questions about the future. Do you want to have (another) child or you prefer not to have more children? Is pregnant: Now I would like to ask you some questions about the future. Do you want to have another child after the one you are expecting or you prefer not to have more children? Have (another) child…..1 Doesn’t want any more…..2 GO TO 604 Cannot get pregnant…..3 GO TO 700 Hesitant/doesn’t know and is pregnant…..4 610 Not pregnant or in doubt…….5 GO TO 608 603 CHECK 216: Is not pregnant or is in doubt: 108 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV How long do you want to wait before the birth of (another) child? Is pregnant: How long do you want to wait to have another child after this one is born? Months…………1 Years…………….2 Now……………..993 GO TO 609 Cannot get pregnant…..994 GO TO 700 After marriage………….995 Other specify 996 GO TO 609 Don’t Know………..998 604 CHECK 216: IF SHE IS PREGNANT Is not pregnant or is in doubt Is pregnant GO TO 610 605 CHECK 408: Is she using a method? Wasn’t asked Is not using contraceptive methods currently Is using contraceptive methods currently GO TO 608 606 CHECK 603: IF SHE WANTS TO WAIT: 24 MONTHS or more/ 2 years or more/ Question 603 wasn’t asked Less than 24 MONTHS or less than 2 years GO TO 610 607 CHECK 602: IF SHE WANTS ANOTHER CHILD: She wants (another) child 602=1: You said you don’t want to have another child for the moment but you are not using any method to avoid pregnancy. Can you tell me why? Any other reason? She doesn’t want to have any more children (602=2): You said you don’t want to have any more children but you are not using any method to avoid pregnancy. Can you tell me why? Any other reason? Circle all answers mentioned Is not married………….A Reasons regarding fertility: Is not having sexual relations………B Rarely has had sexual relations……C Menopause/sterile………….D Infertile……………..E Male infertility…….F Post-partum Amenorrhea…………G Is breast feeding……….H Fatalist……………..I USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 109 Resistance from using the methods Respondent is against the use………..J Husband/partner is against……………K Others are against Religion forbids Lack of knowledge Don’t Know the methods………….N Don’t Know the sources…………..O Reasons regarding the methods: Health concerns……….P Afraid of side effects…………….Q It’s convenient to use………………R Interfere with the normal functioning of the body…S Reasons regarding access and cost: Doesn’t have access/the sources are so far away……T Are expensive………U Another reasonSpecify X Don’t know…………Z 608 If in the next weeks you find out you are pregnant will it be a big problem, a small problem or will it not be a problem for you? Big problem………..1 small problem……..2 No problem…………3 Cannot get pregnant/don’t have sexual relations………4 609 Check 408: Is she using a method? Didn’t ask Is not using Is using GO TO 700 610 Do you think of using any method at any time in the future to delay or avoid pregnancy? Yes………1 No……….2 GO TO 612 Don’t know…….3 GO TO 612 611 Which method would you prefer to use? Sterilization……..1 Vasectomy….2 Contraceptive pill……………..3 IUD ………….4 Contraceptive Injections…………….5 Birth Control Implant……………..5 Male condom………..6 Diaphragm…….7 110 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Foam/gel/eggs………8 Lactation Amenorrhea….9 Periodic Sexual Abstinence…….10 Coitus Interruptus……….11 Other (Specify)…………….96 Hesitant……………..98 In case any of these answers go to 700 612 What is the main reason for not using any contraceptive method? Not married/not in a union……11 Reasons regarding fertility Irregular sexual relations………22 Menopause/hysterectomy…….23 Sterile…………………………………..24 Wants to have more children…..25 Resistance to family planning Respondent is against the use……31 Husband/partner is against……………32 Others are against……………..33 Religion forbids…………….34 Reasons regarding the method Health problems…………51 Side effects………………..52 Access difficulties/it’s far away………..53 It’s very expensive…………..54 Use inconvenience………….55 Interfere with the normal functioning of the body……56 Other specify 96 Don’t know………98 In case any of these answers go to 700 613 If you were married would you use any contraceptive method? Yes…………1 No…………..2 Don’t know…………8 Module 7. KNOWLEDGE ABOUT SEXUAL AND REPRODUCTIVE HEALTH 700 For the health of mother and baby, what is the minimum time a woman should wait to get pregnant again after giving birth? If the woman answers more or equal to 2 years, 24 months, mark option 1 CORRECT If her answer is less than 2 years, 24 months, mark option 2 INCORRECT Greater or equal to 2 years…………1 Less than 2 years………..2 Don’t know………3 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 111 701 For the health of mother and baby, what is the minimum time a woman should wait to get pregnant again after a miscarriage? If the woman answers more or equal to 6 months mark option 1 CORRECT If her answer is less than 6 months mark option 2 INCORRECT CORRECT……….1 INCORRECT……..2 Don’t know………3 Now I’m going to ask you some question that can be true or false 703 Contraceptive pill is effective even if a woman forgets to take it for two or three days True………..1 False………..2 Don’t know……..3 704 When a woman stops taking the contraceptive pill it is possible for her to get pregnant soon True………..1 False………..2 Don’t know……..3 706 The IUD can get stuck forever inside the woman True………..1 False………..2 Don’t know……..3 707 A woman who uses the Depo (injection) should take it every 3 months True………..1 False………..2 Don’t know……..3 708 A woman who uses the Depo (injection) will never be able to have children True………..1 False………..2 Don’t know……..3 709 Long term methods such as implant or IUD can be taken out at any time if a woman decides to get pregnant True………..1 False………..2 Don’t know……..3 710 If a woman wants to postpone a pregnancy without no one knowing, IUD, implant, Depo (injection) can be used. True………..1 False………..2 Don’t know……..3 112 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 711 A man can use the same condom more than once True………..1 False………..2 Don’t know……..3 712 Breast feeding is an effective method to postpone a pregnancy even if the menstrual period of a woman has started after a giving birth True………..1 False………..2 Don’t know……..3 713 Exclusive breast feeding means that a woman breast feed the baby day and night, all the time he asks for it and doesn’t give him any other type of food or water True………..1 False………..2 Don’t know……..3 714 Is it correct for a couple to wait 2 or more years between having a child and getting pregnant again to have another child? (This can be answered with YES, NO, Don’t know) Yes………1 No………..2 Don’t know………3 There are several factors that prevent a woman to go to a healthcare service. If she wants to go to a sanitary unit to get a family planning method what would be a big problem or not 716 To get authorization to go to a sanitary unit Big problem…………1 Not a problem……….2 Don’t know……………3 717 To get the money for the transport Big problem…………1 Not a problem……….2 Don’t know……………3 718 The distance to the sanitary unit Big problem…………1 Not a problem……….2 Don’t know……………3 719 Not wanting to go alone Big problem…………1 Not a problem……….2 Don’t know……………3 720 Afraid of getting to the sanitary unit and not find the method she desires Big problem…………1 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 113 Not a problem……….2 Don’t know……………3 721 Afraid of to be poorly treated by the health care provider when asking for a family planning method Big problem…………1 Not a problem……….2 Don’t know……………3 722 Afraid of being attended by a male health care provider Big problem…………1 Not a problem……….2 Don’t know……………3 723 Afraid of being seen in the sanitary unit for the family planning and be judged by the community Big problem…………1 Not a problem……….2 Don’t know……………3 Now I’m going to read some sentences and would like you to give me your sincere opinion if you agree with them or not. There is no right or wrong answers. We just want to have your opinion. 724 I feel confident with my ability to suggest to my husband for us to wait a few months before having another baby Agree……………1 Partially agree……..2 Totally disagree……….3 Partially disagree………4 Don’t Know…..5 725 I feel confident with my ability to suggest to my husband the family planning method we should use Agree……………1 Partially agree……..2 Totally disagree……….3 Partially disagree………4 Don’t Know…..5 726 I feel confident with my ability to convince my husband about the family planning method we should use Agree……………1 Partially agree……..2 Totally disagree……….3 Partially disagree………4 Don’t Know…..5 727 I feel confident with my ability to convince my husband to authorize me in the use of a family planning method 114 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Agree……………1 Partially agree……..2 Totally disagree……….3 Partially disagree………4 Don’t Know…..5 728 If I wanted to use family planning to avoid pregnancy, I feel confident with my ability to obtain the method I want to use Agree……………1 Partially agree……..2 Totally disagree……….3 Partially disagree………4 Don’t Know…..5 729 Would you be able to ask your husband/partner to use the condom if you like? Yes…………1 No………….2 It depends……..8 Module 8. KNOWLEDGE ABOUT THE PROJECT 801 In the last 12 months did you receive any health activist to talk to you about family planning? Yes………….1 No……………2 GO TO 804 802 If yes, who talked to you about family planning? Traditional birth attendant………1 APE………….2 Activist………..3 803 When was the last time you talked to an activist about family planning? Days………………..1 Weeks………………2 Months…………….3 Never went……..4 Don’t Know……..8 803A If Yes, the activist who talked to you about family planning did refer you to a sanitary unit/APE? Yes………….1 No……………2 804 Did you participate in a community event (community dialogs, lectures) in the last 12 months where the subject of family planning was talked about? Yes………….1 No……………2 805 Apart from the activist did you talk to someone else on how to avoid pregnancy? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 115 Yes………….1 No……………2 go to 805 806 If yes, to whom did you talk? Husband/partner………A Mother/mother in law…….B Daughter………….C Other family member………D Friend………..E Health care provider….F Other Specify G 807 In the last 6 months did any activist come to you to distribute the contraceptive pill in the community? Yes, but doesn’t use contraceptive pill….1 Uses but no one looked for her………2 There is no community distribution in that locality…..3 Doesn’t use and was not visited……..4 808 In the last 6 months did any activist/APE came to you to apply the DEPO in the community? Yes, but doesn’t use DEPO….1 Uses but no one looked for her………2 There is no community distribution in that locality…..3 Doesn’t use and was not visited……..4 809 Did you participate in a mobile brigade in the last 6 months? For example, if you have participated in an event where the health care providers go to the community and distribute family planning methods? Yes………….1 No……………2 810 Have you listened to any radio program which talked about family planning and/or the benefits of healthy spacing between pregnancies? Yes………….1 No……………2 KII GUIDE FOR USAID Instructions A. This is a semi-structured interview guide. The actual questions will be tailored based on the individual(s) interviewed. B. The questions cover the implementation of IFPP activity and are intended to collect data for performance evaluation of this activity. Respondent Background Thank you for agreeing to be interviewed. Please tell us about your involvement IFPP – current role/ position at USAID, the support you have provided the Activity, the length of time you have been working with the Activity/ Activity stakeholders? How would you assess IFPP´s overall performance to date? 116 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV How appropriate is the Activity design – in your view how suitable is the model for the local health systems context in Mozambique? Is it responsive to the unmet need for FP? What about alignment with local FP policy – what efforts have been made to ensure that the support being provided by the project is locally relevant and supports national frameworks? Thinking about other donor funded projects, and those funded by USAID, how has IPFF leveraged on existing Activities/ Programs that are currently working in this area (FP programming)? What factors have contributed to or hindered the implementation of IFPP, and how well has IFPP addressed implementation challenges, both external and internal? In your opinion, which are some noteworthy successes and challenges of IFPP? Please describe. Probe: a. Has there been changes in the community environment to support FP/RH services seeking behaviors as a result of IFPP intervention? If so, what are they? b. Do you know of any local innovations supported by IFPP which helped bolster FP/RH service delivery, demand creation and health systems quality? If yes, what are they? Thinking about key performance indicators of the Activity that relate to increased demand, increased access and improved FP commodity management – how would you describe the project’s performance overall? Probe: a. What are your views of the IPFF M&E system? Are there any aspects that could stand to be improved for better progress tracking? If so, please describe? Do you think that the IFPP interventions contributed to improved access to quality FP services at service delivery points? If so, how and to what extent? Probe: a. What are your views on the supply-side interventions to improve quality of FP services – has this been a major focus of implementation of the Activity? Please elaborate. a. What has worked well/ not so well for IPFF intervention to improve FP service quality? Why? b. In your opinion, what are the main barriers to reaching youth and adolescents with FP/RH information and services? Do you think IFPP interventions helped reaching out to adolescents with FP/RH information and services. If so, in what ways? How has IFPP interventions worked to improve availability of FP/RH commodities at service delivery points? OR: What is your understanding about the FP/RH commodities availability at service delivery points? What might be the reasons for this? Probe: a. To what extent have the forecast and quantities of FP commodities received met the needs at the service points covered by the Activity? b. Do you think there is a high frequency of stock-outs in FP commodity supply chain? If so, at what level? OR: do you know how the supply chain is functioning, and which might be the challenges and/or strengths? Do you have any other comment or suggestion on how the IFPP intervention could be improved? What, if any changes to the project design, would you recommend for the future? Finally, when thinking about sustainability, what has the Activity put in place to ensure that FP interventions continue post- the life of the project (i.e. exit strategy?) Probe: a. How has the Activity ensured that there is local ownership of interventions introduced under the Activity? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 117 b. What needs to be done/ could be done in the next few years to address this? 118 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV KII GUIDE FOR MINISTRY OF HEALTH (MOH)/ MISAU Instructions This is a semi-structured interview guide. The actual questions will be tailored based on the individual(s) interviewed. The questions cover the implementation of IFPP activity and are intended to collect data for performance evaluation of this activity Thank you for agreeing to be interviewed. Please tell us about your involvement IFPP – current role/ position at MISAU, how you work with the Activity, the length of time you have been working with the Activity/ Activity stakeholders? In your view, how appropriate is the Activity design – in your view how suitable is the model for the local health systems context in Mozambique? Is it responsive to the unmet need for FP? What about alignment with local FP policy – what efforts have been made to ensure that the support being provided by the project is locally relevant and supports national frameworks? Thinking about other donor-funded projects, and those funded by USAID, how has IPFF leveraged on existing Activities/ Programs that are currently working in this area (FP programming)? What factors have contributed to or hindered the implementation of IFPP, and how well has IFPP addressed implementation challenges, both external and internal? In your opinion, which are some noteworthy successes and challenges of IFPP? Please describe. Probe: c. Has there been changes in the community environment to support FP/RH services seeking behaviors as a result of IFPP intervention? If so, what are they? d. Do you know of any local innovations supported by IFPP which helped bolster FP/RH service delivery, demand creation and health systems quality? If yes, what are they? What do you think of IFPP’s interventions’ effect on increasing knowledge and awareness of modern contraceptives? How do you think the interventions increased demand, and if yes, how? Probe: a. Do you know of any changes any reported changes here at this level regarding the support FP/RH services as result of IFPP intervention? Do you think that the IFPP interventions contributed to improved access to quality FP at service delivery points? If so, how and to what extent? Probe: a. In your opinion, what are the main barriers to reaching youth and adolescents with FP/RH information and services? Do you think IFPP interventions helped reaching out to adolescents with FP/RH information and services. If so, in what ways? Do you think IFPP interventions improved FP/RH commodities availability at service delivery points? If so, how? OR: What is your understanding about the FP/RH commodities availability at service delivery points? What might be the reasons for this? Probe: a. To what extent have the forecast and quantities of commodities received met the needs at the service point? b. Do you think there is a high frequency of stock outs in FP commodity supply chain? If so, at what level? OR: do you know how the supply chain is functioning, and which might be the challenges and/or strengths? c. What do youth USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 119 On sustainability: Are the elements of this project will eventually gain local ownership and be merged into existing systems in the short and long term? 120 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV KII GUIDE FOR OTHER DONORS AND FP STAKEHOLDERS Instructions C. This is a semi-structured interview guide. The actual questions will be tailored based on the individual(s) interviewed. D. The questions cover the implementation of IFPP activity, and are intended to collect data for performance evaluation of this activity. Introduction of the interviewee/s Are you familiar with the IFPP intervention? How have you been collaborating with IFPP? If so, in which areas? How has IFPP supported local FP efforts? What are some aspects of IFPP’s intervention design that have helped increase knowledge and awareness of modern contraceptives in Mozambique? Probe: b. Has there been changes in the community environment to support FP/RH services seeking behaviors as a result of IFPP intervention? If so, what are they? c. Do you know of any local innovations supported by IFPP which helped bolster FP/RH service delivery, demand creation and health systems quality? If yes, what are they? Do you think that the IFPP interventions contributed to improved access to quality FP services at service delivery points? If so, how and to what extent? Probe: b. In your opinion, what are the main barriers to reaching youth and adolescents with FP/RH information and services? Do you think IFPP interventions helped reaching out to adolescents with FP/RH information and services. If so, in what ways? Do you think IFPP interventions improved FP/RH commodities availability at service delivery points? If so, how? OR: What is your understanding about the FP/RH commodities availability at service delivery points? What might be the reasons for this? Probe: d. To what extent have the forecast and quantities of commodities received met the needs at the service point? e. Do you think there is a high frequency of stock outs in FP commodity supply chain? If so, at what level? OR: do you know how the supply chain is functioning, and which might be the challenges and/or strengths? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 121 KII GUIDE FOR IFPP STAFF AT ALL LEVELS Introduction – Introduce the team members, purpose of the evaluation, confidentiality and right to refuse to answer questions or participate in the interview. Instructions A. This is a semi-structured interview guide. The actual questions will be tailored based on the individual (s) interviewed and their tasks and areas of intervention. B. The questions cover the implementation of IFPP activity and are intended to collect data for performance evaluation of this activity. Introduction of the interviewee/s - What are your tasks and your role in the IFPP? How would you assess IFPP´s overall progress to date? Is the IFPP design the right approach for supporting devolution? What, if any changes to the project design, would you recommend for the future? What factors have contributed to or hindered the implementation of IFPP, and how well has IFPP addressed implementation challenges, both external and internal? In your opinion, which are the main successes and challenges of IFPP? What are the reasons? Questions for each IP individually based on their specific tasks, detailed below, probing for effectiveness, successes and challenges. Pathfinder International Questions regarding the Agreement management, Finance and grants and general coordination of the project. Questions regarding the Contact point with USAID, GoM & partners, and technical leadership for: service provision at the HF and community level; collection of data and reporting; implementation science; M&E N’weti Questions regarding demand management including behavior change & creation of favorable atmosphere, as well as involvement of civil society. Population Services International (PSI) Questions regarding the involvement of private sector, references, and Platform mSaúde – Movercado (mHealth) Abt Associates Inc. Questions regarding strengthening of health systems in FP/RH at province & district level (selected areas) EQ1. What do you think of IFPP’s interventions’ effect on increasing knowledge and awareness of modern contraceptives? Do you think the interventions increased demand, and how? Probe: a. Has there been changes in the community environment to support FP/RH services seeking behaviors as a result of IFPP intervention? If so, what are they? b. Do you know of any local innovations supported by IFPP which helped bolster FP/RH service delivery, demand creation and health systems quality? If so, what are these? EQ2. Do you think that the IFPP interventions contributed to improved access to quality FP services at service delivery points? If so, how and to what extent? Probe: a. In your opinion, what are the main barriers to reaching youth and adolescents with FP/RH information and services? Do you think IFPP interventions helped reaching out to adolescents with FP/RH information and services? If so, in what ways? EQ3. Do you think IFPP interventions improved FP/RH commodities availability at service delivery points? If so, how? 122 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Probe: a. To what extent have the forecast and quantities of commodities received met the needs at the service point? b. Do you think there is a high frequency of stock outs in FP commodity supply chain? If so, at what level? Any other comment or suggestion to improve the IFPP implementation? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 123 KII GUIDE FOR HEALTH FACILITY MANAGERS, STAFF, APEs AND TBAs TRAINED BY IFPP Introduction – Introduce the team members, purpose of the evaluation, confidentiality and right to refuse to answer questions or participate in the interview. Instructions A. This is a semi-structured interview guide. The actual questions will be tailored based on the individual (s) interviewed. B. The questions cover the implementation of IFPP activity and are intended to collect data for performance evaluation of this activity. What are your tasks and role in IFPP? How long have you been working with tasks related to IFPP? 1. Have you received training from the IFPP project? If yes, when and what have been the topic of your training? 2. Have you been practicing the skills you have been trained on? Please give examples. 3. Have you been receiving supervisory/mentoring visit from IFPP? If yes, how often and what are the purposes of the visits? 4.Do you think the training you received helped to increase demand for and access to FP services in your health facility and the communities? If so, how? Please give examples. 5. (EQ3) Have you been experiencing contraceptive stock- outs at your service delivery point? If yes, how often? Has the situation changed over the last two years? If so, how? 6. Do you know of Women who had chosen to have an abortion? Do you know if these women received counseling and services to prevent unplanned pregnancies in the future? In your opinion, what are the main successes and challenges in your work / in IFPP? Do you have any recommendations on how the IFPP could be improved? 124 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV FOCUS GROUP DISCUSSION GUIDE FOR REPRODUCTIVE AGE WOMEN AND THEIR HUSBANDS/PARTNERS We are here to conduct a study to improve access for quality family planning services for women and their partners who use or would like to use a contraceptive method. This group discussion should not take more than 90 minutes and all answers will remain confidential. You may choose to not to answer any questions if you are not comfortable. We hope you will choose to answer as your responses will help to improve family planning knowledge and practices of women and men in Mozambique. May we begin? Start with the introducing yourself and the group. Name of the interviewer: Date: Province / District / Community: Number of individuals participated in this discussion: Starting with warming up questions about the participants and the community. ▪ Have you heard of modern contraceptive methods? If yes, where and when? Probe for condoms, pills, IUDs, implants, injectables, tubal ligation and vasectomy. Probe for where/when and how they got the information. Was it a long time ago or about two years back when the project interventions began? ▪ Do you think women and their partners in your community are using contraceptive methods mentioned above? Is there any difference compared with 2-3 year ago? If so, what is that? ▪ Has some of you or your partners using one of the contraceptive methods mentioned above? ▪ If yes, which one(s) and why? If not, what are the reasons for not using contraception? Probe for the acceptability of using a modern contraceptive method by the spouse/family/community environment. ▪ Have you/or your spouse ever tried to obtain a contraceptive method and were not able to access? If yes, which methods were not available and at which service delivery points and when. Do you think the situation might have improved over the last two years? Probe for different reasons for good/poor access. ▪ Whether you are a user of a contraceptive method or not, what is your opinion about the family planning services in your community? ▪ Do you have any comments to improve family planning services you’ve been receiving, and how? USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 125 FOCUS GROUP DISCUSSION GUIDE FOR ADOLESCENTS INCLUDING ORPHANS AND VULNERABLE CHILDREN (GIRLS AND BOYS) We are here to conduct a study to help increasing the knowledge of and use of family planning methods. This group discussion should not take more than 90 minutes and all answers will remain confidential. You may choose to not to answer any questions if you are not comfortable. We hope you will choose to answer as your responses will help to improve reproductive health in Mozambique. Start with introducing yourself and the group. Name of the interviewer: Date: Province / District / Community: Number of individuals participated in this discussion: Starting with some warming up questions about their background, studies (if in-school adolescents) and whether they are active in the school councils, use SMS Mcena etc… ▪ Have you heard of modern contraceptive methods? If yes, how and where? ▪ Are girls and boys in your locality using contraceptive method/s? If yes, what are the reasons? If no, why not? ▪ Has some of you used any contraceptive method/s? If yes, which one/s? Why did you use that specific method? Where did you receive the method? If no, why didn’t you use any contraceptives? ▪ In your opinion, are the families/communities support/not support adolescents using contraceptives? ▪ Do you know whether there are any contraceptive methods available in the local health clinic? If yes, which methods? Do you think access for contraceptives increased during the last 2-3 years? Probe for different reasons for good/poor access specifically for young people. ▪ If used some contraceptive method, what is your opinion about the quality/friendliness of service where you were attended? ▪ Do you have any other comment/opinion to improve contraceptive knowledge and use among adolescents? 126 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX E: WORK PLAN Table of Contents LIST OF ACRONYMS ............................................................................................................................. 127 INTRODUCTION....................................................................................................... 128 OVERVIEW OF THE HEALTH SYSTEM............................................................................................11 ACTIVITY DESCRIPTION....................................................................................... 128 EVALUATION AUDIENCE AND INTENDED USE .................................................................. 131 EVALUATION PURPOSE, OBJECTIVES AND QUESTIONS................................................. 131 EVALUATION OBJECTIVES ....................................................................................................... 132 EVALUATION QUESTIONS....................................................................................................... 132 EVALUATION METHODOLOGY .................................................................................................... 132 EVALUATION DESIGN......................................................................................................................... 132 QUALITATIVE METHODS .......................................................................................................... 133 QUANTITATIVE METHODS...................................................................................................... 135 ANALYTICAL FRAMEWORK-GETTING TO ANSWERS............................................. 136 LIMITATIONS.................................................................................................................................... 136 GENDER CONSIDERATIONS ........................................................................................................... 136 ETHICAL CONSIDERATIONS AND CONFIDENTIALITY................................................... 136 TEAM COMPOSITION, RESPONSIBILITIES AND ROLES..................................................... 136 FIELD VISIT SCHEDULES...................................................................................................................... 137 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS TABLE ................................ 137 EVALUATION TIMELINE............................................................................................................. 137 EVALUATION REPORT OUTLINE ......................................................................................... 137 DISSEMINATION OF EVALUATION FINDINGS ............................................................. 137 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 127 List of Acronyms APE Agente Polivalente Elementar – Ministry of Health Approved Community Health Worker CBO Community Based Organization CDCS Country Development Coordination Strategy CMAM Central de Medicamentos e Artigos Médicos (National Drugs, Commodities and Supplies Warehouse) CPR Contraceptive Prevalence Rate CYP Couple Year Protected DFID United Kingdom Department of International Development DPM Provincial Medical Store DPS Direcção Provincial de Saúde – Provincial Health Directorate EA Enumeration Area FGD Focus Group Discussion FP Family Planning FP/RH Family Planning/Reproductive Health FTP First-Time Parent GRM Government of the Republic of Mozambique HH Household IFPP Integrated Family Planning Program IMASIDA Inquérito de Indicadores de Imunização, Malária e HIV/SIDA em Moçambique - National Malaria and HIV Indicator Survey INE Instituto Nacional de Estatística – National Institute of Statistics KAP Knowledge, Attitude and Practices KII Key Informant Interview LARC Long-Acting Reversible Contraceptive mCPR Modern Contraceptive Prevalence Rate M&E Monitoring and Evaluation MISAU Ministry of Health NGO Non-government Organization OVC Orphans and vulnerable children PAC Post-abortion care PSI Population Services International RH Reproductive Health SBCC Social and Behavioral Change Communication SDSMAS District and City health Directorate SI Strategic Information SOW Scope of Work UNDP United Nations Development Program USG United States Government USAID United States Agency for International Development UNFPA United Nations Fund for Population Activities WB World Bank 128 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Introduction Mozambique is a low-income country located in south-east Africa. It is ranked 181 of 188 countries on the human development index (UNDP, 2016) and has experienced rapid population growth in recent years, with the total population increasing from 16.1 million in 1997 to an estimated 27.1 million in 2017 (INE, 2017). Overall, Mozambique has improved its reproductive health indicators in recent years. From 2011 to 2015, the total fertility rate decreased from 5.9 to 5.3 births per woman, the modern contraceptive prevalence rate (mCPR) among women in union increased from 11.3% to 25.3%, and unmet need for family planning (FP) decreased from 29% to 23% (IDS 2011, IMASIDA 2015). Antenatal care during pregnancy is high (92.6% of women attend at least one visit), and 70.3% of women deliver in a health facility (IMASIDA 2015). However, this overall progress masks substantial regional variations in reproductive health indicators. Nampula and Sofala provinces have modern CPRs of only 21.8% and 14.4% among women in union, respectively (IMASIDA 2015), and remain in need of program interventions to increase access to reproductive health services. Overview of the Health System Mozambique is moving towards a more decentralized health system. The Ministry of Health (MISAU) is organized into three administrative levels: the central agencies, the District/provincial level (DPS), and the District and City health Directorate (SDSMAS). Service provision in Mozambique is dominated by the public sector through the National Health System (NHS). 95% of health care is provided through NHS government services while only 60% of the population can reach these services. Public sector facility service provision is organized into four levels of care86. It is important to note that FP services are integrated, such that FP services are offered at health facilities alongside reproductive health services (i.e. Antenatal care, maternal and neonatal child health). These services are widely offered across levels 1 to 4. Front-line service providers tend to be nurses based in maternal and child health units/ departments at facility-level. The range of FP services provided at health facilities includes counseling and access to both short and long -acting reversible contraceptives (LARC). At the community level, service provision is provided through the mobile brigades, the APEs and/or NGO supported Community Health Worker (locally known as “activistas”). Only 16% of users access methods through the private sector through private pharmacies and grocery stores. The for-profit private sector is largely confined to major cities. Activity description The Integrated Family Planning Program (IFPP) is a $ 34,560,000 five-year initiative (2016-2021) funded by the United States Agency for International Development to increase use of modern contraceptive methods by target populations in Nampula and Sofala provinces. IFPP is being implemented in all 36 districts of Nampula87 and Sofala88 provinces. The project is led by Pathfinder International with a team of global and local partners—N’weti, Population Services International (PSI), and Abt Associates. The consortium members bring in specific and complementary skillsets to achieve the goals of the IFPP. Pathfinder International leads the team with its experience in family planning service delivery. IFPP will build on N’Weti’s rich local expertise to increase demand among the project’s five target populations, engage male partners, address inequitable gender norms that impede contraceptive use, and cultivate an enabling environment. To foster sustainable impact and reinforce community and facility activities, the project will apply Abt Associates’s health systems strengthening experience to strengthen priority FP/RH provincial and district systems. PSI’s role is expanding access to private sector services. 86 Level 1: Level 2: Level 3: and Level 4: 87 Nampula: Angoche, Eráti, Ilha de Moçambique, Lalaua, Larde, Liúpo, Malema, Meconta, Mecubúri, Memba, Mogincual, Mogovolas, Moma, Monapo, Mossuril, Muecate, Murrupula, Nacala, NacalaVelha, Nacarôa, Nampula Rapale, Ribáuè 88 Sofala: Nhamatanda, Muanza, Marínguè, Sofala, Marromeu, Machanga, Gorongosa, Beira, Búzi, Caia, Chemba Cheringoma, Chibabava, Dondo USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 129 IFPP responds to the United States Government (USG) strategy for development and foreign assistance in Mozambique through the Country Development Coordination Strategy (CDCS). The USAID/Mozambique CDCS outlines an overarching development objective health goal to “Improve the Health Status of Target Populations” through three results: 1) Increased coverage of high impact health and nutrition services, 2) Increased adoption of positive health and nutrition behaviors, and 3) Strengthened systems to deliver health, nutrition, and social services (CDCS, 2013). In alignment with the CDCS, this activity aims to enhance the Government of the Republic of Mozambique (GRM) and Ministry of Health (MISAU) priorities as it relates to FP. The activity’s mandate is to increase the use of modern contraceptive methods by target populations to achieve the following: • Increased access to a wide range of modern contraceptive methods and quality FP/RH services; • Increased demand for modern contraceptive methods and quality FP/RH services; and • Strengthened FP/RH health systems. By the end of the period of performance, IFPP aims to reach an additional 565,000 new contraceptive users (“initiators”89) and spur a 15-percentage point increase in modern contraceptive prevalence (mCPR) as well as a more diverse and effective method mix in the intervention areas. IFPP’s results framework is shown in Figure 1. 89 Initiators – defined as the clients which began using a contraceptive at the service delivery point. 130 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Figure: IFPP Results Framework IFPP Theory of Change The IFPP Activity design provides a mix of both demand-side and supply-side interventions. On the demand-side, interventions are geared towards improving uptake of FP services among the following five target categories of beneficiaries: post-partum women; women living with HIV; adolescents including orphans and vulnerable children (OVC); medium -and high-parity women; young couples; and, post￾abortion women. On the supply-side, the model of service delivery has focused on maximizing reach by focusing on the following service delivery points: health facilities, Agente Polivalente Elementar (APEs), mobile brigades, FP commodity warehouses, schools, among others. Furthermore, the model of FP services under IFPP delivers three “packages of services”, each unique to the district category (urban, peri-urban and rural) to which they were assigned, as depicted in Figure 2. In doing so, key outcomes anticipated at the end of the project performance period includes, positive changes in the perceived quality of FP services – (IR 2); increased uptake of FP services (“initiators”) – (IR 1) and marked improvements in availability of modern contraceptives at service delivery points – (IR 3). USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 131 Figure: Packages of Family Planning services offered by IFPP Evaluation Audience and Intended Use The USAID/Mozambique Health office will use the results of this evaluation to inform continued implementation of the Activity. The audience for this evaluation has been placed into the following categories: External audience: The Government of the Republic of Mozambique; Ministry of Health (MISAU) & Procurement Supply Management (PSM) Internal USAID audience: Front Office, Health Office, DO4 Team Members, Strategic Information (SI) Office, and IFPP Other donors: United Kingdom (UK) Department of International Development (DFID); the United Nations Fund for Population Activities (UNFPA) and the World Bank The external audience will use the evaluation to inform health policy to support FP service delivery and strategies to improve access to modern contraceptives. The internal audience, in addition to using evaluation results to improve IFPP implementation, this audience will also use findings, conclusions and recommendations to build greater synergy and improve coordination across USAID FP/RH programs. Similarly, other donors stand to learn about the efficacy of the approach used by IFPP to improve FP/RH programming in Mozambique. Evaluation Purpose, Objectives and Questions The purpose of the IFPP Mid-Term Performance Evaluation is to assess the effectiveness of the IFPP’s approach to increasing the utilization of family planning and reproductive health services, as well as its efforts at strengthening FP/RH health systems. RURAL PACKAGE: tailored for rural districts, which have limited access to any service delivery entry point; low demand for, and significant sociocultural barriers to, contraception; relatively low uptake; and high ideal number of children. This package combines significant community-based service delivery with intensive SBCC interventions at individual and community levels to generate demand. This package focuses on postpartum women, adolescents and youth first-time parents (FTP), and medium- and high- parity women. COMBO-PACKAGE: This is geared towards districts with high internal variation in access, demand, and contraceptive uptake due to the presence of corridor areas with increasing economic growth and mobile coverage, the combo package combines elements of the rural and the urban packages. This package will focus on postpartum women, adolescents and youth (particularly FTPs), and medium- and high-parity women, as well as reaching post abortion care (PAC) clients through service delivery. URBAN PACKAGE: This is for districts with high FP service demand (including those with a high unmet need for FP services) and physical access to services (but where services are of inadequate quality and the method mix is limited), high mobile and media coverage, a higher concentration of adolescents, secondary schools, and higher rates of sexual and gender-based violence (SGBV) and HIV. The urban package uses existing networks and platforms to increase linkages between priority populations contraceptive and FP/RH services. This package focuses on postpartum women, HIV- positive women, and adolescents, and will reach PAC clients through health facilities (HFs). 132 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV EVALUATION OBJECTIVES • To examine how IFPP has increased demand for modern contraceptives methods of FP • To assess how IFPP interventions have increased access and improved quality of FP service delivery; and • To determine how commodity management had improved following IFPP support in select service delivery points. The evaluation scope of work is included in Annex A. EVALUATION QUESTIONS There are three key thematic areas guiding this evaluation: (i) Increase demand for modern contraceptive use; (ii) Improve access to quality FP services; and (iii) Improving commodity availability & management. Evaluation Question 1. To what extent has IFPP increased demand for modern contraceptives methods of family planning and reproductive health? This question seeks to understand if the knowledge and awareness of modern contraceptive methods has increased over the performance period of the IFPP. Simi larly, evaluators will look to assess if any changes in demand and use of modern contraceptive services in catchment areas surrounding health facilities, including other types of service delivery points, can be linked to the Activity intervention. The team will note other actors that may have influenced changes in knowledge, attitude and practices as it relates to family planning. Evaluation Question 2. To what degree has IFPP interventions increased access and quality of FP/RH services? The evaluation will identify efforts made by the activity to improve access to FP services. Issues related to access will be captured by self-reporting drawn from community member experiences, including households that fall within health facility catchment areas where the interventions have been rolled out. More specifically, the evaluation will document perspectives of target populations (i.e. adolescents, women, and men) to note similarities and differences in experiences when seeking FP services. The evaluation will also seek to document perspectives on successes and challenges in providing quality services from health providers and IFPP staff. Where possible, evaluators will inquire about the efficacy of the activity intervention to improve quality of service provision with both IFPP staff and health service providers. Evaluation Question 3. To what degree has improved commodities management increased the availability of FP/RH commodities at service delivery points? The evaluation will examine how forecasting with respect for FP commodities occurs at both national and sub-national level. Experiences will be captured form both health service providers and supply chain actors to better understanding the process involved, as well as, any bottlenecks to ensuring availability of FP commodities at service delivery points. The team will inquire about availability of the full range of FP commodities supported by the Activity, as well as, the frequency of stock-outs and what measures, if any, providers take when facing this challenge. Evaluation Methodology The evaluation will be conducted between February-April 2019 and will cover the period from activity inception in June 2016 through December 2018. In-country evaluation activities will be conducted between February 25-April 5, 2019. A detailed evaluation timeline, including the timing of key activities and tasks, and due dates of deliverables is presented in Annex G. Evaluation Design The evaluation methodology will employ a mixed method approach combining both qualitative and quantitative approaches. Given the nature of IFPP’s multiple interventions both methods will yield rich USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 133 information that will provide the evidence base for the team to formulate conclusions and recommendations to inform FP programming for the remainder of the Activity period. Subsequent sections will present in detail the qualitative and quantitative methods that will be used in this evaluation. Qualitative Methods This section summarizes the sampling approach, data sources and analysis methods for the qualitative approach. Sampling Approach for Qualitative Methods District Selection The evaluation will select three districts in Nampula and two districts in Sofala, based on the population sizes of the two provinces. In total, five districts will be selected. The following two criteria will be used for the selection of the districts included in the qualitative data collection: District typology: Since IFPP categorized its intervention districts according to the “package of services” to be delivered to the urban, semi urban and rural districts rural, the evaluation’s sample selection will follow suit. A representative sample will be drawn from districts of all three typologies. Intervention duration: IFPP implementation has begun in mid-2016, thus the interventions have been ongoing for about two years. The team will select a mix of districts that represent varying stages of intervention roll out to allow the evaluators to compare implementation experience. Health Facility Selection The evaluation team will aim to select two health facilities per district. Evaluators will collect data within the periphery of targeted health facilities, spanning a catchment area ranging from 2 to 12 kilometers 90, depending on whether the facility is based in an urban, peri-urban or rural area. Health facility selection will also reflect the various levels of the public health system (Level 1 to Level 4). Health facilities will be grouped into two categories as follows: i) Facilities which received assistance within the first year of Activity implementation; and ii) facilities that received intervention within the last six months. Selection of end-beneficiaries As mentioned in earlier sections, IFPP specifically targets six distinct categories of beneficiaries: Postpartum women; women living with HIV; adolescents including orphans and vulnerable children (OVC); medium- and high-parity women; young couples; and, post-abortion women. This evaluation will conduct Focus Group Discussions (FGDs) at the community level with a select group of beneficiaries – this includes postpartum women; men; adolescents; and high-parity women. In addition, evaluators will hold smaller groups consisting of 2 to 3 HIV positive women. The evaluation will not include post￾abortion women. This is discussed under limitations section. Inclusion criteria to guide selection of target beneficiaries is presented below: ▪ Post-partum and high parity women, aged 15-49 in union who are currently using a modern contraceptive method or with an unmet need for FP; ▪ Women living with HIV, aged 15-49 in union who are currently using a modern contraceptive method or with an unmet need for FP; ▪ Husbands/partners of women aged 15-49 in union who are currently using a modern contraceptive method or with an unmet need for FP; ▪ Adolescent girls aged 10-19 90 134 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ▪ Adolescents boys aged 10-19 Beneficiary experiences accessing and using modern contraceptive methods. In addition, evaluators will seek to capture their perceptions of the quality of FP services provided in service delivery points supported by IFPP. Figure 3 summarizes the target sample for qualitative methods: Figure: Health Facilities & FGDs for inclusion in Nampula and Sofala Province District # of Health Facility/Catchment Area FGD per catchment area Nampula Urban 2 4 Peri urban 2 4 Rural 2 4 Sofala Peri urban 2 4 Rural 2 4 Data collection This evaluation will document qualitative data through use of the following techniques: document review, key informant interviews (KII), FGDs, and facility observations. 1. Document review: This method relies on gathering pertinent information from secondary sources include IFPP records, reporting documents and other useful technical references. Key indicators of interest presented in the Activity MEL plan will be reviewed and achievements noted, where possible. The core team has started this process, having reviewed key IFPP documents with plans to expand the list to include additional documents. The list of documents is in Annex B. 2. Key Informant Interviews (KIIs): KIIs will provide insights into the effectiveness of IFPP approaches, successes, and gaps, with related interventions. Semi-structured interviews with a wide range of stakeholders and beneficiaries will be conducted at national and subnational level. In Maputo, evaluators will seek to hold meetings with USAID, IFPP and MOH staff and staff of other family planning program stakeholders. At the provincial and district levels, IFPP, MOH staff and representatives of the private sector will be among key informants. At community level, health providers attached to select facilities will be interviewed along with community leaders (i.e. elected members of the community) who serve as community-point representatives attached to local health facilities. A list of key informants to be interviewed in Maputo is found in Annex C. This list will be expanded to include other informants at sub-national level. 3. Focus Group Discussions (FGDs): The field survey team will conduct FGDs with a range of end￾beneficiaries to gain an in-depth understanding of their experiences. Their perspectives on accessibility, availability and overall experience using modern contraceptives will be discussed. 4. Facility Observations: The team will visit select health facilities to assess general presence of a trained health provider who has received IFPP training. In addition, the team will make observations on whether the provider has the requisite tools or supports to provide quality FP services. Qualitative Data Analysis USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 135 The evaluation will use a combination of thematic and pattern analysis in synthesis of qualitative data. Themes and patterns across data sets will be important to answer to the specific evaluation questions will be noted, and evaluators will triangulate results drawn from KIIs, FGDs, and observations. Quantitative Methods To ensure that findings drawn from the qualitative component can be corroborated and to assess any changes in awareness and use of modern contraceptives, a Knowledge Attitude and Practices (KAP) survey will be conducted as part of this evaluation. This methodology will target districts and Enumeration Areas (EAs) previously targeted by KAP survey that took place at Activity baseline in 2017. In doing so, the evaluation will be able to note any differences, positive or negative, in target communities as it relates to perceptions, attitudes, and use FP services and modern contraceptives. Sampling Approach KAP survey will use a pre- and post-test approach focusing only on areas that have received IFPP intervention (treatment areas). This will allow the evaluators to examine if there are any changes (improvements or regress in KAPs among survey respondents) that can be attributed to the intervention. Target population This survey will seek to recruit households located within catchment areas surrounding service delivery points where the intervention was introduced and implemented. The profile (demographic characteristics) of individuals targeted by the survey will be women of reproductive age, between ages 15 to 49 years. This is in line with the demographic characteristics of IFPP beneficiaries who participated in the baseline survey in 2017. Sample design Probability sampling will be used to inform the KAP survey sampling in Nampula and Sofala. The sampling approach is multi-stage (three tiered) to ensure that every household (HH) has an equal chance of being selected: ▪ Step I: An Enumerator Area (EA) is randomly drawn from the sampling frame using probability proportional to size method (PPS). ▪ Step II: within an EA, HHs are randomly selected. ▪ Step III: a respondent in each sampled HH is randomly selected by using Kish grid. In the case of this survey, woman at reproductive age constitute the target population within a HH. Sampling frame As mentioned earlier in this section, the KAP survey will adopt the same sampling approach used in the baseline study. KAP’s survey target population will target households (HHs) within EAs in Nampula and Sofala. It is important to note that residents of collective housing estates and the homeless will be excluded from the sample. Household Selection This KAP survey will look to sample a total of 816 households will be selected in 34 enumeration areas (of which 10 are urban) in Nampula. In Sofala, the sample will target 600 households selected in 30 enumeration areas (of which 12 are urban). In each HH, only one woman will be selected for participation. In households where there are more than one woman who meets the eligibility criteria for participation, a Kish grid will be used to select the survey respondent. 136 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Analytical Framework-Getting to Answers As mentioned above, during the planning stage, the evaluation team developed a “Getting to Answers" worksheet for identifying the most appropriate designs and methods for answering evaluation questions. The tool maps out the methods to be used, including data collection tools, data sources, and data analysis techniques for each evaluation question. Then the team determined how these different methods and sets of data will be woven together into a coherent mixed method design that meets all the requirements of the evaluation. The analytical framework is found in Annex F. Limitations As is the case with any evaluation, there are known limitations. For qualitative methods, we anticipate the following: ▪ Selection bias: The evaluation team will request names of organizations/ institutions and point persons that have been in contact with the Activity to act as a first step in snow ball sampling. This method, when used in combination with purposive sampling, will enable the team to make independent decisions about who to target for inclusion for KIIs. Similarly, the team will request lists of health facilities reached by IFPP to determine sample selection of the final health facilities. ▪ Recall biases: the evaluation team will seek to speak to interview (through survey and qualitative methods) enough respondents who are knowledgeable in the subject matter. In addition, the evaluators will look to synthesize information noting, where possible, the varying levels of exposure across different respondent groups. In doing so, the similarities in experience, and any marked differences will be noted and considered in the analysis process. The evaluation will ensure that saturation is achieved for qualitative data to address this. ▪ Absence of post abortion women as a beneficiary category targeted in FGDs: The evaluators opted to exclude this group due to the socio-cultural sensitivities related to having an abortion. Rather, this evaluation will seek to capture the experiences of this group by talking to other IFPP staff involved at community level, community members (women) and health providers, and triangulate data to better understand what their FP needs. If needed, the team will also seek to review documents as part of the desk review. Gender Considerations The evaluation team will ensure that the evaluation approach incorporates gender analysis throughout all levels of the evaluation and analyze variables that might place certain individuals or populations at a disadvantage. Specific attention will be given to gender considerations while collecting and analyzing data. Where possible, sex-disaggregated data should be evaluated for outputs and outcomes to determine the degree to which the project affected both women and men. Specifically, the team will make sure that gender is incorporated in (1) evaluation design – both women and men will be interviewed and participate in KIIs and FGDs; (2) the data collection tools – which include specific questions on equal access to both sexes; and (3) data tabulation – analysis of results, and the conclusions and recommendations. Since the primary target for the KAP survey is female, all survey interviewers will be women. A male interviewer will be employed to conduct FGDs with men. Ethical Considerations and Confidentiality The team will obtain verbal consent from all participants of the KIIs and FGD participants according to USAID Evaluation Policy guidelines and in-line with standard ethical practice around voluntary nature of participation. As such, participants will be given the option to opt out of answering specific questions, as well as the whole interview, should they choose to. The consent form will ensure that any information gathered through interviews or discussions will not be linked to any specific person in the reports or raw data sets shared with USAID to maintain anonymity. Verbal consent will be requested of KAP survey respondents prior to interviews. Team Composition, Responsibilities and Roles The evaluation team will be comprised of short-term consultants and fieldwork staff and MMEMS full￾time staff. MMEMS will also provide the required support staff and transport for all parties. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 137 MMEMS/Maputo MEL Specialist (Zenobia Machanguana) will provide project oversight throughout the evaluation stages. She will be responsible for the overall client management. In addition, Quality Management/ DCOP & Collaboration Advisor (Eduarda Cipriano) will provide quality management and assurance. MMEMS COP, (Luís Reves) and the project Technical Director (Mark Renzi), will provide overall technical oversight. The core team includes the Team Leader (Pinar Senlet), an International FP/RH Consultant, and Health Sector Specialist (Ritva Parviainen). The Team Lead will provide technical leadership for the evaluation, including the design and implementation of the methodologies to be applied at field level, overseeing data quality gathered at field level, providing analysis and leading report writing. She has the primary responsibility for the overall completion of activities and deliverables and guide the efforts of the evaluation team. The Health Sector Specialist will play a key role in carrying out the evaluation and ensuring it is implemented according to the evaluation design and in accordance with evaluation quality standards. In addition, the Health Sector Specialist will support the design and implementation of the methodologies, collection and analysis of field data, quality assurance, and drafting of the evaluation report. She will report to the evaluation Team Leader and work collaboratively with the evaluation team. play a key role in carrying out the evaluation. The evaluation team will be supported by a statistician Carlos Lauchande, whose role is to develop the protocol for the KAP survey and conduct statistical analysis of the survey. Two field assistants and a translator will provide field support during data collection of qualitative data. A subcontractor will be recruited for quantitative data collection and management. A team of two experienced field supervisors will supervise quantitative data collection conducted by field enumerators. The supervisors will ensure quality data collection, with oversight from the Team Lead and the MMEMS Intervention Supervisor. Field Visit Schedules To collect qualitative data, the core team will first conduct KIIs in Maputo. By the end of the Maputo interviews, the core team will have aligned their data gathering approaches, and will travel to Nampula and Sofala with the interviewers and translators between March 11-23. It is expected that the team will spend one week in each province. The team’s field trip schedules are depicted in Annex H. The KAP survey will be conducted separately by a contractor and a team of supervisors and enumerators. MMEMS has put in quality control measures to ensure data reliability and quality. See Annex D the KAP Survey protocol for more details. Findings, Conclusions and Recommendations Table Upon returning from the field, the core team will integrate all data into a table summarizing what was learned in the effort. This will be a table that includes all relevant conclusions, detailing the findings on which those conclusions were based, and any recommendations offered by the team. The recommendations will be clearly linked to specific conclusions. This will provide the basis for the evaluation report. Evaluation Timeline The evaluation will be conducted between February-April 2019 and will cover the period from activity inception in June 2016 through December 2018. In-country evaluation activities will be conducted between February 25-April 5, 2019. A detailed evaluation timeline, including the timing of key activities and tasks, and due dates of deliverables is presented in Annex G. Evaluation Report Outline The Evaluation Final Report will follow USAID's Criteria to Ensure the Quality of the Evaluation Report (Found in USAID Evaluation Policy) and the guidance provided in the evaluation SOW. A draft outline of the report is in Annex I. Dissemination of Evaluation Findings Upon the finalization and approval of the evaluation report, a dissemination plan will be developed to share the results strategically. The dissemination activities will focus primarily on USAID and IFPP, beginning with a briefing on the results of the evaluation before submission of the draft report. 138 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Dissemination may expand to include a broader array of interested parties, e.g., the MOH, key partners and other development actors working in family planning and reproductive health programs in Mozambique. In consultation with USAID, we will first identify the groups and/or individuals who are likely to be affected by or interested in the evaluation results. Next, appropriate communication tools will be identified. Tools may include sharing of reports, briefs, press releases. They may also include efforts including presentations, meetings, or facilitated workshops. The selection of communication channels will depend on the most effective ways to reach the identified stakeholders. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 139 ANNEX F: KAP SURVEY METHODOLOGY KAP Survey It was the aim of the to measure changes in knowledge, attitudes, and practices (KAP) of different types of contraceptives. At baseline, a survey was carried out to capture KAPs among populations within catchment areas where the IFPP intervention was expected to be implemented. The specific evaluation question which is related to this objective is EQ. 1: To what extent has IFPP increased demand for modern contraceptives methods of family planning? The KAP survey compared baseline results with existing views of respondents drawn from the same catchment areas targeted at baseline, to assess any changes or variations in overall respondent awareness and views of modern contraceptive methods. More specifically, survey data was a compliment and corroborated qualitative insights on how IFPP contributed to any notable changes. This section describes the survey methodology, including the sample design, data collection, data quality control and data analysis. Survey Methodology Survey design As stated in the previous section, this KAP survey used a pre- and post-test approach focusing only on areas that have received IFPP intervention (treatment areas). This will allow the evaluators to examine if there were any changes (improvements or regress in KAPs among survey respondents) that can be attributed to the intervention. Target population The target population are the households located within catchment areas surrounding service delivery points where the intervention was introduced and implemented. The profile (demographic characteristics) of individuals targeted by the survey will be women of reproductive age, between ages 15 to 49 years. This is in line with the demographic characteristics of IFPP beneficiaries who participated in the baseline survey in 2017. Sample design Probability sampling was used to inform the KAP survey sampling in Nampula. The sampling approach is multi-stage (three tiered) to ensure that every HH has an equal chance of being selected: Step I: a primary sampling unit (PSU) /Enumerator Area (EA) is randomly drawn from the sampling frame using probability proportional to size method (PPS). Step II: within an EA, households are randomly selected. Step III: a respondent in each sampled household is randomly selected by using Kish grid. In the case of this survey, woman at reproductive age constitute the target population within a HH. Sampling frame As mentioned earlier in this section, the KAP survey adopted the same sampling approach used in the baseline study. KAP’s survey targeted population within EAs in Nampula . It is important to note that residents of collective housing estates and the homeless were excluded from the sample. Table 1 below presents the sample frame for this study, with population target distributed by province and location (rural/ urban). HH is the PSU. Table 1 sampling frame Total Urban Rural Province N of HH PSU PSU PSU 140 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Nampula 794431 21.9% 240 182556 22.98% 108 611875 77.02% 132 Sofala 275792 7.6% 140 111249 40.34% 80 164543 59.66% 60 Total 1070223 27.45 380 293805 27.45 188 776418 72.55 192 Source: Baseline sample design; PSU-Primary sample unity Sample size The 2007 census data were used to estimate the proportion of the population of target reproductive age women and the average household size. According to the baseline report, a sample of 816 households and 34 enumeration areas was assumed to serve as a representative sample based in the following assumptions: providing indicators with acceptable level of accuracy with coefficient of variation below 17% and a confidence interval of 95%, for each representative domain (the province); Sample size was estimated to detect a 10% change. In addition, it assumed a non-response rate of 10%, with a design effect of 2 and power of 0.8. Like baseline, this KAP survey looked to sample a total of 816 households in selected in 34 enumeration areas (of which 10 are urban) in Nampula. In each HH, only one woman will be selected for participation. In households where more than one woman who meets the eligibility criteria for participation, a Kish grid was used to select the respondent. Table 2 provides the sample distribution Table 2 Sample distribution for the districts covered by the baseline study. Districts Administrative posts Enumeration Areas (EA) HHs per (EA) Total of HHs Nampula Angoche 3 3 24 72 Cidade Nampula 4 4 24 96 Iha de Moçambique 1 1 24 24 Meconta 2 2 24 48 Memba 1 1 24 24 Mogovolas 3 3 24 72 Mogincual 1 1 24 24 Mecuburi 1 1 24 24 Monapo 4 4 24 96 Mossuril 1 1 24 24 Murrupula 1 1 24 24 Nacala porto 1 2 24 48 Nacala Velha 2 2 24 48 Nacarroa 2 2 24 48 Namapa-Erati 3 3 24 72 Nampula 3 3 24 72 subt-total 33 34 816 Data Collection First, a list of HHs was developed for each EA. The list was used to select randomly households to be interviewed. Only HHs with at least one female member at reproductive age (female 15-49) were eligible for inclusion in the survey. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 141 Systematic random sampling (SRS) was used to select households in each EA. Within the HHs a kish table was used for selection HH woman at reprodutive age. Adititional 4 HHs were included in the list in the event that a HH needs to be substituted (replaced). Field interviewers’ responses were captured electronically on password-protected Android tablets that were pre-programmed with the questionnaire using Kobo Toolbox. The electronic data capture tool mirrored paper questionnaire and present one question per screen. Instructions were included in the tool to guide interviewers to facilitate interview flow. Field Management Pre-field work – Training Prior to the training, 39 data collectors and 7 supervisors were selected for training and recruitment. The recruitment of the data collectors and supervisors was done based on the data collectors' performance during the training, pre-test exercise and team performance evaluation. Table 4 shows the training configuration. Table 4: Training Configuration Days Type of training 1st to 3th day Class room training 1 day general field work interviewing techniques 2 days of reading and comprehension of questionnaire 4th to 5th day Pre-test Pre-test discussion These sessions aimed at training the supervisors and data collectors on the use of data collection procedures, with focus on the KAP questionnaire, use of tablets, interview exercises and procedures, sampling and household selection criteria and ethics issues. A separate training session was conducted for the field supervisors only at the end of each day of the data collectors' training, aiming to train them on their roles, responsibilities and on all the materials and knowledge they needed to perform their task as supervisors. After the training, the team conducted pre-tests, in the EA not included in the sample, with the objective to test the face validity of the questions, as well as the efficiency of the survey methodology and strategy. The pre-tests was conducted by trainee supervisors and data collectors, under the direct supervision of company staff. The pre-test exercise will be followed by pre-test discussion. Data Quality Control Pre-programmed tablets included a skip logic built-in and error messages and caution notices were triggered when faulty data entered to alert interviewers to correct problems. In addition, respondents were interviewed in a private location out of earshot of others including children and other family members. The survey team’s data base manager run daily checks based on a predesigned data cleaning script that included checks for: structure, uniqueness, and external consistency of key identifiers; completeness of data; acceptable data; and unexpected data. An error report was generated from the database and shared with the field team daily. Immediate action/correction (e.g. re-interview, re-visit to households for confirmation, etc.) was taken by the field teams, in consultation with the evaluation team lead, to ensure high quality data were collected. At the end of data collection data consistency and missing data analysis was carried out to ensure reasonable standards of data quality. For data cleaning, frequency distributions of variables was analysed to assess the problems of outliers and values out of the expected range. 142 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Data Analysis The analysis of KAPs survey used descriptive statistics, estimation of Cohen's d and the statistical tests to determine whether the indicator’s differences between the baseline and the Midline evaluation are statistically significant. Descriptive statistics includes the means/percentage for 2017 and 2019, their standard errors, and the mean differences between 2017 and 2019 to evidence of difference. The standard error of the mean difference was computed with the following formula: SE=(SE2 17+SE2 19) 1/2, assuming independence of the variables between 2017 and 2019. The statistical significance of p<0.05 was considered if the difference taken in absolute terms was more than twice its standard error (SE). Finally, Chi-square tests and T-test was used to test differences between sub-groups. For 2x2 tables, p-values from Fisher’s Exact Test (2- sidd) was used. Indicators were broken down by sub groups such as: age, marital status, religion, location (urban-rural) and socioeconomic status (SES). SES was based in estimation of wealth index, approach used when HHs expenditure and consummation data is not available Deaton (1997) Wealth Index or possession Index is a composite measure of the cumulative living standard of a household. The wealth index is calculated using data on a household’s ownership of selected assets, such as televisions and bicycles, crops, livestock, radio materials used for housing construction, and types of water access and sanitation facilities etc. Generated with a statistical procedure known as Principal Components Analysis (PCA)91, the Wealth Index places individual households on a continuous scale of relative wealth. The break points that define wealth quintiles were estimated, as: q1-Lowest, q2-Second, q3-Middle, q4-Fourth, and q5-Highest. Which, tentatively, could be labelled as “very poor” “poor”. “moderate living conditions” “well off”, “very well off”. The available data could not differentiate between q1 and q2. They are presented as one group, representing around 40% of the sample. 91 Principal component analysis (PCA) involves a mathematical procedure that transforms a number of possibly correlated variables into a smaller number of uncorrelated variables called principal components. The first principal component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible. 91 Many indicator variables are categorizations. To determine the weights and apply them to form the index, it is necessary to break these variables into sets of dichotomous variables (dummy variables). Filmer and Pritchett (1998) recommended using principal components analysis (PCA) to assign the indicator weights, the procedure that is used for the DHS wealth index. DHS uses the SPSS factor analysis procedure. This procedure first standardizes the indicator variables (calculating zscores); then the factor coefficient scores (factor loadings) are calculated; and finally, for each household, the indicator values are multiplied by the loadings and summed to produce the household’s index value. In this process, only the first of the factors produced is used to represent the wealth index. The resulting sum is itself a standardized score with a mean of zero and a standard deviation of one USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 143 ANNEX G: SOURCES OF INFORMATION IFPP USAID Documents PD-AID-656-A-16-00005 Pathfinder May 13, Agreement June 07, 2016 Key IFPP Documents IFPP Performance Indicators Matrix IFPP October-November-December 2018 Q1F3 IFPP Y2 Annual Report IFPP April-May-June 2018 report Q3FY2 IFPP January-February-March 2018 report Q2FY2 IFPP October-November-December 2017 report revised 02 09 18 IFPP Y1 Annual Report Annex II IFPP workplan April-May-June Q3FY2 MEL Plan for IFPP – Integrated Family Planning Project IFPP Baseline Survey Report August 17, 2017, final, and related document Pesquisa Formativa IFPP Final, 06 12 17 ▪ Áreas de enumeração ▪ Desenho amostra Planeamento Familiar 2016 ▪ Questionário Agregado Familiar ▪ List of IFPP districts from the IFPP Baseline Survey ▪ Questionário Mulheres ▪ Moz IFPP baseline datasets ▪ US Posto Administrativo Sofala ▪ Postos Administrativos e US Nampula Supplemental IFPP Documents Análise de risco dos contraceptivos, Ministério da Saúde, Grupo de trabalho, fevereiro 2019 Balanço PF 2018, Ministério da Saúde Calendário de distribuição de medicamentos Fornecimento de contraceptivos aos DDM pelo DPM – ano 2018 IFPP Calculations November Sofala and Nampula List of IFFP districts from the IFPP baseline survey 144 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Mozambique Insights report, Learnings Document: Increasing the demand for family planning in rural Mozambique, November 2018 (draft, not for circulation) Posto Administrativo Sofala 27.2.2019 Postos Administrativos e US em Nampula Vale a pena – INCEPTION PHASE DELIVERABLE, National & Provincial Contextual Assessment (draft, not for circulation) US Apoiadas pelo IFPP e sua Classificação Nampula 1.3.19 US Apoiadas e Posto Administrativo Sofala 1.3.19 National Documents Family planning and contraception strategy 2010–2015 (2020) – Maputo, August 2010, Ministry of Health IMASIDA 201 - Inquérito de Indicadores de Imunização, Malária e HIV/SIDA em Moçambique (National Malaria and HIV Indicator Survey) INE 2017 Census- Instituto Nacional de Estatística IDS 2011- Inquérito Demográfico e de Saúde, (Demographic and Health Survey) Pacote de Intervenções no Âmbito de Saúde Escolar, Ministério da Saúde, Gabinete da Ministra, 4 de Março de 2019 Other Documents USAID Family Planning Sustainability check-list – a Project Assessment Tool for Designing and Monitoring Sustainability of Community-based Family Planning Service USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 145 ANNEX H: LIST OF INTERVIEWS Table: Full Data Collection Summary Category # of Individual Interviews # of Group Interviews Female Interviewees (%) Total number of Interviewees USAID 4 - 50% 4 MOH (National-Level) 1 1 100% 3 MOH (Province-Level) 2 - - 2 MOH (District-Level) 4 7 47% 19 IFPP Consortium Partners 5 1 50% 8 Other Donors 2 1 75% 4 Community Leaders 1 2 - 5 Others International Programs 3 1 100% 5 TOTAL 22 13 26 (52%) 50 146 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Table: List of Key Informant Interviewees Category Region Name Role Date Individual or Group Interview Transcription (Y/N) USAID Maputo Division Chief FOR Maternal and Child Health, Family Planning, Malaria and Nutrition 03/06/19 Individual Y USAID Maputo MCH/FP Project Management Specialist 03/06/19 Individual Y USAID Maputo MCH/FP Team Lead 03/06/19 Individual Y USAID Maputo M&E for Strategic Information 03/20/19 Individual N IFPP Consortium Partner Maputo COP of Integrated Family Planning Program (IFPP), Pathfinder 03/05/19 Group interview N IFPP Consortium Partner Maputo Program Director, Pathfinder 03/05/19 Group interview N IFPP Consortium Partner Maputo Senior M&E Officer, Pathfinder 03/05/19 Group interview N IFPP Consortium Partner Maputo Director, PSI 03/06/19 Individual Y IFPP Consortium Partner Nampula, Nacala Porto Supervisor, PSI 03/15/19 Individual Y IFPP Consortium Partner Maputo Manager of the Community Component of IFPP Program, N’weti 03/08/19 Individual Y USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 147 Category Region Name Role Date Individual or Group Interview Transcription (Y/N) IFPP Consortium Partner Maputo Health Systems Strengthening, Senior Technical Advisor, Abt Associates 03/05/19 Individual Y IFPP Consortium Partner Nampula city Adviser at the Provincial Health Directorate, Abt Associates 03/11/19 Individual Y Other Donors Maputo National Program Officer for Sexual and Reproductive Health, UNFPA 03/08/19 Individual Y Other Donors Maputo Specialist in Healthcare and Regional Tuberculosis Program, World Bank 03/08/19 Individual Y Other Donors Maputo Policy Officer for Sexual and Reproductive Health & Rights / HIV/AIDS, Dutch Embassy 03/08/19 Group interview Y Other Donors Maputo First Secretary SRHR/HIV, Dutch Embassy 03/08/19 Group interview Y MOH Maputo National Director of Public Health 03/07/19 Group interview Y MOH Maputo Director of the Family Planning Program 03/07/19 Group interview Y MOH Maputo Head of the Department of 03/07/19 Individual Y 148 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Category Region Name Role Date Individual or Group Interview Transcription (Y/N) Maternal and Child Health MOH Nampula city Provincial Chief Medical Officer 03/15/19 Individual Y MOH Nampula city Manager of Province Medical Warehouse in Nampula 03/11/19 Individual Y MOH Mecuburi district District Chief Medical Officer at the District Health Directorate 03/12/19 Group interview Y MOH Mecuburi district Manager of District Medical Warehouse at Mecuburi Sede 03/12/19 Group interview Y MOH Mecuburi district Director of the Namina Health Center 03/12/19 Group interview Y MOH Mecuburi district MCH Nurse, Focal Point for FP Services at Namina Health Center 03/12/19 Group interview Y MOH Mecuburi district Health Post Nurse, Unidade Moçambicana 03/12/19 Individual Y MOH Monapo district Director of District Health Directorate 03/13/19 Group interview Y MOH Monapo district Preventive Medicine Technician (TMP) 03/13/19 Group interview Y MOH Monapo district MCH/FP Head Nurse of FP at Monapo Sede Health Center 03/13/19 Group interview Y USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 149 Category Region Name Role Date Individual or Group Interview Transcription (Y/N) MOH Monapo district Acting Director of Monapo Sede Health Center 03/13/19 Group interview Y MOH Monapo district Lead Health Technician at Metuchiria Health Center 03/14/19 Group interview Y MOH Monapo district MCH Nurse at Metuchiria Health Center 03/14/19 Group interview Y MOH Monapo district Pharmacist at Metuchiria Health Center 03/14/19 Group interview Y MOH Monapo district Warehouse Manager at the District Medical Warehouse in Monapo Sede 03/14/19 Group interview Y MOH Monapo district Warehouse Clerk at the District Medical Warehouse in Monapo Sede 03/14/19 Group interview Y MOH Nacala Porto district Director of District Health Directorate 03/15/19 Individual Y MOH Nacala Porto district Warehouse Manager, Nacala District Medical Warehouse 03/15/19 Individual Y MOH Nacala Porto district Director of the Health Center Nacala Porto Urbano 03/15/19 Group interview Y MOH Nacala Porto district Head MCH Nurse of FP Services, Health 03/15/19 Group interview Y 150 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Category Region Name Role Date Individual or Group Interview Transcription (Y/N) Center Nacala Porto Urbano MOH Nacala Porto district Lead MCH Nurse of FP Services at Nacala District Hospital 03/16/19 Individual N Other Nampula City Program Director, International Pregnancy Advisory Services (IPAS) 03/11/19 Individual Y Other Nampula city Clinical Director, International Center for AIDS Care and Treatment Program (ICAP) 03/11/19 Group interview Y Other Nampula city Provincial Adviser for Cacum, ICAP 03/11/19 Group interview Y Other Nampula city Provincial Representative, GHSC-PSM Procurement System Management 03/11/19 Individual Y Other Monapo district Head of Broadcasting; Manager of FP Program at Monapo Community Radio 03/13/19 Individual Y Community Leaders Mecuburi district Head of the Administrative Post in Namina, Mecuburi district 03/12/19 Group interview Y USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 151 Category Region Name Role Date Individual or Group Interview Transcription (Y/N) Community Leaders Mecuburi district Community Leader in Namina, Mecuburi district 03/12/19 Group interview Y Community Leaders Monapo district Community Leader at Monapo Sede, Bairro de Micolene 03/13/19 Group interview Y Community Leaders Monapo district Secretary, Monapo Sede 03/13/19 Group interview Y Community Leaders Monapo district President of Assenemo CBO, Monapo Sede 03/13/19 Individual Y 152 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX I: COMPLETE FOCUS GROUP LIST Table: List of Focus Group Interviewees Type of Focus Group District Locality # of Male Participants # of Female Participants Date Transcriptio n (Y/N) Female adults Mecuburi Namina Health Center catchment area - 12 03/12/19 Y Male adults Mecuburi Namina Health Center catchment area 7 - 03/12/19 Y Female adults Mecuburi Unidade Moçambicana catchment area - 11 03/12/19 Y Male adults Mecuburi Unidade Moçambicana catchment area 12 - 03/12/19 Y Female adolescents Mecuburi Unidade Moçambicana catchment area - 3 03/12/19 N Male adolescents Mecuburi Unidade Moçambicana catchment area 3 - 03/12/19 Y Female adults Monapo Monapo Sede Health Center catchment area - 7 03/13/19 Y Male adults Monapo Monapo Sede Health Center catchment area 5 - 03/13/19 Y Female adolescents Monapo Monapo Sede Health Center catchment area - 5 03/13/19 Y Male adolescents Monapo Monapo Sede Health Center catchment area 4 - 03/13/19 Y Female adults Monapo Metuchiria Health Center catchment area - 9 03/14/19 Y Male adults Monapo Metuchiria Health Center catchment area 19 - 03/14/19 Y USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 153 Type of Focus Group District Locality # of Male Participants # of Female Participants Date Transcriptio n (Y/N) Female adolescents Monapo Metuchiria Health Center catchment area - 15 03/14/19 N Male adolescents Monapo Metuchiria Health Center catchment area 10 - 03/14/19 Y Female adults Nacala Porto Urbano Health Center catchment area - 10 03/15/19 Y Male adults Nacala Porto Urbano Health Center catchment area 4 - 03/15/19 Y Female adolescents Nacala Porto Urbano Health Center catchment area - 3 03/16/19 Y Male adolescents Nacala Porto Urbano Health Center catchment area 8 - 03/16/19 Y Female adults Nacala Porto Nacala Porto District Hospital catchment area - 15 03/16/19 N Male adults Nacala Porto Nacala Porto District Hospital catchment area 8 - 03/16/19 Y Female adolescents Nacala Porto Nacala Porto District Hospital catchment area - 10 03/16/19 Y Male adolescents Nacala Porto Nacala Porto District Hospital catchment area 5 - 03/16/19 Y TOTAL 85 (55 adults and 30 adolescents) 100 (64 adults and 36 adolescents) 154 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX J: QUANTITATIVE DATA RESULTS Table E1.1: Key IFPP indicators Key IFPP Indicators Province 2017 2019 Trend ES % % Contraception and unmet need Percent of women aged 15-49 who have ever used a modern contraceptive method Nampula 42.3 39.8 Percent of women of reproductive age (15-49) in target districts who are currently using a modern contraceptive method Nampula 20.2 17.0 Percent of women in union of reproductive age (15-49) in target districts who are currently using a modern contraceptive method Nampula 19.0 16.3 Percent of women in union aged 15–49 with an unmet need for family planning Nampula 22.7 38.4 Percent of demand satisfied for modern contraception among all sexually active women aged 15-49 Nampula 47.4 32.1 Antenatal, Delivery and Postpartum Care Percent of women aged 15-49 with a birth in the last 3 years who had 4 or more ANC visits Nampula 48.1 61.1 0.31 Percent of deliveries with a skilled birth attendant (SBA) in USG-assisted programs Nampula 71.9 78.7 0.21 Percent of women aged 15–49 who delivered in a facility in the past 3 years, who received a contraceptive method upon discharge from the facility Nampula 38.3 27.2 0.24 Percent of women aged 15–49 with a birth in the past 3 years who had a postpartum visit Nampula 60.4 65.9 Percent of women aged 15–49 who had a postpartum visit in the past 3 years and received a contraceptive method during postpartum visit Nampula 37.1 17.8 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 155 Table 2.1 (a) Percent distribution of household heads and number of usual residents. NAMPULA 2017 2019 Trend ES Sex of household head Male 80.7 78.4 Female 19.3 21.6 Mean # of usual household members 1 1.6 2.0 2 10.4 14.7 0.09 3 18.3 21.7 4 20.6 24.3 5 20.6 18.3 6+ 30.9 19.1 0.24 Mean size of households 4.7 4.2 Number of households 802 816 Table 2.2 (a) Housing characteristics. NAMPULA 2017 2019 Trend ES Electricity Yes 21.7 25.0 No 78.3 75.0 Floor Natural 80.4 82.5 Other 19.6 17.5 Roof Natural 72.2 72.7 Other 27.8 27.3 Walls Natural 19.6 26.3 0.14 Rudimentary 28.4 44.6 0.35 Finished 46.8 28.8 0.39 Other 5.3 0.2 0.10 Mean number of rooms for sleeping 2.1 2.1 Mean number of people per room for sleeping 2.4 2.2 Drinking water source Improved source 62.1 69.1 0.19 Piped water into dwelling/yard/ plot 5.7 17.0 0.23 Public tap/ standpipe 41.2 28.9 0.26 Tube well/borehole 7.5 8.3 Protected dug well 7.7 14.7 0.14 Bottled water/ tanker 0.0 0.1 Non-improved source 37.9 30.9 0.15 Unprotected dug well 33.3 23.0 0.21 Unprotected spring 4.5 0.0 0.09 Surface water 0.1 7.8 0.15 Type of toilet/latrine Improved facility 31.3 31.6 Toilet 1.3 0.9 Improved pit latrine 18.0 14.2 0.08 156 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Traditional improved latrine 12.1 16.5 0.09 Non-improved facility 68.7 68.4 Traditional latrine 45.4 45.8 No facility/bush/field 23.3 22.5 Number of households 802 816 Table 2.3 (a) Ownership of household goods. NAMPULA 2017 2019 Trend ES Radio 38.4 22.7 0.33 Television 20.4 19.9 Mobile phone 50.1 31.4 0.41 Refrigerator 6.3 5.8 Candles/lantern 64.6 61.2 Car/Truck 1.2 1.2 Boat 0.4 0.5 Animal Cart 0.1 0.4 Watch 13.0 11.0 Bicycle 31.2 19.2 0.25 Motorcycle/Scooter 18.0 10.7 0.15 Number of households 802 816 Table 2.4 Acceptance of family planning by household heads, by province NAMPULA 2017 2019 Trend ES In favor of using FP to avoid pregnancy 53.3 65.2 0.30 Spoke to a health provider about FP in the past 6 months 2.9 44.6 0.87 Discussed FP with someone else in past 6 months 16.0 33.0 0.35 Partner 5.0 10.0 0.10 Family member 11.6 13.0 Friend/Neighbor 5.4 16.2 0.22 Other 0.4 13.1 0.25 Received info about FP in health consultation 29.9 Number of household heads 802 816 Table 2.5 (a) Percent distribution of women aged 15-49 by background characteristics, Nampula NAMPULA Weighted Unweighted 2017 2019 Trend ES 2017 2019 Trend ES Place of Residence Rural 67.6 67.2 70.6 Urban 32.4 32.8 29.4 Age 15-19 15.3 14.8 24.1 0.19 20-24 21.4 21.5 22.3 25-29 15.9 17.6 20.1 30-34 12.6 12.0 14.3 35-39 15.7 15.0 10.0 0.10 40-44 9.9 10.2 5.5 0.09 45-49 9.2 9.0 3.6 0.11 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 157 Marital status Never married 6.2 6.73 8.2 Married 65.3 65.3 37.3 0.66 Living together 15.8 16.5 41.5 0.52 Divorced/Separated 9.6 8.4 10.8 Widowed 3.1 3.1 2.2 Education Level No education 44.8 47.3 40.9 0.14 Primary 41.8 40.7 45.0 Secondary or higher 12.3 11.4 13.5 Other 1.1 0.8 0.6 Religion Catholicism 53.4 53.0 52.5 Protestantism 1.6 2.0 6.7 0.09 Islam 41.7 42.1 36.3 0.13 Zionism 0.3 0.1 0.1 No religion 1.6 1.6 1.8 Other 1.3 1.1 2.6 0.03 Nationality Mozambican 99.8 99.6 99.9 South African 0.1 0.1 Malawian 0.1 0.3 Congolese - 0.1 Mother tongue Portuguese 1.8 1.9 2.1 Emakhuwa 95.4 95.4 96.2 Xichangana 0.0 0.0 0.0 Elomwe 0.4 0.5 0.4 Cisena 0.0 0.0 0.0 Echuwabo 0.2 0.1 0.4 Shona 0.6 0.3 0.0 Other 1.5 1.9 1.0 Number of women 15-49 802 802 816 Table 2.6 (a) Knowledge of contraception among all women aged 15-49 NAMPULA 2017 2019 Trend ES Knowledge of any method 80.1 92.6 0.51 Knowledge of any modern method 79.6 92.0 0.49 Female sterilization 6.3 15.2 0.18 Male sterilization 1.8 9.1 0.15 Pill 63.2 78.1 0.42 IUD 23.3 34.2 0.23 Injectables 66.2 84.3 0.56 Implants 34.9 58.6 0.54 Male condom 48.2 62.1 0.33 Female condom 28.7 31.5 LAM 6.1 7.0 Emergency contraception 0.1 5.9 0.12 Knowledge of any traditional method 4.9 18.5 Periodic abstinence 1.1 10.7 0.19 Withdrawal 1.7 13.6 0.24 158 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Other 2.2 2.2 0.00 Number of women 15-49 802 816 Table 2.7 (a) Ever use of contraception among women aged 15-49, NAMPULA 2017 2019 Trend ES Ever use of any method 42.6 43.4 Ever use of any modern method 42.3 39.8 Female sterilization 0.7 0.7 Male sterilization 0.0 0.4 Pill 19.2 13.8 0.11 IUD 2.7 1.1 0.03 Injectables 28.3 20.6 0.16 Implants 5.6 4.4 Male condom 7.4 16.3 0.18 Female condom 4.8 1.5 0.07 LAM 0.7 0.5 Emergency contraception 0.0 1.6 0.03 Ever use of any traditional method 1.1 11.8 0.21 Periodic abstinence 0.0 4.0 0.08 Withdrawal 0.3 9.3 0.18 Other 0.8 1.0 Number of women 15-49 802 816 Table 2.8 (a) Current use of contraception among women aged 15-49 (all women in union) NAMPULA 2017 2019 Trend ES Current use of any method (women in union) 20.6 16.5 Current use of any modern method (women in union) 19.0 16.3 Female sterilization 0.2 0.8 Pill 3.9 0.0 0.08 IUD 0.9 4.4 0.07 Injectables 11.6 0.2 0.23 Implants 2.1 9.8 0.15 Male condoms 0.7 1.7 Female condoms 0.0 1.7 0.03 LAM 0.9 0.0 0.02 Current use of any traditional method (women in union) 0.7 0.0 0.01 Periodic abstinence 0.1 0.2 Withdrawal 0.0 0.0 Other 0.6 0.0 0.01 Number of women in union 15-49 656 643 Current use of any method (all women) 20.7 17.2 Current use of any modern method (all women) 20.2 17.0 Female sterilization 0.3 0.6 Male sterilization 0.0 0.0 Pill 3.9 4.5 IUD 0.7 0.1 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 159 Injectables 12.7 9.9 Implants 2.0 2.1 Male condoms 1.1 1.2 Female condoms 0.0 0.1 LAM 0.7 0.0 0.01 Current use of any traditional method (all women) 0.5 0.1 Periodic abstinence 0.1 0.1 Withdrawal 0.1 0.0 Other 0.2 0.0 Number of women 15-49 802 816 160 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Table 2.9.2 (a): Current contraceptive use by women in union aged 15-49 in Nampula, by background characteristics, baseline Table 2.9.2 (b): Current contraceptive use by women in union aged 15-49 in Nampula, by background characteristics, midline Background characteristic Any Method Modern Method Traditional Method Other Not using a method Number of women in union Any Modern Method Fem Sterili￾zation Pill IUD Inject￾ables Implant Male Condom LAM Any Tradi￾tional Method Periodic Absti￾nence With￾drawal Age 15–19 9.1 9.1 0.0 1.3 0.0 6.1 0.0 1.7 0.0 0.0 0.0 0.0 0.0 90.9 103 20–24 17.8 17.8 0.0 4.3 0.0 10.7 2.4 0.0 0.8 0.0 0.0 0.0 0.0 82.2 149 25–29 24.4 24.4 0.0 8.0 0.0 14.5 1.1 0.5 3.0 1.7 0.0 0.0 1.7 75.6 115 30–34 32.7 31.5 0.0 5.7 1.4 16.3 5.3 2.8 0.0 1.2 1.2 0.0 0.0 67.3 79 35–39 22.9 22.9 1.5 3.1 1.6 13.1 4.8 0.0 1.0 0.0 0.0 0.0 0.0 77.1 95 40–44 18.5 18.5 0.0 0.5 4.9 17.1 0.0 0.0 0.9 0.0 0.0 0.0 0.0 81.5 67 45–49 8.9 4.7 0.0 3.0 0.0 1.7 0.0 0.0 0.0 0.0 0.0 0.0 4.1 91.1 48 Education Level No education 15.6 15.3 0.5 4.0 0.4 10.7 0.9 0.0 0.0 0.3 0.3 0.0 0.0 84.4 309 Primary 19.1 18.3 0.0 3.7 0.5 8.7 2.6 1.2 1.7 0.6 0.0 0.0 1.4 80.9 281 Secondary or higher 42.0 42.0 0.0 5.5 5.2 30.4 6.0 2.0 1.9 0.0 0.0 0.0 0.0 58.0 60 Other 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 6 Total 19.5 19.0 0.2 3.9 0.9 11.6 2.1 0.7 0.9 0.4 0.1 0.0 0.6 80.5 656 Background characteristic Any Method Modern Method Traditional Method Other Not using a method Number of women in union Any Modern Method Fem Sterili￾zation Pill IUD Inject￾ables Implant Male Condom LAM Any Tradi￾tional Method Periodic Absti￾nence With￾drawal Age 15–19 18.3 18.3 0.0 5.6 0.0 11.3 2.1 0.7 0.0 0.0 0.0 0.0 0.0 81.7 142 20–24 12.4 12.4 0.0 3.1 0.6 8.1 1.2 1.2 0.0 0.0 0.0 0.0 0.0 87.6 161 25–29 18.8 18.8 1.6 2.3 0.0 12.5 0.8 3.1 0.0 0.0 0.0 0.0 0.0 81.3 128 30–34 21.5 21.5 1.1 5.4 0.0 12.9 4.3 1.1 0.0 0.0 0.0 0.0 0.0 78.5 93 35–39 12.7 12.7 0.0 6.3 0.0 6.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 87.3 63 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 161 Table 2.10.1 (a): Current contraceptive use by all women aged 15–49 in Nampula, by background characteristics, baseline Background characteristic Any Method Modern Method Traditional Method Other Not using a method Number of women Any Modern Method Fem Sterili￾zation Pill IUD Inject￾ables Implant Male Condom LAM Any Traditional Method Periodic Absti￾nence With￾drawal Age 15–19 12.5 12.5 0.0 2.0 0.0 9.1 0.0 1.4 0.0 0.0 0.0 0.0 0.0 87.5 119 20–24 20.2 20.2 0.0 4.9 0.0 12.4 2.0 1.3 0.7 0.0 0.0 0.0 0.0 79.8 172 25–29 25.1 25.1 0.0 8.5 0.0 15.1 0.9 0.4 2.5 1.4 0.0 0.0 1.4 74.9 141 30–34 37.6 36.6 0.0 4.7 1.2 21.7 4.4 4.7 0.0 1.9 0.9 1.0 0.0 62.4 96 35–39 19.7 19.7 1.6 2.4 1.2 11.2 4.3 0.0 0.7 0.0 0.0 0.0 0.0 80.3 120 40–44 17.9 17.9 0.0 0.4 4.2 16.7 0.0 0.0 0.8 0.0 0.0 0.0 0.0 82.1 82 45–49 8.7 5.8 0.0 2.1 0.0 1.2 2.6 0.0 0.0 0.0 0.0 0.0 2.9 91.3 72 Education Level No education 15.0 14.7 0.4 3.6 0.3 10.2 1.2 0.0 0.0 0.3 0.3 0.0 0.0 85.0 379 Primary 19.7 19.1 0.0 4.1 0.4 10.0 2.2 1.3 1.4 0.5 0.0 0.0 1.2 80.3 326 Secondary or higher 46.1 46.1 0.7 4.8 3.4 32.1 4.6 5.1 1.2 1.0 0.0 1.0 0.0 53.9 91 Other 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 6 Total 20.6 20.2 0.3 3.9 0.7 12.7 2.0 1.1 0.7 0.5 0.1 0.1 0.5 79.4 802 Table 2.10.2 (b): Current contraceptive use by all women aged 15–49 in Nampula, by background characteristics, midline Background characteristic Any Method Modern Method Traditional Method Other Not using a method Number of women Any Modern Method Fem Sterili￾zation Pill IUD Inject￾ables Implant Male Condom LAM Any Traditional Method Periodic Absti￾nence With￾drawal Age 15–19 17.8 17.8 0.0 5.1 0.0 9.6 2.5 1.0 0.0 0.0 0.0 0.0 0.0 82.2 197 20–24 14.3 14.3 0.0 3.8 0.5 8.8 2.2 1.1 0.0 0.0 0.0 0.0 0.0 85.7 182 40–44 17.1 14.3 5.7 5.7 0.0 5.7 2.9 0.0 0.0 2.9 2.2 0.0 0.0 82.9 35 45–49 9.5 9.5 0.0 4.8 0.0 0.0 0.0 4.8 0.0 0.0 0.0 0.0 0.0 90.5 21 Education Level No education 15.1 15.1 0.4 3.4 0.0 9.1 1.5 1.1 0.0 0.0 0.0 0.0 0.0 84.9 265 Primary 16.0 15.7 1.0 4.7 0.3 10.3 1.3 1.3 0.0 0.3 0.3 0.0 0.0 84.0 300 Secondary or higher 23.7 23.7 1.3 6.6 0.0 10.5 3.9 2.6 0.0 0.0 0.0 0.0 0.0 76.3 76 Other 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.0 2 Total 16.5 16.3 0.8 4.4 0.2 9.8 1.7 1.4 0.0 0.2 0.1 0.0 0.0 83.5 643 162 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 25–29 19.5 19.5 1.2 3.0 0.0 12.8 1.2 2.4 0.0 0.0 0.0 0.0 0.0 80.5 164 30–34 20.5 20.5 0.9 5.1 0.0 12.8 3.4 0.9 0.0 0.0 0.0 0.0 0.0 79.5 117 35–39 15.9 15.9 0.0 7.3 0.0 7.3 1.2 0.0 0.0 0.0 0.0 0.0 0.0 84.1 82 40–44 17.8 15.6 4.4 4.4 0.0 8.9 2.2 0.0 0.0 2.2 2.2 0.0 0.0 82.2 45 45–49 6.9 6.9 0.0 3.4 0.0 0.0 0.0 3.4 0.0 0.0 0.0 0.0 0.0 93.1 29 Education Level No education 15.0 15.0 0.3 3.6 0.0 8.4 1.8 0.9 0.0 0.0 0.0 0.0 0.0 85.0 334 Primary 15.5 15.3 0.8 4.6 0.3 10.1 1.1 1.1 0.0 0.3 0.3 0.0 0.0 84.5 367 Secondary or higher 27.3 27.3 0.9 6.4 0.0 14.5 4.5 2.7 0.0 0.0 0.0 0.0 0.0 72.7 110 Other 60.0 60.0 0.0 20.0 0.0 0.0 40.0 0.0 0.0 0.0 0.0 0.0 0.0 40.0 5 Total 17.2 17.0 0.6 4.5 0.1 9.9 2.1 1.2 0.0 0.1 0.1 0.0 0.0 82.8 816 Table 2.11: Most recent source of method among modern contraception users aged 15-49, by province Baseline Midline Pill Injectable Male Condom Total Pill Injectable Male Condom Total Nampula Public Sector 96.1 100.0 43.1 95.3 86.1 95.1 80.0 92.1 Private Sector 0.0 0.0 38.0 2.5 8.3 1.2 0.0 2.9 Other source 0.0 0.0 8.1 0.5 5.6 3.7 10.0 4.3 Missing 3.9 0.0 10.7 1.6 0.0 0.0 10.0 0.7 Total 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 163 Table 2.12 (a) Contraceptive initiation and FP counseling received among women aged 15-49 who ever used a contraceptive method, Nampula NAMPULA 2017 2019 Trend ES Mean age at first use of contraception 23.5 22.8 Initiated contraception since 2015 32.7 70.7 0.90 Informed by health provider about side effects or complications of method 40.0 76.3 0.91 Informed about what to do if side effects or complications occur 39.5 71.4 0.78 Number of women who ever used contraception 328 308 Table 2.13 (a) Desire for children among women in union aged 15-49, by province NAMPULA 2017 2019 Trend ES Want another soon (within 2 years) 34.7 18.4 0.34 Want another later (wait 2+ years) 21.1 30.0 0.18 Undecided* 3.3 6.2 0.06 Want no more children 17.2 20.1 Sterilized/declared infecund 8.5 14.5 0.12 Pregnant 13.1 7.0 0.12 Missing 2.0 3.9 0.04 Number of women in union 15-49 656 643 *Undecided/don’t know about timing of next birth or whether they want more children 164 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Table 2.14.2 (a): Need for family planning among women in union aged 15–49 in Nampula, by background characteristics, baseline Background characteristic Unmet need for family planning Met need for family planning Demand for family planning Number of women in union Age For spacing For limiting Total For spacing For limiting Total For spacing For limiting Total % of demand satisfied 15–19 18.5 0.0 18.5 4.6 4.5 9.1 23.2 4.5 27.6 33.0 103 20–24 17.6 5.4 23.0 8.0 9.8 17.8 25.6 15.2 40.9 43.7 149 25–29 22.3 4.4 26.7 10.7 13.7 24.4 33.0 18.0 51.0 47.7 115 30–34 3.4 7.7 11.2 11.6 21.1 32.7 15.0 28.8 43.8 74.5 79 35–39 14.1 18.8 32.9 3.4 19.5 22.9 17.4 38.3 55.8 41.0 95 40–44 4.9 10.4 15.3 6.3 12.2 18.5 11.3 22.6 33.9 54.7 67 45–49 4.2 27.6 31.7 7.1 1.7 8.9 11.3 29.3 40.6 21.8 48 Education Level No education 14.6 8.9 23.5 3.9 11.7 15.6 18.5 20.6 39.1 40.0 309 Primary 13.1 10.4 23.4 7.1 12.0 19.1 20.2 22.3 42.5 44.9 281 Secondary or higher 15.3 1.8 17.2 26.3 15.7 42.0 41.6 17.6 59.2 71.0 60 Other 0.0 15.6 15.6 0.0 0.0 0.0 0.0 15.6 15.6 0.0 6 Total 13.8 8.9 22.7 7.4 12.1 19.5 21.3 21.0 42.2 46.2 656 Table 2.14.2 (b): Need for family planning among women in union aged 15–49 in Nampula, by background characteristics, midline Background characteristic Unmet need for family planning Met need for family planning Demand for family planning Number of women in union Age For spacing For limiting Total For spacing For limiting Total For spacing For limiting Total % of demand satisfied 15–19 28.9 5.6 34.5 16.9 2.1 19.0 45.8 7.7 53.5 35.5 142 20–24 29.2 8.1 37.3 12.4 0.6 13.0 41.6 8.7 50.3 25.9 161 25–29 30.5 10.2 40.6 14.1 5.5 19.5 44.5 15.6 60.2 32.5 128 30–34 19.4 18.3 37.6 11.8 9.7 21.5 31.2 28.0 59.1 36.4 93 35–39 20.6 20.6 41.3 4.8 7.9 12.7 25.4 28.6 54.0 23.5 63 40–44 8.6 42.9 51.4 0.0 17.1 17.1 8.6 60.0 68.6 25.0 35 45–49 0.0 33.3 33.3 0.0 14.3 14.3 0.0 47.6 47.6 30.0 21 Education Level USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 165 No education 23.4 14.7 38.1 10.6 5.7 16.2 34.0 20.4 54.3 29.9 265 Primary 27.7 12.7 40.3 11.3 4.7 16.0 39.0 17.3 56.3 28.4 300 Secondary or higher 19.7 11.8 31.6 18.4 6.6 25.0 38.2 18.4 56.6 44.2 76 Other 50.0 0.0 50.0 0.0 0.0 0.0 50.0 0.0 50.0 0.0 2 Total 25.0 13.4 38.4 11.8 5.3 17.1 36.9 18.7 55.5 30.8 643 Table 2.15.2 (a): Demand for modern contraception among all sexually active women aged 15–49 in Nampula, by background characteristics, baseline Background characteristic Unmet need for modern FP Met need for modern FP Demand for modern FP Number of women in union Age For spacing For limiting Total For spacing For limiting Total For spacing For limiting Total % of demand satisfied 15–19 18.4 0.0 18.4 4.5 4.3 8.8 22.9 4.3 27.2 32.3 107 20–24 17.9 5.6 23.6 9.3 11.4 20.8 27.2 17.1 44.3 46.8 165 25–29 22.1 5.0 27.1 10.9 13.5 24.4 32.9 18.5 51.5 47.4 128 30–34 3.1 7.5 10.6 11.3 24.7 36.0 15.5 32.1 46.6 77.7 87 35–39 12.6 19.3 31.8 3.7 18.9 22.6 16.3 38.2 54.5 41.6 107 40–44 4.7 11.6 16.4 6.5 12.6 19.1 11.3 24.3 35.5 53.9 72 45–49 4.5 28.8 33.3 6.4 1.6 4.2 10.9 30.3 37.6 19.2 56 Education Level No education 14.1 9.6 23.7 3.9 11.7 15.6 18.4 21.2 39.3 40.1 340 Primary 12.6 11.3 23.9 7.8 12.1 19.2 20.4 23.4 43.1 45.5 300 Secondary or higher 16.4 1.4 17.9 23.1 21.9 45.0 39.5 23.3 62.8 71.6 76 Other 0.0 15.6 15.6 0.0 0.0 0.0 0.0 15.6 15.6 0.0 6 Total 13.6 9.5 23.0 7.7 12.9 20.3 21.4 22.3 43.3 47.4 722 Table 2.15.2 (b): Demand for modern contraception among all sexually active women aged 15–49 in Nampula, by background characteristics, midline Background characteristic Unmet need for modern FP Met need for modern FP Demand for modern FP Sexually active Age For spacing For limiting Total For spacing For limiting Total For spacing For limiting Total % of demand satisfied 15–19 25.9 6.2 32.1 16.0 3.7 19.8 42.0 9.9 51.9 38.1 162 20–24 27.6 7.6 35.3 11.8 1.8 13.5 39.4 9.4 48.8 27.7 170 25–29 26.2 10.7 36.9 12.8 6.0 18.8 38.9 16.8 55.7 33.7 149 166 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV 30–34 17.8 17.8 35.6 10.9 9.9 20.8 28.7 27.7 56.4 36.8 101 35–39 21.2 19.7 40.9 4.5 10.6 15.2 25.8 30.3 56.1 27.0 66 40–44 7.9 42.1 50.0 0.0 13.2 13.2 7.9 55.3 63.2 20.8 38 45–49 0.0 30.4 30.4 0.0 8.7 8.7 0.0 39.1 39.1 22.2 23 Education Level No education 21.8 13.8 35.6 9.3 6.2 15.6 31.1 20.1 51.2 30.4 289 Primary 26.0 12.7 38.7 10.8 4.6 15.5 36.8 17.3 54.2 28.6 323 Secondary or higher 16.0 12.8 28.7 18.1 8.5 26.6 34.0 21.3 55.3 48.1 94 Other 33.3 0.0 33.3 0.0 33.3 33.3 33.3 33.3 66.7 50.0 3 Total 23.0 13.1 36.1 11.1 5.9 17.1 34.1 19.0 53.2 32.1 709 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 167 Table 2.16 (a): Preferred future method of contraception among women union aged 15–49 who are not pregnant and not currently using a contraceptive method, baseline Women who are considering using FP in the future prefer: Any method LAPM Injectables Pills Other methods Not sure Number of women Nampula 29.4 25.9 21.9 4.8 0.7 2.3 542 Table 2.16 (b): Preferred future method of contraception among women union aged 15–49 who are not pregnant and not currently using a contraceptive method, midline Women who are considering using FP in the future prefer: Any method LAPM Injectables Pills Other methods Not sure Number of women Nampula 52.0 7.0 28.3 20.0 1.0 2.8 400 Table 2.17 (a) Family planning consultations during last health visit among women who had an appointment for themselves or their children in the last 12 months, by province NAMPULA 2017 2019 Trend ES Women who had any health appointment in past 12 months 34.9 50.6 0.35 Healthy Child Consultation 18.9 24.5 0.11 Triage 11.2 25.1 0.28 RH Consultation 9.5 9.7 HAART 0.3 0.5 Other 2.1 1.8 Number of women aged 15-49 802 816 Women who discussed FP (among women who had an appointment) Any Appointment 57.3 66.8 0.24 Healthy Child Consultation 35.0 32.0 Triage 6.7 24.9 0.37 RH Consultation 20.2 17.7 HAART 0.2 0.7 Other 0.5 1.0 Number of women who had a health appointment in the past 12 months 247 413 *Percentages for sub-categories add up to more than overall percentage of those who had a health appointment, because women could give multiple responses for type of health appointment. Table 2.18 (a) Antenatal care use by women with a pregnancy in the past 3 years, by province NAMPULA 2017 2019 Trend ES # ANC consultations None 0.3 4.6 0.09 1 3.6 4.6 2 – 3 48.0 29.8 0.40 4+ 48.1 61.1 0.31 Counseled on contraception during last pregnancy 168 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Yes 64.0 61.7 No 24.3 30.0 Don’t know 11.4 8.3 Received recommendation for specific contraceptive method during last pregnancy* 64.3 61.7 IUD 12.6 12.0 Implant 29.9 28.7 Sterilization 4.6 3.1 Pills 54.8 44.9 0.23 Condoms 21.6 15.8 0.12 Injectables 61.8 55.6 LAM 0.0 0.4 Other 1.4 0.9 No specific method 36.0 38.3 Number of women with a birth in the past 3 years 417 457 *Percentages add up to more than 100% because some women were recommended more than one method Table 2.19 (a) Delivery location of last birth in the past 3 years and FP services received prior to discharge NAMPULA 2017 2019 Trend ES Delivered in a health facility Yes 68.7 75.9 0.21 No 30.7 23.9 0.14 Missing 0.5 0.2 Delivery assisted by skilled provider 71.9 78.7 0.21 Doctor 8.6 2.4 0.12 Nurse/ Midwife 24.8 37.3 0.26 Auxiliary Midwife 38.5 54.2 0.36 Traditional midwife 13.7 5.0 0.17 Friends/ Family 13.2 16.2 Other 1.0 2.6 No one 0.2 2.9 0.05 DK/Missing 8.6 4.6 0.08 Number of women with a birth in the past 3 years 417 457 Received FP counseling after delivery (prior to discharge) 62.8 46.6 0.39 Doctor 49.7 1.1 1.02 Nurse/ Midwife 43.6 55.6 0.28 Other 0.0 0.9 0.02 DK/Missing 6.7 42.5 0.74 Received recommendation for specific contraceptive method after delivery (prior to discharge) 58.9 51.6 0.18 IUD 9.2 6.6 Implant 18.5 19.1 Sterilization 0.4 0.6 Pills 39.3 31.6 0.16 Condoms 14.7 4.8 0.20 Injectables 51.1 41.6 0.21 LAM 0.0 0 Other 0.0 0.0 No specific method 41.1 51.6 0.24 Received contraceptive method prior to discharge* Yes 38.3 27.2 0.24 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 169 No 61.1 72.3 0.30 Don’t Know/Missing 0.6 0.5 Advised about when to come back to initiate FP method Yes 52.5 77.4 0.66 No 9.2 18.1 0.18 Don’t Know/Missing 38.3 4.5 0.70 Number of women with a birth in the past 3 years who delivered in a facility 274 351 *Calculated among women who indicated that they had been counseled on FP Table 2.20 (a) Postpartum care after last birth in the past 3 years. *Percentages add up to more than 100% because some women were counseled on multiple methods **Calculated among women who reported being counseled on FP Table 2.21 (a) HTSP knowledge and attitudes among women aged 15-49 NAMPULA 2017 2019 Trend ES Knows minimum time to wait to conceive after having a birth, for health of mother and baby 2+ years (accurate) 86.8 86.4 Less than 2 years (inaccurate) 3.7 3.7 Don’t Know 9.5 9.9 Knows minimum time to wait after abortion before conceiving again 6+ months (accurate) 79.4 79.3 Less than 6 months (inaccurate) 3.4 5.3 Don’t Know 17.2 15.4 Believes it is right that a couple should wait 2+ years after having a child to conceive another Agree 76.2 81.3 0.17 Disagree 2.6 3.7 Don’t Know 21.2 15.1 0.12 Number of women aged 15-49 802 816 Table 2.22 (a) Knowledge about specific contraceptive methods among women aged 15-49 NAMPULA 2017 2019 Trend ES The pill is effective even if a woman forgets to take it for 2-3 days True 7.5 14.6 0.14 False 38.8 52.8 0.32 Don’t Know 53.7 32.6 0.47 A woman can get pregnant as soon as she stops taking the pill True 42.4 67.0 0.59 False 4.9 5.0 Don’t Know 52.7 27.9 0.54 The IUD can be stuck forever inside the woman True 2.4 5.1 0.05 False 26.7 50.6 0.52 Don’t Know 70.8 44.2 0.66 170 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV A woman using DMPA (injection) should take an injection every 3 months True 51.6 71.8 0.52 False 2.0 2.2 Don’t Know 46.4 26.0 0.44 A woman who uses DMPA (injection) will never again have children in her life True 6.3 12.9 0.13 False 41.9 55.3 0.31 Don’t Know 51.8 31.9 0.44 Long-acting methods such as implant or IUD can be removed anytime when the woman wants to get pregnant True 32.6 57.7 0.57 False 4.3 5.9 Don’t Know 63.1 36.4 0.62 If a woman wants to delay a pregnancy, she can use the IUD, implant or Depo (injection) without anyone in the family knowing True 31.0 54.8 0.53 False 7.7 11.4 0.07 Don’t Know 61.3 33.8 0.63 A man may use the same condom more than once True 0.4 2.7 0.05 False 46.5 68.4 0.54 Don’t Know 53.1 28.9 0.53 Breastfeeding is an effective way of delaying a pregnancy after childbirth once a woman’s menstrual period has begun True 29.6 28.2 False 35.8 45.2 0.21 Don’t Know 34.6 26.6 0.17 Exclusive breastfeeding means that the woman suckles the baby day and night on demand, and gives no other food or water True 50.7 54.0 False 16.6 22.1 0.11 Don’t Know 32.7 23.9 0.18 Number of women aged 15-49 802 816 Table 2.23 (a) Perceived barriers to seeking contraceptive services among women aged 15-49. NAMPULA 2017 2019 Trend ES Obtain permission to go to health facility Big problem 13.3 28.4 0.31 Not a problem 67.8 66.2 Don’t Know 18.9 5.4 0.27 Obtain money for transportation Big problem 49.7 49.6 Not a problem 41.3 46.1 Don’t Know 9.0 4.3 0.09 Distance to health facility Big problem 48.1 53.7 0.13 Not a problem 43.9 41.8 Don’t Know 8.0 4.5 0.07 Do not want to go alone Big problem 10.0 28.4 0.38 Not a problem 79.1 63.5 0.45 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 171 Don’t Know 10.9 8.1 Afraid health facility will not have contraceptive method she wants Big problem 16.8 36.5 0.41 Not a problem 62.7 53.4 0.23 Don’t Know 20.4 10.0 0.21 Afraid of being treated badly by health provider when asking for contraception Big problem 18.6 37.3 0.39 Not a problem 60.2 50.9 0.22 Don’t Know 21.2 11.9 0.19 Afraid of being served by male health provider Big problem 15.0 29.9 0.31 Not a problem 69.6 60.2 0.25 Don’t Know 15.4 9.9 0.11 Afraid someone will see her getting contraceptive method at the facility and she will be judged by the community Big problem 9.8 23.9 0.29 Not a problem 72.4 63.8 0.24 Don’t Know 17.8 12.3 0.11 Number of women aged 15-49 802 816 Table 2.24 (a) Ability to negotiate contraceptive use with partner among women aged 15-49, by province NAMPULA 2017 2019 Trend ES I feel confident in my ability to suggest to my husband to wait a few months before having another baby Strongly Agree 44.0 63.4 0.46 Agree 16.3 14.0 Disagree 4.5 6.5 Strongly Disagree 11.1 2.2 0.18 Don’t Know 24.1 14.0 0.21 I feel confident in my ability to suggest to my husband the FP method that we should use Strongly Agree 46.6 60.4 0.33 Agree 17.4 13.5 0.08 Disagree 4.1 6.7 0.05 Strongly Disagree 10.2 2.5 0.15 Don’t Know 21.6 16.9 0.10 I feel confident in my ability to convince my husband on which FP method to use Strongly Agree 37.4 58.0 0.47 Agree 18.0 14.3 0.07 Disagree 9.6 7.2 Strongly Disagree 11.4 3.4 0.16 Don’t Know 23.6 17.0 0.13 I feel confident in my ability to convince my husband to give permission to use a FP method Strongly Agree 38.7 57.6 0.43 Agree 19.8 15.2 0.09 Disagree 8.7 7.1 Strongly Disagree 9.0 3.1 0.12 Don’t Know 23.7 17.0 0.14 172 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV If I wanted to use a FP method to avoid pregnancy, I feel confident in my ability to obtain the method I want to use Strongly Agree 41.4 58.8 0.40 Agree 16.2 13.7 Disagree 8.3 6.0 Strongly Disagree 10.6 2.8 0.16 Don’t Know 23.5 18.6 0.10 I am able to ask my husband to use a condom, if I want to Yes 25.3 47.7 0.48 No 52.5 28.8 0.52 It depends (on the situation) 22.1 23.5 Number of women aged 15-49 802 816 Table 2.25 (a) Decision-making power among women aged 15-49. NAMPULA 2017 2019 Trend ES Main decision-maker about use of contraception Woman alone 25.5 24.8 Husband/partner alone 20.1 23.7 Both woman and husband/partner 36.9 44.4 0.16 Others 17.5 7.2 0.21 Number of women aged 15-49 802 816 Husband/partner approves of respondent using contraception to prevent pregnancy Yes 42.1 40.0 No 28.4 36.9 0.18 Don’t Know 29.4 23.2 0.13 Number of women in union aged 15-49 656 643 Table 2.26 (a) Sexual practices among women aged 15-49, by province NAMPULA 2017 2019 Trend ES Median age at first sex 14 15 Had a non-regular partner (higher risk intercourse) Yes 11.7 6.0 0.11 No 88.3 94.0 0.28 Number of women aged 15-49 802 801 Used a condom at last sex with a non-regular partner in the past 12 months Yes 18.6 27.1 No 77.7 72.9 Missing 3.8 0.0 0.08 Number of women aged 15-49 who had a non￾regular partner 101 48 Table 2.27 (a) Discussed FP with CHW in the past 12 months, among women aged 15-49. NAMPULA 2017 2019 Trend ES Visited by a Community Health Worker in the past 12 months who discussed FP 12.1 24.4 0.25 TBA 0.8 1.1 APE 1.4 10.5 0.18 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 173 Activist 9.9 12.7 Number of women aged 15- 49 802 816 Most recent contact with CHW to discuss FP <1 month ago 44.2 29.1 0.33 1-5 months ago 16.9 28.1 0.23 6+ months ago 19.5 25.6 Missing/DK 19.4 17.1 Activist referred woman to a health facility Yes 94.5 73.9 0.81 No 5.5 26.1 0.42 Number of women aged 15- 49 who had contact with CHW in past 12 months 95 199 Table 2.28 (a) Discussed FP in other community venues or with family/friends in the past 12 months, among women aged 15-49, by province NAMPULA 2017 2019 Trend ES Participated in community event in the past 12 months where FP was discussed Yes 14.1 17.3 No 85.9 82.7 Discussed FP with someone else (not health activist) in past 12 months 13.8 20.7 0.14 Husband/partner 6.7 6.6 Mother/mother-in-law 4.8 3.9 Daughter 0.2 1.1 0.02 Other family member 1.4 6.0 0.09 Friend 3.6 9.1 0.11 Health worker 3.2 6.3 0.06 Other 0.5 0.1 Number of women aged 15-49 802 816 Table 2.29 (a) Participation in community-based mobile brigades and exposure to other FP messages in the past 6 months, among women aged 15-49, by province NAMPULA 2017 2019 Trend ES Attended mobile brigade in the past 6 months where FP was offered Yes 10.1 7.7 No 89.9 92.3 Heard radio program discussing FP/HTSP in the past 6 months Yes 32.4 25.1 0.15 No 67.6 74.9 0.21 Number of women aged 15-49 802 816 174 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV SELECTED FP INDICATORS BY AGE, RELIGION, MARITAL STATUS AND SES-NEW VERSION TABLE 3 KNOWLEDGE AND CURRENT USE OS MODERN METHOD BY AGE, RELIGION, MARITAL STATUS AND SES Knowledge of any modern method Baseline Midline % % Age 15-19 75.4 88.3 20-24 77.8 93.4 25-49 80.4 93.1 Religion Islam 82.9 92.9 Others 76.2 91.5 Marital status In union 79.6 91.9 Others 76.2 92.5 SES Q1,2 64.8 87.9 Q5 88.0 96.3 Zone Urban 86.7 97.9 Rural 80.1 91.6 Total 79.6 92.0 Current use modern method Baseline Midline % % Age 15-19 12.5 17.8 20-24 20.2 14.3 25-49 20.6 17.8 Religion Islam 20.6 18.6 Others 18.5 16.2 Marital status In union 19.0 16.3 Others 23.1 19.7 SES Q1,2 12.5 11.2 Q5 27.8 27.8 Zone Urban 26.5 27.9 Rural 17.7 12.5 Total 20.7 17.2 TABLE 4 UNMET NEED AND DEMAND BY AGE, RELIGION, MARITAL STATUS AND SES Unmet need for modern FP Met need for modern FP Demand for modern FP Demand satisfied % % % % Age 15-19 32.1 19.8 51.9 38.1 20-24 35.3 13.5 48.8 27.7 25-49 38.2 17.5 55.7 31.4 Religion Islam 34.2 19.1 53.3 35.8 Others 37.2 15.9 53.1 30.0 Marital status In union 39.8 16.3 56.1 29.1 Others 0.0 24.2 24.2 100.0 SES Q1,2 40.3 10.8 51.1 21.2 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 175 Q5 29.9 27.0 56.9 47.4 Zone Urban 29.9 28.4 58.3 48.7 Rural 38.6 12.5 51.1 24.4 Total 36.1 17.1 53.2 32.1 TABLE 5 WOMAN WHO DISCUSSED FP IN HEALTH FACILITY BY AGE, RELIGION, MARITAL STATUS AND SES % of Women who discussed FP (among women who had an appointment) Age Baseline Midline 15-19 38.3 69.1 20-24 52.7 71.7 25-49 36.8 62.6 Religion Islam 50.5 71.7 Others 39.0 64.1 Marital status In union 42.0 67.4 Others 35.6 63.3 SES Q1,2 39.1 67.5 Q5 66.7 68.6 Zone Urban 74.2 63.3 Rural 35.1 68.8 Total 57.3 66.8 176 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV TABLE 6 PERCEIVED BARRIERS BY AGE, RELIGION, MARITAL STATUS AND SES Afraid health facility will not have contraceptive method she wants Afraid of being treated badly by health provider when asking for contraception Big problem Not a problem Do not know Big problem Not a problem Do not know Age 15-19 35.0 49.2 15.7 40.6 41.6 17.8 20-24 33.5 54.4 12.1 33.0 55.5 11.5 25-49 38.4 54.9 6.6 37.5 53.1 9.4 Religion Islam 38.9 52.0 9.1 37.5 51.4 11.1 Others 35.2 54.2 10.6 37.1 50.6 12.3 Marital status In union 35.0 56.0 9.0 36.5 51.9 11.5 Others 42.2 43.9 13.9 39.9 46.8 13.3 SES Q1,2 37.8 50.7 11.5 41.6 44.1 14.2 Q5 27.2 64.2 8.6 23.5 69.1 7.4 Zone Urban 31.3 59.2 9.6 27.9 62.9 9.2 Rural 38.7 51.0 10.2 41.1 45.8 13.0 Total 36.5 53.4 10.0 37.3 50.9 11.9 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 177 TABLE 7 PERCEIVED BARRIERS BY AGE, RELIGION, MARITAL STATUS AND SES Afraid of being served by male health provider Afraid someone will see her getting contraceptive method at the facility and she will be judged by the community Big problem Not a problem Do not know Big problem Not a problem Do not know % % % % % % Age 15-19 35.5 50.3 14.2 30.5 53.8 15.7 20-24 26.9 63.2 9.9 18.1 70.9 11.0 25-49 28.6 63.4 8.0 23.3 65.4 11.2 Religion Islam 30.7 58.8 10.5 23.3 63.2 13.5 Others 29.4 61.0 9.6 24.2 64.2 11.5 Marital status In union 28.9 61.3 9.8 22.6 65.2 12.3 Others 33.5 56.1 10.4 28.9 59.0 12.1 SES Q1,2 34.0 54.2 11.8 28.8 55.6 15.6 Q5 27.2 64.2 8.6 13.0 81.5 5.6 Zone Urban 22.1 69.2 8.8 12.5 76.7 10.8 Rural 33.2 56.4 10.4 28.6 58.5 12.8 Total 29.9 60.2 9.9 23.9 63.8 12.3 178 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV TABLE 8 ABILITY TO NEGOTIATE CONTRACEPTIVE USE WITH PARTNER AMONG WOMEN AGED 15-49 I feel confident in my ability to suggest to my husband to wait a few months before having another baby I feel confident in my ability to suggest to my husband the FP method that we should use I feel confident in my ability to convince my husband on which FP method to use I feel confident in my ability to convince my husband to give permission to use a FP method If I wanted to use a FP method to avoid pregnancy, I feel confident in my ability to obtain the method I want to use I am able to ask my husband to use a condom, if I want to strongly agree strongly agree strongly agree strongly agree strongly agree yes Age 15-19 65.5% 59.4% 56.9% 57.9% 58.9% 44.7% 20-24 65.4% 64.3% 62.1% 62.6% 65.4% 56.0% 25-49 61.6% 59.3% 56.8% 55.4% 56.1% 45.5% Religion Islam 63.9% 61.1% 58.8% 56.1% 58.8% 49.0% Others 63.1% 60.0% 57.5% 58.5% 58.8% 46.9% Marital status In union 64.9% 61.0% 59.3% 58.6% 59.9% 46.3% Others ) boy￾friend ) 57.8% 58.4% 53.2% 53.8% 54.9% 52.6% SES Q1,2 59.5% 55.6% 54.5% 53.7% 54.5% 42.5% Q5 72.2% 72.2% 69.1% 68.5% 67.3% 70.4% Zone Urban 70.0% 67.5% 65.8% 64.6% 66.3% 66.3% Rural 60.6% 57.5% 54.7% 54.7% 55.7% 39.9% Total 63.4% 60.4% 58.0% 57.6% 58.8% 47.7% USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 179 Table 9 Discussed FP in other community venues or with family/friends in the past 12 months a Participated in community event in the past 12 months where FP was discussed Discussed FP with someone else (not health activist) in past 12 months Age 15-19 20.8 18.8 20-24 19.2 24.7 25-49 14.9 19.9 Religion Islam 18.6 23.6 Others 16.5 19.0 Marital status In union 16.6 21.0 Others 19.7 19.7 SES Q1,2 13.4 13.7 Q5 17.9 32.1 Zone Urban 15.4 28.3 Rural 18.1 17.5 Total 17.3 20.7 Table 10 Participation in community-based mobile brigades and exposure to other FP messages in the past 6 months, among women aged 15-49 a Attended mobile brigade in the past 6 months where FP was offered Heard radio program discussing FP/HTSP in the past 6 months Age 15-19 8.6 26.4 20-24 8.8 25.8 25-49 6.9 24.3 Religion Islam 10.1 27.4 Others 6.3 23.8 Marital status In union 7.2 24.6 Others 9.8 27.2 SES Q1,2 7.4 17.3 Q5 9.3 29.6 Zone Urban 6.7 28.3 Rural 8.2 23.8 Total 7.7 25.1 TABLE 11 BARRIERS BY AGE Get permission to go to the health facility Getting money for transport Distance to health facility Big problem Not a problem Do not know Big problem Not a problem Do not know Big problem Not a problem Do not know 15-19 30.3% 61.0% 8.7% 55.0% 39.0% 6.0% 54.7% 38.3% 7.0% 20-24 30.7% 65.4% 3.9% 49.3% 47.9% 2.9% 51.1% 46.1% 2.9% 24-49 28.1% 67.3% 4.7% 49.8% 46.0% 4.2% 54.8% 40.8% 4.4% 180 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Correspondent analysis results Multiple correspondence analysis (MCA) was used to detect and represent underlying structures in a data set. It does this by representing data as points in a low-dimensional Euclidean space. The results suggest that variables more associated with first dimension are : current use of modern methods, wealth and urban/rural . While the second dimension is likely to be associated with knowledge and age (See Table 1). Table 1 Discrimination Measures Dimension 1 2 Mean knowledge Modern Methods .112 .317 .214 current use of Modern methods .161 .127 .144 AGE .036 .144 .090 URBANO / RURAL .779 .040 .410 quintile .749 .487 .618 Active Total 1.838 1.115 1.477 The Graphs below, suggest that, in dimension 1, right side , woman from quintile 5, living in urban areas are currently, more likely to use modern method (CU.YES). In the opposite side, left side of dimension 1, woman living in rural areas, from poor families (q12) are currently, less prone to use modern methods (CU.NO). USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 181 CU.YES Current use Modern methods CU. NO no current use of modern K.YES Knowledge of modern methods K.NO no knowledge of modern method Table 1 Discrimination Measures 182 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Dimension 1 2 Mean knowledge Modern Methods .112 .317 .214 current use of Modern methods .161 .127 .144 AGE .036 .144 .090 URBANO / RURAL .779 .040 .410 quintile .749 .487 .618 Active Total 1.838 1.115 1.477 USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 183 ANNEX K: DEFINITIONS AND SYNTAX FOR CALCULATION OF DEMAND AND NEED FOR FAMILY PLANNING Need and Demand for Family Planning Percentage of women with a) unmet need for family planning, b) met need for family planning, c) total demand for family planning by whether for spacing, limiting, and total, and the percentage of the demand for family planning that is satisfied Definition Percentage of currently married women, all women, and sexually active unmarried women with 1) unmet need for family planning, 2) met need for family planning, 3) total demand for family planning by whether for spacing, limiting, and total, and 4) Percentage of total demand for family planning that is satisfied 5) Percentage of total demand for family planning that is satisfied by modern methods Coverage: Population base: 1) All women (IR file) 2) Currently married (v502 = 1) (IR file) 3) Sexually active unmarried women: Includes women who are not currently married or in a consensual union (single, divorced, widowed, and separated) and who had sexual intercourse within the last 30 days (v502 ≠ 1 & v528 <= 30) (IR file) Time period: Current status at time of survey. Numerators: 1) Unmet need for contraception: Number of women who are not using a method of contraception and are: a) For spacing (v626a = 1): fecund and say they want to wait two or more years for their next birth; fecund and are unsure whether they want another child; fecund, want another child, but are unsure when to have the birth; pregnant and wanted current pregnancy later; or postpartum amenorrheic and wanted last birth later b) For limiting (v626a = 2): fecund and do not want any more children; pregnant and did not want current pregnancy; or postpartum amenorrheic and did not want the last birth c) Total unmet need: Number of women with an unmet need for family planning for spacing or limiting (v626a = 1 or v626a = 2) 2) Met need for contraception: Number of women who are using a method of contraception and are a) For spacing (v626a = 3): not considered to be limiting (see below) 184 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV b) For limiting (v626a = 4): want no more children; are sterilized; or say they cannot get pregnant when asked about the desire for future children c) Total met need (v626a = 3 or v626a = 4): Number of women with met need for spacing or limiting 3) Total demand for contraception: Number of women who have a met need or unmet need: a) For spacing (v626a = 1 or v626a = 3) b) For limiting (v626a = 2 or v626a = 4) c) Total (v626a in 1, 2, 3, 4) 4) Demand satisfied: Number of women who are using any contraceptive method (v626a in 3,4) 5) Demand satisfied by modern methods: Number of women who are using any modern contraceptive method including female sterilization, male sterilization, pill, IUD, injectables, implants, male condom, female condom, emergency contraception, vaginal methods, lactational amenorrhea method, or other modern methods (v313 = 3) Denominators: Unmet need, met need, and total demand indicators: Number of All women age 15-49 Currently married women (v502 = 1) Sexually active unmarried women – including women who are not currently married or in a consensual union (single, divorced, widowed, and separated) and who had sexual intercourse within the last 30 days (v502 ≠ 1 & v528 in 0:30) Demand satisfied indicators: Number of All women age 15-49 Currently married women (v502 = 1) Sexually active unmarried women – including women who are not currently married or in a consensual union (single, divorced, widowed, and separated) and who had sexual intercourse within the last 30 days (v502 ≠ 1 & v528 in 0:30). that have either unmet need for family planning or met need for family planning (v626a in 1, 2, 3, 4) Variables: IR file. v626a Unmet need for contraception (Definition 3) v313 Current use by method type v502 Currently/formerly/never in union v528 Time since last sex (in days) v005 Woman’s individual sample weight Calculation Unmet need, met need, and total demand indicators: Numerator divided by the denominator multiplied by 100. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 185 The calculation of numerators for the met and unmet need indicators first takes into account a woman’s contraceptive status. Women who are using contraception are considered to have a met need for family planning. For women who are not using contraception, the determination of the need for family planning involves several additional steps. Nonusers are separated into those who are currently married and those who are not married but who are sexually active. An unmarried woman is considered to be sexually active if the woman had sexual intercourse within 30 days prior to the survey. Unmarried women who are not sexually active are considered not to be exposed to the risk of pregnancy and, therefore, have no need for family planning. To assign need status to married and sexually active unmarried nonusers, these women are separated into two groups: (1) those who are pregnant or postpartum amenorrheic and (2) those who are not pregnant or postpartum amenorrheic. Women are considered to be postpartum amenorrheic if their period had not returned since their last live birth in the two-year period prior to the survey. Married and sexually active unmarried nonusers who are not pregnant or postpartum amenorrheic are further separated into those who are considered fecund and those who are infecund. Women are classified as infecund if they fall into any of the following categories: married 5+ years, had no children in the past 5 years and never used contraception; responded “can’t get pregnant” to question on desire for future children; responded “menopausal/hysterectomy” on reason for not using contraception; response to time since last period is > 6 months and not postpartum amenorrheic (0-59 months); response to time since last period is “menopausal/hysterectomy” or “never menstruated”; or response to time since last period is “last period was before last birth” and last birth was 5+ years ago. Infecund women have no need for family planning. Other women who have no need include married or sexually active unmarried women who are not using contraception and are: pregnant and wanted the current pregnancy; postpartum amenorrheic and wanted their last birth; or fecund and want another child within 2 years. Married and sexually active unmarried nonusers have unmet need for spacing if they are: pregnant and wanted the pregnancy later; postpartum amenorrheic and wanted their last birth later; or fecund and want the next child in 2+ years, want another child but are undecided on the timing or are undecided if they want another child. Married and sexually active unmarried nonusers have an unmet need for limiting if they are: pregnant and did not want the current pregnancy at all; postpartum amenorrheic and did not want their last birth at all their last birth; or fecund and want no more children. 186 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV The calculation of unmet need for currently married women is summarized in the diagram below: Demand satisfied indicators: Numerator divided by the denominator multiplied by 100. The numerator for the proportion of demand satisfied include the number of women currently using any contraceptive method while the numerator for the demand satisfied by modern methods include the number of women using any modern contraceptive method. The denominators include those women with a met need (those using contraception irrespective whether the method is a modern method or a traditional method) and those with an unmet need for family planning. Notes and Considerations Details of the calculation of the unmet need variable can be found on the Unmet Need page of the DHS Program website at https://www.dhsprogram.com/topics/Unmet-Need.cfm, including survey-specific code in Stata and SPSS for the construction of v626a for surveys that do not have the variable in the USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 187 dataset. The results of the change in Definition are also summarized in Revising Unmet Need: In Brief at https://www.dhsprogram.com/pubs/pdf/OD63/OD63[12June2012_English].pdf The proportion of demand satisfied by modern methods indicator corresponds to SDG indicator 3.7.1 and UNAIDS indicator 5.2. Handling of Missing Values If responses are missing on questions with respect to the wantedness of births or the current pregnancy, women are assigned a value of missing on the unmet need variable. This represents a change from the previous practice of assuming that pregnant, postpartum amenorrheic, or fecund nonusers for whom information on the wantedness of the current pregnancy/last birth was missing had an unmet need for family planning (Bradley et al. 2012). Changes over Time The Definition of unmet need for family planning was simplified during the DHS VI project to improve comparability over time and between surveys within the DHS Program and to facilitate comparability with data collected in other surveys including the MICS surveys undertaken in many countries with UNICEF support. Bradley et al. (2012) detail the changes involved in revising the unmet need Definition in DHS Analytical Study No. 25. Several changes have occurred over time in the calculation of unmet need for family planning. Due to these changes, comparisons of unmet need and demand for family planning between surveys may not be valid if based on country reports. References Bradley, S.E.K., T.N. Croft, J.D. Fishel, and C. Westoff. 2012. Revising Unmet Need for Family Planning. DHS Analytical Studies No. 25. Calverton, Maryland, USA, ICF International. https://www.dhsprogram.com/publications/publication-AS25-Analytical-Studies.cfm Westoff, C.F. 2012. Unmet need for modern contraceptive methods. DHS Analytical Studies No. 28. Calverton, Maryland, USA: ICF International. https://www.dhsprogram.com/publications/publication￾AS28-Analytical-Studies.cfm MacQuarrie, K.L.D. 2014. Unmet need for family planning among young women: levels and trends. DHS Comparative Reports No. 34. Rockville, Maryland, USA: ICF International. https://www.dhsprogram.com/publications/publication-CR34-Comparative-Reports.cfm Wang, W., S. Staveteig, R. Winter, and C. Allen. 2017. Women’s Marital Status, Contraceptive Use, and Unmet Need in Sub-Saharan Africa, Latin America, and the Caribbean. DHS Comparative Reports No. 44. Rockville, Maryland, USA: ICF. https://www.dhsprogram.com/publications/publication-CR44- Comparative-Reports.cfm Westoff, C.F. 2006. New estimates of unmet need and the demand for family planning. DHS Comparative Reports No. 14. Calverton, Maryland, USA: Macro International. https://www.dhsprogram.com/publications/publication-CR14-Comparative-Reports.cfm 188 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV Westoff, C.F. 2001. Unmet need at the end of the century. DHS Comparative Reports No. 1. Calverton, Maryland, USA: ORC Macro. https://www.dhsprogram.com/publications/publication-CR1-Comparative￾Reports.cfm Westoff, C.F. and A. Bankole. 1995. Unmet need: 1990-1994. DHS Comparative Studies No. 16. Calverton, Maryland, USA: Macro International. https://www.dhsprogram.com/publications/publication￾CS16-Comparative-Reports.cfm Westoff, C.F. and L.H. Ochoa. 1991. Unmet need and the demand for family planning. DHS Comparative Studies No. 5. Columbia, Maryland, USA: Institute for Resource Development. https://www.dhsprogram.com/publications/publication-CS5-Comparative-Reports.cfm UNAIDS. 2017. Global AIDS Monitoring 2018: Indicators for monitoring the 2016 United Nations Political Declaration on Ending AIDShttp://www.unaids.org/en/resources/documents/2018/Global-AIDS-Monitoring Resources DHS Program website: Unmet Need: https://www.dhsprogram.com/topics/Unmet-Need.cfm DHS Indicator Snapshot: Demand Satisfied by Modern Methods: https://www.youtube.com/watch?v=RceOuLjJwKY&list=PLagqLv￾gqpTMx2Q10C_prJRnCtM55C0o8&index=29&t=0s DHS-7 Tabulation plan: Tables 7.13.1, 7.13.2 and 15.15 API Indicator IDs: Currently married women: FP_NADM_W_UNS, FP_NADM_W_UNL, FP_NADM_W_UNT, FP_NADM_W_MNS, FP_NADM_W_MNL, FP_NADM_W_MNT, FP_NADM_W_TDS, FP_NADM_W_TDL, FP_NADM_W_TDT, FP_NADM_W_PDS, FP_NADM_W_PDM (API link, STATcompiler link) All women: FP_NADA_W_UNS, FP_NADA_W_UNL, FP_NADA_W_UNT, FP_NADA_W_MNS, FP_NADA_W_MNL, FP_NADA_W_MNT, FP_NADA_W_TDS, FP_NADA_W_TDL, FP_NADA_W_TDT, FP_NADA_W_PDS, FP_NADA_W_PDM (API link, STATcompiler link) Sexually active unmarried women: FP_NADU_W_UNS, FP_NADU_W_UNL, FP_NADU_W_UNT, FP_NADU_W_MNS, FP_NADU_W_MNL, FP_NADU_W_MNT, FP_NADU_W_TDS, FP_NADU_W_TDL, FP_NADU_W_TDT, FP_NADU_W_PDS, FP_NADU_W_PDM (API link, STATcompiler link) SDG Indicator 3.7.1: Proportion of women of reproductive age (aged 15-49 years) who have their need for family planning satisfied with modern methods WHO 100 Core Health Indicators: Demand for family planning satisfied with modern methods MICS6 Indicator TM.4: Need for family planning satisfied with modern contraception UNAIDS Indicator 5.2: Demand for family planning satisfied by modern methods USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 189 Syntax Encoding: windows-1252. * Women in union ECIVIL=2,3 NOT using Contraception Q408=2 Pregnant Q216=1 Postpartum amenorrheic Amenorrheic=1 Pregnant or Postpartum amenorrheic PPA=1 Did not want current pregnancy Q217=2 AND Q218=2 Did not want last birth at all Q302A=2 AND Q302B=2 Wanted current pregnancy later Q217=2 and Q218=1 Wanted last birth later Q302A=2 AND Q302B=1 NOT (Pregnant or Postpartum amenorrheic) PPA=2 Infecund Fecund=2 Married 5+ years ago, had no children in past 5 years, and never used contraception= infecund1 Can't get pregnant on wantedness of futute children Q602A=3 Menopausal/hysterectomy on reason not using contraception Q612=23 Time since last period >=6 months and not postpartum amenorrheic Time_LastPeriod=1 AND Amenorrheic=2 Time since last period is Menopausal/hysterectomy/never menstruated Q219=994, 996 Time since last period is last period was before last birth and last birth is 5+ years ago Q219=995 AND Time_Children>=60 Wants no more children Q602A=2 Wants next child in 2+ years Q602A=1 AND Q603A_TEMPO=2; wants child and undecided timing Q602A=1 AND Q603_A=998 or undecided if wants child Q602A=4,5 Demand for Family planning New_Unmet=1 OR Q408=1 Proportion of demand satisfied Q408=1/New_Demand=1 Proportion of demand satisfied by modern methods Q409R4/New_Demand=1 Women sexually active ACTIVA=1. *Amenorrheic. DO IF (Q219=995 AND Time_LastBirth>=24). COMPUTE Amenorrheic=1. ELSE. COMPUTE Amenorrheic=2. END IF. EXECUTE. *Infecund1. IF (ANY (Q501,1,2) AND (Q201=2 AND Q402R2=2 AND Time_married>=60)) Infecund1=1. EXECUTE. *Time since last period. IF ((Q219=1 AND Q219_0 >= 180) OR (Q219=2 AND Q219_0>=24) OR (Q219=3 AND Q219_0>=6) OR (Q219=4 AND Q219_0>=1)) Time_LastPeriod=1. EXECUTE. *Fecund. 190 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV IF (Infecund1=1 OR Q602A=3 OR ANY(Q612R,6,7) OR (Time_LastPeriod=1 AND Amenorrheic=2) OR (ANY (Q219,994, 996)) OR (Q219=995 AND Time_LastBirth>=60)) Fecund=2. RECODE Fecund (2=2) (ELSE=1) INTO Fecund_R. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q408=2 AND (Q216=1 AND ((Q217=2 AND Q218=1) OR (Amenorrheic=1 AND (Q302A=2 AND Q302B=2))))))) New_Unmet_Limit1=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q408=2 AND (Q216=1 AND ((Q217=2 AND Q218=2) OR (Amenorrheic=1 AND (Q302A=2 AND Q302B=1))))))) New_Unmet_Spacing1=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q408=2 AND ((Amenorrheic=2 OR Q216=2) AND FECUND_R=1) AND (Q602A=2)))) New_Unmet_Limit2=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q408=2 AND ((Amenorrheic=2 OR Q216=2) AND FECUND_R=1) AND ((Q602A=1 AND Q603A_TEMPO=2) OR (Q602A=1 AND Q603_A=998) OR (ANY(Q602A,4,5)))))) New_Unmet_Spacing2=1. EXECUTE. IF (New_Unmet_Limit1=1 OR New_Unmet_Limit2=1) New_Unmet_Limit=1. IF (New_Unmet_Spacing1=1 OR New_Unmet_Spacing2=1) New_Unmet_Spacing=1. IF (New_Unmet_Limit=1 OR New_Unmet_Spacing=1) New_Unmet=1. RECODE New_Unmet_Limit New_Unmet_Spacing New_Unmet (1=1) (ELSE=2) INTO Unmet_Limit Unmet_Spacing Unmet_Total. EXECUTE. IF (ANY(ECIVIL,2,3)) ULimit=Unmet_Limit. IF (ANY(ECIVIL,2,3)) USpacing=Unmet_Spacing. IF (ANY(ECIVIL,2,3)) Uunmet=Unmet_Total. EXECUTE. IF (Q408=1 AND ((ANY (Q602A,2,3)) OR Q409_LAQUEAÇÃOFEMININA_A=1)) New_Meet_LimitFP=1. RECODE New_Meet_LimitFP (1=1) (ELSE=2) INTO New_Meet_Limit. IF (Q408=1 AND New_Meet_Limit=2) New_Meet_SpacingFP=1. RECODE New_Meet_SpacingFP (1=1) (ELSE=2) INTO New_Meet_Spacing. IF (New_Meet_LimitFP=1 OR New_Meet_SpacingFP=1) New_MeetFP=1. RECODE New_MeetFP (1=1) (ELSE=2) INTO New_Meet. EXECUTE. IF (ANY(ECIVIL,2,3)) MLimit=New_Meet_Limit. IF (ANY(ECIVIL,2,3)) MSpacing=New_Meet_Spacing. IF (ANY(ECIVIL,2,3)) Mmeet=New_Meet. EXECUTE. IF (Ulimit=1 OR Mlimit=1) New_DemandLFP=1. IF (USpacing=1 OR MSpacing=1) New_DemandSFP=1. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 191 IF (Uunmet=1 OR Mmeet=1) New_DemandFP=1. RECODE New_DemandLFP New_DemandSFP New_DemandFP (1=1) (ELSE=2) INTO DemandLFP DemandSFP DemandFP. IF (ANY(ECIVIL,2,3)) DLimit=DemandLFP. IF (ANY(ECIVIL,2,3)) DSpacing=DemandSFP. IF (ANY(ECIVIL,2,3)) Ddemand=DemandFP. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q409R5=2 AND (Q216=1 AND ((Q217=2 AND Q218=1) OR (Amenorrheic=1 AND (Q302A=2 AND Q302B=2))))))) New_Unmet_Limit1M=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q409R5=2 AND (Q216=1 AND ((Q217=2 AND Q218=2) OR (Amenorrheic=1 AND (Q302A=2 AND Q302B=1))))))) New_Unmet_Spacing1M=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q409R5=2 AND ((Amenorrheic=2 OR Q216=2) AND FECUND_R=1) AND (Q602A=2)))) New_Unmet_Limit2M=1. EXECUTE. IF ((ANY(ECIVIL,2,3) AND (Q409R5=2 AND ((Amenorrheic=2 OR Q216=2) AND FECUND_R=1) AND ((Q602A=1 AND Q603A_TEMPO=2) OR (Q602A=1 AND Q603_A=998) OR (ANY(Q602A,4,5)))))) New_Unmet_Spacing2M=1. EXECUTE. IF (New_Unmet_Limit1M=1 OR New_Unmet_Limit2M=1) New_Unmet_LimitM=1. IF (New_Unmet_Spacing1M=1 OR New_Unmet_Spacing2M=1) New_Unmet_SpacingM=1. IF (New_Unmet_LimitM=1 OR New_Unmet_SpacingM=1) New_UnmetM=1. RECODE New_Unmet_LimitM New_Unmet_SpacingM New_UnmetM (1=1) (ELSE=2) INTO Unmet_LimitM Unmet_SpacingM Unmet_TotalM. EXECUTE. IF (ACTIVA=1) ULimit_Sex=Unmet_LimitM. IF (ACTIVA=1) USpacing_Sex=Unmet_SpacingM. IF (ACTIVA=1) Uunmet_Sex=Unmet_TotalM. EXECUTE. IF (Q409R4=1 AND ((ANY (Q602A,2,3)) OR Q409_LAQUEAÇÃOFEMININA_A=1)) New_Meet_LimitFPM=1. RECODE New_Meet_LimitFPM (1=1) (ELSE=2) INTO New_Meet_LimitM. IF (Q409R4=1 AND New_Meet_Limit=2) New_Meet_SpacingFPM=1. RECODE New_Meet_SpacingFPM (1=1) (ELSE=2) INTO New_Meet_SpacingM. IF (New_Meet_LimitFPM=1 OR New_Meet_SpacingFPM=1) New_MeetFPM=1. RECODE New_MeetFPM (1=1) (ELSE=2) INTO New_MeetM. EXECUTE. IF (ACTIVA=1) MLimit_Sex=New_Meet_LimitM. IF (ACTIVA=1) MSpacing_Sex=New_Meet_SpacingM. IF (ACTIVA=1) Mmeet_Sex=New_MeetM. EXECUTE. 192 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV IF (Ulimit_Sex=1 OR Mlimit_Sex=1) New_DemandLFP_Sex=1. IF (USpacing_Sex=1 OR MSpacing_Sex=1) New_DemandSFP_Sex=1. IF (Uunmet_Sex=1 OR Mmeet_Sex=1) New_DemandFP_Sex=1. RECODE New_DemandLFP_Sex New_DemandSFP_Sex New_DemandFP_Sex (1=1) (ELSE=2) INTO DemandLFP_Sex DemandSFP_Sex DemandFP_Sex. IF (ACTIVA=1) DLimit_Sex=DemandLFP_Sex. IF (ACTIVA=1) DSpacing_Sex=DemandSFP_Sex. IF (ACTIVA=1) Ddemand_Sex=DemandFP_Sex. EXECUTE. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 193 ANNEX L: DISCLOSURE OF CONFLICTS OF INTEREST 194 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 195 196 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 197 198 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 199 200 | IFPP MIDTERM PERFORMANCE EVALUATION USAID.GOV ANNEX M: REFERENCE FOR ADS CHAPTER 201 MANDATORY REFERENCE FOR ADS CHAPTER 201 CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Pursuant to 201.3.5.17, draft evaluation reports must undergo a peer review organized by the office managing the evaluation. The following criteria should serve as the basis against which the report is reviewed. To help ensure a high-quality evaluation report, these criteria must be included in the evaluation SOW to communicate to evaluators USAID’s quality criteria. Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. Evaluation methodology should be explained in detail and sources of information properly identified. Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. USAID.GOV IFPP MIDTERM PERFORMANCE EVALUATION | 201 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523 USA