Endline Evaluation Report of the Agir pour la Planification Familiale Project (AgirPF) September 2018 Endline Evaluation Report of the Agir pour la Planification Familiale Project (AgirPF) September 2018 By Macoumba Thiam Reviewed by Alioune Diagne Andre Koalaga About EngenderHealth and AgirPF EngenderHealth is a leading global women’s health organization committed to working toward a world where sexual and reproductive rights are respected as human rights and women and girls have the freedom to reach their full potential. In nearly 20 countries around the world, EngenderHealth creates lasting change by training health care professionals and partnering with governments and communities to make high-quality family planning and sexual and reproductive health services available today and for all generations to come. In 2013, to support advancement toward both FP2020 and the Ougadougou Partnership, the United States Agency for International Development (USAID)/West Africa Regional Health Office awarded a five-year, $29 million project, Agir pour la Planification Familiale (AgirPF), to EngenderHealth with its core partner, Avenir Health (formerly Futures Institute). The goal of AgirPF is to enable individuals and couples to make, and voluntarily act on, informed decisions about FP, in selected urban and peri￾urban areas of Burkina Faso, Côte d’Ivoire, Mauritania, Niger, and Togo. The AgirPF Project Endline Evaluation Report i Contents Acronyms iii List of tables iv List of figures v Executive summary 1 Performance against PMP indicators 1 AgirPF SBCC campaigns 2 Impact on FP 2 Introduction 3 Overview and methodology 3 Beneficiaries 3 Geographic scope 4 Description of the project intervention 4 Monitoring and evaluation 5 Baseline assessment 7 Mid-term performance evaluation 8 Endline evaluation 9 Overall objectives 9 Specific objectives 9 Methodology 10 Sampling procedures 10 Data collection and entry 11 Data analysis 12 SBCC campaign evaluation overview 15 Results 16 Progress against PMP indicators 16 Output Results 17 Outcome results 22 SBCC custom indicator and outcome indicators 13, 14, and 15. 24 Endline survey and SBCC campaign evaluation 28 Background characteristics of endline survey respondents 28 The AgirPF SBCC campaign: Messaging reach by activity and country 31 Conclusion on the evaluation of the SBCC campaign 39 AgirPF impact on knowledge and use of family planning methods 39 Knowledge of family planning 40 Use of family planning methods 43 Evaluation of the net impact of AgirPF interventions on current use of modern FP 49 Discussion and conclusion 56 The AgirPF Project Endline Evaluation Report ii Performance against the PMP indicators 56 Strategic objective indicators 56 Intermediate results 1, 2, and 3 57 Results of the AgirPF SBCC campaign 58 AgirPF’s link to FP knowledge and use by population-level indicators 59 FP use 59 Method mix 60 Reason for not using an FP method 60 Bibliography 61 Appendices 63 Appendix A: Questionnaires (Male and Female) 63 Appendix B: Number of intervention and nonintervention health facilities by health district 73 Appendix C: Statistics for population sample calculation 75 Appendix D: AgirPF PMP Indicators’ Results Table 76 Output Indicator Results 76 Outcome Indicator Results 81 The AgirPF Project Endline Evaluation Report iii Acronyms AgirPF Agir pour la Planification Familiale CAPI Computer-assisted personal interviewing CHWs Community health workers COPE Client-oriented provider-efficiency CPR Contraceptive prevalence rate CSO Civil society organization CSPro Census and survey processing system CYP Couple year of protection DHIS2 District health information software DHS Demographic and health survey E4D Evidence for Development EA Enumeration area FP Family planning FPSD Family planning special days HF Health facility HIP/BP High impact practice/best practice IUD Intrauterine device IVR Interactive voice response KAP Knowledge, attitudes, and practices LARC/PM Long-acting reversible contraceptives/permanent method LOP Life of project MAP Men as Partners mCPR Modern contraceptive prevalence rate MICS Multiple indicator cluster surveys MNCH Maternal and neonatal child health MOH Ministry of Health M&E Monitoring and evaluation PMP Performance Monitoring Plan SDM Standard days method SDP Services delivery point SBCC Social and behavior change communication SRH Sexual and reproductive health SWT Site walk-through UNFPA United Nations Population Fund USAID United States Agency for International Development USG United States Government WHO World Health Organization WRA Women of reproductive age The AgirPF Project Endline Evaluation Report iv List of tables Table 1. Number of AgirPF intervention and nonintervention sites by country 4 Table 2. AgirPF timeline by country 7 Table 3. Final sample sizes for the household survey 11 Table 4. Main languages spoken in the study areas 12 Table 5. Twenty-five AgirPF PMP indicators, data collection areas, and sources 13 Table 6. List of SBCC campaign activities captured in the endline evaluation by country 15 Table 7. Strategic objective output indicators, regional totals 18 Table 8. IR 1 Output indicators, regional totals 19 Table 9. IR 2 Output indicators, regional totals 20 Table 10. IR 3 Output indicators, regional totals 21 Table 11. AgirPF implemented High-impact Practices (HIPs) by country 24 Table 12. Comparison of the background characteristics of the female respondents living in the intervention and nonintervention areas, by country 29 Table 13. Comparison of the background characteristics of the male respondents living in the intervention and nonintervention areas, by country 30 Table 14. AgirPF SBCC campaign interactive voice response (IVR) data in Niger, November 1, 2017 to February 6, 2018 36 Table 15. Percentage of the respondents who heard about specific method, by sex, area, and country 42 Table 16. FP method by zone and country 44 Table 17. Modern FP methods among women by background characteristic, zone, and country 48 Table 18. Modern FP methods among men by background characteristic, zone, and country 49 Table 19. Regression on background characteristics, zone, and country 50 Table 20. Method mix among female users of modern contraceptives, by country 52 Table 21. Method mix among male users of modern contraceptives, by country 53 Table 22. Main reasons cited by women for not using an FP method, by country 54 Table 23. Main reasons cited by men for not using an FP method, by country 54 Table 24. Number of intervention and nonintervention health facilities by health district in the four countries 73 Table 25. Statistics for population sample calculation for the four countries 75 The AgirPF Project Endline Evaluation Report v List of figures Figure 1. Study design 6 Figure 2. Study hypothesis 7 Figure 3. AgirPF PMP indicator distribution by performance level as of July 2017 16 Figure 4. Contraceptive uptake in AgirPF intervention zones by group age, years 2-5 18 Figure 5. Contraceptive method mix among modern FP users in AgirPF intervention sites 19 Figure 6. Achieved mCPR in AgirPF intervention zone compared to the LOP targets 22 Figure 7. Percentage of supervised providers technically deemed competent in REDI counselling, implants and IUD insertion and removal, during PY5Q2, by country 23 Figure 8. Percentage of men who heard or saw a USG-supported FP/RH message by zone and country 26 Figure 9. Percentage of women who heard or saw a USG-supported FP/RH message by zone and country 25 Figure 10. Percentage of the respondents citing lack of knowledge as a key barrier to FP use (either because they do not know about FP at all or they know about FP but lack more information) 27 Figure 11. Percentages of women and men reporting having discussed FP with their partners during the past three months, by zone and country 28 Figure 12. Nigeria segmentation example 32 Figure 13. Percentage of women and men reached by different SBCC activities implemented in Burkina Faso 33 Figure 14. Percentage of women and men reached by different SBCC activities implemented in Côte d’Ivoire 35 Figure 15. Percentage of women and men reached by SBCC activities implemented in Niger 36 Figure 16. Distribution of the calls about basic information for FP methods by method 37 Figure 17. Distribution of the calls to listen to users’ testimonials by method 37 Figure 18. Distribution of the calls inquiring about the methods according to the information requested 37 Figure 19. Percentage of women and men reached by different SBCC activities implemented in Togo 39 Figure 20. Percentage of women and men who have heard about FP methods by zone and country40 Figure 21. Comparison between the percentage of women who know an FP method between the baseline and the endline, by zone and country 40 Figure 22. Percentage of women and men who have heard about modern FP methods by zone and country 41 Figure 23. Percentage of women and men who have heard about LARCs/PM by zone and country41 Figure 24. Contraceptive Prevalence Rate (CPR) among women and men, by zone and country 45 Figure 25. Modern Contraceptive Prevalence Rate (mCPR) among women and men, by zone and country 46 Figure 26. Percentage of women and men currently using LARCs/PMs, by zone and country 46 The AgirPF Project Endline Evaluation Report vi Figure 27. Percentage of the respondents citing lack of knowledge as key barrier to use of FP (either because they do not know about FP at all or they know about FP but lack sufficient information), by zone and country 55 The AgirPF Project Endline Evaluation Report 1 Executive summary Agir pour la Planification Familiale (AgirPF) was a five-year family planning (FP) project funded by USAID and implemented by EngenderHealth in five West African francophone countries. The objective of AgirPF was to increase access to and use of quality FP services in the selected urban and peri-urban areas of Burkina Faso, Côte d’Ivoire, Mauritania, Niger, and Togo. This endline evaluation included endline household surveys conducted in each country (except Mauritania), baseline assessment data, FP service statistics, and related reports. The evaluators combined descriptive and multivariate analytical methods to discern how AgirPF performed against its 25 PMP indicator targets, the effectiveness of AgirPF’s strategic behavior change communication campaigns (SBCC), and whether the intervention impacted FP knowledge and use. Performance against PMP indicators AgirPF’s performance against PMP indicators was positive. The program met or exceeded targets for 78% of the 25 PMP indicators for which data are available across the program’s strategic objective (SO) and three intermediate result areas, despite challenges related to the broad reach of the project, high levels of complexity, and a range of implementation obstacles ranging from office registration issues, to political turmoil, to complex differences in culture and demographic characteristics. All four of the program’s overall targets for SO indicators were met or exceeded by the program, including couple years of protection (CYP), modern contraception prevalence rates (mCPR), total FP users, and new modern contraceptive users. The SO output indicators were particularly impressive for Burkina Faso and Togo, which exceeded all their targets by significant margins. In the case of Burkina Faso, however, this performance masked one of the strangest findings of significance in the evaluation—a low mCPR that actually declined from the baseline assessment (47% to 38%) in intervention areas despite otherwise high-level performance. While this finding matches other recent studies across the country, more research must be done to understand the underlying causes for the drop in the mCPR rate despite the strong performance of the program. Beyond the SO indicators, the program largely excelled under the intermediate result areas. Of particular note under IR 1, the program reached 99% of its training targets, with 5,699 people trained in FP and RH. The program was also particularly successful at reaching youth, with over 318,101 participating in education programs (118% of targets). AgirPF was even more successful at meeting targets for evidence-based service delivery as defined under IR 2. Facilities in Burkina Faso, Côte d’Ivoire, Niger, and Togo accepted and welcomed innovations in their facilities and implemented 10 high-impact practices (HIPs), many of which were integrated into national health protocols. The program also over doubled the target number of regional technical trainings over the life of the program. Targets under IR 3 were all met or exceeded by the program, which highlighted AgirPF’s extremely effective approach to removing policy barriers around FP and contraceptives. Advocacy conducted under IR 3 led to 19 policies or guidelines changed to improve access to FP and RH services The AgirPF Project Endline Evaluation Report 2 (317% of targets), 41 formal agreements signed at the regional level (273% of targets), and over 101 advocacy activities conducted (289%), among other achievements. AgirPF SBCC campaigns In addition to PMP performance, the endline evaluation also had a special focus on evaluating the impact of AgirPF’s multi-faceted SBCC campaigns based on an innovative approach of segmenting the population according to their attitudes, behaviors, and potential in terms of FP use. The campaign reached wide audiences (up to 75% of those living in Niger’s intervention areas were reached with a USG-supported FP/RH message, 82% in the nonintervention areas) through a variety of methods, yet evaluating the SBCC campaign was virtually impossible given the contamination of the nonintervention areas by the activities implemented by AgirPF. Most of the SBCC activities (posters, radio spots, Facebook pages, comic books, etc.) could not be confined to a specific area given that intervention and nonintervention areas were generally in the same urban/peri-urban districts. Because the campaigns began in years 3 and 4 of the program, there was a lack of sufficient baseline data with which to track changes in attitudes and behavior post-campaign. The extremely varied techniques, themes, and targeted audiences also defied easy analysis. Further research to examine whether the SBCC campaigns contribute to increased levels of FP information-seeking, positive attitudes about FP, and actual FP use need to be conducted to definitively establish effectiveness. Impact on FP Despite strong performance on many output and outcome indicators in the PMP, it was difficult to demonstrate that AgirPF made a difference in terms of FP knowledge and contraceptive prevalence because of the proximity of intervention and nonintervention areas. For example, an AgirPF beneficiary could easily attend health facilities and events in a nearby nonintervention area led by another Ministry of Health (MOH) partner. What is more, nonintervention areas were not control zones, and were often heavily targeted by other programs related to improving FP methods. In large part due to these contamination effects, logistic regression could not show any difference in FP outcomes based on living in an AgirPF intervention zone on mCPR after controlling for background characteristics, except for a negative finding for women in Burkina Faso. There were some small statistically significant variances in mCPR, FP method mix, and use of long-acting reversible contraceptives/permanent methods among and between countries based on some background characteristics. Analysis by zone does show more opposition to contraceptive use in the nonintervention than in the intervention areas in Côte d’Ivoire (16.6% vs 6.8%), Niger (17.6% vs 13%), and Togo (11% vs 8.8%), much may be a result of AgirPF educational activities. The evaluators recommend further research to determine whether the SBCC campaigns and broader education activities contributed to increased levels of FP information-seeking, positive attitudes about FP, and actual FP use. It will be critical to take into account evaluation challenges such as baseline data collection and potential contamination effects on nonintervention sites (and vice versa) during the design stage of future projects to best understand USAID impact. The AgirPF Project Endline Evaluation Report 3 Introduction Despite the dramatic increase in the availability and use of family planning (FP) worldwide over the past fifty years, contraceptive use in much of francophone West Africa remains low while unmet need remains high. According to the most recent 2013 Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), which coincided with the launch of the AgirPF project, the modern contraceptive prevalence rate (mCPR) was 14% to 34% in the urban areas of Burkina Faso, Côte d’Ivoire, Mauritania, Niger, and Togo, while unmet need ranged from 21% to 37%. On July 5, 2013, the USAID/West Africa Regional Health Office awarded a five-year, $29 million cooperative agreement called the Agir Pour la Planification Familiale (AgirPF) project to EngenderHealth and its partner, Avenir Health (former Futures Institute). The goal of AgirPF was to enable women of reproductive age (15-49 years old) to make and voluntarily act on informed decisions about family planning (FP). Zones of intervention included the urban and peri-urban areas of five francophone West African countries including Burkina Faso, Côte d’Ivoire, Mauritania, Niger, and Togo. The project team worked closely with Ministries of Health (MOHs) and other local partners to support the national action plans for strengthening FP that followed the February 2011 Francophone West Africa Regional Conference on Population, Development, and Family Planning in Ouagadougou, Burkina Faso. The current report analyzes data from the endline evaluation study along with other sources of information generated by the project to assess AgirPF performance and effectiveness through changes over time in key services, enabling environment, and demand indicators. The first part of the report gives a detailed overview of the project and presents the objectives and methodology of the evaluation. The second part presents the results, conclusions, and lessons learned. Overview and methodology As the objective of the AgirPF project was to increase access to and use of quality FP services in the five participating francophone West African countries, the project strengthened public, private, and NGO facilities to provide a wide range of FP services, including integrated FP/MNCH services and services for youth and men. Beneficiaries The target beneficiaries of AgirPF interventions were women of reproductive age (WRA) from 15-49 years old living in the urban and peri-urban areas of the 10 largest cities in the 5 countries (see Geographic scope below). These beneficiaries included post-partum women, women seeking post￾abortion care services, and first-time parents. Target beneficiaries also included men and youth, The AgirPF Project Endline Evaluation Report 4 especially through EngenderHealth’s work with male FP service professionals and through the project’s emphasis on “youth-friendly” services. Geographic scope EngenderHealth implemented AgirPF in the 10 largest cities (80,000+ population) of the five participating countries, with the exception of Zinder, Niger, which was not selected due to safety concerns. Cities included Ouagadougou, Bobo-Dioulasso, and Koudougou in Burkina Faso; Abidjan in Côte d’Ivoire; Nouakchott in Mauritania; Niamey and Maradi in Niger; and Lomé, Sokodé, and Kara in Togo. In consultation with MOH authorities, AgirPF selected 263 intervention and 148 nonintervention health facilities (HFs). Table 1 provides the number of cities, health districts, and intervention and nonintervention health facilities (HFs) by country. Table 24 in Annex B gives a complete list of the intervention and nonintervention HFs. Table 1. Number of AgirPF intervention and nonintervention sites by country Countries # cities # Health Districts # Intervention Health Facilities # Nonintervention Health Facilities Burkina Faso 3 8 57 31 Côte d’Ivoire 1 10 79 48 Mauritania 1 9 43 13 Niger 2 9 36 32 Togo 3 9 48 24 TOTAL 10 45 263 148 The team determined the intervention zones of the project based on a set of five criteria.1 Similarly, the nonintervention zone consisted of the catchment areas of the nonintervention HFs. Apart from the intervention HFs, AgirPF also supported 21 private clinics including six in Burkina Faso, 10 in Niger, and five in Togo, upon request from USAID. Description of the project intervention 1 AgirPF’s intervention zones selection criteria: - Population size of the district: 5,000 people or more, - Level of CPR in the city districts: less than 10%, - Activities carried out in the Health Facilities (facilities that do not offer counseling, offering only short acting methods (pills, injectable, condoms and spermicides), poor quality of services, and/or poor infection prevention practices), and - The absence of other international development donors/partners in the zones and their domain of intervention to avoid duplication and emphasize complementarity The AgirPF Project Endline Evaluation Report 5 To achieve its objectives, AgirPF’s intervention consisted of five broad activities: i. Improve FP service quality by establishing Centers of Excellence in each capital city to train providers and supervisors in the following areas: a. Provision of gender sensitive, male and youth-friendly, rights-based FP services b. COPE® quality improvement process c. Facilitative supervision d. Training of nurses, doctors, and midwives to provide implants and IUDs in addition to short-acting methods ii. Bring FP services to underserved communities through: a. Mobile outreach services and FP special days, offering a wide range of contraceptives including implants and IUDs b. Community health workers (CHWs) providing first-offer condoms, pills, and injectables (in Togo and Niger) and then immediately transitioning clients to sustainable sources of resupply. CHWs also led community discussions on sexual and reproductive health and FP, as well as reflection sessions about gender norms as they relate to FP. iii. Educate and empower clients and grassroots advocates, by using gender transformative social and behavior change communication (SBCC); men as partners (MAP) tools; and Avenir Health training for CSO/district advocates to deliver FP information to CHWs and clients. iv. Reduce financial barriers by collaborating with MOHs and other partners to provide dedicated FP services at low or no cost (e.g. via FP special days and mobile services) in AgirPF countries where services are not free v. Solve logistics issues and estimate commodity needs in collaboration with the USAID DELIVER project by training: a. Facility staff in the Client-Oriented Provider-Efficiency (COPE) for Contraceptive Security (CS) method b. Contraceptive procurement table (CPT) teams to use Reality Check (√), a tool for assisting FP professionals to plan and advocate based on informed estimates of need, by examining the relationship of contraceptive prevalence and population to numbers of family planning users, adopters, and commodities.2 Monitoring and evaluation To measure the performance and effectiveness of activities, AgirPF planned three studies including the baseline assessment (2013-2014), mid-term performance evaluation (2016), and endline evaluation (2018).3 The monitoring and evaluation (M&E) team used quasi-experimental methods with non￾equivalent control groups to measure changes between the baseline and endline via population-level indicators following the AgirPF intervention. The team matched the experimental and control groups 2 https://www.engenderhealth.org/pubs/family-planning/reality-check/ 3 The evaluation team did not conduct a mid-term performance evaluation in Mauritania due to delays in project start￾up. The AgirPF Project Endline Evaluation Report 6 by sample age distribution for the household surveys and type of health facility for the facility audits. Matching reduced study bias and increased the level of confidence in measuring the final impact of the intervention. However, the selection of intervention and nonintervention health facilities in the same health district resulted in increased spillover effects of the intervention activities on the nonintervention group, as is discussed at length in the Results and Discussion and Conclusion chapters. The study design (figure 1) depicts the nature and timing of intervention activities introduced by AgirPF (X) as well as measurement of dependent variables (0). The team measured 01 and 04 during the baseline assessment, and 03 and 06 during the endline evaluation. Due to changes in the mid-term evaluation methodology, the team did not measure the 02 and 05 household variables. Figure 1. Study design Time 48 months Experimental Group 01 Xa 02 Xa 03 ------------------------------------------- Nonequivalent Control Group 04 05 06 Xa : Intervention activities introduced by AgirPF 01 & 04 : Baseline measurement of dependent variables, representing the existing situation and providing control measures for the experimental group (AgirPF intervention zones) and nonequivalent control group (nonintervention zones) 02 & 05 : Mid-term measurement of dependent variables, representing changes in variables in experimental and comparison zones, respectively. 03 & 06 : Endline measurement of dependent variables, representing changes in variables in experimental and comparison zone, respectively. -------- : Indicates that the two groups were not randomly assigned : Intervention duration The AgirPF project lasted five years but intervention time varied by country from 24 months in Mauritania to 48 months in Togo. Table 2 shows the duration of start-up, intervention, and close-out activities in each country. The AgirPF Project Endline Evaluation Report 7 Table 2. AgirPF timeline by country Country Start Date End Date Duration Start- (months) up Intervention Activities Begin / Reporting in DHIS2 Intervention Activities End Close-out Burkina Faso Jul. 2013 Oct. 2014 Mar. 2018 Jul. 2018 42 Côte d’Ivoire Nov. 2014 Mar. 2015 Mar. 2018 Jul. 2018 37 Mauritania Feb. 2014 Mar. 2016 Mar. 2018 Jul. 2018 25 Niger Jul. 2013 Oct. 2014 Mar. 2018 Jul. 2018 42 Togo Jul. 2013 Oct. 2014 Mar. 2018 Jul. 2018 42 Figure 2. Study hypothesis O1 = O4 ≤ O5 ≤ O6 < O2 < O3 The study hypothesis (figure 2) posits the following: a. Women living in urban and peri-urban areas AgirPF intervention areas will have higher levels of family planning knowledge and utilization of family planning and other reproductive health services after their introduction than before: O1 < O2 < O3. b. Knowledge and use of reproductive health services among women will be higher in urban and peri-urban areas where AgirPF introduces community-level and clinic-strengthening interventions than where there are no AgirPF intervention activities: O5 < O2 andO6 < O3. c. Men and women living in urban and peri-urban areas where AgirPF intervenes will have higher levels of dialogue with their partner about FP and other reproductive health services after their introduction than before: O1 < O2 < O3. d. The proportion of men reporting dialogue about FP with their wife/partner will be higher in urban and peri-urban areas where AgirPF intervenes than where there are no AgirPF intervention activities: O5 < O2 andO6 < O3. Baseline assessment AgirPF conducted the baseline assessment study in four of its five implementation countries (Burkina Faso, Niger, Togo and Côte d’Ivoire)4 in order to inform the development of baseline indicators and to guide future AgirPF programming around three identified areas: (1) supply, (2) enabling environment, and (3) demand. The baseline assessment identified priority areas and existing strengths 4 Mauritania was not part of this study due to registration challenges in the country that prevented programming from starting until later in the project. The AgirPF Project Endline Evaluation Report 8 and best practices on which to build and comprised (i) a health facility audit; (ii) a key informant qualitative survey; (iii) a community health worker qualitative survey; and (iv) a quantitative household survey. The household survey participants included women at reproductive age (15-49 years old) and men (aged 15-59) and examined their knowledge, attitudes, and practices (KAP) related to FP as well as their gender attitudes. Specific indicators included (i) mCPR, (ii) family planning discussion among partners, and (iii) exposure to family planning messages. The findings formed the basis for a range of program intervention recommendations in each country. The assessment recommending bringing Côte d’Ivoire, which lagged behind other project countries, up to expected standards by (1) creating demand for FP through SBCC-related activities for men and women; (2) equipping facilities with IEC materials; (3) improving the quality of basic training and conducting in-service training of providers in FP; (4) improving availability of supplies; and (5) strengthening the waste disposal system and training staff in infection prevention. For Burkina Faso, the recommendations included: (1) increasing the availability of IUDs, in particular postpartum IUDs, by providing staff training and equipment and supplies and (2) equipping facilities with equipment needed for implants and PAC. Focus areas identified for Niger included achieving a higher CPR and a more balanced contraceptive method mix. Specific recommended interventions included (1) increasing knowledge and demand for condoms; (2) increasing knowledge of and demand for the standard days method (SDM) and natural family planning; and (3) investigating and removing demand￾side barriers to use of hormonal methods. Last, for Togo, recommendations included: (1) training/re￾training providers of PAC services; (2) equipping the facilities with PAC equipment; (3) improving supplies of cycle beads and female condoms; and (4) increasing demand and awareness for certain FP methods such as implants, SDM, and emergency contraceptives. Mid-term performance evaluation The USAID Evidence for Development (E4D) project carried out the mid-term performance evaluation in all countries except Mauritania where the team completed the baseline study a few months prior to the midterm evaluation. The purpose of the mid-term evaluation was to determine the extent to which the AgirPF portfolio met its overarching objectives. The evaluators sought to document whether the AgirPF project (i) was on track for achieving its intended results; (ii) advanced select high impact practices; (iii) achieved the intermediate results necessary for meeting the final expected results; and (iv) had relevant successes, challenges, and lessons learned. As such, the mid-term evaluation used a mixed methods approach, which included quantitative data from the program record files, AgirPF District Health Information System 2 (DHIS2) database, as well as direct data collection on FP service delivery from a representative sample of AgirPF and comparison sites. Qualitative data in the form of purposive stakeholder interviews (including AgirPF staff, AgirPF trained providers, MOH partners, district and regional health managers, local partners and consortium partners) and analysis of quarterly reports were also used to triangulate and verify The AgirPF Project Endline Evaluation Report 9 quantitative findings and to answer specific evaluation questions. The evaluation concluded that the AgirPF project was on track to meet its pre-set family planning service targets. The evaluation also found that AgirPF had contributed important support to the regional advancement of improved family planning access and utilization through a combination of regional training, networking and exchanges as well as harmonized in-country efforts aimed at training providers, supporting policy advocacy, creating or increasing domestic budgets dedicated to FP, improving logistics management, and behavior change activities. Endline evaluation The endline evaluation assessed the project’s effectiveness and changes over time in key services, the enabling environment, and demand indicators in four of the five implementation countries of Burkina Faso, Cote d’Ivoire, Niger, and Togo. Mauritania was not included as project implementation began two years later than the other countries and was limited for policy reasons on some project objectives and activities. The study consisted of a light household survey that used the same methodology as the baseline assessment but focused on the measurement of the following three PMP indicators as well as a USAID demand generation indicator used for reporting to the US Congress: • Indicator 2: Modern Contraceptive Prevalence Rate (mCPR); • Indicator 14: Proportion of women citing lack of information on FP methods as a key barrier to use; • Indicator 15: Proportion of women and men reporting increased dialogue with their partner about FP; and • USAID Custom Indicator: Percent of audience who recall hearing or seeing a specific USG￾supported Family Planning/Reproductive Health (FP/RH) message. The household survey also included questions regarding respondents’ age, sex, and marital status, duration of residence, education, and number of living children. Overall objectives The overall objective of the evaluation was to assess the performance and effectiveness of the project’s intervention and changes over time in key services, the enabling environment, and demand indicators in line with the baseline study and the mid-term evaluation results and recommendations. Specific objectives • Measure AgirPFs progress against Performance Monitoring Plan (PMP) indicators over the course of the project • Analyze how changes in the population-level indicators are linked to AgirPF’s intervention • Contribute to the evaluation of the AgirPF SBCC campaign The AgirPF Project Endline Evaluation Report 10 Methodology The main data sources for the endline evaluation are household data, facility audit data, and key respondent interviews (NGOs managers and DHMTs). Other sources include the following: • AgirPF baseline assessment (for the trend analysis of some population-based indicators such as the CPR); • Server of the toll-free line set up in Niger by AgirPF to provide information on FP; • AgirPF DHIS2 database containing FP service statistics collected monthly in AgirPF intervention facilities; and • AgirPF monthly, quarterly, and annual reports as well as Mid-Term Evaluation Results. Sampling procedures The endline evaluation utilized the same methodology as the household survey of the baseline study, thus allowing for similar sampling sizes and methodologies. For example, both surveys included multi￾stage stratified sampling for women whereby each health facility catchment area represented a stratum. First, the team selected enumeration areas (EA) in each stratum.5 Second, the evaluators randomly selected a minimum of 20 households in the EA. Within the household, the evaluators selected all females aged 15-49 years old. The team systematically sampled men by inviting the partner or spouse of the woman interviewed to participate in the survey to demonstrate whether the AgirPF intervention had a significant impact on men’s dialogue with their partners about FP at the country-level. Appendix C provides reporting based on the available estimates of each variable. To calculate the minimum sample sizes needed to demonstrate whether AgirPF had a statistically significant effect on increasing modern contraceptive use among women of reproductive age, the evaluators: • Utilized the key indicator, “proportion of women currently using a modern contraceptive method”; and • Employed the Epi Info Stat-Calc Sample Size and Power program for simple random sampling assuming: (i) an average annual women’s CPR increase of 2 percent, (ii) a two-sided confidence interval of 95 percent, and (iii) a power of 80 percent. Finally, the evaluators adjusted the minimum sample sizes needed for each focus city to account for the estimated response rate for sampling a minimum of 20 households at the final stage. 6 Appendix C includes sample size calculations as well as estimates of CPR and other parameters. 5 The national statistical offices in each country delineate enumeration areas. 6 In a two-stage sampling design, DHS survey data suggest that the optimal sample size is 22-25 households where there are 0.90 women of reproductive age per household and 14-16 where there are 1.40 women of reproductive age per household. A loss of precision will result where the sample size is less than optimal. See http://dhsprogram.com/pubs/pdf/WP30/WP30.pdf, last accessed 26 March 2014. The AgirPF Project Endline Evaluation Report 11 Table 3 shows the final sample sizes for the household survey in each country. The table gives the distribution of facilities concerned by the survey by country and by zone of intervention (intervention and nonintervention). It also provides information on the number of men and women entering the country by country. Table 3. Final sample sizes for the household survey City/ Health Districts Intervention Nonintervention Total Number of Facilities Minimum Sample Needed* Actual Sample Needed** Number of Facilities Minimum Sample Needed* Actual Sample Needed** Women Men All Burkina Faso Ouagadougou 33 253 660 22 253 440 1,100 550 1,650 Bobo Dioulasso 13 298 298 8 298 298 596 298 894 Koudougou 8 202 202 8 202 202 404 202 606 Total Burkina Faso 54 753 1,160 38 753 940 2,100 1,050 3,150 Côte d’Ivoire Abidjan 8 270 8 270 360 180 540 Total Côte d’Ivoire 82 2,802 50 1,688 2,994 1,496 4,490 Niger Maradi 21 293 420 14 293 294 714 358 1,072 Niamey 15 255 300 15 255 300 600 300 900 Total Niger 36 548 720 29 548 594 1,314 658 1,972 Togo Lomé 19 202 380 9 202 201 581 291 872 Sokodé 14 176 280 7 176 176 456 228 684 Kara 15 198 300 8 198 198 498 249 747 Total Togo 48 576 960 24 576 575 1,535 768 2,303 TOTAL 110 1,641 3,240 87 1,641 2,409 5,649 2,825 8,475 * Minimum sample is based on sample size calculation by city with proportional distribution by clinic catchment area. ** Actual needed based on minimum 20 households per clinic catchment area. If (# clinics in district * 20) < than minimum sample, then actual sample = minimal sample. Data collection and entry Four private firms (one for each country) collected the household data for the endline survey between April and June 2018.7 For security purposes, each company underwent a background check and signed an agreement not to share any information about the project gathered from the surveys. EngenderHealth conducted a rigorous in-house ethical review process of study protocols and tools as did the USAID mission and local host country ethics committees. The firms worked under the supervision of AgirPF Country Managers, the M&E/Research Advisor, and the Regional M&E/Research Officer. AgirPF staff checked data collection process quality and supplied predetermined information and survey tools to reduce the potential risk of data reporting 7 Two of the endline household data collection firms also conducted the baseline assessment in Burkina Faso and Togo. The AgirPF Project Endline Evaluation Report 12 bias. Orientation sessions for the firms included training in mandatory ethics, study protocols, data collection guides, and survey tools.8 After household enumeration and the selection of an eligible respondent in each household surveyed, trained interviewers conducted private structured interviews with respondents in French or the local languages listed in table 4.9 Table 4. Main languages spoken in the study areas Country Local languages Burkina Faso French, More, and Dioula Côte d’Ivoire French, Dioula Niger French, Haoussa, and Zarma (ou Djerma) Togo French, Ewe, Mina, Cotocoli, and Kabye The interviewers emphasized that respondent participation was voluntary and obtained the informed consent of each participant. The team stored all completed data collection forms and data files without the names of respondents to maintain confidentiality. Oversampling compensated for respondents that declined to be interviewed. AgirPF provided the tablets and CSPro computer-assisted personal interviewing (CAPI) applications for the interviewers. The CAPI application integrated quality controls checked in real time during the interview. Data analysis Data were collected at the population level during the baseline assessment and the endline evaluation surveys by interviewing women in the reproductive age (15-49 years) and men aged 15-59 years living in the project intervention and nonintervention areas. Further, a portion of the data collection occurred routinely at the project’s intervention health facilities, while the remaining indicators came from activity reports provided by project staff or by its partners. The measurement of the performance against the PMP Indicators is described in the Results chapter below. Household survey data analysis The analysis combined descriptive and explicative methods, based on univarate, bivariate, and multivariate techniques (chi-2 test, T-test, and logistic regressions). The AgirPF M&E/Research Advisor and M&E/Research Officer began the household data analysis following the data collection and cleaning performed using SPSS software packages. To assess whether 8 AgirPF translated the data collection instruments from English to French and local languages. 9 French and local language fluency was one of the selection criteria for the supervisors and interviewers in each country. The AgirPF Project Endline Evaluation Report 13 or not AgirPF reached its objectives, the team (i) entered and analysed the household data, and (ii) compared the indicators with their respective life-of-project targets as well as across the intervention and nonintervention zones while controlling for confounding factors such as the respondent’s background characteristics. DHIS2 and report analysis The AgirPF M&E team computed the 25 remaining performance monitoring plan (PMP) indicators using data retrieved from AgirPF DHIS2 database and activity reports(see table 5). These country- or regional￾level indicators were disaggregated by country and, depending on the indicator, other variables such as sex, service delivery type, contraceptive method, and training area. The team calculated the level of performance based on the percentage of the LOP targets achieved by the last reporting month of the project (June 2018). In a few cases10, the team could not determine progress against indicators because of missing indicator targets or unavailable data. For instance, without data from the endline evaluation survey (completed in July 2018), the team could not measure population-level indicators. Performance was measured as follows: • Good: 80% or more of the LOP target achieved; • Moderate: 70% to 79% of the LOP target achieved; • Poor: Less than 70% of the LOP target achieved; and • Not Measured: Target not set, data missing, or indicator not applicable. Table 5. Twenty-five AgirPF PMP indicators, data collection areas, and sources # Indicator Data collection area Data Sources Region Burkina Faso Côte d’Ivoire Mauri￾tania Niger Togo 1 Number of CYP achieved in AgirPF supported areas X X X X X X Service statistics 2 Modern contraceptive prevalence rate (mCPR) n/a X X X X X Baseline & Endline surveys 3 Total number of FP method users X X X X X X Service statistics 4 Number of additional users of modern methods of contraception X X X X X X Service statistics 5 Percentage of FP service providers deemed technically competent based on an assessment according to national, international or other defined standards X X X X X X Supervision reports 6 Number of local organizations with improved organizational and management capacity as X X X X X X Activity reports 10 The cases where AgirPF was unable to determine the progress of the indicators include: (i) the number of HIV positive women who received comprehensive FP services, (ii) the proportion of men and women with gender-equitable attitudes and (iii) the number of service delivery points (SDPs) reporting stock-outs of contraceptives per quarter. The AgirPF Project Endline Evaluation Report 14 # Indicator Data collection area Data Sources Region Burkina Faso Côte d’Ivoire Mauri￾tania Niger Togo measured by a defined organizational assessment tool 7 Number of FP curricula updated to include gender sensitivity, couple counseling, youth￾and male-friendly services X X X X X X Activity reports 8 Number of people trained in FP and reproductive health with USG funds X X X X X X Activity reports 9 Number of HIV positive women who received comprehensive FP services n/a n/a X n/a n/a n/a Service statistics & Activity reports 10 Number of FP special days conducted X X X X X X Activity reports 11 Number of additional USG-assisted community health workers (CHWs) providing family planning information and/or services during the year X n/a X n/a X X Activity reports 12 Proportion of women and men reporting increased dialogue with their partner about FP n/a X X X X X Baseline & Endline surveys 13 Proportion of men and women with gender￾equitable attitudes n/a X X X X X Baseline & Endline surveys 14 Percent of women citing lack of information on FP methods as a key barrier to use n/a X X X X X Baseline & Endline surveys 15 Percent of women that discussed FP with husbands/partners, friends/family within the last 15 days n/a X X X X X Baseline & Endline surveys 16 Number of youth that participate in educational program on gender, FP, and SRH X X X X X X Activity reports 17 Number of site walk-throughs (SWTs) conducted X X X X X X Activity reports 18 Number of HIPS/BPs for family planning and maternal and child health and/or HIV/AIDS incorporated into national health protocols or standards X X X X X X Activity reports 19 Number of HIPs piloted through implementation research X X X X X X Activity Reports 20 Number of regional technical meetings organized and supported by AgirPF and its partners X n/a n/a n/a n/a n/a Activity reports 21 Number of policies or guidelines developed or changed with USG assistance to improve access to and use of family planning and reproductive health services X X X X X X Activity reports 22 Number of advocacy presentations created or X X X X X X Activity The AgirPF Project Endline Evaluation Report 15 # Indicator Data collection area Data Sources Region Burkina Faso Côte d’Ivoire Mauri￾tania Niger Togo updated with support from AgirPF reports 23 Number of advocacy activities conducted X X X X X X Activity reports 24 Number of formal agreements (MoU, policy, declaration, etc.) signed at the regional level X X X X X X Activity reports 25 Number of service delivery points (SDPs) reporting stock-outs of contraceptives per quarter X X X X X X Service statistics SBCC campaign evaluation overview One of the objectives of the endline survey was to evaluate the AgirPF SBCC campaign via the following indicator: “Percent of audience who recall hearing or seeing a specific USG-supported family planning/reproductive health (FP/RH) message.” This indicator was not part of the PMP, but was discussed with and accepted by USAID specifically for the SBCC activities. As such, the team prepared survey questions for the target audience in each country, administered the survey by phone, and recorded the phone calls on the IVR server. Recorded data included the number of calls, completion status, duration, and topics. Table 6 includes the list of SBCC campaign activities captured in the endline evaluation by country. Further analysis of the SBCC campaign evaluation can be found in the corresponding section of the Results chapter below. Table 6. List of SBCC campaign activities captured in the endline evaluation by country Countries SBCC Activities Burkina Faso 1. The Nintriga campaign that consisted of radio talk, a song, and a video clip on FP. 2. The “Ali et Ténè” comic book about youth and adolescent sexuality. 3. The “Ali et Ténè” Facebook page containing posts and videos about Nintriga and about FP in general. Côte d’Ivoire 1. Couples discussion and experience sharing about FP using videos about couple interaction relating to FP. 2. The “Safi et Leo” comic book about youth and adolescent sexuality. 3. The “Safi et Leo” Facebook page that contains posts and videos about FP. 4. Youth talk (“Causerie avec les jeunes”) about FP. 5. The "Plan My Life" mobile application. Niger 1. The Inter Voice Reponses system (IVR), a toll-free number one can call to obtain free information about FP. 2. Distribution of coupons to obtain FP services free of charge at health centers. 3. Posters and radio spots about the management of side effects of FP methods. Togo 1. Couples discussion and experience sharing about FP using videos about couple interaction relating to FP. 2. The “Assibi et Salifou” comic book about youth and adolescent sexuality. 3. Posters and radio spots about the management of side effects of FP methods. The AgirPF Project Endline Evaluation Report 16 Results Progress against PMP indicators The project team developed a Performance Monitoring Plan (PMP) to measure indicators across the project based on USAID’s global monitoring guidelines and framework.11 AgirPF’s approach to monitoring prioritized data from existing sources, such as service statistics and existing data collection tools. As such, AgirPF coordinated with partners to conduct a participatory review of processes, flow, and reporting systems for FP data in the Health Management Information System (HMIS) of each country and developed monitoring tools and the District Health Information System version 2 database (DHIS2). USAID approved the AgirPF PMP reflecting the USAID/WA Regional 2015 Cooperation Development Strategy (RCDS), global monitoring guidelines, and accompanying list of indicators. AgirPF developed 25 output and outcome indicators to evaluate the impact of its technical assistance at the intervention site level grouped under the Strategic Objective (SO) of increasing access to and use of quality FP services in the selected urban and peri-urban areas of the five countries and three Intermediate Results (IRs): (1) delivery of quality FP information, products, and services strengthened and expanded, (2) evidence-based service delivery approaches selected, adapted, and implemented, and (3) efforts to remove policy barriers and improve contraceptive commodity security coordinated. On average, AgirPF performed well on 78% of the 25 PMP indicators but moderately and poorly on 6% and 17%, respectively.12 Figure 3 provides details regarding indicator performance against targets (see Annex D for more detail, including indicators for which data were unavailable). There were significant variances in meeting targets across the countries. For example, Togo performed well on 88% of its indicators, followed by Niger at 76%; and Burkina Faso at 71%. Côte d’Ivoire (29%) and Mauritania (57%) had the lowest percentages indicators with good performance, which is consistent with their later program implementation start dates and more 11 The Performance Monitoring Plan (PMP) outlines the criteria that will be used to assess the outputs and outcomes of the AgirPF project in five francophone countries of West Africa between July 5, 2013 and July 4, 2018. It is based on USAID’s DQA tools and MEASURE’s Data Demand and Use Toolkit—in particular, the proven Performance of Routine Information System Management (PRISM) Framework and tools evaluated by the MEASURE Evaluation. 12 Performance levels include those for which data are available Figure 3. AgirPF PMP indicator distribution by performance level as of July 2017 The AgirPF Project Endline Evaluation Report 17 significant barriers to effective implementation (discussed below). Moderate performance was less common in each country, ranging from 0% in Togo to 12% in Côte d’Ivoire and Niger. Output Results Throughout implementation, the project tracked 18 output indicators across the SO and three project IRs; AgirPF met or exceeded 15 of the 18 indicators (within a USAID/WA and EngenderHealth agreed upon acceptable margin of 20%), reflecting an 83% achievement of its expected results. AgirPF’s combined work in capacity building and evidence generation based on the implementation of HIPs by health authorities enabled this high performance. The project maintained a results-oriented approach throughout implementation and ensured that each activity contributed to the SO. Note that the output results are obtained from AgirPF intervention sites only via DHIS2 data and therefore do not concern the non-intervention sites. Results from Mauritania are also not taken into account given that implementation started much later than at other sites and that there were limitations on programming (see Overview above). Strategic objective: output results Three output indicators were related to AgirPF’s primary SO of increasing the access to and use of quality FP methods: the number of CYP achieved (indicator 1), the total number of FP method users (indicator 3), and the number of new users of modern methods of contraception/FP (indicator 4). Table 7 describes AgirPF performance towards these three indicators. The endline evaluation results show that AgirPF performed well across most of the four indicators but with marked variations between countries. The data demonstrates that indicator performance across the three SO output indicators (indicators 1, 3, and 4) was exceptionally strong in Burkina Faso (152%, 106%, and 123%, respectively) and Togo (133%, 105%, 129%), mostly successful in Mauritania (35%, 101%, and 78.5%) and Niger (63%, 93%, and 90%), and weak in Côte d’Ivoire (34%, 64%, and 41%). The reasons for these discrepancies can largely be found in the unique challenges and cultural contexts of each of the countries, and did not represent failures of program implementation. For example, in Côte d’Ivoire 24% of sites (20 out of 84 sites) had never provided FP services of any kind. AgirPF therefore spent considerable time integrating FP service provision in these facilities by training facility management teams and providers as well as equipping and supervising the facilities. These activities required a minimum of six months to complete, and while critical for laying the groundwork to achieve project results, meant that AgirPF did not have time to progress to the range of activities implemented in the other three countries, particularly because Côte d’Ivoire joined the project one year later than Burkina Faso, Niger, and Togo. Political unrest, including the 2016 terrorist attacks, also disrupted programming. The AgirPF Project Endline Evaluation Report 18 In Mauritania and Niger, AgirPF interventions had strong performances resulting in 83% and 92% average achievement rates for total and additional users, respectively. However, in terms of CYP, the project achieved much lower results (35% in Mauritania and 63% in Niger). One of the main reasons for low CYP performance in Mauritania and Niger was the preference for short-acting contraceptives as opposed to LARCs/PMs. CYP is the estimated protection provided by contraceptive methods during a one-year period and is based upon the volume of all contraceptives sold or distributed free of charge to clients during that period. 13 CYP is calculated by multiplying the quantity of each method distributed to clients by a conversion factor which is based on how a method is used, failure rates, wastage, and how many units of the method are typically needed to provide one year of contraceptive protection for a couple. Therefore, the more the clients use short-acting methods, the lower the CYP. Table 7. Strategic objective output indicators, regional totals Indicator Number Indicator Target Achieved Rate 1 Number of CYP achieved in AgirPF supported areas 1,683,000 1,367,980 81% 3 Total number of FP method users 1,658,396 1,505372 91% 4 Number of additional users of modern methods of contraception 687,193 574,974 84% IR 1: Output results Results for IR 1 (“Delivery of Quality FP Information, Products, and Services Strengthened and Expanded) outputs are largely captured by SO output indicators, as well as indicator 8 which tracks the number of people trained in FP and RH with USG funds. Together, these indicators demonstrate the reach and expansion of the program along with other indicators in table 8. Apart from general expansion, it is interesting to note the increase in youth (age 10 to 24 years old) contraceptive uptake and participation in related activities held at schools, youth associations, youth clubs, among other places. Figure 4 shows that from PY2 to PY5, 45% of the new clients in the region were young people. New users between 10 and 24 years old at the country level comprised 50% in Burkina Faso, 44% in Côte d’Ivoire, 56% in Niger, and 31% in Togo. These results can be clearly seen in 13 https://www.usaid.gov/what-we-do/global-health/family-planning/couple-years-protection-cyp Figure 4. Contraceptive uptake in AgirPF intervention zones by group age, years 2-5 45% 50% 44% 56% 31% 55% 50% 56% 44% 69% Region Burkina Faso Cote d'Ivoire Niger Togo Group Age of 10-24 Years Group Age of 25-50+ years The AgirPF Project Endline Evaluation Report 19 indicator 16 tracking youth participation; AgirPF reached 318,101 youths in the region which was 118% of the target. The shift towards LARCs in the method mix also suggests a positive change regarding the quality of FP information, access to services, and service provision. Figure 5 shows that the contraceptive method mix changed significantly between the 2015 baseline and December 2017. The main change is the 20% decrease in the share of pills (from 44% to 24%) balanced by a substantial increase in the percentage of injectables (from 27% to 40%), implants (from 17% to 28%) and IUDs (from 4% to 8%). The increased use of LARCs is likely due to expanded availability and affordability. For example, LARCs are more widely available now than at the start of project, in terms of both the commodities and trained service providers. AgirPF sites also offered LARCs free of charge during FP special days (indicator 10). Table 8. IR 1 Output indicators, regional totals Indicator Number Indicator Target Achieved Rate 6 Number of local organizations with improved organizational and management capacity (OCAT) 42 21 50% 7 Number of FP curricula updated to include gender sensitivity, couple counseling, youth and male friendly services 5 5 100% 8 Number of people trained in family planning and reproductive health with USG funds 5,784 5,699 99% 9 Number of HIV positive women who received comprehensive FP services 1,300 651 50% 10 Number of special FP days conducted 2,235 1,854 83% 11 Number of additional USG-assisted CHWs providing family planning information and/or services during the year 505 792 156% Figure 5. Contraceptive method mix among modern FP users in AgirPF intervention sites The AgirPF Project Endline Evaluation Report 20 Indicator Number Indicator Target Achieved Rate 16 Number of youths who participate in educational programs on gender, FP, and SRH 270,000 318,101 118% 17 Number of Site Walk-Throughs (SWT) conducted 329 136 41% IR 2: Output results Under IR 2, AgirPF sought to select, adapt, and implement evidence-based service delivery approaches. IR 2 output indicators demonstrate the extent to which the MOH, DHMTs, and facilities collaborated with the project. Burkina Faso, Côte d’Ivoire, Niger, and Togo accepted and welcomed innovations in their facilities and implemented 10 HIPs. All indicators exceeded the agreed acceptable margin of 20 percent. Table 9 shows that AgirPF met both output indicators, and greatly exceeded expectations in terms of organizing regional technical meetings (269% over targets). Table 9. IR 2 Output indicators, regional totals Indicator Number Indicator Target Achieved Rate 19 Number of HIPs piloted through implementation research 10 10 100% 20 Number of regional technical meetings organized and supported by AgirPF and its partners 16 43 268.7% IR 3: Output results Under IR 3, AgirPF sought to coordinate efforts to remove policy barriers and improve contraceptive commodity security. Improving the policy enabling environment for FP programs is part of the EngenderHealth SEED model which was incorporated into the program. To address the weak political support for FP programs, as well as socio-cultural obstacles for FP access and use, AgirPF, in close collaboration with the five implementing country governments, the Ouagadougou Partnership (OP), the West Africa Health Organization (WAHO), and other partners achieved or exceeded every IR 3 indicator target. Table 9 indicates that AgirPF exceeded the results for the five output indicators for IR 3 (see table 10). This success in fostering enabling environments was due in part to: i. An advocacy strategy focused on the two key determinants of the policy environment of FP— political support and the socio-cultural climate—so that government officials are less fearful of seeing their efforts in favor of FP contradicted by religious leaders; ii. The element of “ownership,” which is critical for the credibility of advocacy messages and key stakeholder tools and advocacy plan production; The AgirPF Project Endline Evaluation Report 21 iii. Pertinent and compelling data presented with RAPID tools that persuaded public administrators and officials, as well as religious leaders, that investments in FP would strengthen socioeconomic development in their respective countries; iv. Institutional strengthening and skills transfer, which assured continuing FP advocacy; and v. The methodological approach of AgirPF and the regional partnership with WAHO, which increased the legitimacy and credibility of efforts at the country level EngenderHealth’s vision when developing the AgirPF project was to “achieve the reproductive intentions of women and couples in urban/peri-urban areas of Burkina Faso, Côte d’Ivoire, Mauritania, Niger, and Togo using FP services.” The project’s success under IR 3 was almost certainly due in part to opportune timing for fulfilling this vision, due to global FP coverage intentions. By the start of the project, there was broad support for the FP2020 objective of reaching 20 million FP users in developing countries, the Ouagadougou Partnership objective of reaching 2.2 million additional FP users in Francophone West Africa by 2020, and government willingness to consider FP a national priority that would both combat maternal and infant mortality and foster development, which are clearly reflected in the over-performance of output indicators under IR 3. Table 10. IR 3 Output indicators, regional totals Indicator Number Indicator Target Achieved Rate 21. a Number of policies or guidelines developed or changed with USG assistance to improve access to and use of FP and RH services 6 19 317% 21. b Number of countries with a line item in the national budget for FP 5 5 100% 22 Number of advocacy presentations created or updated (with Avenir Health and HP+) 6 47 783% 23 Number of advocacy activities conducted 35 101 288.6% 24 Number of formal agreements (MoU, policy, declaration, etc.) signed at the regional level 15 41 273% 25 Number of SDP reporting stock-outs of contraceptives per quarter14 n/a n/a n/a Challenges In sum, AgirPF met 83%, or 15 of 18 of its output result targets over the LOP. The project did not reach the remaining targets (indicators 6, 9, and 17) due to delayed project start-up, which took about six months in each country due to the need to complete the baseline evaluation prior to implementation. In Côte d’Ivoire, government delays in finalizing the National Budgeted FP Action Plan meant that implementation activities did not get started until November 2014. In Mauritania, AgirPF encountered administrative issues, namely registration, which delayed implementation until October 2015. Furthermore, challenges faced in securing government support in Mauritania, iterative strikes in Côte d’Ivoire and Togo, political and military unrest in Côte d’Ivoire and Burkina Faso, and 14 The indicator 25 was never defined and target set. USAID/WA was expected to finalize the indicator definition and never come back to AgirPF. Therefore, there are no baseline target nor endline results. The AgirPF Project Endline Evaluation Report 22 the institutional unavailability of FP products in Côte d’Ivoire and Burkina Faso, negatively impacted results. While these delays impeded the achievement of LOP indicator targets, AgirPF proactively managed shifting constraints and opportunities in each country. Outcome results AgirPF, working in partnership with government officials, achieved outcomes in terms of systems improvements, increases in service quality, access and use, and positive changes in health behaviors across the targeted states and districts. The following are AgirPF outcome results throughout the project life. Indicator 2 (SO): Modern contraceptive prevalence rate (mCPR) AgirPF measured the current use of FP methods during the endline survey (see Endline survey and SBCC campaign evaluation section below for more). Figure 6 compares the mCPR, a key outcome indicator for the program’s SO, among women aged 15-49 years living in the project’s intervention zones by the end of the project and the LOP target by country. The survey revealed that AgirPF achieved the target in Togo (48%), surpassed targets in Côte d’Ivoire (48% vs. 34%) and Niger (62% vs. 47%), but did not meet the target in Burkina Faso (38% vs. 47%). Not reaching the target in Burkina Faso was surprising given that in Burkina Faso, the project over￾performed against most of the indicators related to mCPR (number of additional users, total users). However, this result seems to be consistent with the most recent estimates of the mCPR in Burkina Faso. According to the 2017-2018 round of PMA2020, the mCPR is 26.4% among all women age 15- 49 years old and 30.1% among married women. The data is not disaggregated by area of residence, but such a national prevalence may correspond to an incidence in urban and peri-urban areas of 38%. In addition, a yet unpublished household survey conducted by the USAID E4D project in charge of evaluation and research found a similar mCPR in 2018 (40%) in the AgirPF intervention zone. Another surprising finding is the high (62%) mCPR in Niger. For example, in Maradi the mCPR in the AgirPF intervention zone was 44.2% according to the baseline assessment versus 6.9% found by the DHS 2012. Likewise, the mCPR in the AgirPF intervention area in Niamey was 44.3% at baseline compared to 31.8% (DHS 2012). According to the most recent 2016 PMA2020 round, mCPR in Niamey was 18.8% among all women and 27.8% among married women. Indeed, further research is Figure 6. Achieved mCPR in AgirPF intervention zone compared to the LOP targets The AgirPF Project Endline Evaluation Report 23 needed to identify what makes the intervention zones in Niger so different from the rest of the country including the developments that occurred before AgirPF started its interventions. In conclusion, AgirPF generally performed well for mCPR with marked variations across the countries. The evaluation team found unexpected results in Burkina Faso with a mCPR far below the target and Niger with a very high mCPR. While external sources corroborate the accuracy of the AgirPF results, the evaluators recommend further research to gain a deeper understanding of the questions the data present for the FP community. Indicator 5 (IR 1): Percent of FP service providers deemed technically competent based on an assessment according to national international or other defined standards (USAID RDCS). As an outcome under IR 1, AgirPF sought to build the capacity of FP service providers. This outcome, coupled with the provision of equipment and facilitative supervision, contributed to a substantial increase in provider competence. AgirPF measured provider competence with the EngenderHealth service quality evaluation tool across three dimensions of service quality including counseling based on REDI, and implant and IUD insertion and removal. The project defined competent providers as those scoring at least 85% for each of the four quality dimensions. AgirPF carried out competence evaluations on a quarterly basis during facilitative supervision. Figure 7 indicates the performance trends by country in PY5, demonstrating that only facilities in Niger failed to meet the 85% threshold (79% across the three components). Facilities in Niger were particularly weak at IUD insertion and removal, which dragged down overall scores. 85% 81% 81% 100% 87% 95% 85% 99% 87% 72% 70% 99% BURKINA FASO (N= 251) COTE D'IVOIRE (N=86) NIGER (N=43) TOGO (N=75) REDI Counselling Implant Insertion & Removal IUD Insertion & Removal Figure 7. Percentage of supervised providers technically deemed competent in REDI counselling, implants and IUD insertion and removal, during PY5Q2, by country The AgirPF Project Endline Evaluation Report 24 Indicator 18 (IR 2): Number of HIPs for family planning and maternal and child health and/or HIV/AIDS incorporated into national health protocols or standards (USAID RDCS). As part of IR 2, AgirPF selected and introduced nine FP High-Impact Practices (HIPs) in all of the intervention countries (see table 11) in the second year of the program. AgirPF implemented HIPs and continued their demonstration through PY5. AgirPF selected the HIPs based on demonstration and magnitude of impact on service utilization, including contraceptive use and continuation contribution to ensuring informed choice and volunteerism; and potential replication in a wide range of settings. The HIPs were instrumental to the main quantitative achievements as well as qualitative achievements regarding countries’ acceptance and willingness to continue their implementation and replication in other facilities. Table 11. AgirPF implemented High-impact Practices (HIPs) by country HIPs Burkina Faso Côte d’Ivoire Mauritania Niger Togo Service delivery • CHW/Task shifting X No n/a X X • PACFP X X n/a X X • FP/MCNH services X X X X X • Outreach services (FPSD, mobile services) X X X No X • SWT X X X No X • COPE for CS X X No X • Facilitative supervision X X X X X Enabling environment • Advocacy and policy development X X X X X Enhancement • Youth-friendly service provision X No n/a No X SBCC custom indicator and outcome indicators 13, 14, and 15. During the endline survey, the evaluation team measured a custom USAID indicator not found in the PMP and outcome indicators 13-15 (all IR 1) to evaluate AgirPF SBCC campaign results. These indicators include: • USAID Custom Indicator: Percent of audience who recall hearing or seeing a specific USG￾supported Family Planning/Reproductive Health (FP/RH) message; • Indicator 13: Percent of men and women with gender-equitable attitudes (indicator 13); • Indicator 14: Proportion of women citing lack of information on FP methods as a key barrier to use; and The AgirPF Project Endline Evaluation Report 25 • Indicator 15: Percent of women who discussed FP with husbands/partners, friends/family within the last 3 months. These indicators were developed in collaboration specifically for the SBCC activities that began half way through the project. In many cases, the program therefore lacked baseline data from which to measure behavior change. Capturing change was further complicated by the diversity of SBCC activities and the audiences they targeted (Endline survey and SBCC campaign evaluation section). USAID Custom Indicator: Percent of audience who recall hearing or seeing a specific USG-supported Family Planning/Reproductive Health (FP/RH) message Figure 8 and figure 9 show the percentage of women and men, respectively, who reported being reached by at least one of the SBCC activities (disaggregated by intervention zone and nonintervention zone). Figure 8 shows that the percentage of women reached with a USG-supported FP/RH message varied significantly across the four countries with significant differences between the intervention and the nonintervention areas. The campaign in Niger reached 75% of women in the intervention area and 82% in the nonintervention area, which was far greater than the other countries. However, the campaign lasted longer in Niger than in the other countries, especially the radio broadcasts, which is consistent with the data. SBCC campaign activities reached fewer than 16% of women in Côte d’Ivoire, as well as 14% of the women in the intervention area and 23% in the nonintervention area in Burkina Faso. In Togo, 40% of the surveyed women heard or saw a USG-supported FP/RH message. The various differences in reach are the result of the highly tailored and varied SBCC strategies used across the countries (see the The AgirPF SBCC campaign: messaging reach by activity and country section for more). Figure 9 patterns for men are very similar to those for women, except that Figure 8. Percentage of women who heard or saw a USG-supported FP/RH message by zone and country The AgirPF Project Endline Evaluation Report 26 the difference between the intervention and nonintervention area is significant only in Burkina Faso, where twice as many men were reached in the nonintervention area than in the intervention area. This is largely a reflection of one of the most successful SBCC activities in Burkina Faso targeting men specifically. Indicator 13 (IR 1): Percent of men and women with gender-equitable attitudes (USAID RDCS) This modified gender indicator measures to what extend gender-related issues impeded FP use in AgirPF implementing countries. The endline evaluation results revealed that gender-related issues (own opposition, husband/partner opposition, others’ opposition, religious prohibition) are still relatively frequent reasons for non-use in some countries (17% among women in the nonintervention areas of Burkina Faso and 11% among women in the intervention areas of Burkina Faso and the nonintervention area of Côte d’Ivoire). Among men, gender-related issues were cited by 10% to 18% in nonintervention zone in Côte d’Ivoire, Niger, and Togo and the intervention areas of Burkina Faso and Niger. It is notable that there was less opposition to FP in every country among women and men in the intervention zones, except for women in Niger, which suggests that program may have had an impact at reducing stigma and improving gender-equitable attitudes, at least as they pertain to FP. Indicator 14 (IR 1): Percent of women citing lack of information on FP methods as a key barrier to use In AgirPF implementing countries, women and men citing lack of information as a key barrier to FP use was still high. Figure 10 shows that a high percentage of the population is not using an FP method for lack of information, especially in Burkina Faso and Côte d’Ivoire (both sexes) and in Niger (among Figure 9. Percentage of men who heard or saw a USG-supported FP/RH message by zone and country The AgirPF Project Endline Evaluation Report 27 men). There is no significant difference between the intervention and the nonintervention areas. In particular, the percentage of women not using FP for lack of knowledge varies from 11% to 13% in Burkina Faso and Côte d’Ivoire and is under 4% in Niger and Togo. Among men, the percentage is higher: from 15% to 17% in Burkina and Côte d’Ivoire, 8 to 10% in Niger and 3% in Togo. The data indicates that there is still a long way to go in terms of sensitization about FP and SBC in Burkina, Côte d’Ivoire, and Niger. Figure 10. Percentage of the respondents citing lack of knowledge as a key barrier to FP use (either because they do not know about FP at all or they know about FP but lack more information) (*) Statistically significant at 5% (-) Not statistically significant at 5% Indicator 15: Percent of women who discussed FP with husbands/partners, friends/family within the past three months One of the expected results of the SBCC campaign was that couples would discuss FP more frequently than before to facilitate joint decision making. The evaluators asked respondents how often they discussed FP with their partner over the past three months. Figure 11 gives the percentages of couples who reported at least one discussion with the partner over the period. The results show that the prevalence of couple’s discussion varies a lot across the countries and, in some cases, significantly between the intervention and nonintervention areas. In contrast, there are only small gender variations. Discussion about FP is relatively frequent in Niger where half of the women reported it (55% in the intervention area and 50% in the nonintervention area). One-third of the women in the intervention area of Côte d’Ivoire and Togo reported dialogue about FP with their partners, compared to 23% in the nonintervention area in Côte d’Ivoire and 30% in Togo. Men in Togo report couple’s discussions about FP far more (46% and 41% in the two areas) than women (33% and 30%). Burkina Faso stands out with: (i) the lowest percentage of women and men reporting The AgirPF Project Endline Evaluation Report 28 discussion with the partner about FP (30% and below); and (ii) being the only country where couple discussion is more frequent in the nonintervention area. Endline survey and SBCC campaign evaluation In order to assess whether or not AgirPF SBCC interventions made a difference, the evaluation team compared performance indicator results in the intervention and nonintervention areas. The endline survey measured knowledge, attitudes, and FP practices through exposure to FP messages (through AgirPF SBCC Campaign), knowledge of FP (knowledge of any method, modern method, LARCs/PMs), use of FP methods (past and current use, method mix, reasons not using FP), Discussions about FP between partners/spouses Background characteristics of endline survey respondents The endline survey also included women’s and men’s background characteristics linked to FP knowledge, attitudes, and practices such as duration of residence in the surveyed area, age group, marital status, education, and number of living children. The evaluators controlled for these variables when evaluating the differences between the intervention and the nonintervention areas (see table 12). The time duration in the intervention area is an important element in explaining the chances of exposure to program interventions. As shown in the table above, most women surveyed in different countries (from 60% in Togo to 80% in Niger) resided in the intervention area at least five years (or since the beginning of the project in 2013). But the duration of residence varied by country. For example, 70% of women in Burkina Faso, 72% of women in Côte d’Ivoire, and 76% of women in Niger resided in the area of intervention for more than five years. There was no significant difference in the duration of residence Figure 11. Percentages of women and men reporting having discussed FP with their partners during the past three months, by zone and country (*) Significant at 5% (-) Not significant at 5%. The AgirPF Project Endline Evaluation Report 29 between women in the intervention and nonintervention areas, except in Burkina Faso, where 70% of the women in the intervention area resided for five years or more compared to 66% in the nonintervention area. In terms of age distribution, there was no significant difference between age groups in the intervention and the nonintervention zones except in Burkina Faso where 4% less women were in the 40-49 years old age group (11%) than in the nonintervention area (15%). This difference could be simply due to hazard in recruiting respondents. Table 12. Comparison of the background characteristics of the female respondents living in the intervention and nonintervention areas, by country Background characteristics Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 2,045 813 2,205 922 746 455 1,047 525 Duration of residence (*) (-) (-) (-) 0-1 year 13.1% 12.7% 10.1% 8.9% 6.7% 4.8% 0.0% 0.0% 2-4 years 16.7% 20.9% 18.0% 15.8% 16.6% 15.2% 36.8% 40.3% 5+ years 70.2% 66.4% 72.0% 75.3% 76.7% 80.0% 63.2% 59.7% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Age group (*) (-) (-) (-) 15-19 20.4% 22.1% 18.7% 16.8% 14.6% 10.3% 16.2% 14.1% 20-24 19.6% 18.1% 18.8% 18.9% 23.1% 23.1% 21.3% 19.2% 25-39 48.5% 44.4% 49.6% 51.4% 51.2% 52.5% 46.2% 51.9% 40-49 11.5% 15.4% 12.8% 12.9% 11.1% 14.1% 16.3% 14.8% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Marital status (-) (-) (*) (*) Currently in union 62.1% 62.9% 54.5% 50.4% 97.6% 93.4% 65.8% 72.8% Single 34.6% 32.3% 42.6% 47.1% 1.1% 3.1% 29.6% 22.4% Formerly in union 3.3% 4.8% 2.9% 2.5% 1.3% 3.5% 4.6% 4.8% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Education (*) (*) (-) (*) No education 31.6% 38.0% 21.1% 23.6% 54.0% 57.6% 13.2% 16.3% Primary 26.2% 27.1% 25.9% 31.7% 25.6% 22.9% 27.1% 32.0% Secondary 20.5% 21.2% 22.6% 19.0% 16.0% 13.8% 38.3% 35.4% High school 15.1% 8.5% 19.2% 16.6% 2.7% 4.0% 12.6% 10.9% University 6.6% 5.3% 11.2% 9.1% 1.7% 1.8% 8.8% 5.4% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Number of living children (-) (-) (-) (*) Mean number of living children 1.8 1.9 1.8 1.7 3.6 3.6 1.8 2.1 (-) Difference between the zones not significant at the 5% level (*) = Difference significant at the 5% level The distribution of the surveyed population by marital status varied widely in the selected countries. In Niger nearly 98% of the surveyed women lived in a union (either married or in a consensual union), which is a reflection of the country’s highly Islamic culture. As such, reproduction and fertility in Niger are viewed and discussed almost exclusively though the context of marriage, and entry into unions occurs much earlier than in the other study countries. In contrast, the percentage of women living in a union in Côte d’Ivoire (50%), Burkina Faso (62%), and Togo (73%), where common-law unions are much more permissible, demonstrated a much wider range. The difference in the distribution between the intervention and the nonintervention areas was not significant in Burkina Faso and Côte d’Ivoire, The AgirPF Project Endline Evaluation Report 30 while there were significant differences in the percentage of women living in a union in Niger (98% in the intervention area, 94% in the nonintervention area) and Togo (66% in the intervention area, 73% in the nonintervention area (94%). The level of education of the surveyed women varied significantly between the intervention and the nonintervention areas in Burkina Faso, Côte d’Ivoire, and Togo. Education varied from one country to another, but also within the same country according to the area of residence. Overall, women living in the nonintervention areas were less educated than those living the intervention areas. Taking into account the countries of residence, Niger’s education rate is lowest. Indeed, more than the half of women (54%-58%) had no education at all. This is in contrast to much lower rates of no education in Togo (13%-16%), Côte d’Ivoire (21%-24%), and Burkina Faso (32%-38%). The average number of living children per woman varied significantly, ranging from Niger (3.6) to Burkina Faso, Côte d’Ivoire, and Togo (between 1.8 and 2.1). The differences between the intervention and the nonintervention areas were not significant, except in Togo where women in the intervention area have a significantly lower number of children (1.8) than those living in the nonintervention area (2.1). The high average number of children per women in Niger may be explained by the aforementioned differences in marital status patterns between Niger and the other countries. Table 13 compares selected background characteristics of the men living in the intervention and nonintervention zone including duration of residence, age, marital status, and education. Differences in the background characteristics of men in Burkina Faso and Côte d’Ivoire are not significantly different, but they are in Niger and Togo, thus the need to control for the background characteristics when analyzing mCPR. Table 13. Comparison of the background characteristics of the male respondents living in the intervention and nonintervention areas, by country Background characteristics Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 886 377 1,184 461 331 240 537 272 Duration of residence (-) (-) (*) (-) 0-1 year 8.7% 7.7% 7.3% 6.1% 6.3% 2.1% 18.2% 15.8% 2-4 years 14.6% 11.7% 17.1% 14.5% 16.0% 12.5% 22.5% 16.5% 5+ years 76.7% 80.6% 75.6% 79.4% 77.6% 85.4% 59.2% 67.6% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Age group (-) (-) (-) (*) 15-19 17.9% 18.8% 15.5% 15.4% .6% 0.0% 13.0% 10.3% 20-24 18.5% 17.8% 14.3% 14.8% 6.0% 4.6% 20.9% 15.4% 25-39 44.9% 42.2% 45.0% 46.2% 49.8% 47.9% 46.6% 47.8% 40-49 18.6% 21.2% 25.3% 23.6% 43.5% 47.5% 19.6% 26.5% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Marital status (-) (-) (*) (*) The AgirPF Project Endline Evaluation Report 31 Background characteristics Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Currently in union 48.5% 54.1% 48.1% 48.2% 96.4% 99.2% 51.0% 60.7% Single 50.7% 45.4% 50.5% 50.1% 3.6% .8% 48.2% 36.4% Formerly in union .8% .5% 1.4% 1.7% 0.0% 0.0% .7% 2.9% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Education (-) (-) (-) (*) No education 21.1% 27.9% 10.8% 9.1% 43.2% 47.1% 2.8% 5.5% Primary 17.6% 20.2% 15.9% 21.0% 20.2% 15.8% 16.6% 21.7% Secondary 22.0% 25.7% 22.5% 26.7% 16.0% 20.0% 31.3% 36.4% High school 21.2% 14.9% 28.5% 23.4% 12.1% 8.8% 25.9% 23.2% University 18.1% 11.4% 22.4% 19.7% 8.5% 8.3% 23.5% 13.2% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% (-) Difference between the two zones not significant at the 5% level (*) = Difference significant at the 5% level The AgirPF SBCC campaign: Messaging reach by activity and country The objective of AgirPF’s SBCC component was to provide a unifying strategic framework for social and behavior change around FP in the region. In each country, the goal was to define a strategy that would excite and inspire stakeholders to adopt new behaviors. During its last two years, AgirPF implemented various SBCC activities in four of its five intervention countries (with the exception of Mauritania) selected based on the findings of an innovative research approach. In 2013, the William & Flora Hewlett Foundation commissioned Hope Consulting to study how FP programs could improve communication strategies using commercial segmentation: identifying segments of women with common FP needs, attitudes, and behaviors, rather than common demographic characteristics. The study identified five segments: Modern Elites, Healthy Proactive, Traditional Autonomists, Conservative Passives, and Sheltered Skeptics. With technical support from Camber Collective, AgirPF used this approach to pilot an SBCC strategy for Niger. During the segmentation process, women fell into five groups, each with different FP needs, attitudes and behaviors as seen figure 12. AgirPF decided to extend this new approach to the other implementation countries with the rationale that similarities in population size and culture would allow the approach to be successful. AgirPF then used a series of qualitative research studies, in combination with a re-analysis of key components of the project’s baseline results, to validate the segments in Burkina Faso, Togo, and Côte d’Ivoire and develop a systematic and evidenced-based SBCC strategy, which targeted women, men, youth and other influencers. Applying commercial marketing principles, this SBCC strategy continued in Niger, followed by Togo, Burkina Faso and Côte d’Ivoire. A diverse set of approaches were used to target these different priority segments and their influencers, including printed materials, social media, radio shows, videos, community outreach activities, interpersonal communication and an Inter Voice Response system (IVR). In each country, the strategy led to an SBCC campaign. AgirPF organized workshops and meetings with ministers and other stakeholders to validate strategies, launch SBCC campaigns, and ensure that The AgirPF Project Endline Evaluation Report 32 the segmentation approach supported and strengthened existing SBCC campaigns. AgirPF also signed sub-grants with local communication agencies and marketing organizations to develop campaign tools and implemented the main activities. For the ease of data collection, in each country survey participants were asked a limited number of questions to capture the exposure to these different activities of the campaign (the list is provided in Table 6 in the Overview section). Burkina Faso In Burkina Faso, SBCC activities consisted of the following: • Activities that surrounded promoting the behavior of an “upstanding man” or Nintringa.”15 These activities included: o Interactive debate sessions on two local radio and three TV programs promoting the desirable behavior of an “upstanding man” planning his family 15 An upstanding man is a man of integrity, caring for his family and already using family planning; a man that everyone in his community wants to emulate and called in local language “Nintiriga”. Figure 12. Nigeria segmentation example The AgirPF Project Endline Evaluation Report 33 o Recorded 55 reactions (15 of which were on video) during the radio debate sessions, to promote the "upstanding man" concept and to share personal experiences in SRH including PF o Rebroadcasted and continued discussions around the “upstanding man" concept on the two radio stations and Facebook TV (@nintiriga ou l’homme modèle) o Pre-tested, validated, and used three posters of the "upstanding man" and his family to moderate discussions during the TV broadcasts o Developed a song and a video clip to promote the “upstanding man” concept on TV and radio and during FP special days’ events; created a Facebook page to display campaign tools, group discussion reports with couples and youth, photos, and other campaign information • AgirPF developed a three-episode comic book, “Assibi et Salifou: la première fois” to improve youth and adolescent knowledge on SRH information and services in Togo. This comic book was adapted to the country context of Burkina Faso as “Tene et Ali; la premiere fois.” The Tene et Ali Facebook page was also used to host discussions related to youth SRH. Figure 13 shows that the Nintriga activities reached the highest target audience percentage in the intervention (12%) and the nonintervention areas (28%), while the two Ali & Tene activities reached an average of 3% of all respondents. Finally, the percentage of men and women reached is significantly higher in the nonintervention than in the intervention area for the Nintringa activities and for the Ali & Tene Facebook page. The rational explanation of this result is contamination given the near impossibility of concentrating the campaign on the targeted facilities only in an urban or peri-urban area. Figure 13. Percentage of women and men reached by different SBCC activities implemented in Burkina Faso The AgirPF Project Endline Evaluation Report 34 Côte d’Ivoire AgirPF implemented a diverse set of SBCC activities in Côte d’Ivoire (see figure 14 and below): • Produced, pretested, and used five videos to facilitate couple’s discussion sessions on FP in Abidjan; organized group discussion sessions for 600 couples using the videos on FP couple interaction • AgirPF developed a three-episode comic book, “Assibi et Salifou: la première fois” to improve youth and adolescent knowledge on SRH information and services in Togo. This comic book was adapted to each country context Cote d’Ivoire and became known as “Safi et Léo: la premiere fois.” • Organized young people's group sessions on sexual and reproductive health for 200 young people (20 to 25 years old) about how to start an FP conversation with parents and how to deal with peer pressure; seventy couples participated in these sessions via the Safi et Léo Facebook page (https://www.facebook.com /SAFIetLEO/) and national radio (radio CÔTE D’IVOIRE) during three talk shows • Developed a free, fun Android application, Plan my Life, for contraceptive decision-making and to encourage young people to create open and honest lines of communication about FP; promoted the app, which is available on Google Play, with Facebook advertisements developed by IROLINE and flyers placed in trendy locations in Abidjan Youth talks had the highest reach among women (11%) and men (16%) in the intervention and nonintervention areas. Couple discussions about FP reached 9% of women and 11% of men. The remaining three activities reached less than 6% of the surveyed population. Overall, the activities reached a higher percentage of men than women in the intervention and the nonintervention zones. All five activities reached more women in the intervention zones than in the nonintervention zones, although the difference is not statistically significant for couple talks and the Safi & Leo Facebook page. For men, there is no statistically significant difference between the percentages reached in the zones except for the “Plan My Life” mobile app. Indeed, significantly more men in the nonintervention area reported having heard about the app (5.2%), compared to 4.2% in the intervention area. In Côte d’Ivoire, AgirPF also supported the diffusion of messages about responsible parenting and birth spacing in places of worship such as mosques, churches, and temples. The endline evaluation survey reveals that among women, 14.4% of respondents in the intervention area and 21.5% in the nonintervention heard about responsible parenting and birth spacing in places of worship, whereas the messaging reached 17.8% of men in the intervention area and 23.1% in the nonintervention area. However, it difficult to know if these messages were USG-supported. For instance, it is possible that The AgirPF Project Endline Evaluation Report 35 an imam talked about responsible parenting, and even FP. As such, the evaluation team did not account for its FP messaging in places of worship when computing the percentage of women and men that heard or saw a USG-supported FP message. Niger In Niger, AgirPF conducted the following SBCC activities: • Developed and tested five posters to help the Sheltered Skeptics and Healthy Proactive segments to manage the side effects of contraceptives; distributed 400 copies to 36 intervention sites and five private health facilities. Produced five radio spots in two local languages (Haoussa and Djerma) broadcasted by five community radio stations (two in Niamey and three in Maradi) to help Sheltered Skeptics discuss FP with their spouses and peers • Organized a word of mouth campaign whereby 140 Healthy Proactives (80 in Niamey and 60 in Maradi) provided testimonials about their experience with FP and how they handled difficulties such as side effects and stigmatization • Recorded 12 testimonials and 16 messages on contraceptive methods in Haoussa and Djerma; tested and placed them on the IVR platform in collaboration with Voto Mobile, international agency for interactive voice response (IVR). The interviewers asked respondents in Niger about their exposure to AgirPF SBCC activities; figure 15 summarizes the responses to these questions. A majority of the respondents reported seeing or hearing a message about side effects management including 66% of women and 77% of men in the intervention area and 80% of women and men in the nonintervention area. The percentage is significantly higher in the nonintervention area; note that this information was aired through radio talks and was likely heard in intervention and nonintervention zones alike. The second most popular Figure 14. Percentage of women and men reached by different SBCC activities implemented in Côte d’Ivoire The AgirPF Project Endline Evaluation Report 36 activity, “received a free FP services coupon,” reached 22% of the women in the intervention area. Less than 9% of the men received free coupons. In West Africa in general and in Niger in particular, men usually do not go to FP facilities. It is therefore not suprising men received fewer coupons. The differences between the percentage of respondents reached in the intervention and the nonintervention area are not statistically significant for the free FP services coupon and the IVR. Twelve percent of women and 16% of men heard about the IVR in the intervention zones. In addition to conducting the household survey, the evaluation team analyzed first-hand data from the IVR server related to the calls received (see table 14). The system connected 94% or 1,027 calls out of 1,089 trials, resulting in 2,702 question-answer exchanges between the server and the callers over the 98 day period. Out of 766 calls that reached the introductory message, 62% selected the first choice offered by the server, “basic information about the FP methods,” compared to 38% for the second choice, “FP users’ testimonials”. This discrepancy in number of subscribers compared to the completed calls is due to the frequent trouble in internet connection. The high number of questions on primary FP information Table 14. AgirPF SBCC campaign interactive voice response (IVR) data in Niger, November 1, 2017 to February 6, 2018 Indicator Value Total number of calls received 1,089 Number of calls connected 1,027 Number of calls completed 618 Number of unique subscribers 392 Number of first-time Subscribers 384 Number of question-answers 2,702 Mean duration for all calls 1.4 minutes Mean duration for completed calls 1.7 minutes Multiple choice questions Dial 1 : To receive information about the FP methods Dial 2 : To listen to FP users’ testimonials Answers 1. FP methods 62.3% 477 2. Users’ testimonials 37.7% 289 Total 100% 766 Figure 15. Percentage of women and men reached by SBCC activities implemented in Niger The AgirPF Project Endline Evaluation Report 37 corroborate recent research findings from the Camber Collective highlighting that women and their partners have a critical need for information about RH/FP from reliable sources.16 Figure 16 shows the distribution of the callers inquiring about basic information about FP methods by method selected. Out of the 458 callers to receive information about the FP methods offered in Niger, 41% asked about pills, followed by injectable contraceptives and male condoms (16% respectively). Few callers asked about female condoms (3%) and LARCs/PMs—implants (9%), tubal ligation (8%), IUD (6%), and vasectomy (1.5%). This is consistent with the popularity of pills and injectable contraceptives among female FP users and weak interest about vasectomy in the country. The same pattern is observed for the calls that selected the second option related to FP users’ testimonials (figure 17). In fact, out of the 271 calls received, 61% selected user testimonials from pill users, 17% from injectable contraceptive users, and 14% from implant users. Noticeably, the callers were more interested in how to use the methods rather than on their side effects, except for the permanent methods (tubal litigation and vasectomy) for which the majority wanted to learn more about the side effects. Out of 458 calls, 302 selected reversible 16 Planification Familiale, “Vive mobilisation pour le projet d’Analyse de Demande Nationale de la Fondation Hewlett “https://partenariatouaga.org/vive-mobilisation-pour-le-projet-danalyse-de-demande-nationale-de-la-fondation￾hewlett/ Figure 16. Distribution of the calls about basic information for FP methods by method Figure 17. Distribution of the calls to listen to users’ testimonials by method Figure 18. Distribution of the calls inquiring about the methods according to the information requested The AgirPF Project Endline Evaluation Report 38 methods and 36 permanent methods. For the reversible methods, between 56% and 77% wanted to know about the definitions and the instructions for use of the method. In contrast, for the permanent methods, the side effects mattered more: 53% of the callers asked about them for tubal ligation and 50% for vasectomy. Togo For SBCC activities in Togo, AgirPF carried out the following: • Developed and tested five posters and five flyers to help Healthy Proactive segments and health professionals to manage the side effects of contraceptives; printed 1,100 posters and 24,000 flyers for 48 AgirPF supported sites and five private health facilities; produced and broadcasted 20 radio programs including interactive FP debate sessions in Lomé (10), Sokodé, (5) and Kara (5) • Recorded five videos of sketches about problematic family planning interactions between partners to facilitate 20 couple’s discussion sessions in the three cities. Overall, 300 couples participated in the sessions including 150 in Lomé, 75 in Kara and 75 in Sokodé • With the support of the US Embassy in Lomé, AgirPF developed a three-episode comic book, “Assibi et Salifou: la première fois” to improve youth and adolescent knowledge on SRH information and services (adapted as above in Burkina Faso and Côte d’Ivoire). In Togo, the respondents were asked about their exposure to the AgirPF SBCC campaign: diffusion of messages on the management of FP method side effects, couples’ discussions about FP, and the “Assibi et Salifou” comic book (see figure 19). As in Niger, the messages on the management of FP methods’ side effects reached more women and men in the intervention and the nonintervention areas than the other campaign elements. Overall, between 7% and 11% of women and men participated to the couples’ discussions in the two areas. Similar percentages read the comic book. The differences between the intervention and nonintervention areas are not significant among men and women for all activities except for the comic book for which significantly more men in the intervention areas have read it (9%) compared to men in the nonintervention area (7%). There are slight gender differences in the percentage reached by the SBCC activities in the intervention area. In contrast, in the nonintervention area, more men have heard about the messages on the management of FP method side effects than women and fewer read the comic book. Men also reported less participation to the couples’ discussions. The AgirPF Project Endline Evaluation Report 39 Figure 19. Percentage of women and men reached by different SBCC activities implemented in Togo Conclusion on the evaluation of the SBCC campaign In Year 3 and 4, AgirPF used an innovative method to design its SBCC activities and target the right audience for each activity. Because the campaign started late in the project, the SBCC campaigns lacked baseline data from which to track overall patterns of change. The so-called segmentation approach, originally a marketing approach, yielded activities designed reach each segment with a clear idea and a clear objective rather than a one-size-fists-all approach. In doing so, this approach can be a powerful tool in a context of limited resources; decision makers and implementers can focus resources and programming on the segments most likely to change their behavior. The implementation of this approach in the AgirPF context helped reach target populations (living in the intervention zone) and beyond (living in the nonintervention zone) with messages and experiences that have the potential to increase acceptance and use of FP use, including messages countering false rumors related to FP methods; experience sharing with users; and education on youth and adolescent sexuality through youth friendly channels (social media). However, further research to examine whether the SBCC campaigns contributed to increased levels of FP information-seeking, positive attitudes about FP, and actual FP use needs to be conducted to definitively establish its effectiveness. AgirPF impact on knowledge and use of family planning methods This section’s analysis goes into deeper findings about the differences in the knowledge and use of FP between the intervention and the nonintervention areas and across the respondent’s background characteristics in order to evaluate AgirPF’ impact. The AgirPF Project Endline Evaluation Report 40 Knowledge of family planning Knowledge of FP is measured through three indicators: the percentage of respondents who have ever heard about FP methods of any kind, the percentage who have heard about a modern FP method(s), and the percentage who have heard about long-acting reversible contraceptives and permanent methods (LARCs/PMs). Knowledge of any type of FP method Knowledge of FP was widespread in all four countries. The percentage of men and women who have heard about FP varies between 91% and 98% in Niger and Togo and between 84% and 90% in Burkina Faso and Côte d’Ivoire. Knowledge is slightly higher among women than among men in all countries and in the intervention and nonintervention areas (figure 20). Knowledge of FP does not vary significantly across the intervention and nonintervention areas except among women in Togo, where the difference is tiny (97% in the intervention area vs. 98% in the nonintervention area). Knowledge of FP evolved differently over time across the countries (figure 21). In Togo, knowledge was already widespread at baseline but increased from 93% to 97% in the intervention area and from 94% to 98% in the nonintervention area. In Niger, knowledge increased from 96% to 97% in the intervention area and 98% in the non-intervention area, respectively. FP knowledge increased significantly in the nonintervention area in Côte d’Ivoire from 83% to 90%, yet decreased slightly in the intervention zone (90% to 88%). Figure 20. Percentage of women and men who have heard about FP methods by zone and country (*) Significant at 5% (-) Not significant at 5% Figure 21. Comparison between the percentage of women who know an FP method between the baseline and the endline, by zone and country The AgirPF Project Endline Evaluation Report 41 In Burkina Faso, there was a slight decline in FP knowledge in both intervention and non-intervention areas, which mirrors some of the puzzling findings about mCPR discussed in the Results section. Near-universal knowledge of FP in most of the areas surveyed at baseline left little room for improvement for new interventions. Therefore, the impact of the project is not captured efficiently through this indicator on overall FP knowledge, but rather through knowledge of specific FP methods (see Methods known, below). Knowledge of modern methods Knowledge of modern FP methods is nearly as widespread in the intervention and nonintervention areas as knowledge of FP methods of any type (see figure 22). The percentage of men and women who have heard about modern FP methods varies from 96% to 98% in Togo, 90% to 98% in Niger 84% to 89% in Côte d’Ivoire and 83% to 88% in Burkina Faso. Women know more about modern FP than men, but differences are small. Differences in knowledge of modern FP do not vary significantly across the intervention and nonintervention areas except among women in Togo, where the difference is tiny (96% in the intervention area vs. 98% in the nonintervention area). Knowledge of long-acting reversible contraceptives and permanent methods (LARCs/PMs) Knowledge of long-acting reversible contraceptives and permanent methods (LARCs/PMs) is less widespread than knowledge of any type of methods and of modern methods (see figure 23). However, the percentage having heard about LARCs/PMs is relatively high in Togo (80-87%) and among women in Niger (88-92%) and Burkina Faso (74-77%). LARCs/PMs are less known in Côte d’Ivoire (43-61%) and among men in Burkina Faso and Niger (63-70%). Knowledge of LARCs/PMs is significantly higher in the intervention area than in the nonintervention area in Côte d’Ivoire (61% vs. 54% among Figure 23. Percentage of women and men who have heard about LARCs/PM by zone and country (*) Significant at 5 (-) Not significant at 5% Figure 22. Percentage of women and men who have heard about modern FP methods by zone and country (*) Significant at 5 (-) Not significant at 5% The AgirPF Project Endline Evaluation Report 42 women and 49% vs. 43% among men). It is also higher in Burkina Faso, but the difference is not statistically significant. In Niger and Togo, the percentage of women and men that know about LARCs/PMs is higher in the nonintervention areas, but the difference is not significant. Methods known Table 15. Percentage of the respondents who heard about specific method, by sex, area, and country Method Burkina Faso Côte d’Ivoire Niger Togo Women Men Women Men Women Men Women Men Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 2,073 718 886 377 2,205 922 1,184 461 746 455 331 240 1,047 525 537 272 Female sterilization 18% 19% 16% 15% 18% 16% 22% 19% 26% 23% 21% 15% 32% 36% 34% 35% Male sterilization 13% 11% 12% 11% 8% 8% 15% 12% 17% 14% 17% 8% 21% 25% 24% 24% IUD 45% 47% 30% 23% 23% 21% 20% 20% 63% 55% 25% 24% 53% 55% 39% 42% Implants 69% 67% 59% 55% 55% 48% 39% 31% 75% 81% 47% 64% 82% 83% 72% 77% Injectable 73% 73% 59% 61% 74% 70% 65% 59% 89% 93% 65% 80% 86% 90% 81% 85% Pills 82% 82% 75% 77% 84% 84% 79% 73% 95% 97% 88% 90% 82% 82% 75% 74% Male condoms 58% 56% 73% 72% 74% 80% 83% 81% 39% 40% 53% 55% 91% 93% 96% 96% Female condoms 34% 30% 34% 26% 52% 54% 55% 54% 25% 23% 21% 15% 75% 79% 80% 78% Emerg. Contr. 18% 13% 22% 11% 28% 29% 24% 21% 3% 3% 2% 2% 21% 18% 26% 17% Spermicides /Foam 9% 12% 8% 10% 11% 10% 13% 11% 6% 3% 3% 3% 13% 16% 17% 15% SDM 22% 25% 17% 16% 20% 25% 20% 24% 6% 5% 4% 4% 34% 30% 29% 30% LAM 11% 14% 6% 6% 13% 13% 8% 7% 39% 44% 24% 25% 26% 30% 20% 22% Rhythm method 23% 32% 20% 20% 38% 44% 32% 43% 4% 3% 6% 4% 66% 70% 64% 69% Natural FP 11% 18% 12% 13% 11% 14% 10% 9% 4% 5% 4% 6% 30% 28% 31% 31% Withdrawal 13% 14% 21% 12% 20% 23% 30% 37% 4% 7% 10% 13% 40% 43% 56% 63% Percentage for which difference between the intervention and the nonintervention areas is statistically significant at the 5% level (T-test) are highlighted in grey For each FP method, table 15 compares the percentages of respondents in the intervention and nonintervention areas who have heard about it by sex and country. The results reveal the following: • The most known methods by women are pills (from 82% to 97%), injectable contraceptives (70%-93%), and implants (67%-83%, except in Côte d’Ivoire where the percentage is 55% in the intervention area and 48% in the nonintervention area). Male condoms are also well known The AgirPF Project Endline Evaluation Report 43 by women as a contraceptive method in Côte d’Ivoire (74%-80%), as well as male and female condoms in Togo (75%-93%). • The most known methods by men are pills (73%-90%), injectable contraceptives (65%-80% in Niger and Togo, 59%-65% in Burkina Faso and Côte d’Ivoire), male condoms (72%-96%, except in Niger), female condoms, and implants in Togo (72%-80%). • The least known methods (by less than 20% of the respondents) are spermicides/foam, emergency contraception (except in Côte d’Ivoire), male sterilization, and natural FP (except in Togo). • For the majority of the methods, there is no significant difference between the intervention and nonintervention areas. • By country, Burkina Faso and Côte d’Ivoire have more significant differences between the two areas, followed by Niger. In Togo, knowledge of FP does not vary by area. The comparison between the intervention and nonintervention areas shows that in case of significant differences, the percentage is generally higher in AgirPF zone. Some exceptions exist however, for instance in Niger where some methods are most known outside AgirPF intervention zone (injectables, implants, withdrawal) and in Côte d’Ivoire (male condoms, SDM, rhythm method, withdrawal). The fact that injectables are most known outside AgirPF zones may be due to the introduction of Sayana press in Niger that was an opportunity of mass sensitization on self￾administrated injectables. Use of family planning methods Past use of FP methods The percentage of women and men having ever used an FP method is slightly higher in the nonintervention than in the intervention zones, except in Burkina Faso. The percentage of women who reported they have used an FP method in the past is relatively high in Côte d’Ivoire, Niger, and Togo (from 72% to 87%) as shown in table 16. Those that have ever used FP methods is less widespread in Burkina Faso (61% in the intervention zone and 59% in the nonintervention area). The usage of modern contraception is less widespread but follows the same pattern. There are significant differences in methods that have been used for FP. In Burkina Faso, male condoms and pills were equally popular (around 26% of the women have used them as an FP strategy, respectively), followed by implants and injectable contraceptives (between 14% and 18%). In Togo, male condoms are by far the most popular method with 58% of women in the intervention area and 55% in the nonintervention area having used them as an FP strategy. Condoms are followed by implants, injectables, and pills, with 14% to 22% of the surveyed women reporting use. In Côte d’Ivoire, three methods are the most used: male condoms (45% in the intervention area and 60% in the nonintervention area), followed by pills (around 30%), and injectables (19% in the intervention area and 11% in the nonintervention area). In Niger, pills (around 49%), injectables (26% in the The AgirPF Project Endline Evaluation Report 44 intervention area and 31% in the nonintervention area) and implants (18% in the intervention area and 22% in the nonintervention area) are popular, yet male condoms were used at a much lower rate than other countries (2% in the intervention area, 5% in non-intervention areas). In all countries, permanent methods are not popular, with less than 0.5% of women its use. Table 16. FP method by zone and country Indicators Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 2,045 813 2,205 922 746 455 1,047 525 % who ever used an FP method 61.0% 59.4% 72.2% 75.9% 77.3% 80.0% 86.1% 86.7% % who ever used a modern FP method 59.5% 57.8% 68.5% 73.2% 76.4% 78.0% 77.3% 80.2% Modern Methods Female sterilization 0.4% 0.2% 0.3% 0.2% 0.3% 0.2% 0.1% 0.2% Male sterilization 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.1% 0.2% IUD 4.2% 4.4% 0.5% 1.0% 4.6% 3.5% 3.8% 3.2% Implant 17.9% 14.5% 5.9% 5.9% 18.4% 21.8% 16.0% 17.5% Injectable 17.8% 17.6% 18.9% 10.7% 26.0% 31.2% 17.5% 21.9% Pill 26.6% 26.7% 30.2% 31.0% 50.0% 48.8% 13.9% 17.1% Male condom 26.3% 24.4% 44.9% 60.2% 2.0% 5.3% 58.3% 54.5% Female condom 0.6% 0.5% 2.4% 1.8% 1.2% 1.5% 2.9% 2.5% Emergency contraceptive pill 5.1% 1.7% 8.1% 8.7% 0.1% 0.4% 3.7% 1.3% Spermicide/vaginal foaming tablet 0.2% 0.7% 1.2% 0.8% 0.5% 0.0% 0.9% 0.6% SDM 4.7% 9.0% 12.7% 21.4% 1.7% 2.0% 18.0% 16.8% LAM 1.2% 1.4% 5.9% 6.9% 10.6% 14.9% 10.6% 11.4% Traditional methods Natural FP 2.0% 3.6% 4.9% 7.0% 0.9% 2.9% 17.0% 13.9% Rhythm/counting days 8.6% 9.7% 27.0% 36.2% 0.1% 1.5% 50.0% 52.8% Withdrawal 2.4% 1.8% 7.4% 8.2% 0.9% 4.4% 17.1% 19.4% Other Other/Unspecified 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% Current Use of FP Methods The current use of FP methods was measured through three indicators: i. The percentage of respondents using any type of FP method by the time of the survey, i.e. the contraceptive prevalence rate (CPR); ii. The percentage of respondents using a modern FP method by the time of the survey, that i.e modern contraceptive prevalence rate (mCPR); and iii. The percentage currently using LARCs/PMs. The AgirPF Project Endline Evaluation Report 45 Current use of any type of method The CPR among men and among women varied widely across the four countries, but not significantly between the intervention and the nonintervention areas within each country (figure 24). The CPR among women ranges from 39% in the intervention zone in Burkina Faso to 64% in the intervention zone in Niger. It is higher in Niger, followed by Togo, then by Côte d’Ivoire. The lowest prevalence is recorded in Burkina Faso (39% and 42% for the two zones). The patterns of CPR among men differed from CPR among women across the four counties. Men in Togo have the highest prevalence (64%), followed closely by Niger (60% in the intervention area and 63% in the nonintervention area). Côte d’Ivoire follows with a prevalence between 50 and 52%. Again, Burkina Faso had the lowest prevalence (44% in the intervention area and 39% in the nonintervention area) Current use of modern FP methods The current use of modern contraception follows the same pattern as the use of any method, both among women and men (figure 25). Modern methods vary significantly across countries, but not significantly by zone. Among women, Niger has the highest mCPR (62% in the intervention area and 57% in the nonintervention area), followed by Côte d’Ivoire and Togo (48% to 51%). Burkina Faso lags behind with 38% in the intervention area and 42% in the nonintervention area. Among men, Togo has the highest prevalence, closely followed by Niger with mCPR ranging from 57% to 61%. Niger has an mCPR around 50% while Burkina records the lowest prevalence (44% in the intervention area and 39% in the nonintervention). (-) Significant at 5% (*) Not significant at 5% Figure 24. Contraceptive Prevalence Rate (CPR) among women and men, by zone and country The AgirPF Project Endline Evaluation Report 46 Current use of LARCs/PMs One of the objectives of AgirPF was to support MOHs to improve the availability of the full range of FP methods in its supported facilities, including LARCs and permanent methods. Figure 26 below examines the usage of LARCs/PMs among women and men in the intervention zone and compares their use by those living in the nonintervention area. The results show marked variations in the prevalence of the LARCs/PMs across the four countries and no significant disparities by zone except in Togo. The results also reveal that contrary to findings related to CPR and mCPR prevalence, Burkina Faso, along with Niger, are leading countries in the use of LARCs/PMs. Among women living in the intervention area, 13% in Burkina are currently using a LARC/PM and 17% in Niger. These percentages reach 18% in the nonintervention area. The (-) Significant at 5% (*) Not significant at 5% Figure 25. Modern Contraceptive Prevalence Rate (mCPR) among women and men, by zone and country (-) Significant at 5% (*) Not significant at 5% Figure 26. Percentage of women and men currently using LARCs/PMs, by zone and country The AgirPF Project Endline Evaluation Report 47 percentage using LARCs/PMs is also high in Togo (11% to 12%), but very low in Côte d’Ivoire (less than 6%). The same pattern is observed among men but at lower rates. Current use of modern FP methods by background characteristics The mCPR varies according to the background characteristics of respondents (tables 17-18). Regarding the duration of residence, it is interesting to note that in Burkina Faso, Côte d’Ivoire and Niger, the mCPR is higher in the intervention area for those residing in the area since the beginning of the project and smaller for those who moved to the area recently (1 year ago or less). Variation in mCPR by age group depends on the country. In all countries but Niger, the mCPR is higher in the age group 20-39 years and lower among adolescents. Niger is the exception with the lowest prevalence recorded among the oldest age bracket of WRA (40-49 years). Interestingly, the mCPR among the adolescents in the intervention area (66%) is even higher than among those 25-39 years old (62%). The use of modern contraception varies substantially by marital status in all countries and in the intervention and nonintervention areas as well. In all countries except Togo, the mCPR is higher among those living in union and lower among women formerly in union. In Togo, the situation is quite different. The highest prevalence is recorded among the single (56% vs. 48 and 49% among the other groups). The variation of the mCPR by education is consistent with what is already known about the relationship between education and contraceptive use: modern contraceptive use is higher among better-educated women and less widespread among the less educated. The use of modern contraception also depends on the number of living children per woman. In general, it is lowest among women with no children (perhaps due to a desire for children) and among women with five living children or more. Despite their potential need for contraception, this low prevalence may reflect the fact they use traditional methods or they do not use FP methods at all for other reasons linked to older age such as lower levels of education and a lack of information about modern FP. The prevalence is higher among women with 1-4 living children. The situation in Niger varies somewhat from this general pattern, where mCPR is very low among childless women (10% in the intervention area and 6% in the nonintervention) but relatively high among women with 5 living children or plus (51% in the intervention area and 49% in the nonintervention), albeit still less than those with 1-4 living children. The AgirPF Project Endline Evaluation Report 48 Table 17. Modern FP methods among women by background characteristic, zone, and country Background Characteristic Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 2,045 813 2,205 922 746 455 1,047 525 Total 38.1% 41.5% 48.2% 47.2% 61.7% 57.1% 48.0% 50.5% Duration of residence 0-1 year 32.5% 40.6% 39.6% 43.9% 44.0% 31.8% 50.6% 50.5% 2-4 years 39.2% 38.8% 50.3% 51.4% 69.4% 46.4% 53.2% 54.5% 5+ years 38.9% 42.6% 48.9% 46.7% 61.5% 60.7% 45.7% 49.1% Age-group 15-19 18.4% 25.7% 32.9% 40.6% 66.1% 44.7% 37.5% 49.3% 20-24 44.4% 44.3% 49.4% 48.3% 68.0% 66.7% 56.5% 63.4% 25-39 46.3% 50.3% 56.2% 52.7% 62.0% 62.3% 51.4% 49.8% 40-49 28.5% 33.6% 37.8% 31.9% 41.0% 31.3% 37.6% 37.2% Marital status Currently in union 42.1% 48.0% 49.5% 47.5% 62.6% 60.0% 49.9% 49.1% Single 32.2% 31.7% 47.4% 48.4% 25.0% 28.6% 48.9% 55.5% Formerly in union 24.6% 20.0% 35.9% 17.4% 20.0% 6.3% 16.7% 48.0% Education No education 29.0% 33.7% 36.7% 30.7% 48.1% 54.6% 32.4% 37.9% Primary 42.2% 41.8% 46.0% 43.8% 80.6% 62.5% 44.2% 45.2% Secondary 38.4% 42.3% 47.0% 52.0% 74.8% 65.1% 49.4% 52.1% High school 46.0% 63.1% 56.1% 62.7% 65.0% 50.0% 57.3% 64.3% University 47.4% 52.5% 64.0% 63.1% 76.9% 25.0% 64.5% 82.1% Number of children 0 child 24.1% 26.1% 41.0% 44.3% 10.3% 6.3% 42.3% 52.3% 1 child 49.7% 49.2% 54.6% 52.5% 74.0% 65.6% 55.2% 49.5% 2 children 46.1% 52.2% 57.3% 54.8% 77.2% 71.0% 50.5% 47.9% 3 children 48.6% 53.6% 50.4% 51.1% 71.1% 68.4% 45.8% 50.6% 4 children 43.8% 50.0% 50.6% 36.4% 61.1% 64.6% 53.0% 59.1% 5+ children 33.2% 29.2% 41.3% 32.0% 50.6% 48.6% 47.6% 41.8% Table 18 reveals that current use of modern FP also varies according to men’s background characteristics and for each subgroup of men between the intervention and the nonintervention zones. In Burkina Faso, the mCPR is higher in the intervention area for those residing in the area for 2 years or more among adolescents and young adults, the single and formerly married, and men who attained secondary school and university. In Côte d’Ivoire, it is only among men with primary level of education that the mCPR is higher in the intervention zone (50% vs. 38%) and with secondary education (44% vs. 40%). In Niger, mCPR is higher in the intervention area than in the nonintervention among those arrived recently in their area, the aged 20-24, the singles and men who attained high school. In Togo, mCPR is higher in the intervention area only among those who lived in their area for 5 years or more, men aged 40-49 years, and men with no education or with high school level. The AgirPF Project Endline Evaluation Report 49 These findings defy easy analysis on the basis of background characteristics. In order to disentangle these background characteristics to determine the impact of AgirPF, multivariate analysis is discussed in the section below. Table 18. Modern FP methods among men by background characteristic, zone, and country Background Characteristic Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Inter￾vention Interv. Non￾interv. Interv. N 886 377 1,184 461 331 240 537 272 Total 43.6% 39.0% 48.8% 50.5% 57.1% 60.0% 59.6% 61.4% Duration of residence 0-1 year 42.9% 44.8% 44.2% 42.9% 47.6% 20.0% 52.0% 76.7% 2-4 years 45.7% 34.1% 50.7% 55.2% 62.3% 60.0% 62.0% 62.2% 5+ years 43.2% 39.1% 48.8% 50.3% 56.8% 61.0% 61.0% 57.6% Age-group 15-19 17.0% 5.6% 33.3% 39.4% 50.0% n/a 38.6% 39.3% 20-24 42.1% 37.3% 53.3% 54.4% 40.0% 36.4% 70.5% 71.4% 25-39 59.0% 56.0% 56.1% 56.8% 61.2% 66.1% 63.6% 69.2% 40-49 33.3% 36.3% 42.8% 43.1% 54.9% 56.1% 52.4% 50.0% Marital status Currently in union 47.2% 50.5% 47.8% 50.5% 58.6% 60.5% 58.0% 60.6% Single 40.3% 25.7% 50.8% 50.2% 16.7% 0.0% 61.8% 64.6% Formerly in union 28.6% 0.0% 11.8% 62.5% n/a n/a 25.0% 37.5% Education No education 31.6% 39.0% 31.3% 35.7% 49.0% 52.2% 60.0% 46.7% Primary 39.1% 42.1% 50.0% 38.1% 55.2% 68.4% 44.9% 50.8% Secondary 30.8% 20.6% 43.6% 39.8% 54.7% 66.7% 56.0% 60.6% High school 51.6% 55.4% 52.2% 65.7% 72.5% 61.9% 69.1% 58.7% University 68.1% 53.5% 57.4% 67.0% 85.7% 70.0% 64.3% 91.7% Evaluation of the net impact of AgirPF interventions on current use of modern FP The results presented in the previous sections revealed that use of modern FP does not vary significantly between AgirPF intervention and nonintervention areas. However there were significant gains in mCPR from baseline to endline in Niger (47% to 62%), Cote d’Ivoire (34% vs 48%); and Togo (40% vs 48%). In Burkina Faso, there was a dramatic and inexplicable demobilization (47% vs 38%) despite the good performance in related indicators such as the number of FP new users, number of total FP users, and CYP. However, differences may exist and be hidden by differences in background characteristics of the respondents in the two areas. In order to control for these confounding variables and measure the net impact of living in the intervention zone versus living in the nonintervention area, this section applies multivariate analysis. Evaluators used the logistic regression method to estimate the impact of the AgirPF intervention (captured through residency in the intervention zone) on the likelihood of using a modern FP method after controlling for backgrounding characteristics. Table 19 below presents the results of eight regression models, two per country (one for the women and one for the men). All models are statistically significant. The AgirPF Project Endline Evaluation Report 50 Table 19. Regression on background characteristics, zone, and country Variables & Modalities Burkina Faso Côte d’Ivoire Niger Togo Women Men Women Men Women Men Women Men Significance of the model at 5% level (Omnibus test) * * * * * * * * Zone * ns Ns ns ns Ns ns ns Intervention 0.806* 1.060 1.033 0.908 1.061 0.943 0.887 0.827 Nonintervention (Ref.) 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Duration of residence ns ns Ns ns * * ns ns 0-1 year 0.949 0.898 0.831 0.714 0.467* 0.347* 0.993 0.779 2-4 years 0.951 0.989 1.075 1.048 0.753 0.753 1.156 0.965 5+ years (Ref.) 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Age Group * * * * * Ns * * 15-19 (Ref.) 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 20-24 2.082* 3.257* 1.654* 1.996* 1.033 0.062 1.666* 3.743* 25-39 1.741* 7.221* 2.024* 2.520* 0.872 0.104 1.061 3.774* 40-49 (W) / 40-59 (M) 0.917 3.029* 1.122 1.621* 0.375* 0.076 0.654 2.573* Marital Status * ns * ns * Ns * ns Currently in union (Ref.) 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Single 1.570* 0.765 1.269* 1.201 0.565 0.049* 1.649* 1.376 Formerly in union 0.455* 0.475 0.602* 0.452 0.062* n/a 0.418* 0.427 Education * * * * * * * * No education 0.360* 0.261* 0.311* 0.363* 0.356 0.230* 0.176* 0.540 Primary 0.593* 0.378* 0.471* 0.585* 1.000 0.334* 0.268* 0.463* Secondary 0.740 0.335* 0.639* 0.585* 0.941 0.385* 0.395* 0.725 High school 1.080 0.843 0.908 0.935 0.821 0.744 0.619 0.985 University (Ref.) 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 Number of children * * * * 0 child 0.252* 0.724 0.054* 0.221* 1 child 0.859 1.155 1.341 0.502* 2 children 1.019 1.396* 1.877* 0.603* 3 children 1.276 1.231 1.498* 0.666 4 children 1.369 1.125 1.323 1.234 5+ children (Ref.) 1.000 1.000 1.000 1.000 * Significant at 5% The logistic regression results confirm that living in AgirPF intervention areas has no net impact on the use of modern contraception vis-à-vis the non-intervention area after controlling for background characteristics, except among women in Burkina Faso. In that case, women living in the intervention area were 20% less likely to use a modern FP method than their counterparts living in the nonintervention area. We obtain similar findings when we conduct a logistic regression on the probability of currently using a LARC/PM. In other words, the impact of AgirPF interventions is not perceivable by comparing modern FP use in the two area, except among women in Burkina Faso. This does not necessarily mean that AgirPF The AgirPF Project Endline Evaluation Report 51 did not make a difference. This finding confirms once again the difficulty of evaluating the net impact of health interventions in urban and peri-urban settings where intervention and nonintervention areas are close by or intermingled. There was certainly contamination between the two areas since it is easy for an urban dweller to attend a health facility other than the closest one to his/her house, if for instance they organize special events like FP special days, are more appreciated by clients, are known for better service quality, etc. Moreover, the nonintervention areas are not control zones. They are not “sterile” from all interventions since other MoH partners were also implementing activities in the nonintervention areas to improve availability, accessibility, and quality of services. This is particularly true in Niger. Finally, the results of the logistic regressions reveal that for women, age, marital status, education and number of living children each have an individual and net impact on use of modern FP, independent of the other characteristics. Women of middle age (defined as 25-39 years), single, better educated, with living children are more likely to use modern contraception than the others. In addition, in Niger, women residing in the intervention area since the beginning of the project have greater probability to use modern FP than those who moved to the zone more recently. For men, the characteristics that matter are age and education, and, in Niger, duration of residence. Marital status has no significant effect on men’s use of modern FP. Method Mix Another way to evaluate whether AgirPF made a difference in terms of modern contraceptive use is to compare the method mix among modern FP users between the intervention and the nonintervention areas. Table 20 presents the results for female users by country and table 21 the results for male users by country. The method mix for women is significantly different between the two areas in Côte d’Ivoire (mainly with injectable contraceptive used most frequently in the intervention area and condoms used most in the nonintervention area) and in Niger (pills in the intervention area and injectable contraceptives and implants in the nonintervention area). In each country, the method mix is heavily skewed towards short-acting methods, especially in Côte d’Ivoire, Niger and Togo (ranging from 67% in nonintervention areas in Niger to 90% in nonintervention areas in Côte d’Ivoire). The situation is quite different in Burkina Faso, where only 65% of women in the intervention area and 56% in the nonintervention area resort to short-acting methods. The AgirPF Project Endline Evaluation Report 52 Table 20. Method mix among female users of modern contraceptives, by country The method mix varies substantially by country. In Burkina Faso, implants are the most used method (26% in the intervention area and 34% in the nonintervention), followed by condoms (27% and 19% in the two area), injectable contraceptives (16% and 20%), and pills (17% and 10%). Condoms are the most used method in Côte d’Ivoire, (36% in the intervention zone and 48% in the nonintervention zone) and Togo (48% and 43% in the two zones respectively), followed by pills and injectable contraceptives in Côte d’Ivoire and implants and injectable contraceptives in Togo. In Niger, pills and injectable contraceptives are the most popular methods (from 28% to 38%), followed by implants (22% in the intervention zone and 30% in the nonintervention). Condoms are rarely used (less than 4%). In all countries, IUDs are not popular (less than 10%), but sterilization and spermicides are the most unpopular methods, with a maximum of 1% of the users resorting to those methods. The method mixes among men also reveal significant differences between the two zones only in Côte d’Ivoire (in pills and injectable contraceptives most used in the intervention zone, and condoms most used in the nonintervention zone) and Niger (pills most used in the intervention zone, injectable contraceptives and implants most used in the nonintervention zone). Except in Niger, condoms are the most used method by men (from 52% in Burkina Faso to 74% in Togo). In Niger, pills are the most currently used method by men with their partners (45% in the intervention area and 35% in the nonintervention area). Other frequently used methods as reported by men are implants in Burkina Faso (19%-24%), pills in Côte d’Ivoire (20%-12%), injectable contraceptives in Niger (21%-27%) and implants in Togo (9%). Methods Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Inter￾vention Interv. Non￾interv. Interv. N 790 298 1,063 435 460 260 503 265 Sig. (-) (*) (*) (-) Short Acting methods 65.4% 56.0% 88.8% 90.1% 72.2% 66.9% 75.0% 77.7% LAM 0.8% 0.7% 1.6% 0.9% 5.4% 6.5% 2.0% 1.9% Emerg. contraception 0.1% 0.0% 0.7% 0.7% 0.0% 0.0% 0.6% 0.4% SDM 4.4% 6.4% 7.0% 6.2% 0.2% 0.0% 8.5% 9.4% Spermicides/Foam 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.4% 0.0% Condom 26.8% 19.1% 35.5% 47.6% 3.0% 0.8% 47.5% 42.6% Pills 17.0% 10.4% 23.2% 25.3% 37.6% 27.7% 5.2% 6.8% Injectable 16.2% 19.5% 20.9% 9.4% 25.9% 31.9% 10.7% 16.6% LARCs/PMs 34.6% 44.0% 11.2% 9.9% 27.8% 33.1% 25.0% 22.3% Implants 27.5% 33.9% 10.1% 8.7% 22.2% 29.6% 19.9% 18.5% IUD 6.6% 9.1% 0.6% 1.1% 5.2% 3.1% 5.0% 3.4% Sterilization 0.5% 1.0% 0.6% 0.0% 0.4% 0.4% 0.2% 0.4% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% The AgirPF Project Endline Evaluation Report 53 Table 21. Method mix among male users of modern contraceptives, by country Methods Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. Interv. Non￾interv. N 386 147 578 233 189 144 320 167 Sig. (-) (*) (*) (-) Short Acting methods 79.0% 70.1% 95.5% 96.1% 79.4% 77.8% 86.3% 90.4% LAM 0.0% 0.7% 0.0% 0.0% 3.7% 7.6% 0.3% 0.6% Emerg. contraception 0.5% 0.0% 0.0% 0.4% 0.0% 0.0% 0.9% 0.0% SDM 3.1% 1.4% 4.3% 3.0% 0.0% 0.0% 5.6% 3.0% Spermicides/Foam 0.0% 0.0% 0.2% 0.0% 0.5% 0.0% 0.3% 0.0% Condom 60.6% 52.4% 59.3% 75.1% 9.0% 7.6% 70.0% 73.7% Pills 8.8% 8.8% 19.6% 11.6% 45.0% 35.4% 3.4% 4.2% Injectable 6.0% 6.8% 12.1% 6.0% 21.2% 27.1% 5.6% 9.0% LARCs/PMs 21.0% 29.9% 4.5% 3.9% 20.6% 22.2% 13.8% 9.6% Implants 18.7% 25.9% 3.8% 3.4% 15.9% 22.2% 9.1% 9.0% IUD 1.8% 2.7% 0.5% 0.4% 3.7% 0.0% 3.4% 0.0% Sterilization 0.5% 1.4% 0.2% 0.0% 1.1% 0.0% 1.3% 0.6% Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% Main reasons for not using an FP method Women and men who know about FP but are not currently using an FP method were asked the main reasons of their non-use. The possible answers were grouped in five categories: • Fertility-related issues: not having sex, infrequent sex, menopausal/ hysterectomy, subfecund/infecund, - postpartum amenorrheic, breastfeeding, being fatalistic/having no control, desire to get pregnant; • Opposition to use: own opposition, husband/partner opposition, others' opposition, religious prohibition; • Lack of knowledge: know no method, know no source; • Method-related reasons: health concerns, fear of side effects, lack of access/too far, costs too much, inconvenient to use, interferes with body's normal processes; and • Other reasons/don't know/no response. Table 22 contains the percentage of women not using an FP method that cited each reason among women. Table 23 provides the same information for male non-users. Both tables reveal that fertility￾related concerns are by far the most cited reason (from 70% to 85% among women and from 68% to 86% among men). In comparison, the second most frequent reasons are cited less than 18%. The AgirPF Project Endline Evaluation Report 54 Table 22. Main reasons cited by women for not using an FP method, by country Background characteristics Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Inter￾vention Interv. Non￾interv. Interv. N 1,030 327 822 370 250 171 462 222 Fertility-related issues 74.5% 73.1% 78.5% 73.2% 69.6% 82.5% 84.8% 77.5% Opposition to use 10.6% 16.8% 6.4% 11.4% 9.6% 4.1% 4.5% 5.4% Lack of knowledge 0.3% 0.3% 2.8% 1.6% 0.8% 0.0% 0.6% 1.8% Method-related reasons 13.0% 17.1% 8.0% 13.2% 14.8% 9.4% 11.7% 12.6% Other reasons/Don't know/No response 9.0% 11.6% 8.8% 11.9% 8.4% 7.0% 3.7% 3.2% Table 23. Main reasons cited by men for not using an FP method, by country Background characteristics Burkina Faso Côte d'Ivoire Niger Togo Interv. Non￾interv. Interv. Non￾interv. Inter￾vention Interv. Non￾interv. Interv. N 361 167 433 151 100 74 181 90 Fertility-related issues 77.0% 80.8% 77.1% 66.9% 76.0% 67.6% 86.2% 81.1% Opposition 9.1% 6.0% 6.9% 16.6% 13.0% 17.6% 8.8% 11.1% Lack of knowledge 1.1% 1.8% 5.5% 2.6% 1.0% 4.1% 2.2% 0.0% Method-related reasons 10.2% 13.2% 5.1% 11.3% 10.0% 9.5% 8.8% 10.0% Other reasons/Don't know/No response 7.8% 9.0% 10.6% 14.6% 9.0% 10.8% 2.2% 0.0% Opposition to use and method-related issues are relatively frequent reasons for non-use in some countries (17% among women in the nonintervention area of Burkina Faso and 11% among women in the intervention area of Burkina Faso and the nonintervention area of Côte d’Ivoire). Among men, opposition to use is cited by 10% to 18% in nonintervention zone in Côte d’Ivoire, Niger and Togo and in the intervention area of Burkina Faso and Niger. Method-related reasons are equally cited except in Togo where it was more often cited (12%) than opposition to use (around 5 %). Non-use because of lack of knowledge was rarely cited (less than 3% among women and less than 6% among men). However, a more accurate measure of lack of knowledge as a key barrier to current use of FP should also include those who answered not knowing about FP at all, rather than just focusing on those who know but are not currently using as done in the two tables below. The results of this more accurate measure is presented in figure 27 below. Comparisons by zone reveal more opposition to use in the nonintervention than in the intervention areas in Burkia Faso, Côte d’Ivoire and Togo among women and in Côte d’Ivoire, Niger and Togo. Women and men in all countries except Niger also more frequently cite method-related reasons in the nonintervention area than in the intervention area. Figure 27 shows that a high percentage of the population is not using an FP method for lack of information, especially in Burkina Faso and Côte d’Ivoire (both sexes) and in Niger (among men). There is no significant difference between the intervention and the nonintervention area. The AgirPF Project Endline Evaluation Report 55 The percentage of women not using FP for lack of knowledge varies from 11% to 13% in Burkina Faso and Côte d’Ivoire and is under 4% in Niger and Togo. Among men, the percentage is higher: from 15% to 17% in Burkina and Côte d’Ivoire, 8 to 10% in Niger, and 3% in Togo. These findings indicate that there is still a long way to go in terms of sensitization about FP and SBC in Burkina, Côte d’Ivoire and Niger. Conclusion on knowledge and use of FP This evaluation of the impact of AgirPF in terms on FP knowledge and use do not lead to definitive conclusions mainly because it was not possible to overcome two major confounding effects. First, significant contamination of the nonintervention area by AgirPF’s interventions is highly likely because nonintervention facilities generally belong to the same district and are situated in urban or peri-urban areas where it is easy to reach several health facilities. The second confounding factor is the fact that the nonintervention area was actually not a control zone. Other partners of the MoHs were implementing activities in the nonintervention area to improve the accessibility and quality of health services. As the result of this contamination and interference, knowledge of FP does not vary significantly across the intervention and nonintervention areas except among women in Togo, where the difference is tiny (97% in the intervention area vs. 98% in the nonintervention area). The CPR among men and among women varies a lot across the four countries, but not significantly between the intervention and the nonintervention areas within each country. Marked variations in the prevalence of the LARCs/PMs across the four countries also exist but there is no significant disparity by zone, (-) Significant at 5% (*) Not significant at 5% Figure 27. Percentage of the respondents citing lack of knowledge as key barrier to use of FP (either because they do not know about FP at all or they know about FP but lack sufficient information), by zone and country The AgirPF Project Endline Evaluation Report 56 except in Togo. The logistic regression results confirm that living in AgirPF intervention has no net impact on the use of modern contraception after controlling for the background characteristics, except among women in Burkina Faso. In terms of method mix, there is no difference by zone except in Côte d’Ivoire. The main reason for non-use of FP is the exception to the lack of sensitivity to AgirPF interventions. In fact, the comparison by zone reveals more opposition to use in the nonintervention than in the intervention areas in Burkina Faso, Côte d’Ivoire, and Togo among women and in Côte d’Ivoire, Niger and Togo among men. Women and men in all countries except Niger also more frequently cite method-related reasons in the nonintervention area than in the intervention area. Discussion and conclusion The overall objective of the evaluation was to assess the performance and effectiveness of the project’s interventions and changes over time in key services, the enabling environment, and demand indicators in line with the baseline study and the mid-term evaluation results and recommendations. Specific objectives included: • Measuring AgirPFs progress against Performance Monitoring Plan (PMP) indicators over the course of the project • Contributing to the evaluation of the AgirPF SBCC campaign • Analyzing how changes in the population-level indicators are linked to AgirPF’s intervention Performance against the PMP indicators AgirPF’s performance against PMP indicators was highly positive. The program met or exceeded 15 of the 18 output indicators for the project, despite the broad reach of the project, high levels of complexity, and a range of implementation challenges ranging from office registration issues, to political turmoil, to complex differences in culture and demographic characteristics. Strategic objective indicators Indicators for the program’s strategic objective show a high level of performance by the program through meeting or exceeding overall targets for CYP, mCPR (outcome indicator), total FP users, and new modern contraceptive users. The SO output indicators were particularly impressive for Burkina Faso and Togo, which exceeded all their targets by significant margins. In the case of Burkina Faso, however, this performance masked one of the most confounding findings of significance in the evaluation—a mCPR that actually declined from the baseline assessment (47% to 34%) in intervention areas. However, other studies have found similar findings. According to the 2017-2018 round of PMA2020, the mCPR in the country is 26.4% among all women age 15-49 years The AgirPF Project Endline Evaluation Report 57 old and 30.1% among married women. The data is not disaggregated by area of residence, but such a national prevalence may correspond to an incidence in urban and peri-urban areas of 38%. Additionally, a yet unpublished household survey conducted by the USAID E4D project found a similar mCPR in 2018 (40%) in the AgirPF intervention zones. Further research is suggested to better understand this the underpinnings of this finding. Conversely, the program found unexpectedly high mCPR in Niger. In one city, for example, a DHS 2012 assessment found an mCPR rate of 6.9%, yet the AgirPF baseline found an mCPR of 44.2%. Across interventions zones in the country, the mCPR rate rose from 47% at the baseline to 62% at the time of the endline survey. Further research is required to understand why intervention zones differ so greatly from the rest of the country and/or to understand deviations from the DHS study. Judged purely on performance on SO indicators, Côte d’Ivoire had the lowest performance (29% good) followed by Mauritania (57%). However, the program also had to overcome particularly significant challenges in these countries, include 1- and 2- year delays in project start-up outside control of the program, respectively. In Côte d’Ivoire, the program also encountered particularly low capacity at intervention facilities—24% had never provided FP services of any kind. This low capacity required considerable remediation before the program could even begin program activities. Intermediate results 1, 2, and 3 Beyond the SO indicators, the program largely excelled under the intermediate result areas. Of particular note under IR 1, the program met 99% of its target training target, with 5,699 people trained in FP and RH. The program was also particularly successful at reaching youth, with over 318,101 participating in education programs (118% of targets). This positive engagement of youth could be seen in the demographic profile of beneficiaries with 45% falling in the 10-24 year age bracket. The program was also mostly successful with ensuring with technical competence of FP service providers, with providers in every country except Niger reaching EngenderHealth quality standards across several components. A range of IR 1 outcome indicators are covered in the AgirPF SBCC campaign section below. AgirPF was also successful at meeting targets for evidence-based service delivery as defined under IR 2. Facilities in Burkina Faso, Côte d’Ivoire, Niger, and Togo accepted and welcomed innovations in their facilities and implemented 10 HIPs, many of which were integrated into national health protocols. The program also over doubled the target number of regional technical trainings over the life of the program. Targets under IR 3 were all met or exceeded by the program, which highlighted AgirPF’s extremely effective approach to removing policy barriers around FP and contraceptives. Advocacy conducted under IR 3 led to 19 policies or guidelines changed to improve access to FP and RH services (317% The AgirPF Project Endline Evaluation Report 58 of targets), 41 formal agreements signed at the regional level (273% of targets), and over 101 advocacy activities conducted (289%), among others. Results of the AgirPF SBCC campaign AgirPF implemented a multi-faceted SBCC campaign based on findings from an innovative approach of segmenting the population according to their attitudes, behaviors, and potential in terms of FP use. The campaign reached the most people in Niger (75% in the intervention area were reached with a USG-supported FP/RH message, 82% in the nonintervention), where the SBCC strategy was first piloted and rolled-out. The SBCC campaign reached far fewer people in Côte d’Ivoire (less than 17% were reached) and Burkina Faso, where 14% of the women in the intervention area and 23% in the nonintervention were reached. Overall, in Togo, two fifths of the female respondents have heard or seen a USG-supported FP/RH message. In Côte d’Ivoire and Togo, more women living in the intervention areas were reached than in the nonintervention area. In Burkina Faso and Niger, the reverse was observed. Evaluating the SBCC campaign was not an easy task due to the contamination of the nonintervention areas by the activities implemented by AgirPF, on the one hand, and the fact that the nonintervention area was not a control area, since they also received many interventions by other partners of the MoH, especially in Niger. While this contamination frustrates the ability to attribute change to AgirPF, it likely means that the impact of the project was substantially greater than an anticipated as it reached many far beyond the narrow intervention zone. Most of the SBCC activities (posters, radio spots, Facebook pages, and comic books) could not simply be confined into the specific intervention area, since the intervention and the nonintervention areas are generally in the same districts. Because the campaigns began in years 3 and 4 of the program, there was likewise not sufficiently accurate baseline data with which to track changes in attitudes post-campaign. The extremely varied techniques, themes, and targeted audiences also defy easy analysis. Further research to examine whether the SBCC campaigns contribute to increased levels of FP information-seeking, positive attitudes about FP, and actual FP use need to be conducted to definitively establish its effectiveness. The SBCC was an opportunity for testing innovative communication approaches, beyond the overall strategic approach. In Niger, the use of IVR led to 1,027 calls successfully connected to the server out of 1,089 trials, resulting in 2,702 questions-answers between the server and the callers between 09/11/2017 and 02/06/2018. Overall 62% selected the first choice offered by the server “Basic information about the FP methods”, compared to 38% for the second choice: “FP users’ testimonials”. For a first experience, these figures are encouraging and confirmed recent research findings from Camber Collective highlighting that women belonging to the different segments and theirs partners have critical need for information about RH, including FP from reliable sources. The AgirPF Project Endline Evaluation Report 59 AgirPF’s link to FP knowledge and use by population-level indicators Knowledge of FP does not vary significantly across the intervention and nonintervention areas except among women in Togo, where the difference is tiny (97% in the intervention area vs. 98% in the nonintervention area). Near-universal knowledge of FP in most of the areas surveyed at baseline left little room for improvement for new interventions. Therefore, the impact of the project is not captured efficiently through this indicator of overall knowledge, but through the knowledge of specific methods. For the majority of the methods, there is no significant difference between the intervention and nonintervention areas. In Burkina Faso and Côte d’Ivoire there are more significant differences between the two areas exist. In Togo, knowledge of FP does not vary by area. In case of significant differences, knowledge is generally higher in AgirPF zone. Differences in knowledge of modern FP use does not vary significantly across the intervention and nonintervention areas except among women in Togo. Knowledge of LARCs/PMs is significantly higher in the intervention area than in the nonintervention area in Côte d’Ivoire (61% vs. 54% among women and 49% vs. 43% among men). It is also higher in Burkina Faso, but the difference is not statistically significant. In Niger and Togo, the percentage knowing LARCs/PMs is higher in the nonintervention areas, but the difference is not significant. FP use In Burkina Faso, Côte d’Ivoire and Niger, the mCPR is higher in the intervention area for those residing in the area since the beginning of the project and lower for those who moved to the area recently (1 year ago or less). The results show marked variations in the prevalence of the LARCs/PMs across the four countries and no significant disparities by zone except in Togo. Though mCPR is low in Burkina Faso, the country, along with Niger, is a leading country in the use of LARCs/PMs. The logistic regression results confirm that living in AgirPF intervention has no net impact on the use of modern contraception after controlling for the background characteristics, except among women in Burkina Faso. In that particular case, women living in the intervention area have 20% less chance of using a modern FP method than their counterparts living in the nonintervention area. We obtain similar findings when we conduct a logistic regression on the probability of currently using a LARC/PM. In other words, the impact of AgirPF interventions is not perceivable by comparing modern FP use in the two area, except among women in Burkina Faso. This does not necessarily mean that AgirPF did not make a difference, but is a consequence of the contamination of and interference in the nonintervention zone. The AgirPF Project Endline Evaluation Report 60 Method mix The women’s method mix is significantly different between the two areas in Côte d’Ivoire (mainly in injectable most used in the intervention area and condom most used in the nonintervention area) and in Niger. In each country, the method mix is heavily skewed towards short-acting methods, especially in Côte d’Ivoire, Niger and Togo (from 67% in nonintervention area in Niger to 90% in nonintervention area in Côte d’Ivoire use these methods). The situation is quite different in Burkina Faso, where only 65% of women in the intervention area and 56% in the nonintervention area resort to short-acting methods. The method mixes among men also reveal significant differences between the two zones only in Côte d’Ivoire (in pills and injectable most used in the intervention zone, and condom most used in the nonintervention zone) and Niger. Reason for not using an FP method Fertility-related concerns is by far the most cited reason (from 70% to 85% among women and from 68% to 86% among men) for not using FP methods. In comparison, the second most frequent reasons are cited less than 18% of the time. Comparisons by zone reveal more opposition to use in the nonintervention than in the intervention area in Burkina, Côte d’Ivoire and Togo among women and in Côte d’Ivoire, Niger and Togo. Women and men in all countries except Niger also more frequently cite method-related reasons in the nonintervention area than in the intervention area. A high percentage of the population is not using an FP method for lack of information, especially in Burkina Faso and Côte d’Ivoire (both sexes) and in Niger (among men). There is no significant difference between the intervention and the nonintervention area. The percentage of women not using FP for lack of knowledge varies from 11% to 13% in Burkina Faso and Côte d’Ivoire and is under 4% in Niger and Togo. Among men, the percentage is higher: from 15% to 17% in Burkina and Côte d’Ivoire, 8 to 10% in Niger and 3% in Togo. This gives indication that there is still a long way to go in terms of sensitization about FP and SBC in Burkina, Côte d’Ivoire and Niger. In sum, AgirPF performed well, but it is difficult to demonstrate that it made a difference in terms of FP knowledge and contraceptive prevalence. This does not mean that AgirPF interventions were not effective, but rather highlight the difficulty to evaluate the net impact of health interventions in urban and peri-urban settings with close intervention and nonintervention areas. There were certainly contamination between the two areas since it is easy for an urban dweller to attend a health facilities other than the closest one to his/her house, if for instance they organize special events like FP special days, are more appreciated by clients, are known for better service quality, etc. Moreover, the nonintervention areas are not control zone It is therefore important to pay special attention to the evaluation of future project at the time of the design of the project. The AgirPF Project Endline Evaluation Report 61 Bibliography 1. John Caldwell and Pat Caldwell “Fertility Transition in sub-Saharan Africa” (paper presented at Conference on Fertility and the Current South African Issues of Poverty, HIV/AIDS and Youth, Pretoria, South Africa, Oct. 24, 2002). 2. Donna Clifton, communication with author, 2008 3. Bongaarts J. Fertility transitions in developing countries: progress or stagnation? Stud Fam Plann 2008; 39: 105-10 doi: 10.1111/j.1728-4465.2008.00157.x pmid: 18678174. 4. Ezeh AC, Mberu BU, Emina JO. Stall in fertility decline in Eastern African countries: regional analysis of patterns, determinants and implications. Philos Trans R Soc Lond B Biol Sci 2009; 364: 2991-3007 doi: 10.1098/rstb.2009.0166pmid: 19770151. 5. Casterline JB. The pace of fertility transition: national patterns in the second half of the twentieth century. Popul Dev Rev 2001; 27: 17-52. 6. Coale AJ. The demographic transition reconsidered. In: International Union for the Scientific Study of Population. Proceedings of the international population conference 1973. Liège: Editions Ordina; 1973. 7. Lesthaeghe R, Vanderhoeft C. Ready, willing, and able: a conceptualization of transitions to new behavioral forms. In: Casterline JB, editor. Diffusion processes and fertility transition: selected perspectives. Washington: National Academy Press; 2001. pp. 240-64. 8. Bawah AA, Akweongo P, Simmons R, Phillips JF. Women’s fears and men’s anxieties: the impact of family planning on gender relations in northern Ghana. Stud Fam Plann 1999; 30: 54-66 doi: 10.1111/j.1728-4465.1999.00054.x pmid: 10216896. 9. Castle S, Konaté MK, Ulin PR, Martin S. A qualitative study of clandestine contraceptive use in urban Mali. Stud Fam Plann 1999; 30: 231-48 doi: 10.1111/j.1728- 4465.1999.00231.x pmid: 10546314. 10. Rutenberg N, Watkins SC. The buzz outside the clinics: conversations and contraception in Nyanza Province, Kenya. Stud Fam Plann 1997; 28: 290-307 doi: 10.2307/2137860 pmid: 9431650. 11. Hulton LA, Cullen R, Khalokho SW. Perceptions of the risks of sexual activity and their consequences among Ugandan adolescents. Stud Fam Plann 2000; 31: 35-46 doi: 10.1111/j.1728-4465.2000.00035.x pmid: 10765536. 12. Demographic and Health Surveys [Internet]. Calverton: Measure DHS; 2010. Available from: http://www.measuredhs.com/accesssurveys/ [accessed 2 November 2010]. 13. Tsui AO, Ochoa LH. Service proximity as a determinant of contraceptive behaviour: evidence for cross-national studies of survey data. In: Phillips J, Ross JA, editors. Family planning programmes and fertility. Oxford: Clarendon Press; 1992. pp. 222-56. The AgirPF Project Endline Evaluation Report 62 14. Cleland J, Ali MM. Sexual abstinence, contraception, and condom use by young African women: a secondary analysis of survey data. Lancet 2006; 368: 1788-93 doi: 10.1016/S0140- 6736(06)69738-9 pmid: 17113428. 15. Becker S. Couples and reproductive health: a review of couple studies. Stud Fam Plann 1996; 27: 291-306 doi: 10.2307/2138025 pmid: 8986028. 16. Becker S, Costenbader E. Husbands’ and wives’ reports of contraceptive use. Stud Fam Plann 2001; 32: 111-29 doi: 10.1111/j.1728-4465.2001.00111.x pmid: 11449861. 17. Miller K, Zulu EM, Watkins SC. Husband-wife survey responses in Malawi. Stud Fam Plann 2001; 32: 161-74 doi: 10.1111/j.1728-4465.2001.00161.x pmid: 11449864. 18. World population prospects, 2008 revision. New York: United Nations Department of Social and Economic Affairs; 2009. 19. Shah I, Ahman E. Unsafe abortion: global and regional incidence, trends, consequences, and challenges. J Obstet Gynaecol Can 2009; 31: 1149-58 pmid: 20085681. 20. Sedgh G, Henshaw S, Singh S, Ahman E, Shah IH. Induced abortion: estimated rates and trends worldwide. Lancet 2007; 370: 1338-45 doi: 10.1016/S0140-6736(07)61575- X pmid: 17933648. 21. Ross J, Stover J. The family planning program effort index: 1999 cycle. Int Fam Plan Perspect 2001; 27: 119-29 doi: 10.2307/2673833. The AgirPF Project Endline Evaluation Report 63 Appendices Appendix A: Questionnaires (Male and Female) Appendix A1: Women Questionnaire ENQUETE MENAGE POUR L’EVALUATION FINALE DU PROJET AgirPF IDENTIFICATION Nom Ville : _______________________________ Nom Région Sanitaire : _______________________________ Nom District Sanitaire : _______________________________ Nom Formation Sanitaire : _______________________________ Nom Localité/Quartier : _______________________________ CODE VILLE CODE REGION CODE DISTRICT CODE FS CODE LOCALITE/QUARTIER NUMERO DE ZD |__|__|__| NUMERO DU MENAGE NUMERO DE LA REPONDANTE Nom Enquêteur: _______________________________ Nom Superviseur: _______________________________ Date Interview (jj/mm/aa): /___/___/ /___/___/ /__1_/_8__/ Heure Début interview (hh/mn) :/___/___/ /___/___/ SECTION 1: CARACTERISTIQUES SOCIODEMOGRAPHIQUESET SOCIOECONOMIQUES DU REPONDANT Tout d’abord, je voudrais vous poser des questions sur vous et votre famille No Questions Modalités de réponses et codes 100 Depuis combien de temps habitez￾vous à [Nom de la localité/Quartier] ? Nombre d’années ............................................. Depuis toujours 66 Pas de réponse/Refus 77 Ne sait pas 99 101 Quel âge aviez-vous lors de votre dernier anniversaire ? ENCERCLEZ LE GROUPE D’AGE CORRESPONDANT 15-19ans ............................... 1 20-24ans ............................... 2 25-29ans ............................... 3 30-34ans ............................... 4 35-39ans................................ 5 40-44ans................................ 6 45-49ans................................ 7 Ne sait pas...........................99 Questionnaire des Femmes en âge de procréer (15-49 ans) The AgirPF Project Endline Evaluation Report 64 No Questions Modalités de réponses et codes 102 Quel est votre statut matrimonial ? SI MARIEE, SONDEZ POUR CONNAITRE LE NOMBRE DE COEPOUSES Mariée, monogame............... 1 Mariée, polygame ................. 2 Vivons ensemble................... 3 Célibataire ............................. 4 Divorcée................................. 5 Séparée ................................. 6 Veuve..................................... 7 Pas de réponse/Refus.........77 103 Avez-vous déjà fait des études ? SI OUI, quel est votre plus haut niveau d’éducation ? Pas été à l’école.................... 1 Primaire................................. 2 Secondaire 1......................... 3 Secondaire 2.......................... 4 Universitaire........................... 5 Pas de réponse/Refus........... 7 104 En dehors de votre propre ménage, faites-vous actuellement un autre travail ?(SONDER) Oui 1 Non 2 Pas de réponse/Refus 7 105 Combien de fils et de fillessurvivants avez-vous ? Nombre de garçon(s)...................................... Nombre de fille (s)........................................... SECTION 2: CONTRACEPTION Maintenant, je voudrais que nous parlions de la planification familiale, c’est-à-dire les différents moyens ou méthodes qu'un couple peut utiliser pour retarder ou éviter une grossesse. No Questions Modalités de réponses et codes 200 Avez-vous entendu parler de méthodes qu’un couple peut utiliser pour retarder ou éviter une grossesse ? Oui ...................................................................... 1 Non ..................................................................... 2 (Aller à300) Pas de réponse/Refus ........................................ 7 (Aller à300) Si « Oui » à 200, de quelle(s) voie(s)/méthode(s) permettant de retarder ou éviter les grossesses avez-vous entendu parler ? Ecrire le code 1 dans 201 pour chaque méthode mentionnée spontanément.Ensuite lisez le nom et la description de chaque méthode qui n’est pas mentionnée spontanément. Ecrire le code 1si la méthode est reconnue et le code 2 si elle n’est pas reconnue. Ensuite pour chaque méthode avec le code 1, posez 202 avant d’aller à la méthode suivante. PLUSIEURS REPONSES POSSIBLES.SONDER: D’autres méthodes ? 201. Avez￾vous déjà entendu parler de {méthode} ? 1 = Oui 2 = Non 7 = Refus 202. Avez￾vous déjà utilisé {méthode} ? 1 = Oui 2 = Non 7 = Refus a. PILULES : Les femmes peuvent prendre une pilule chaque jour pour éviter de tomber enceinte. b. DIU : Les femmes peuvent se faire placer un stérilet par une infirmière ou un docteur pour leur éviter les grossesses pendant plusieurs années. c. INJECTABLES: Les femmes peuvent avoir des injections qui peuvent leur éviter de tomber enceinte pendant plusieurs mois. d. COMPRIMES MOUSSANTS / GELEE: Les femmes peuvent se placer des cachets mousseux dans le vagin avant les rapports sexuels. The AgirPF Project Endline Evaluation Report 65 Si « Oui » à 200, de quelle(s) voie(s)/méthode(s) permettant de retarder ou éviter les grossesses avez-vous entendu parler ? Ecrire le code 1 dans 201 pour chaque méthode mentionnée spontanément.Ensuite lisez le nom et la description de chaque méthode qui n’est pas mentionnée spontanément. Ecrire le code 1si la méthode est reconnue et le code 2 si elle n’est pas reconnue. Ensuite pour chaque méthode avec le code 1, posez 202 avant d’aller à la méthode suivante. PLUSIEURS REPONSES POSSIBLES.SONDER: D’autres méthodes ? 201. Avez￾vous déjà entendu parler de {méthode} ? 1 = Oui 2 = Non 7 = Refus 202. Avez￾vous déjà utilisé {méthode} ? 1 = Oui 2 = Non 7 = Refus e. PRESERVATIF MASCULIN : Les hommes peuvent mettre une capote en caoutchouc sur leur pénis avant les rapports sexuels. f. PRESERVATIF FEMININ : Les femmespeuvent placer un fourreau dans leur vagin avant les rapportssexuels. g. Méthodes des Jours Fixes (MJF) : Une femme peut se servir de perles pour compter les jours de son cycle et éviter d'avoir des rapports sexuels les jours où elle est susceptible de tomber enceinte h. LA STERILISATION FEMININE: Les femmes peuvent subir une opération pour ne plus avoir d’enfants. i. LA STERILISATION MASCULINE: Les hommes peuvent subir une opération pour ne plus avoir d’enfants. j. IMPLANT : Les femmes peuvent se faire insérer par un médecin ou une infirmière un bâtonnet ou plus sous la peau du haut du bras pour les empêcher de tomber enceinte, pendant une annéeou plus. k. RYTHME, COMPTAGE DES JOURS: Une femme peut compter les jours de son cycle et éviter d’avoir des rapports sexuels les jours où il est probable qu’elle tombe enceinte. l. PLANIFICATION FAMILIALE NATURELLE:Une femme peut prendre sa température tous les jours ou vérifier sa mucosité vaginale pour déterminer les jours où il faut éviter des rapports sexuels. m. COIT INTERROMPU: L’homme peut faire attention et se retirer de la femme avant d’éjaculer. n. MAMA: L’allaitement exclusif au sein jusqu’à 6 mois peut constituer une protection naturelle contre les grossesses. o. CONTRACEPTION D’URGENCE : Une femme peut prendre une forte dose de la pilule dans les 24 heures qui suivent les rapports sexuels non protégés. p. Avez-vous entendu parler d’autres voies ou méthodes utilisées pour éviter les grossesses ? Précisez__________________________________ No Questions Modalités de réponses et codes 203 Vous et votre mari/partenaire faites-vous quelque chose ou utilisez-vous actuellement une méthode quelconque pour retarder ou éviter une grossesse ? Oui ..................................................1 Non ............................................. 2 (Aller à 205) Pas de réponse/Refus................ 7 (Aller à 205) The AgirPF Project Endline Evaluation Report 66 204 Si oui, quelle méthode utilisez-vous ? METTRE 1=OUI LORSQUE LA METHODE EST UTILISEE ET 0=NON DANS LE CAS CONTRAIRE PLUSIEURS REPONSES POSSIBLES. PASSER ENSUITE A Q300 Pilules combinées ou à progestatif seul………… DIU …………………….. Injectables……………... Implants……………….. Préservatif masculin….. Préservatifféminin…… Spermicides…………… Stérilisationféminine … Stérilisation masculine … Méthode des jours fixes (MJF)………. MAMA …………… Contraception d’urgence ………... Contraception naturelle …………. Refus…………….. Ne sait pas………. 205 Vérifiez 203. Si aucune méthode n'est utilisée, demandez : Vous avez dit que vous et votre mari/ partenaire n'utilisiez aucune méthode pour retarder ou éviter une grossesse. Pouvez￾vous me dire pourquoi vous et votre mari/ partenaire n'utilisez pas une méthode ? SONDEZ: Ya-t-il d’autres raisons ? METTRE 1=OUI LORSQU'UNE RAISON EST EVOQUEE PAR L'ENQUETEE ET 0=NON SI LA RAISON/MODALITE N'EST PAS EVOQUEE PLUSIEURS RÉPONSES POSSIBLES PROBLEMES DE FECONDITE 1. N’a pas de rapports sexuels ....................... 2. Rapports sexuels peu fréquents ................. 3. Ménopause/hystérectomie.......................... 4. Peu féconde ................................................ 5. Aménorrhée postpartum ............................. 6. Allaitement maternel ................................... 7. Fataliste/pas de contrôle............................. 8. Veut tomber enceinte.................................. OPPOSITION A L’UTILISATION 9. Répondant est opposé................................ 10.Mari/Partenaire est opposé .......................... 11.Autre personne opposée............................. 12.Interdiction religieuse .................................. MANQUE DE CONNAISSANCE 13.Ne connait pas de méthode ........................ 14.Ne connait pas de source ........................... RAISONS LIEES AUX METHODES 15.Problèmes de santé .................................... 16.Peur d’effets secondaires............................. 17.Manque d’accès/trop loin ............................ 18.Coûtent trop cher ........................................ 19.Incommode à utiliser................................... 20.Interfère avec les processus normaux du corps....................................................... 30. Autre (Précisez) : ....................................... 77. Pas de réponse/Refus ............................... 99. Aucune idée/Ne sait pas ............................. The AgirPF Project Endline Evaluation Report 67 SECTION 3: SBCC / ACTIVITES GENERATRICES DE DEMANDE No Questions Modalités de réponses et codes 300 Au cours des trois derniers mois, combien de fois avez-vous parlé à votre mari/partenaire d'une méthode de planification familiale ? Jamais 1 Une ou deux fois ......................................................... 2 Plus de deux fois......................................................... 3 Pas de réponse/Refus ................................................ 7 Ne sait pas 9 301 Au cours des 12 derniers mois, avez-vous vu des affiches ou entendu des messages radiophoniques sur la gestion des effets secondaires des méthodes de planification familiale? Oui ………1 Non 2 Pas deréponse/Refus ................................................. 7 302 Au cours des 12 derniers mois, avez-vous entendu parler d'un numéro gratuit que l'on peut appeler pour obtenir des informations gratuites sur la planification familiale? Oui ………1 Non 2 Pas de réponse/Refus ................................................7 303 Au cours des 12 derniers mois, avez-vous été contactée par quelqu'un qui vous a donné un coupon pour obtenir des services de planification familiale gratuitement dans les centres de santé? Oui ………1 Non 2 Pas de réponse/Refus 7 Merci d'avoir repondu à mes questions. Fin de l'interview de temps(hh/mn): The AgirPF Project Endline Evaluation Report 68 Appendix A 2 : Men Questionnaire ENQUETE MENAGE POUR L’EVALUATION FINALE DU PROJET AgirPF IDENTIFICATION Nom Ville : _______________________________ Nom Région Sanitaire : _______________________________ Nom District Sanitaire : _______________________________ Nom Formation Sanitaire : _______________________________ Nom Localité/Quartier : _______________________________ CODE VILLE CODE REGION CODE DISTRICT CODE FS CODE LOCALITE/QUARTIER NUMERO DE ZD |__|__|__| NUMERO DU MENAGE NUMERO DU REPONDANT Nom Enquêteur: _______________________________ Nom Superviseur: _______________________________ Date Interview (jj/mm/aa): /___/___/ /___/___/ /___/___/ Heure Déut interview (hh/mn) :/___/___/ /___/___/ SECTION 1: CARACTERISTIQUES SOCIODEMOGRAPHIQUES ET SOCIOECONOMIQUES DU REPONDANT Tout d’abord, je voudrais vous poser des questions sur vous et votre famille No Questions Modalités de réponses et codes 100 Depuis combien de temps habitez￾vous à [Nom de la localité/Quartier] ? Nombre d’années ............................................. Depuis toujours 66 Pas de réponse/Refus 77 Ne sait pas 99 101 Quel âge aviez-vous lors de votre dernier anniversaire ? ENCERCLEZ LE GROUPE D’AGE CORRESPONDANT 15-19ans ............................... 1 20-24ans ............................... 2 25-29ans ............................... 3 30-34ans ............................... 4 35-39 ans............................... 5 40-44ans................................6 45-49ans................................ 7 50-54 ans............................... 8 55-59 ans............................... 9 Ne sait pas...........................99 102 Quel est votre statut matrimonial ? SI MARIE, SONDEZ POUR CONNAITRE LE NOMBRE D’EPOUSES Marié, monogame................. 1 Marié, polygame.................... 2 Vivons ensemble................... 3 Célibataire ............................. 4 Divorcé................................... 5 Séparé ................................... 6 Veuf........................................ 7 Pas de réponse/Refus.........77 Questionnaire des Hommes âgés de 15 à 59 ans The AgirPF Project Endline Evaluation Report 69 No Questions Modalités de réponses et codes 103 Avez-vous déjà fait des études ? SI OUI, quel est votre plus haut niveau d’éducation ? Pas été à l’école.................... 1 Primaire................................. 2 Secondaire 1......................... 3 Secondaire 2.......................... 4 Universitaire........................... 5 Pas de réponse/Refus........... 7 104 En dehors de votre propre ménage, faites-vous actuellement un autre travail ?(SONDER) Oui 1 Non 2 Pas de réponse/Refus 7 SECTION 2: CONTRACEPTION Maintenant, je voudrais que nous parlions de la planification familiale, c’est-à-dire les différents moyens ou méthodes qu'un couple peut utiliser pour retarder ou éviter une grossesse. No Questions Modalités de réponses et codes 200 Avez-vous entendu parler de méthodes qu’un couple peut utiliser pour retarder ou éviter une grossesse ? Oui ...................................................................... 1 Non ..................................................................... 2 (Aller à300) Pas de réponse/Refus ........................................ 7 (Aller à300) Si « Oui » à 200, de quelle(s) voie(s)/méthode(s) permettant de retarder ou éviter les grossesses avez-vous entendu parler ? Ecrire le code 1 dans 201 pour chaque méthode mentionnée spontanément.Ensuite lisez le nom et la description de chaque méthode qui n’est pas mentionnée spontanément. Ecrire le code 1si la méthode est reconnue et le code 2 si elle n’est pas reconnue. Ensuite pour chaque méthode avec le code 1, posez 202 avant d’aller à la méthode suivante. PLUSIEURS REPONSES POSSIBLES.SONDER: D’autres méthodes ? 201. Avez￾vous déjà entendu parler de {méthode} ? 1 = Oui 2 = Non 7 = Refus 202. Avez￾vous déjà utilisé {méthode} ? 1 = Oui 2 = Non 7 = Refus a. PILULES : Les femmes peuvent prendre une pilule chaque jour pour éviter de tomber enceinte. b. DIU : Les femmes peuvent se faire placer un stérilet par une infirmière ou un docteur pour leur éviter les grossesses pendant plusieurs années. c. INJECTABLES: Les femmes peuvent avoir des injections qui peuvent leur éviter de tomber enceinte pendant plusieurs mois. d. COMPRIMES MOUSSANTS / GELEE: Les femmes peuvent se placer des cachets mousseux dans le vagin avant les rapports sexuels. e. PRESERVATIF MASCULIN : Les hommes peuvent mettre une capote en caoutchouc sur leur pénis avant les rapports sexuels. f. PRESERVATIF FEMININ : Les femmespeuvent placer un fourreau dans leur vagin avant les rapportssexuels. g. Méthodes des Jours Fixes (MJF) : Une femme peut se servir de perles pour compter les jours de son cycle et éviter d'avoir des rapports sexuels les jours où elle est susceptible de tomber enceinte h. LA STERILISATION FEMININE: Les femmes peuvent subir une opération pour ne plus avoir d’enfants. i. LA STERILISATION MASCULINE: Les hommes peuvent subir une opération pour ne plus avoir d’enfants. The AgirPF Project Endline Evaluation Report 70 Si « Oui » à 200, de quelle(s) voie(s)/méthode(s) permettant de retarder ou éviter les grossesses avez-vous entendu parler ? Ecrire le code 1 dans 201 pour chaque méthode mentionnée spontanément.Ensuite lisez le nom et la description de chaque méthode qui n’est pas mentionnée spontanément. Ecrire le code 1si la méthode est reconnue et le code 2 si elle n’est pas reconnue. Ensuite pour chaque méthode avec le code 1, posez 202 avant d’aller à la méthode suivante. PLUSIEURS REPONSES POSSIBLES.SONDER: D’autres méthodes ? 201. Avez￾vous déjà entendu parler de {méthode} ? 1 = Oui 2 = Non 7 = Refus 202. Avez￾vous déjà utilisé {méthode} ? 1 = Oui 2 = Non 7 = Refus j. IMPLANT : Les femmes peuvent se faire insérer par un médecin ou une infirmière un bâtonnet ou plus sous la peau du haut du bras pour les empêcher de tomber enceinte, pendant une annéeou plus. k. RYTHME, COMPTAGE DES JOURS: Une femme peut compter les jours de son cycle et éviter d’avoir des rapports sexuels les jours où il est probable qu’elle tombe enceinte. l. PLANIFICATION FAMILIALE NATURELLE:Une femme peut prendre sa température tous les jours ou vérifier sa mucosité vaginale pour déterminer les jours où il faut éviter des rapports sexuels. m. COIT INTERROMPU: L’homme peut faire attention et se retirer de la femme avant d’éjaculer. n. MAMA: L’allaitement exclusif au sein jusqu’à 6 mois peut constituer une protection naturelle contre les grossesses. o. CONTRACEPTION D’URGENCE : Une femme peut prendre une forte dose de la pilule dans les 24 heures qui suivent les rapports sexuels non protégés. p. Avez-vous entendu parler d’autres voies ou méthodes utilisées pour éviter les grossesses ? Précisez__________________________________ No Questions Modalités de réponses et codes 203 Vous et votre femme/partenaire faites-vous quelque chose ou utilisez-vous actuellement une méthode quelconque pour retarder ou éviter une grossesse ? Oui ..................................................1 Non ................................................. 2 (Aller à 205) Pas de réponse/Refus.................... 7 (Aller à 205) 204 Si oui, quelle méthode utilisez-vous ? METTRE 1=OUI LORSQUE LA METHODE EST UTILISEE ET 0=NON DANS LE CAS CONTRAIRE PLUSIEURS REPONSES POSSIBLES. PASSER ENSUITE A Q300 Pilules combinées ou à progestatif seul………… DIU …………………….. Injectables……………... Implants……………….. Préservatif masculin….. Préservatifféminin…… Spermicides…………… Stérilisation féminine … Stérilisation masculine … Méthode des jours fixes (MJF)………. MAMA …………… Contraception d’urgence ………... Contraception naturelle …………. Refus…………….. Ne sait pas………. The AgirPF Project Endline Evaluation Report 71 205 Vérifiez 203. Si aucune méthode n'est utilisée, demandez : Vous avez dit que vous et votre femme/ partenaire n'utilisiez aucune méthode pour retarder ou éviter une grossesse. Pouvez￾vous me dire pourquoi vous et votre femme/ partenaire n'utilisez pas une méthode ? SONDEZ: Ya-t-il d’autres raisons ? METTRE 1=OUI LORSQU'UNE RAISON EST EVOQUEE PAR L'ENQUETEE ET 0=NON SI LA RAISON/MODALITE N'EST PAS EVOQUEE PLUSIEURS RÉPONSES POSSIBLES PROBLEMES DE FECONDITE 1. N’a pas de rapports sexuels................. 2. Rapports sexuels peu fréquents ................. 3. Ménopause/hystérectomie.......................... 4. Peu féconde ................................................ 5. Aménorrhée postpartum ............................. 6. Allaitement maternel ................................... 7. Fataliste/pas de contrôle............................. 8. Veut tomber enceinte.................................. OPPOSITION A L’UTILISATION 9. Répondant est opposé................................ 10.Mari/Partenaire est opposé .......................... 11.Autre personne opposée............................. 12.Interdiction religieuse .................................. MANQUE DE CONNAISSANCE 13.Ne connait pas de méthode ........................ 14.Ne connait pas de source ........................... RAISONS LIEES AUX METHODES 15.Problèmes de santé .................................... 16.Peur d’effets secondaires............................. 17.Manque d’accès/trop loin ............................ 18.Coûtent trop cher ........................................ 19.Incommode à utiliser................................... 20.Interfère avec les processus normaux du corps....................................................... 30. Autre (Précisez) : ....................................... 77. Pas de réponse/Refus ............................... 99. Aucune idée/Ne sait pas ............................. SECTION 3: SBCC / ACTIVITES GENERATRICES DE DEMANDE No Questions Modalités de réponses et codes 300 Au cours des trois derniers mois, combien de fois avez-vous parlé à votre femme/partenaire d'une méthode de planification familiale ? Jamais 1 Une ou deux fois ......................................................... 2 Plus de deux fois......................................................... 3 Pas de réponse/Refus ................................................ 7 Ne sait pas 9 The AgirPF Project Endline Evaluation Report 72 No Questions Modalités de réponses et codes 301 Au cours des 12 derniers mois, avez-vous vu des affiches ou entendu des messages radiophoniques sur la gestion des effets secondaires des méthodes de planification familiale? Oui ………1 Non 2 Pas deréponse/Refus ................................................. 7 302 Au cours des 12 derniers mois, avez-vous entendu parler d'un numéro gratuit que l'on peut appeler pour obtenir des informations gratuites sur la planification familiale? Oui ………1 Non 2 Pas de réponse/Refus ................................................7 303 Au cours des 12 derniers mois, avez-vous été contactée par quelqu'un qui vous a donné un coupon pour obtenir des services de planification familiale gratuitement dans les centres de santé? Oui ………1 Non 2 Pas de réponse/Refus 7 Merci d'avoir repondu à mes questions. Fin de l'interview de temps(hh/mn): The AgirPF Project Endline Evaluation Report 73 Appendix B: Number of intervention and nonintervention health facilities by health district Table 24. Number of intervention and nonintervention health facilities by health district in the four countries City Health Districts Intervention health facilities Control health facilities Total District Health Facilities Number Percent Number Percent Number Percent Burkina Faso Ouaga￾dougou Baskuy 8 80.0 2 20.0 10 100 Bogodogo 10 58.8 5 29.4 17 100 Boulmiougou 9 81.8 2 18.2 11 100 Nongremassom 6 40.0 6 40.0 15 100 Signoghin 0 - 7 87.5 8 100 Sub-total Ouagadougou 33 54.1 22 36.1 61 100 Bobo Dioulasso Do 9 69.2 4 30.8 13 100 Dafra 4 50.0 4 50.0 8 100 Sub-total Bobo Dioulasso 13 61.9 8 38.1 21 100 Koudougou Koudougou 8 42.1 8 42.1 20 100 Sub-total Koudougou 8 42.1 8 42.1 19 100 Total Burkina Faso 54 53.5 38 37.6 101 100 Côte d’Ivoire Abidjan Abobo-Est 8 50.0 8 50.0 16 100 Abobo-Ouest 7 77.8 2 22.2 9 100 Anyama 9 50.0 9 50.0 18 100 Cocody-Bingerville 7 53.8 6 46.2 13 100 Koumassi-Port-Bouët-Vridi 11 64.7 6 35.3 17 100 Marcory-Treichville 2 33.3 4 66.7 6 100 Adjamé-Plateau-Attécoubé 10 76.9 3 23.1 13 100 Dabou 8 61.5 5 38.5 13 100 Yopougon-Est 10 83.3 2 16.7 12 100 Yopougon-Ouest 10 66.7 5 33.3 15 100 Total Côte d’Ivoire 8217 62.1 50 37.9 132 100 Niger Maradi District 1 Madarounfa 6 10.5 6 10.5 57 100 District 2 Maradi Commune 9 81.8 2 18.2 11 100 District 3 Aguie 3 4.3 3 4.3 69 100 District 5 Guidan Roumdji 3 2.4 3 2.4 126 100 Sub-total Maradi 21 8.0 14 5.3 263 100 Niamey District 7 Niamey I 1 5.3 1 5.3 19 100 District 6 Niamey II 10 45.5 10 45.5 22 100 District 8 Niamey IV 1 20.0 1 20.0 5 100 District 9 Niamey V 3 30.0 3 30.0 10 100 Sub-total Niamey 15 26.8 15 26.8 56 100 Total Niger 36 11.3 29 9.1 319 100 Togo Lomé District Sanitaire No 1 1 33.3 1 33.3 3 100 District Sanitaire No 2 3 75.0 1 25.0 4 100 District Sanitaire No 3 2 33.3 2 33.3 6 100 District Sanitaire No 4 1 33.3 0 0.0 3 100 District Sanitaire No 5 6 75.0 2 25.0 8 100 District du Golfe 6 60.0 3 30.0 10 100 Sous-total Lomé 19 55.9 9 26.5 34 100 Sokodé Tchaoudjo 14 46.7 7 23.3 30 100 Sous-total Sokodé 14 46.7 7 23.3 30 100 1717 The number felt down to 79 since 3 facilities were no longer part of the project for diverse reasons. The AgirPF Project Endline Evaluation Report 74 Kara Kozah 14 35.9 7 17.9 39 100 Binah 1 7.1 1 7.1 14 100 Sous total Kara 15 28.3 8 15.1 53 100 Total Togo 48 41.0 24 20.5 117 100 The AgirPF Project Endline Evaluation Report 75 Appendix C: Statistics for population sample calculation Table 25. Statistics for population sample calculation for the four countries Country City Response rate (%) CPR for Women (%) Proportion of men reporting dialogue with their partner about FP (%) Population estimates (number) Design effect Women (number) 15-49 Men 15-59 Women 15-49 Men 15-59 Burkina Faso Ouagadougu 95.7 93.3 32.6 46.0 361,601 363,964 1.288 Bobo 98.2 96.3 26.8 46.0 133,276 134,147 1.643 Koudougou 98.2 96.3 09.6 46.0 25,312 25,478 1.643 Côte d’Ivoire Abidjan 90.6 86.2 21.3 49.4 1,065,115 1.148 Niger Niamey 92.4 81.3 31.8 37.9 177,622 177,622 1.449 Maradi 93.2 80.9 23.6 37.9 37,136 37,136 1.399 Togo Lome 90.6 86.3 12.0 51.0 256,451 258,349 1.305 Sokode 93.9 90.6 13.7 51.0 29,091 29,091 1.199 Kara 93.9 90.6 18.6 51.0 29,032 29033 1.199 The AgirPF Project Endline Evaluation Report 76 Appendix D: AgirPF PMP Indicators’ Results Table Output Indicator Results Number Indicator Description and Type Indicator Definition (including how measured, disaggregation) Source of information / Data collection method and frequency Country Target Achievement Rate SO: Increase access to and use of quality FP services in select urban and peri-urban areas of five francophone West African countries 1 Number of CYP achieved in AgirPF supported areas (output indicator) (USAID RDCS) The estimated protection by family planning (FP) services during a one￾year period, based upon the volume of all contraceptives sold or distributed free of charge to clients during that period. MOH supported sites data from DHIS2, FP special days, Sub-grantee activity, WAAF, etc. Monthly Quarterly Burkina Faso 420,842 640,530 152% Côte d’Ivoire 643,882 219,512 34% Mauritania 71,997 25,394 35% Niger 351,444 221,420 63% Togo 194,836 258,882 133% Total 1,683,000 1,367,980 81% 3 Total number of FP method users (output indicator) The number of persons during a defined reference period (e.g., one year) who use a modern contraceptive method. These include all users accessing project supported sites/services for re￾supply, method changes, and/or new users. MOH supported sites data from DHIS2, FP special days, sub-grantee activity, WAAF, etc. Collect monthly Report quarterly Burkina Faso 528,454 562,361 106% Côte d’Ivoire 481,548 309,404 64% Mauritania 86,690 87,530 101% Niger 347,833 322,421 93% Togo 213,871 223,656 105% Total 1,658,396 1,505,372 91% 4 Number of additional users of modern methods of contraception (USAID RDCS) Number of new users of a modern method of contraception defined as someone who was not using a modern method of contraception when they received their method, including people who previously used a method, stopped, and are now starting a method. MOH supported sites data from DHIS2, FP special days, sub-grantee activity, WAAF, etc. Collect monthly Report quarterly Burkina Faso 173,824 214,016 123% Côte d’Ivoire 265,948 107,844 40.5% Mauritania 21,786 17,116 78.5% Niger 145,160 130,058 89.5% Togo 80,475 103,562 129% Total 687,193 574,974 84% Result 1: Delivery of quality FP information, products, and services strengthened and expanded Sub-result 1.1: Partners strengthened to implement evidence-based approaches and deliver quality FP services 6 Number of local organizations with improved The number of AgirPF local partners implementing AgirPF activities which are improving organizational and Routine supervision reports Burkina Faso 10 6 60% Côte d’Ivoire 12 3 40% Mauritania 5 3 60% The AgirPF Project Endline Evaluation Report 77 Number Indicator Description and Type Indicator Definition (including how measured, disaggregation) Source of information / Data collection method and frequency Country Target Achievement Rate organizational and management capacity (OCAT) Output indicator (USAID RDCS) managerial capacity. This improvement will be measured by using the OCAT tool. Annually Niger 8 3 37.5% Togo 7 6 86% Total 42 21 50% 7 Number of FP curricula updated to include gender sensitivity, couple counseling, youth and male friendly services Output indicator FP curriculum integrating gender sensitivity, couple counseling, youth and male friendly services utilized Activity reports Collect monthly report quarterly Burkina Faso 1 1 100% Côte d’Ivoire 1 1 100% Mauritania 1 1 100% Niger 1 1 100% Togo 1 1 100% Total 5 5 100% 8 Number of people trained in family planning and reproductive health with USG funds (output indicator) Number of people (health professionals, primary health care workers, community health workers, volunteers, non-health personnel) trained in FP/RH (including training in service delivery, communication, policy and systems, research, etc.). Activity reports Collect monthly report quarterly Burkina Faso 1296 1525 118% Côte d’Ivoire 1992 1727 87% Mauritania 480 424 88% Niger 864 1250 145% Togo 1152 1071 93% Total 5784 5699 98% 9 Number of HIV positive women who received comprehensive FP services (output indicator) This indicator informs about level of integration of FP services into HIV services. Activity reports, WAAF Collect monthly report quarterly Burkina Faso n/a n/a n/a Côte d’Ivoire 1200 389 32% Mauritania n/a n/a n/a Niger n/a n/a n/a Togo 100 262 262% Total 1300 651 50% 10 Number of special FP days conducted (output indicator) Special FP days are days where the range of FP services are offered free of charge by dedicated providers. Activity reports, daily consultation registers Burkina Faso 520 668 128% Côte d’Ivoire 500 282 56% Mauritania 300 165 55% Niger 395 223 56% The AgirPF Project Endline Evaluation Report 78 Number Indicator Description and Type Indicator Definition (including how measured, disaggregation) Source of information / Data collection method and frequency Country Target Achievement Rate monthly report quarterly Togo 520 516 99% Total 2235 1854 83% Sub-result 1.2: Local leaders, civil society, service providers, municipal government support and promote FP 11 Number of additional USG-assisted CHWs providing family planning information and/or services during the year (output indicator) (USAID RDCS) CHWs supported (trained, equipped with kits) and supervised Activity reports Collect monthly report quarterly Burkina Faso 75 127 282% Côte d’Ivoire 100 100 100% Mauritania18 N/A N/A N/A Niger 150 200 133% Togo 180 365 203% Total 505 792 156% 16 Number of youth who participate in educational program on gender, FP, and SRH (output indicator) Peer educators will lead discussions as moderators with enhanced knowledge on FP. This indicator also includes those reached by group discussion on FP services, WAAF and sub-grantee organizations Sign-in sheets, activity reports Collect monthly report quarterly Burkina Faso 60000 150 579 251% Côte d’Ivoire 45000 46 403 103% Mauritania 45000 442 9,8% Niger 60000 81 352 135% Togo 60000 39 325 66% Total 270,000 318,101 118% 17 Number of Site Walk-Throughs (SWT) conducted (output indicator) Guided tour in facilities by Local community leaders aiming at contributing to solve health issues Sign-in sheets, activity reports Collect monthly report quarterly Burkina Faso 84 37 44% Côte d’Ivoire 100 46 46% Mauritania 20 0 0 Niger 56 4 7% Togo 59 49 83% Total 329 136 41% Result 2: Evidence-based service delivery approaches selected, adapted, and implemented Sub-result 2.1: Efficiency and effectiveness enhanced through adaptation and implementation 19 Number of HIPs piloted through implementation research Implementation research (IR) have been conducted by the project. Implementation research focuses on understanding how programs are Implementation research reports Annually Burkina Faso 2 10 500% Côte d’Ivoire 2 10 500% Mauritania 1 3 300% Niger 3 5 167% 18 N/A stands for Not Applicable. It was not previewed to train CHWs in Mauritania The AgirPF Project Endline Evaluation Report 79 Number Indicator Description and Type Indicator Definition (including how measured, disaggregation) Source of information / Data collection method and frequency Country Target Achievement Rate (output indicator) (USAID RDCS) implemented, translated, replicated, and disseminated in “real-world” settings. Togo 4 10 250% Total n/a n/a n/a Sub-result 2.2: Lessons documented and disseminated from adaptation and implementation 20 Number of regional technical meetings organized and supported by AgirPF and its partners Output indicator (USAID RDCS) Use USAID Definition once it becomes available Reports Annually Burkina Faso n/a n/a n/a Côte d’Ivoire n/a n/a n/a Mauritania n/a n/a n/a Niger n/a n/a n/a Togo n/a n/a n/a Total 17 43 268.7% Result 3: Efforts to remove policy barriers and improve contraceptive commodity security coordinated Sub-result 3.1: Policy barriers identified and new/revised policies adopted and implemented 21a Number of policies or guidelines developed or changed with USG assistance (output indicator) (USAID RDCS) Number of policies, laws and guidelines introduced or updated as a result of USG-assistance related to improvement in family planning and reproductive health services Policies and guidelines Annually Burkina Faso 3 2 67% Côte d’Ivoire 3 1 33% Mauritania 3 2 67% Niger 3 0 0 Togo 3 3 100% Total 15 19 127% 21b Number of countries with a line item in the national budget for FP (USAID RDCS) Use USAID definition once it becomes available. Indicator activities begin in PY3 and are only deemed a success if because of AgirPF: (i) A country that had no line item adds a line item, or (ii) a country that had a line item increases the budgeted amount. Monitoring Report Collected quarterly reported annually Burkina Faso 1 1 100% Côte d’Ivoire 1 1 100% Mauritania 1 1 100% Niger 1 1 100% Togo 1 1 100% Total 5 5 100% 22 Number of advocacy presentations created AgirPF will support countries to develop or update country-specific Reports Burkina Faso 1 21 210% Côte d’Ivoire 1 4 400% The AgirPF Project Endline Evaluation Report 80 Number Indicator Description and Type Indicator Definition (including how measured, disaggregation) Source of information / Data collection method and frequency Country Target Achievement Rate or updated (in collaboration with Avenir Health and HP+) (output indicator) advocacy presentations, including RAPID models. Disaggregated by theme of advocacy presentation Quarterly Mauritania 1 4 400% Niger 1 7 700% Togo 1 11 1100% Total 5 47 783% 23 Number of advocacy activities conducted (output indicator) AgirPF will support the initial launch of advocacy activities for the RAPID presentation to policy makers at the country-level. Reports Quarterly Burkina Faso 6 25 4167% Côte d’Ivoire 5 17 340% Mauritania 7 15 214% Niger 6 12 200% Togo 7 32 457% Total 31 101 288.6% 24 Number of formal agreements (MoU, policy, declaration, etc.) that are signed at the regional level (USAID RDCS) Use USAID Definition once it becomes available Monitoring Report Collected quarterly reported annually Burkina Faso n/a n/a n/a Côte d’Ivoire n/a n/a n/a Mauritania n/a n/a n/a Niger n/a n/a n/a Togo n/a n/a n/a Total 15 41 273% Sub-result 3.2: Contraceptive commodity needs identified and coordinated among partners and country commodity security and logistics management 25 Number of SDP reporting stock-outs of contraceptives per quarter (output indicator) SDP reporting stock-outs of contraceptives per quarter. Disaggregated by method. Health facility stock reports, inventory reports Collect monthly report quarterly Burkina Faso TBD Côte d’Ivoire TBD Mauritania TBD Niger TBD Togo TBD Total TBD19 19 The targets failed to be defined due to no existing clear definition of the indicator. The AgirPF Project Endline Evaluation Report 81 Outcome Indicator Results Indicator Number Indicator Description and Type Indicator Definition Source of information / Data collection method and frequency Country Annual Target Notes Y1 Y2 Y3 Y4 Y5 Total LOP SO: Increase access to and use of quality FP services in select urban and peri-urban areas of five francophone West African countries 2 Contraceptive Prevalence Rate (CPR) (outcome indicator) (USAID RDCS) The proportion of women of reproductive age (WRA, age 15-49) who are using (or whose partner is using) a contraceptive method at a given point in time Numerator: number of WRA who self-report using FP Denominator: number of WRA surveyed For setting targets, the projected growth rate is 2% per year Baseline and end-line household Surveys Y120 & Y5 Burkina Faso 38.5 40.5 42.5 44.5 46.5 38 The results of Burkina Faso are surprising Côte d’Ivoire n/a 27.9 29.9 31.9 33.9 48 Mauritania n/a n/a 19.3 21.3 23.3 n/a Niger 39.2 41.2 43.2 45.2 47.2 62 Togo 40.3 42.3 44.3 46.3 48.3 48 Total n/a n/a n/a n/a n/a n/a Result 1: Delivery of quality FP information, products, and services strengthened and expanded 5 Percent of FP service providers deemed technically competent based on an assessment according to national international or other defined standards Outcome indicator (USAID RDCS) Health providers will be supervised performing the FP services and counseling they were trained in by AgirPF. They will be assessed based on international standards for competency. “Performing up to standards” will be defined as receiving a score of at least 85%. Facility audits, facilitative supervision, sub-grantees Quarterly Burkina Faso 80 n/a n/a n/a 97 121% Côte d’Ivoire 80 n/a n/a n/a 79 99% Mauritania21 80 n/a n/a n/a n/a n/a Niger 80 n/a n/a n/a 92 115% Togo 80 n/a n/a n/a 100 125% Total 80 n/a n/a n/a 92 115% 20 Y1 mCPR are estimates from baseline surveys conducted in the four AgirPF countries (Burkina, Côte d’Ivoire, Niger, and Togo). For Mauritania, data presented is an estimation from the MICS conducted in 2011 in this country. AgirPF baseline survey will be conducted in Nouakchott is year 3. 21 n/a stands for not available. Mauritania began supervising providers late in PY4 and closed the project before the other countries in February 2018. The AgirPF Project Endline Evaluation Report 82 Indicator Number Indicator Description and Type Indicator Definition Source of information / Data collection method and frequency Country Annual Target Notes Y1 Y2 Y3 Y4 Y5 Total LOP Sub-result 1.2: Local leaders, civil society, service providers, municipal government support and promote FP 12 Proportion of women and men reporting increased dialogue with their partner about FP (outcome indicator) A man or a woman is reporting dialoguing with their partners if during the last three months they discussed at least once FP issues including the choice and/or use of a given FP method. Pre and post￾Household KAPB surveys Y1 baseline, Y5 end-line Burkina Faso 52 n/a n/a n/a 65 20.4 Burkina Faso results surprisingly low Côte d’Ivoire n/a 40 n/a n/a 55 34.4 Mauritania n/a n/a n/a22 n/a n/a n/a Niger 39 n/a n/a n/a 50 54.9 Togo 44 n/a n/a n/a 55 32.6 Total n/a n/a n/a n/a n/a n/a 13 Percent of men and women with gender-equitable attitudes (outcome indicator) (USAID RDCS) (Modified gender indicator) Attitudes of women and men will be assessed to determine attitudes in terms of: their support for women’s sexual and reproductive rights; their support for women’s right to practice a contraceptive method; their support for men’s involvement in the promotion of women’s sexual and reproductive health; their support for joint decision￾making about FP; their support for consensual sex in a relationship; their support for women’s involvement in decision-making at the household level; their support for men’s involvement in child care; their resistance to all forms of violence against Household KAPB survey Y1 baseline, Y5 end-line Burkina Faso 41 n/a n/a n/a n/a 11 Côte d’Ivoire n/a 45 n/a n/a n/a 10 Mauritania n/a n/a n/a n/a n/a n/a Niger 22 n/a n/a n/a n/a 10 Togo 43 n/a n/a n/a n/a 10 Total n/a n/a n/a n/a n/a n/a 22 Mauritania’s target will be updated when Baseline data will be available. The AgirPF Project Endline Evaluation Report 83 Indicator Number Indicator Description and Type Indicator Definition Source of information / Data collection method and frequency Country Annual Target Notes Y1 Y2 Y3 Y4 Y5 Total LOP women; their support for women’s human rights. 14 Percent of women citing lack of information on FP methods as a key barrier to use (outcome indicator) Numerator: number of women citing lack of information on FP methods as a key barrier to use Baseline/Endline survey Y1 & Y5 Burkina Faso n/a n/a n/a n/a n/a 11.2 Côte d’Ivoire n/a n/a n/a n/a n/a 12.6 Mauritania n/a n/a n/a n/a n/a n/a Niger n/a n/a n/a n/a n/a 2.8 Togo n/a n/a n/a n/a n/a 3.6 Total n/a n/a n/a n/a n/a n/a 15 Percent of women who have discussed FP with husbands/partners, friends/family within the last three months (outcome indicator) Numerator: number of women who have discussed FP with husband/partner, friends/family within the last three months Baseline/Endline survey Y1 & Y5 Burkina Faso n/a n/a n/a n/a n/a 14.3 Côte d’Ivoire n/a n/a n/a n/a n/a 16.4 Mauritania n/a n/a n/a n/a n/a n/a Niger n/a n/a n/a n/a n/a 75.1 Togo n/a n/a n/a n/a n/a 41.7 Total n/a n/a n/a n/a n/a Result 2: Evidence-based service delivery approaches selected, adapted, and implemented 18 Number of HIPs for family planning and maternal and child health and/or HIV/AIDS incorporated into national health Evidence of HIPs for family planning and maternal and child health and/or HIV/AIDS incorporated into national health protocols or standards Step 1: HIP accepted by MOH Step 2: HIP replicated by MOH National health protocols or standards Collect monthly report quarterly Burkina Faso 5 n/a n/a n/a 10 200% Côte d’Ivoire 4 n/a n/a n/a 10 250% Mauritania 3 n/a n/a n/a 3 100% Niger 5 n/a n/a n/a 7 140% Togo 5 n/a n/a n/a 10 100% The AgirPF Project Endline Evaluation Report 84 Indicator Number Indicator Description and Type Indicator Definition Source of information / Data collection method and frequency Country Annual Target Notes Y1 Y2 Y3 Y4 Y5 Total LOP protocols or standards23 (outcome indicator) (USAID RDCS) Step 3: HIP scaled-up Step 4: HIP integrated into national guidelines Total n/a n/a n/a n/a n/a n/a 23 AgirPF counts a target reached once the HIP is integrated into national guidelines, not for any previous steps.